2:07 p.m.
This is a multi-part message in MIME format.
--------------04175AE2D1C2AC0FFF61E4CE
Content-Type: text/plain; charset=utf-8
Content-Transfer-Encoding: 7bit
Hi,
We have a failure in test post-002_bootstrap.py
from the logs I can see that lago has an unhandled exception from ovirt
sdk which makes sense since the patch reported on CQ is a change to the
sdk.
**
*Link to suspected patches: https://gerrit.ovirt.org/#/c/82022/*
*
Link to Job:
http://jenkins.ovirt.org/job/ovirt-master_change-queue-tester/2734
Link to all logs:
http://jenkins.ovirt.org/job/ovirt-master_change-queue-tester/2734/artifa...
(Relevant) error snippet from the log:
<error>
2017-09-20
09:17:15,676::testlib.py::assert_equals_within::227::ovirtlago.testlib::ERROR::
* Unhandled exception in <function _host_is_up_4 at 0x3dd25f0>
Traceback (most recent call last):
File "/usr/lib/python2.7/site-packages/ovirtlago/testlib.py", line
219, in assert_equals_within
res = func()
File
"/home/jenkins/workspace/ovirt-master_change-queue-tester/ovirt-system-tests/basic-suite-master/test-scenarios/002_bootstrap.py",
line 338, in _host_is_up_4
host_status = host_service.get().status
File "/usr/lib64/python2.7/site-packages/ovirtsdk4/services.py", line
31529, in get
return self._internal_get(headers, query, wait)
File "/usr/lib64/python2.7/site-packages/ovirtsdk4/service.py", line
202, in _internal_get
return future.wait() if wait else future
File "/usr/lib64/python2.7/site-packages/ovirtsdk4/service.py", line
53, in wait
return self._code(response)
File "/usr/lib64/python2.7/site-packages/ovirtsdk4/service.py", line
199, in callback
self._check_fault(response)
File "/usr/lib64/python2.7/site-packages/ovirtsdk4/service.py", line
121, in _check_fault
body = self._internal_read_body(response)
File "/usr/lib64/python2.7/site-packages/ovirtsdk4/service.py", line
307, in _internal_read_body
self._connection.check_xml_content_type(response)
File "/usr/lib64/python2.7/site-packages/ovirtsdk4/__init__.py", line
800, in check_xml_content_type
response.headers
File "/usr/lib64/python2.7/site-packages/ovirtsdk4/__init__.py", line
840, in _check_content_type
raise Error(msg)
Error: The response content type 'text/html; charset=iso-8859-1' isn't
the expected XML
2017-09-20
09:17:15,683::utils.py::wrapper::480::lago.utils::DEBUG::Looking for a
workdir
</error>
Thanks,
*
Dafna*
*
--------------04175AE2D1C2AC0FFF61E4CE
Content-Type: text/html; charset=utf-8
Content-Transfer-Encoding: 8bit
<html>
<head>
<meta http-equiv="content-type" content="text/html;
charset=utf-8">
</head>
<body text="#000000" bgcolor="#FFFFFF">
<p>Hi, <br>
</p>
<p>We have a failure in test post-002_bootstrap.py</p>
<p>from the logs I can see that lago has an unhandled exception from
ovirt sdk which makes sense since the patch reported on CQ is a
change to the sdk. <br>
</p>
<p><b style="font-weight:normal;"
id="docs-internal-guid-5859b7a1-9ef6-91f4-38de-b7313f855cc2">
<p dir="ltr"
style="line-height:1.38;margin-top:0pt;margin-bottom:0pt;"><span
style="font-size:11pt;font-family:Arial;color:#000000;background-color:transparent;font-weight:400;font-style:normal;font-variant:normal;text-decoration:none;vertical-align:baseline;white-space:pre-wrap;">Link
to suspected patches: <a class="moz-txt-link-freetext"
href="https://gerrit.ovirt.org/#/c/82022/">https://gerrit.ov...
<br>
<p dir="ltr"
style="line-height:1.38;margin-top:0pt;margin-bottom:0pt;"><span
style="font-size:11pt;font-family:Arial;color:#000000;background-color:transparent;font-weight:400;font-style:normal;font-variant:normal;text-decoration:none;vertical-align:baseline;white-space:pre-wrap;">Link
to Job: <a class="moz-txt-link-freetext"
href="http://jenkins.ovirt.org/job/ovirt-master_change-queue-tester/...
<br>
<p dir="ltr"
style="line-height:1.38;margin-top:0pt;margin-bottom:0pt;"><span
style="font-size:11pt;font-family:Arial;color:#000000;background-color:transparent;font-weight:400;font-style:normal;font-variant:normal;text-decoration:none;vertical-align:baseline;white-space:pre-wrap;">Link
to all logs: <a class="moz-txt-link-freetext"
href="http://jenkins.ovirt.org/job/ovirt-master_change-queue-tester/...
<br>
<p dir="ltr"
style="line-height:1.38;margin-top:0pt;margin-bottom:0pt;"><span
style="font-size:11pt;font-family:Arial;color:#000000;background-color:transparent;font-weight:400;font-style:normal;font-variant:normal;text-decoration:none;vertical-align:baseline;white-space:pre-wrap;">(Relevant)
error snippet from the log: </span></p>
<p dir="ltr"
style="line-height:1.38;margin-top:0pt;margin-bottom:0pt;"><span
style="font-size:11pt;font-family:Arial;color:#000000;background-color:transparent;font-weight:400;font-style:normal;font-variant:normal;text-decoration:none;vertical-align:baseline;white-space:pre-wrap;"><error></span></p>
<br>
2017-09-20
09:17:15,676::testlib.py::assert_equals_within::227::ovirtlago.testlib::ERROR::
* Unhandled exception in <function _host_is_up_4 at
0x3dd25f0><br>
Traceback (most recent call last):<br>
File "/usr/lib/python2.7/site-packages/ovirtlago/testlib.py",
line 219, in assert_equals_within<br>
res = func()<br>
File
"/home/jenkins/workspace/ovirt-master_change-queue-tester/ovirt-system-tests/basic-suite-master/test-scenarios/002_bootstrap.py",
line 338, in _host_is_up_4<br>
host_status = host_service.get().status<br>
File
"/usr/lib64/python2.7/site-packages/ovirtsdk4/services.py", line
31529, in get<br>
return self._internal_get(headers, query, wait)<br>
File
"/usr/lib64/python2.7/site-packages/ovirtsdk4/service.py", line
202, in _internal_get<br>
return future.wait() if wait else future<br>
File
"/usr/lib64/python2.7/site-packages/ovirtsdk4/service.py", line
53, in wait<br>
return self._code(response)<br>
File
"/usr/lib64/python2.7/site-packages/ovirtsdk4/service.py", line
199, in callback<br>
self._check_fault(response)<br>
File
"/usr/lib64/python2.7/site-packages/ovirtsdk4/service.py", line
121, in _check_fault<br>
body = self._internal_read_body(response)<br>
File
"/usr/lib64/python2.7/site-packages/ovirtsdk4/service.py", line
307, in _internal_read_body<br>
self._connection.check_xml_content_type(response)<br>
File
"/usr/lib64/python2.7/site-packages/ovirtsdk4/__init__.py", line
800, in check_xml_content_type<br>
response.headers<br>
File
"/usr/lib64/python2.7/site-packages/ovirtsdk4/__init__.py", line
840, in _check_content_type<br>
raise Error(msg)<br>
Error: The response content type 'text/html; charset=iso-8859-1'
isn't the expected XML<br>
2017-09-20
09:17:15,683::utils.py::wrapper::480::lago.utils::DEBUG::Looking
for a workdir<br>
<br>
<br>
<p dir="ltr"
style="line-height:1.38;margin-top:0pt;margin-bottom:0pt;"><span
style="font-size:11pt;font-family:Arial;color:#000000;background-color:transparent;font-weight:400;font-style:normal;font-variant:normal;text-decoration:none;vertical-align:baseline;white-space:pre-wrap;"></error></span></p>
<br>
Thanks, <br>
</b></p>
<p><span style="font-weight:normal;">Dafna</span><b
style="font-weight:normal;"
id="docs-internal-guid-5859b7a1-9ef6-91f4-38de-b7313f855cc2"><br>
</b></p>
</body>
</html>
--------------04175AE2D1C2AC0FFF61E4CE--
3:11 p.m.
New subject: [ovirt-devel] Subject: [ OST Failure Report ] [ master ] [20-09-2017 ] [ post-002_bootstrap.py ]
Ondro, could you please take a look?
On Wed, Sep 20, 2017 at 1:07 PM, Dafna Ron <dron(a)redhat.com> wrote:
Hi,
We have a failure in test post-002_bootstrap.py
from the logs I can see that lago has an unhandled exception from ovirt
sdk which makes sense since the patch reported on CQ is a change to the
sdk.
*Link to suspected patches: https://gerrit.ovirt.org/#/c/82022/
<https://gerrit.ovirt.org/#/c/82022/>*
* Link to Job:
http://jenkins.ovirt.org/job/ovirt-master_change-queue-tester/2734
<http://jenkins.ovirt.org/job/ovirt-master_change-queue-tester/2734> Link
to all logs:
http://jenkins.ovirt.org/job/ovirt-master_change-queue-tester/2734/artifa...
<http://jenkins.ovirt.org/job/ovirt-master_change-queue-tester/2734/artifa...
(Relevant) error snippet from the log: <error> 2017-09-20
09:17:15,676::testlib.py::assert_equals_within::227::ovirtlago.testlib::ERROR::
* Unhandled exception in <function _host_is_up_4 at 0x3dd25f0> Traceback
(most recent call last): File
"/usr/lib/python2.7/site-packages/ovirtlago/testlib.py", line 219, in
assert_equals_within res = func() File
"/home/jenkins/workspace/ovirt-master_change-queue-tester/ovirt-system-tests/basic-suite-master/test-scenarios/002_bootstrap.py",
line 338, in _host_is_up_4 host_status = host_service.get().status
File "/usr/lib64/python2.7/site-packages/ovirtsdk4/services.py", line
31529, in get return self._internal_get(headers, query, wait) File
"/usr/lib64/python2.7/site-packages/ovirtsdk4/service.py", line 202, in
_internal_get return future.wait() if wait else future File
"/usr/lib64/python2.7/site-packages/ovirtsdk4/service.py", line 53, in wait
return self._code(response) File
"/usr/lib64/python2.7/site-packages/ovirtsdk4/service.py", line 199, in
callback self._check_fault(response) File
"/usr/lib64/python2.7/site-packages/ovirtsdk4/service.py", line 121, in
_check_fault body = self._internal_read_body(response) File
"/usr/lib64/python2.7/site-packages/ovirtsdk4/service.py", line 307, in
_internal_read_body self._connection.check_xml_content_type(response)
File "/usr/lib64/python2.7/site-packages/ovirtsdk4/__init__.py", line 800,
in check_xml_content_type response.headers File
"/usr/lib64/python2.7/site-packages/ovirtsdk4/__init__.py", line 840, in
_check_content_type raise Error(msg) Error: The response content type
'text/html; charset=iso-8859-1' isn't the expected XML 2017-09-20
09:17:15,683::utils.py::wrapper::480::lago.utils::DEBUG::Looking for a
workdir </error> Thanks, *
Dafna
_______________________________________________
Devel mailing list
Devel(a)ovirt.org
http://lists.ovirt.org/mailman/listinfo/devel
3:32 p.m.
New subject: [ovirt-devel] Subject: [ OST Failure Report ] [ master ] [20-09-2017 ] [ post-002_bootstrap.py ]
That SDK patch is for the *Ruby* SDK, which isn't used in OST, so I
think it isn't related.
On 09/20/2017 02:11 PM, Martin Perina wrote:
Ondro, could you please take a look?
On Wed, Sep 20, 2017 at 1:07 PM, Dafna Ron <dron(a)redhat.com> wrote:
> Hi,
>
> We have a failure in test post-002_bootstrap.py
>
> from the logs I can see that lago has an unhandled exception from ovirt
> sdk which makes sense since the patch reported on CQ is a change to the
> sdk.
>
> *Link to suspected patches: https://gerrit.ovirt.org/#/c/82022/
> <https://gerrit.ovirt.org/#/c/82022/>*
>
> * Link to Job:
> http://jenkins.ovirt.org/job/ovirt-master_change-queue-tester/2734
> <http://jenkins.ovirt.org/job/ovirt-master_change-queue-tester/2734> Link
> to all logs:
>
http://jenkins.ovirt.org/job/ovirt-master_change-queue-tester/2734/artifa...
>
<http://jenkins.ovirt.org/job/ovirt-master_change-queue-tester/2734/artifa...
> (Relevant) error snippet from the log: <error> 2017-09-20
> 09:17:15,676::testlib.py::assert_equals_within::227::ovirtlago.testlib::ERROR::
> * Unhandled exception in <function _host_is_up_4 at 0x3dd25f0> Traceback
> (most recent call last): File
> "/usr/lib/python2.7/site-packages/ovirtlago/testlib.py", line 219, in
> assert_equals_within res = func() File
>
"/home/jenkins/workspace/ovirt-master_change-queue-tester/ovirt-system-tests/basic-suite-master/test-scenarios/002_bootstrap.py",
> line 338, in _host_is_up_4 host_status = host_service.get().status
> File "/usr/lib64/python2.7/site-packages/ovirtsdk4/services.py", line
> 31529, in get return self._internal_get(headers, query, wait) File
> "/usr/lib64/python2.7/site-packages/ovirtsdk4/service.py", line 202, in
> _internal_get return future.wait() if wait else future File
> "/usr/lib64/python2.7/site-packages/ovirtsdk4/service.py", line 53, in
wait
> return self._code(response) File
> "/usr/lib64/python2.7/site-packages/ovirtsdk4/service.py", line 199, in
> callback self._check_fault(response) File
> "/usr/lib64/python2.7/site-packages/ovirtsdk4/service.py", line 121, in
> _check_fault body = self._internal_read_body(response) File
> "/usr/lib64/python2.7/site-packages/ovirtsdk4/service.py", line 307, in
> _internal_read_body self._connection.check_xml_content_type(response)
> File "/usr/lib64/python2.7/site-packages/ovirtsdk4/__init__.py", line 800,
> in check_xml_content_type response.headers File
> "/usr/lib64/python2.7/site-packages/ovirtsdk4/__init__.py", line 840, in
> _check_content_type raise Error(msg) Error: The response content type
> 'text/html; charset=iso-8859-1' isn't the expected XML 2017-09-20
> 09:17:15,683::utils.py::wrapper::480::lago.utils::DEBUG::Looking for a
> workdir </error> Thanks, *
>
> Dafna
>
> _______________________________________________
> Devel mailing list
> Devel(a)ovirt.org
> http://lists.ovirt.org/mailman/listinfo/devel
>
4:20 p.m.
New subject: [ovirt-devel] Subject: [ OST Failure Report ] [ master ] [20-09-2017 ] [ post-002_bootstrap.py ]
The http returned 503:
192.168.201.1 - - [20/Sep/2017:05:17:10 -0400] "GET
/ovirt-engine/api/hosts/06b9cdf3-5bcb-4be8-9e11-8ad5123749f0 HTTP/1.1"
200 5249
192.168.201.1 - - [20/Sep/2017:05:17:13 -0400] "GET
/ovirt-engine/api/hosts/06b9cdf3-5bcb-4be8-9e11-8ad5123749f0 HTTP/1.1"
503 299
192.168.201.4 - - [20/Sep/2017:05:17:11 -0400] "POST
/ovirt-engine/sso/oauth/token-info HTTP/1.1" 500 527
At that time postgres was killed by oom:
Sep 20 05:17:15 lago-basic-suite-master-engine kernel: postgres
invoked oom-killer: gfp_mask=0x280da, order=0, oom_score_adj=0
So from Python SDK perpepective all looks OK.
On Wed, Sep 20, 2017 at 2:11 PM, Martin Perina <mperina(a)redhat.com> wrote:
Ondro, could you please take a look?
On Wed, Sep 20, 2017 at 1:07 PM, Dafna Ron <dron(a)redhat.com> wrote:
>
> Hi,
>
> We have a failure in test post-002_bootstrap.py
>
> from the logs I can see that lago has an unhandled exception from ovirt
> sdk which makes sense since the patch reported on CQ is a change to the sdk.
>
> Link to suspected patches: https://gerrit.ovirt.org/#/c/82022/
>
>
> Link to Job:
> http://jenkins.ovirt.org/job/ovirt-master_change-queue-tester/2734
>
>
> Link to all logs:
>
http://jenkins.ovirt.org/job/ovirt-master_change-queue-tester/2734/artifa...
>
>
> (Relevant) error snippet from the log:
>
> <error>
>
>
> 2017-09-20
> 09:17:15,676::testlib.py::assert_equals_within::227::ovirtlago.testlib::ERROR::
> * Unhandled exception in <function _host_is_up_4 at 0x3dd25f0>
> Traceback (most recent call last):
> File "/usr/lib/python2.7/site-packages/ovirtlago/testlib.py", line 219,
> in assert_equals_within
> res = func()
> File
>
"/home/jenkins/workspace/ovirt-master_change-queue-tester/ovirt-system-tests/basic-suite-master/test-scenarios/002_bootstrap.py",
> line 338, in _host_is_up_4
> host_status = host_service.get().status
> File "/usr/lib64/python2.7/site-packages/ovirtsdk4/services.py", line
> 31529, in get
> return self._internal_get(headers, query, wait)
> File "/usr/lib64/python2.7/site-packages/ovirtsdk4/service.py", line
> 202, in _internal_get
> return future.wait() if wait else future
> File "/usr/lib64/python2.7/site-packages/ovirtsdk4/service.py", line 53,
> in wait
> return self._code(response)
> File "/usr/lib64/python2.7/site-packages/ovirtsdk4/service.py", line
> 199, in callback
> self._check_fault(response)
> File "/usr/lib64/python2.7/site-packages/ovirtsdk4/service.py", line
> 121, in _check_fault
> body = self._internal_read_body(response)
> File "/usr/lib64/python2.7/site-packages/ovirtsdk4/service.py", line
> 307, in _internal_read_body
> self._connection.check_xml_content_type(response)
> File "/usr/lib64/python2.7/site-packages/ovirtsdk4/__init__.py", line
> 800, in check_xml_content_type
> response.headers
> File "/usr/lib64/python2.7/site-packages/ovirtsdk4/__init__.py", line
> 840, in _check_content_type
> raise Error(msg)
> Error: The response content type 'text/html; charset=iso-8859-1' isn't
the
> expected XML
> 2017-09-20
> 09:17:15,683::utils.py::wrapper::480::lago.utils::DEBUG::Looking for a
> workdir
>
>
> </error>
>
>
> Thanks,
>
> Dafna
>
>
> _______________________________________________
> Devel mailing list
> Devel(a)ovirt.org
> http://lists.ovirt.org/mailman/listinfo/devel
5:09 p.m.
New subject: [ovirt-devel] Subject: [ OST Failure Report ] [ master ] [20-09-2017 ] [ post-002_bootstrap.py ]
This is a multi-part message in MIME format.
--------------76F54851C80F784CD6DAB3CE
Content-Type: text/plain; charset=utf-8
Content-Transfer-Encoding: 7bit
We just had a second failure which CQ is reporting to be from the same
patch:
failed job:
http://jenkins.ovirt.org/job/ovirt-4.1_change-queue-tester/1034/console
if you look at the console output you can see this:
*12:27:15* ovirt-engine-sdk-ruby_4.1_build-artifacts-fc24-ppc64le (122) failed building
This is the job that failed on build:
http://jenkins.ovirt.org/job/ovirt-engine-sdk-ruby_4.1_build-artifacts-fc...
I re-ran the sdk build and it succeeded this time:
http://jenkins.ovirt.org/job/ovirt-engine-sdk-ruby_4.1_build-artifacts-fc...
please try to re-submit the patch? maybe we have a coincident but if it fails again it
would be 3ed time...
Thanks,
Dafna
On 09/20/2017 02:20 PM, Ondra Machacek wrote:
The http returned 503:
192.168.201.1 - - [20/Sep/2017:05:17:10 -0400] "GET
/ovirt-engine/api/hosts/06b9cdf3-5bcb-4be8-9e11-8ad5123749f0 HTTP/1.1"
200 5249
192.168.201.1 - - [20/Sep/2017:05:17:13 -0400] "GET
/ovirt-engine/api/hosts/06b9cdf3-5bcb-4be8-9e11-8ad5123749f0 HTTP/1.1"
503 299
192.168.201.4 - - [20/Sep/2017:05:17:11 -0400] "POST
/ovirt-engine/sso/oauth/token-info HTTP/1.1" 500 527
At that time postgres was killed by oom:
Sep 20 05:17:15 lago-basic-suite-master-engine kernel: postgres
invoked oom-killer: gfp_mask=0x280da, order=0, oom_score_adj=0
So from Python SDK perpepective all looks OK.
On Wed, Sep 20, 2017 at 2:11 PM, Martin Perina <mperina(a)redhat.com> wrote:
> Ondro, could you please take a look?
>
> On Wed, Sep 20, 2017 at 1:07 PM, Dafna Ron <dron(a)redhat.com> wrote:
>> Hi,
>>
>> We have a failure in test post-002_bootstrap.py
>>
>> from the logs I can see that lago has an unhandled exception from ovirt
>> sdk which makes sense since the patch reported on CQ is a change to the sdk.
>>
>> Link to suspected patches: https://gerrit.ovirt.org/#/c/82022/
>>
>>
>> Link to Job:
>> http://jenkins.ovirt.org/job/ovirt-master_change-queue-tester/2734
>>
>>
>> Link to all logs:
>>
http://jenkins.ovirt.org/job/ovirt-master_change-queue-tester/2734/artifa...
>>
>>
>> (Relevant) error snippet from the log:
>>
>> <error>
>>
>>
>> 2017-09-20
>> 09:17:15,676::testlib.py::assert_equals_within::227::ovirtlago.testlib::ERROR::
>> * Unhandled exception in <function _host_is_up_4 at 0x3dd25f0>
>> Traceback (most recent call last):
>> File "/usr/lib/python2.7/site-packages/ovirtlago/testlib.py", line
219,
>> in assert_equals_within
>> res = func()
>> File
>>
"/home/jenkins/workspace/ovirt-master_change-queue-tester/ovirt-system-tests/basic-suite-master/test-scenarios/002_bootstrap.py",
>> line 338, in _host_is_up_4
>> host_status = host_service.get().status
>> File "/usr/lib64/python2.7/site-packages/ovirtsdk4/services.py",
line
>> 31529, in get
>> return self._internal_get(headers, query, wait)
>> File "/usr/lib64/python2.7/site-packages/ovirtsdk4/service.py", line
>> 202, in _internal_get
>> return future.wait() if wait else future
>> File "/usr/lib64/python2.7/site-packages/ovirtsdk4/service.py", line
53,
>> in wait
>> return self._code(response)
>> File "/usr/lib64/python2.7/site-packages/ovirtsdk4/service.py", line
>> 199, in callback
>> self._check_fault(response)
>> File "/usr/lib64/python2.7/site-packages/ovirtsdk4/service.py", line
>> 121, in _check_fault
>> body = self._internal_read_body(response)
>> File "/usr/lib64/python2.7/site-packages/ovirtsdk4/service.py", line
>> 307, in _internal_read_body
>> self._connection.check_xml_content_type(response)
>> File "/usr/lib64/python2.7/site-packages/ovirtsdk4/__init__.py",
line
>> 800, in check_xml_content_type
>> response.headers
>> File "/usr/lib64/python2.7/site-packages/ovirtsdk4/__init__.py",
line
>> 840, in _check_content_type
>> raise Error(msg)
>> Error: The response content type 'text/html; charset=iso-8859-1'
isn't the
>> expected XML
>> 2017-09-20
>> 09:17:15,683::utils.py::wrapper::480::lago.utils::DEBUG::Looking for a
>> workdir
>>
>>
>> </error>
>>
>>
>> Thanks,
>>
>> Dafna
>>
>>
>> _______________________________________________
>> Devel mailing list
>> Devel(a)ovirt.org
>> http://lists.ovirt.org/mailman/listinfo/devel
>
--------------76F54851C80F784CD6DAB3CE
Content-Type: text/html; charset=utf-8
Content-Transfer-Encoding: 7bit
<html>
<head>
<meta http-equiv="Content-Type" content="text/html;
charset=utf-8">
</head>
<body text="#000000" bgcolor="#FFFFFF">
<div class="moz-cite-prefix">We just had a second failure which CQ
is reporting to be from the same patch: <br>
<br>
failed job:
<a class="moz-txt-link-freetext"
href="http://jenkins.ovirt.org/job/ovirt-4.1_change-queue-tester/103...
<br>
if you look at the console output you can see this: <br>
<br>
<pre class="console-output" style="box-sizing: border-box;
white-space: pre-wrap; word-wrap: break-word; margin: 0px; color: rgb(51, 51, 51);
font-size: 14px; font-style: normal; font-variant-ligatures: normal; font-variant-caps:
normal; font-weight: normal; letter-spacing: normal; orphans: 2; text-align: start;
text-indent: 0px; text-transform: none; widows: 2; word-spacing: 0px;
-webkit-text-stroke-width: 0px; text-decoration-style: initial; text-decoration-color:
initial;"><span class="timestamp" style="box-sizing:
border-box;"><b style="box-sizing: border-box;">12:27:15</b>
</span>ovirt-engine-sdk-ruby_4.1_build-artifacts-fc24-ppc64le (122) failed building
This is the job that failed on build: <a class="moz-txt-link-freetext"
href="http://jenkins.ovirt.org/job/ovirt-engine-sdk-ruby_4.1_build-a...
I re-ran the sdk build and it succeeded this time: <a
class="moz-txt-link-freetext"
href="http://jenkins.ovirt.org/job/ovirt-engine-sdk-ruby_4.1_build-a...
please try to re-submit the patch? maybe we have a coincident but if it fails again it
would be 3ed time...
Thanks,
Dafna
</pre>
<br>
<br>
On 09/20/2017 02:20 PM, Ondra Machacek wrote:<br>
</div>
<blockquote type="cite"
cite="mid:CAJCqMsV9tTemu+5s-BdY9PLu_E3HW8pO8=--RVH0TyFCjopXzg@mail.gmail.com">
<pre wrap="">The http returned 503:
192.168.201.1 - - [20/Sep/2017:05:17:10 -0400] "GET
/ovirt-engine/api/hosts/06b9cdf3-5bcb-4be8-9e11-8ad5123749f0 HTTP/1.1"
200 5249
192.168.201.1 - - [20/Sep/2017:05:17:13 -0400] "GET
/ovirt-engine/api/hosts/06b9cdf3-5bcb-4be8-9e11-8ad5123749f0 HTTP/1.1"
503 299
192.168.201.4 - - [20/Sep/2017:05:17:11 -0400] "POST
/ovirt-engine/sso/oauth/token-info HTTP/1.1" 500 527
At that time postgres was killed by oom:
Sep 20 05:17:15 lago-basic-suite-master-engine kernel: postgres
invoked oom-killer: gfp_mask=0x280da, order=0, oom_score_adj=0
So from Python SDK perpepective all looks OK.
On Wed, Sep 20, 2017 at 2:11 PM, Martin Perina <a
class="moz-txt-link-rfc2396E"
href="mailto:mperina@redhat.com"><mperina@redhat.com></a>
wrote:
</pre>
<blockquote type="cite">
<pre wrap="">Ondro, could you please take a look?
On Wed, Sep 20, 2017 at 1:07 PM, Dafna Ron <a class="moz-txt-link-rfc2396E"
href="mailto:dron@redhat.com"><dron@redhat.com></a>
wrote:
</pre>
<blockquote type="cite">
<pre wrap="">
Hi,
We have a failure in test post-002_bootstrap.py
from the logs I can see that lago has an unhandled exception from ovirt
sdk which makes sense since the patch reported on CQ is a change to the sdk.
Link to suspected patches: <a class="moz-txt-link-freetext"
href="https://gerrit.ovirt.org/#/c/82022/">https://gerrit.ov...
Link to Job:
<a class="moz-txt-link-freetext"
href="http://jenkins.ovirt.org/job/ovirt-master_change-queue-tester/...
Link to all logs:
<a class="moz-txt-link-freetext"
href="http://jenkins.ovirt.org/job/ovirt-master_change-queue-tester/...
(Relevant) error snippet from the log:
<error>
2017-09-20
09:17:15,676::testlib.py::assert_equals_within::227::ovirtlago.testlib::ERROR::
* Unhandled exception in <function _host_is_up_4 at 0x3dd25f0>
Traceback (most recent call last):
File "/usr/lib/python2.7/site-packages/ovirtlago/testlib.py", line 219,
in assert_equals_within
res = func()
File
"/home/jenkins/workspace/ovirt-master_change-queue-tester/ovirt-system-tests/basic-suite-master/test-scenarios/002_bootstrap.py",
line 338, in _host_is_up_4
host_status = host_service.get().status
File "/usr/lib64/python2.7/site-packages/ovirtsdk4/services.py", line
31529, in get
return self._internal_get(headers, query, wait)
File "/usr/lib64/python2.7/site-packages/ovirtsdk4/service.py", line
202, in _internal_get
return future.wait() if wait else future
File "/usr/lib64/python2.7/site-packages/ovirtsdk4/service.py", line 53,
in wait
return self._code(response)
File "/usr/lib64/python2.7/site-packages/ovirtsdk4/service.py", line
199, in callback
self._check_fault(response)
File "/usr/lib64/python2.7/site-packages/ovirtsdk4/service.py", line
121, in _check_fault
body = self._internal_read_body(response)
File "/usr/lib64/python2.7/site-packages/ovirtsdk4/service.py", line
307, in _internal_read_body
self._connection.check_xml_content_type(response)
File "/usr/lib64/python2.7/site-packages/ovirtsdk4/__init__.py", line
800, in check_xml_content_type
response.headers
File "/usr/lib64/python2.7/site-packages/ovirtsdk4/__init__.py", line
840, in _check_content_type
raise Error(msg)
Error: The response content type 'text/html; charset=iso-8859-1' isn't the
expected XML
2017-09-20
09:17:15,683::utils.py::wrapper::480::lago.utils::DEBUG::Looking for a
workdir
</error>
Thanks,
Dafna
_______________________________________________
Devel mailing list
<a class="moz-txt-link-abbreviated"
href="mailto:Devel@ovirt.org">Devel@ovirt.org</a>
<a class="moz-txt-link-freetext"
href="http://lists.ovirt.org/mailman/listinfo/devel">http://...
</pre>
</blockquote>
<pre wrap="">
</pre>
</blockquote>
</blockquote>
<p><br>
</p>
</body>
</html>
--------------76F54851C80F784CD6DAB3CE--
5:43 p.m.
New subject: [ovirt-devel] Subject: [ OST Failure Report ] [ master ] [20-09-2017 ] [ post-002_bootstrap.py ]
I am aware of that build failure. The tests of the *Ruby* SDK hang
apparently random in ppc64le environments. I am trying to find out why,
but not making much progress, to be honest.
Meanwhile I wonder why this is a failure to run OST tests, as the Ruby
SDK isn't used at all in those tests, as far as I know.
On 09/20/2017 04:09 PM, Dafna Ron wrote:
We just had a second failure which CQ is reporting to be from the
same
patch:
failed job:
http://jenkins.ovirt.org/job/ovirt-4.1_change-queue-tester/1034/console
if you look at the console output you can see this:
*12:27:15* ovirt-engine-sdk-ruby_4.1_build-artifacts-fc24-ppc64le (122) failed building
This is the job that failed on build:
http://jenkins.ovirt.org/job/ovirt-engine-sdk-ruby_4.1_build-artifacts-fc...
I re-ran the sdk build and it succeeded this time:
http://jenkins.ovirt.org/job/ovirt-engine-sdk-ruby_4.1_build-artifacts-fc...
please try to re-submit the patch? maybe we have a coincident but if it fails again it
would be 3ed time...
Thanks,
Dafna
On 09/20/2017 02:20 PM, Ondra Machacek wrote:
> The http returned 503:
>
> 192.168.201.1 - - [20/Sep/2017:05:17:10 -0400] "GET
> /ovirt-engine/api/hosts/06b9cdf3-5bcb-4be8-9e11-8ad5123749f0 HTTP/1.1"
> 200 5249
> 192.168.201.1 - - [20/Sep/2017:05:17:13 -0400] "GET
> /ovirt-engine/api/hosts/06b9cdf3-5bcb-4be8-9e11-8ad5123749f0 HTTP/1.1"
> 503 299
> 192.168.201.4 - - [20/Sep/2017:05:17:11 -0400] "POST
> /ovirt-engine/sso/oauth/token-info HTTP/1.1" 500 527
>
> At that time postgres was killed by oom:
>
> Sep 20 05:17:15 lago-basic-suite-master-engine kernel: postgres
> invoked oom-killer: gfp_mask=0x280da, order=0, oom_score_adj=0
>
>
> So from Python SDK perpepective all looks OK.
>
>
> On Wed, Sep 20, 2017 at 2:11 PM, Martin Perina <mperina(a)redhat.com> wrote:
>> Ondro, could you please take a look?
>>
>> On Wed, Sep 20, 2017 at 1:07 PM, Dafna Ron <dron(a)redhat.com> wrote:
>>> Hi,
>>>
>>> We have a failure in test post-002_bootstrap.py
>>>
>>> from the logs I can see that lago has an unhandled exception from ovirt
>>> sdk which makes sense since the patch reported on CQ is a change to the sdk.
>>>
>>> Link to suspected patches: https://gerrit.ovirt.org/#/c/82022/
>>>
>>>
>>> Link to Job:
>>> http://jenkins.ovirt.org/job/ovirt-master_change-queue-tester/2734
>>>
>>>
>>> Link to all logs:
>>>
http://jenkins.ovirt.org/job/ovirt-master_change-queue-tester/2734/artifa...
>>>
>>>
>>> (Relevant) error snippet from the log:
>>>
>>> <error>
>>>
>>>
>>> 2017-09-20
>>>
09:17:15,676::testlib.py::assert_equals_within::227::ovirtlago.testlib::ERROR::
>>> * Unhandled exception in <function _host_is_up_4 at 0x3dd25f0>
>>> Traceback (most recent call last):
>>> File "/usr/lib/python2.7/site-packages/ovirtlago/testlib.py",
line 219,
>>> in assert_equals_within
>>> res = func()
>>> File
>>>
"/home/jenkins/workspace/ovirt-master_change-queue-tester/ovirt-system-tests/basic-suite-master/test-scenarios/002_bootstrap.py",
>>> line 338, in _host_is_up_4
>>> host_status = host_service.get().status
>>> File "/usr/lib64/python2.7/site-packages/ovirtsdk4/services.py",
line
>>> 31529, in get
>>> return self._internal_get(headers, query, wait)
>>> File "/usr/lib64/python2.7/site-packages/ovirtsdk4/service.py",
line
>>> 202, in _internal_get
>>> return future.wait() if wait else future
>>> File "/usr/lib64/python2.7/site-packages/ovirtsdk4/service.py",
line 53,
>>> in wait
>>> return self._code(response)
>>> File "/usr/lib64/python2.7/site-packages/ovirtsdk4/service.py",
line
>>> 199, in callback
>>> self._check_fault(response)
>>> File "/usr/lib64/python2.7/site-packages/ovirtsdk4/service.py",
line
>>> 121, in _check_fault
>>> body = self._internal_read_body(response)
>>> File "/usr/lib64/python2.7/site-packages/ovirtsdk4/service.py",
line
>>> 307, in _internal_read_body
>>> self._connection.check_xml_content_type(response)
>>> File "/usr/lib64/python2.7/site-packages/ovirtsdk4/__init__.py",
line
>>> 800, in check_xml_content_type
>>> response.headers
>>> File "/usr/lib64/python2.7/site-packages/ovirtsdk4/__init__.py",
line
>>> 840, in _check_content_type
>>> raise Error(msg)
>>> Error: The response content type 'text/html; charset=iso-8859-1'
isn't the
>>> expected XML
>>> 2017-09-20
>>> 09:17:15,683::utils.py::wrapper::480::lago.utils::DEBUG::Looking for a
>>> workdir
>>>
>>>
>>> </error>
>>>
>>>
>>> Thanks,
>>>
>>> Dafna
>>>
>>>
>>> _______________________________________________
>>> Devel mailing list
>>> Devel(a)ovirt.org
>>> http://lists.ovirt.org/mailman/listinfo/devel
>>
9:26 a.m.
New subject: [ovirt-devel] Subject: [ OST Failure Report ] [ master ] [20-09-2017 ] [ post-002_bootstrap.py ]
בתאריך 20 בספט׳ 2017 05:44 PM, "Juan Hernández" <jhernand(a)redhat.com>
כתב:
I am aware of that build failure. The tests of the *Ruby* SDK hang
apparently random in ppc64le environments. I am trying to find out why, but
not making much progress, to be honest.
Meanwhile I wonder why this is a failure to run OST tests, as the Ruby SDK
isn't used at all in those tests, as far as I know.
This is not actually failing OST, its making CQ not run OST at all, since
it verifies builds pass before trying.
Barak Korren
bkorren(a)redhat.com
RHCE, RHCi, RHV-DevOps Team
https://ifireball.wordpress.com/
>
2620
days inactive
2621
days old
6 comments
5 participants
participants (5)
-
Barak Korren -
Dafna Ron -
Juan Hernández -
Martin Perina -
Ondra Machacek