This is a multi-part message in MIME format. --------------76F54851C80F784CD6DAB3CE Content-Type: text/plain; charset=utf-8 Content-Transfer-Encoding: 7bit We just had a second failure which CQ is reporting to be from the same patch: failed job: http://jenkins.ovirt.org/job/ovirt-4.1_change-queue-tester/1034/console if you look at the console output you can see this: *12:27:15* ovirt-engine-sdk-ruby_4.1_build-artifacts-fc24-ppc64le (122) failed building This is the job that failed on build: http://jenkins.ovirt.org/job/ovirt-engine-sdk-ruby_4.1_build-artifacts-fc24-... I re-ran the sdk build and it succeeded this time: http://jenkins.ovirt.org/job/ovirt-engine-sdk-ruby_4.1_build-artifacts-fc24-... please try to re-submit the patch? maybe we have a coincident but if it fails again it would be 3ed time... Thanks, Dafna On 09/20/2017 02:20 PM, Ondra Machacek wrote:
The http returned 503:
192.168.201.1 - - [20/Sep/2017:05:17:10 -0400] "GET /ovirt-engine/api/hosts/06b9cdf3-5bcb-4be8-9e11-8ad5123749f0 HTTP/1.1" 200 5249 192.168.201.1 - - [20/Sep/2017:05:17:13 -0400] "GET /ovirt-engine/api/hosts/06b9cdf3-5bcb-4be8-9e11-8ad5123749f0 HTTP/1.1" 503 299 192.168.201.4 - - [20/Sep/2017:05:17:11 -0400] "POST /ovirt-engine/sso/oauth/token-info HTTP/1.1" 500 527
At that time postgres was killed by oom:
Sep 20 05:17:15 lago-basic-suite-master-engine kernel: postgres invoked oom-killer: gfp_mask=0x280da, order=0, oom_score_adj=0
So from Python SDK perpepective all looks OK.
On Wed, Sep 20, 2017 at 2:11 PM, Martin Perina <mperina@redhat.com> wrote:
Ondro, could you please take a look?
On Wed, Sep 20, 2017 at 1:07 PM, Dafna Ron <dron@redhat.com> wrote:
Hi,
We have a failure in test post-002_bootstrap.py
from the logs I can see that lago has an unhandled exception from ovirt sdk which makes sense since the patch reported on CQ is a change to the sdk.
Link to suspected patches: https://gerrit.ovirt.org/#/c/82022/
Link to Job: http://jenkins.ovirt.org/job/ovirt-master_change-queue-tester/2734
Link to all logs: http://jenkins.ovirt.org/job/ovirt-master_change-queue-tester/2734/artifact/...
(Relevant) error snippet from the log:
<error>
2017-09-20 09:17:15,676::testlib.py::assert_equals_within::227::ovirtlago.testlib::ERROR:: * Unhandled exception in <function _host_is_up_4 at 0x3dd25f0> Traceback (most recent call last): File "/usr/lib/python2.7/site-packages/ovirtlago/testlib.py", line 219, in assert_equals_within res = func() File "/home/jenkins/workspace/ovirt-master_change-queue-tester/ovirt-system-tests/basic-suite-master/test-scenarios/002_bootstrap.py", line 338, in _host_is_up_4 host_status = host_service.get().status File "/usr/lib64/python2.7/site-packages/ovirtsdk4/services.py", line 31529, in get return self._internal_get(headers, query, wait) File "/usr/lib64/python2.7/site-packages/ovirtsdk4/service.py", line 202, in _internal_get return future.wait() if wait else future File "/usr/lib64/python2.7/site-packages/ovirtsdk4/service.py", line 53, in wait return self._code(response) File "/usr/lib64/python2.7/site-packages/ovirtsdk4/service.py", line 199, in callback self._check_fault(response) File "/usr/lib64/python2.7/site-packages/ovirtsdk4/service.py", line 121, in _check_fault body = self._internal_read_body(response) File "/usr/lib64/python2.7/site-packages/ovirtsdk4/service.py", line 307, in _internal_read_body self._connection.check_xml_content_type(response) File "/usr/lib64/python2.7/site-packages/ovirtsdk4/__init__.py", line 800, in check_xml_content_type response.headers File "/usr/lib64/python2.7/site-packages/ovirtsdk4/__init__.py", line 840, in _check_content_type raise Error(msg) Error: The response content type 'text/html; charset=iso-8859-1' isn't the expected XML 2017-09-20 09:17:15,683::utils.py::wrapper::480::lago.utils::DEBUG::Looking for a workdir
</error>
Thanks,
Dafna
_______________________________________________ Devel mailing list Devel@ovirt.org http://lists.ovirt.org/mailman/listinfo/devel
--------------76F54851C80F784CD6DAB3CE Content-Type: text/html; charset=utf-8 Content-Transfer-Encoding: 7bit <html> <head> <meta http-equiv="Content-Type" content="text/html; charset=utf-8"> </head> <body text="#000000" bgcolor="#FFFFFF"> <div class="moz-cite-prefix">We just had a second failure which CQ is reporting to be from the same patch: <br> <br> failed job: <a class="moz-txt-link-freetext" href="http://jenkins.ovirt.org/job/ovirt-4.1_change-queue-tester/1034/console">http://jenkins.ovirt.org/job/ovirt-4.1_change-queue-tester/1034/console</a><br> <br> if you look at the console output you can see this: <br> <br> <pre class="console-output" style="box-sizing: border-box; white-space: pre-wrap; word-wrap: break-word; margin: 0px; color: rgb(51, 51, 51); font-size: 14px; font-style: normal; font-variant-ligatures: normal; font-variant-caps: normal; font-weight: normal; letter-spacing: normal; orphans: 2; text-align: start; text-indent: 0px; text-transform: none; widows: 2; word-spacing: 0px; -webkit-text-stroke-width: 0px; text-decoration-style: initial; text-decoration-color: initial;"><span class="timestamp" style="box-sizing: border-box;"><b style="box-sizing: border-box;">12:27:15</b> </span>ovirt-engine-sdk-ruby_4.1_build-artifacts-fc24-ppc64le (122) failed building This is the job that failed on build: <a class="moz-txt-link-freetext" href="http://jenkins.ovirt.org/job/ovirt-engine-sdk-ruby_4.1_build-artifacts-fc24-ppc64le/122/">http://jenkins.ovirt.org/job/ovirt-engine-sdk-ruby_4.1_build-artifacts-fc24-ppc64le/122/</a> I re-ran the sdk build and it succeeded this time: <a class="moz-txt-link-freetext" href="http://jenkins.ovirt.org/job/ovirt-engine-sdk-ruby_4.1_build-artifacts-fc24-ppc64le/123/">http://jenkins.ovirt.org/job/ovirt-engine-sdk-ruby_4.1_build-artifacts-fc24-ppc64le/123/</a> please try to re-submit the patch? maybe we have a coincident but if it fails again it would be 3ed time... Thanks, Dafna </pre> <br> <br> On 09/20/2017 02:20 PM, Ondra Machacek wrote:<br> </div> <blockquote type="cite" cite="mid:CAJCqMsV9tTemu+5s-BdY9PLu_E3HW8pO8=--RVH0TyFCjopXzg@mail.gmail.com"> <pre wrap="">The http returned 503: 192.168.201.1 - - [20/Sep/2017:05:17:10 -0400] "GET /ovirt-engine/api/hosts/06b9cdf3-5bcb-4be8-9e11-8ad5123749f0 HTTP/1.1" 200 5249 192.168.201.1 - - [20/Sep/2017:05:17:13 -0400] "GET /ovirt-engine/api/hosts/06b9cdf3-5bcb-4be8-9e11-8ad5123749f0 HTTP/1.1" 503 299 192.168.201.4 - - [20/Sep/2017:05:17:11 -0400] "POST /ovirt-engine/sso/oauth/token-info HTTP/1.1" 500 527 At that time postgres was killed by oom: Sep 20 05:17:15 lago-basic-suite-master-engine kernel: postgres invoked oom-killer: gfp_mask=0x280da, order=0, oom_score_adj=0 So from Python SDK perpepective all looks OK. On Wed, Sep 20, 2017 at 2:11 PM, Martin Perina <a class="moz-txt-link-rfc2396E" href="mailto:mperina@redhat.com"><mperina@redhat.com></a> wrote: </pre> <blockquote type="cite"> <pre wrap="">Ondro, could you please take a look? On Wed, Sep 20, 2017 at 1:07 PM, Dafna Ron <a class="moz-txt-link-rfc2396E" href="mailto:dron@redhat.com"><dron@redhat.com></a> wrote: </pre> <blockquote type="cite"> <pre wrap=""> Hi, We have a failure in test post-002_bootstrap.py from the logs I can see that lago has an unhandled exception from ovirt sdk which makes sense since the patch reported on CQ is a change to the sdk. Link to suspected patches: <a class="moz-txt-link-freetext" href="https://gerrit.ovirt.org/#/c/82022/">https://gerrit.ovirt.org/#/c/82022/</a> Link to Job: <a class="moz-txt-link-freetext" href="http://jenkins.ovirt.org/job/ovirt-master_change-queue-tester/2734">http://jenkins.ovirt.org/job/ovirt-master_change-queue-tester/2734</a> Link to all logs: <a class="moz-txt-link-freetext" href="http://jenkins.ovirt.org/job/ovirt-master_change-queue-tester/2734/artifact/exported-artifacts/basic-suit-master-el7/test_logs/basic-suite-master/post-002_bootstrap.py/">http://jenkins.ovirt.org/job/ovirt-master_change-queue-tester/2734/artifact/exported-artifacts/basic-suit-master-el7/test_logs/basic-suite-master/post-002_bootstrap.py/</a> (Relevant) error snippet from the log: <error> 2017-09-20 09:17:15,676::testlib.py::assert_equals_within::227::ovirtlago.testlib::ERROR:: * Unhandled exception in <function _host_is_up_4 at 0x3dd25f0> Traceback (most recent call last): File "/usr/lib/python2.7/site-packages/ovirtlago/testlib.py", line 219, in assert_equals_within res = func() File "/home/jenkins/workspace/ovirt-master_change-queue-tester/ovirt-system-tests/basic-suite-master/test-scenarios/002_bootstrap.py", line 338, in _host_is_up_4 host_status = host_service.get().status File "/usr/lib64/python2.7/site-packages/ovirtsdk4/services.py", line 31529, in get return self._internal_get(headers, query, wait) File "/usr/lib64/python2.7/site-packages/ovirtsdk4/service.py", line 202, in _internal_get return future.wait() if wait else future File "/usr/lib64/python2.7/site-packages/ovirtsdk4/service.py", line 53, in wait return self._code(response) File "/usr/lib64/python2.7/site-packages/ovirtsdk4/service.py", line 199, in callback self._check_fault(response) File "/usr/lib64/python2.7/site-packages/ovirtsdk4/service.py", line 121, in _check_fault body = self._internal_read_body(response) File "/usr/lib64/python2.7/site-packages/ovirtsdk4/service.py", line 307, in _internal_read_body self._connection.check_xml_content_type(response) File "/usr/lib64/python2.7/site-packages/ovirtsdk4/__init__.py", line 800, in check_xml_content_type response.headers File "/usr/lib64/python2.7/site-packages/ovirtsdk4/__init__.py", line 840, in _check_content_type raise Error(msg) Error: The response content type 'text/html; charset=iso-8859-1' isn't the expected XML 2017-09-20 09:17:15,683::utils.py::wrapper::480::lago.utils::DEBUG::Looking for a workdir </error> Thanks, Dafna _______________________________________________ Devel mailing list <a class="moz-txt-link-abbreviated" href="mailto:Devel@ovirt.org">Devel@ovirt.org</a> <a class="moz-txt-link-freetext" href="http://lists.ovirt.org/mailman/listinfo/devel">http://lists.ovirt.org/mailman/listinfo/devel</a> </pre> </blockquote> <pre wrap=""> </pre> </blockquote> </blockquote> <p><br> </p> </body> </html> --------------76F54851C80F784CD6DAB3CE--