- Infra - oVirt List Archives

[JIRA] (OVIRT-1938) [OST] Build fail - mom >= 0.5.12 require, 0.5.11-1 available
by Barak Korren (oVirt JIRA) 20 Mar '18

20 Mar '18

[ https://ovirt-jira.atlassian.net/browse/OVIRT-1938?page=com.atlassian.jira.… ] Barak Korren commented on OVIRT-1938: ------------------------------------- The backing-repo you've used probably does not include the version of mom that the version of VDSM you're testing requires. The non-master suits typically used the official release repo is a fallback repo, so this kind of this is to be expected as the delta from the official release grows and dependencies change. Please pass a more recent build of mom in the list of pacjages that are included in the test. On 19 March 2018 at 22:08, Nir Soffer <nsoffer(a)redhat.com> wrote: > OST run with engine 4.2 failed with missing mom. Is this a known issue? > > 00:04:54.387 - STDERR > 00:04:54.387 + yum -y install ovirt-host > 00:04:54.387 Error: Package: vdsm-4.20.22-8.git4f691c1.el7.centos.x86_64 > (alocalsync) > 00:04:54.388 Requires: mom >= 0.5.12 > 00:04:54.388 Available: mom-0.5.11-1.el7.centos.noarch > (alocalsync) > 00:04:54.389 mom = 0.5.11-1.el7.centos > 00:04:54.390 > > > commit c103010f8b5ae1ccee93f5611a198162f4ccc018 (tag: v4.20.22) > Author: Andrej Krejcir <akrejcir(a)redhat.com> > AuthorDate: Mon Feb 26 11:37:02 2018 +0100 > Commit: Francesco Romani <fromani(a)redhat.com> > CommitDate: Wed Mar 14 08:44:04 2018 -0400 > > mom: Configure mom to use the new json rpc client > > Also updating used version of mom to 0.5.12 > > Change-Id: I44a35129dd80f85785625b822947ee138e67da00 > Bug-Url: https://bugzilla.redhat.com/1544476 > Signed-off-by: Andrej Krejcir <akrejcir(a)redhat.com> > (cherry picked from commit b5a036cd628763fb0770eaf4cb6595eca9c64148) > > > Build: > http://jenkins.ovirt.org/job/ovirt-system-tests_manual/2412/consoleFull > > Here is another build with same issue: > http://jenkins.ovirt.org/job/ovirt-system-tests_manual/2404/console > > Nir > > _______________________________________________ > Infra mailing list > Infra(a)ovirt.org > http://lists.ovirt.org/mailman/listinfo/infra > -- Barak Korren RHV DevOps team , RHCE, RHCi Red Hat EMEA redhat.com | TRIED. TESTED. TRUSTED. | redhat.com/trusted > [OST] Build fail - mom >= 0.5.12 require, 0.5.11-1 available > ------------------------------------------------------------ > > Key: OVIRT-1938 > URL: https://ovirt-jira.atlassian.net/browse/OVIRT-1938 > Project: oVirt - virtualization made easy > Issue Type: By-EMAIL > Reporter: Nir Soffer > Assignee: infra > -- This message was sent by Atlassian Jira (v1001.0.0-SNAPSHOT#100082)

1 0

[OST] Build fail - mom >= 0.5.12 require, 0.5.11-1 available
by Nir Soffer 20 Mar '18

20 Mar '18

OST run with engine 4.2 failed with missing mom. Is this a known issue? *00:04:54.387* - STDERR*00:04:54.387* + yum -y install ovirt-host*00:04:54.387* Error: Package: vdsm-4.20.22-8.git4f691c1.el7.centos.x86_64 (alocalsync)*00:04:54.388* Requires: mom >= 0.5.12*00:04:54.388* Available: mom-0.5.11-1.el7.centos.noarch (alocalsync)*00:04:54.389* mom = 0.5.11-1.el7.centos*00:04:54.390* commit c103010f8b5ae1ccee93f5611a198162f4ccc018 (tag: v4.20.22) Author: Andrej Krejcir <akrejcir(a)redhat.com> AuthorDate: Mon Feb 26 11:37:02 2018 +0100 Commit: Francesco Romani <fromani(a)redhat.com> CommitDate: Wed Mar 14 08:44:04 2018 -0400 mom: Configure mom to use the new json rpc client Also updating used version of mom to 0.5.12 Change-Id: I44a35129dd80f85785625b822947ee138e67da00 Bug-Url: https://bugzilla.redhat.com/1544476 Signed-off-by: Andrej Krejcir <akrejcir(a)redhat.com> (cherry picked from commit b5a036cd628763fb0770eaf4cb6595eca9c64148) Build: http://jenkins.ovirt.org/job/ovirt-system-tests_manual/2412/consoleFull <http://jenkins.ovirt.org/job/ovirt-system-tests_manual/2412/console> Here is another build with same issue: http://jenkins.ovirt.org/job/ovirt-system-tests_manual/2404/console Nir

2 1

[CQ]: 88605, 4 (ovirt-engine) failed "ovirt-master" system tests, but isn't the failure root cause
by oVirt Jenkins 20 Mar '18

20 Mar '18

A system test invoked by the "ovirt-master" change queue including change 88605,4 (ovirt-engine) failed. However, this change seems not to be the root cause for this failure. Change 88777,3 (ovirt-engine) that this change depends on or is based on, was detected as the cause of the testing failures. This change had been removed from the testing queue. Artifacts built from this change will not be released until either change 88777,3 (ovirt-engine) is fixed and this change is updated to refer to or rebased on the fixed version, or this change is modified to no longer depend on it. For further details about the change see: https://gerrit.ovirt.org/#/c/88605/4 For further details about the change that seems to be the root cause behind the testing failures see: https://gerrit.ovirt.org/#/c/88777/3 For failed test results see: http://jenkins.ovirt.org/job/ovirt-master_change-queue-tester/6459/

1 0

[CQ]: 89171, 2 (ovirt-engine) failed "ovirt-4.2" system tests, but isn't the failure root cause
by oVirt Jenkins 19 Mar '18

19 Mar '18

A system test invoked by the "ovirt-4.2" change queue including change 89171,2 (ovirt-engine) failed. However, this change seems not to be the root cause for this failure. Change 89165,2 (ovirt-engine) that this change depends on or is based on, was detected as the cause of the testing failures. This change had been removed from the testing queue. Artifacts built from this change will not be released until either change 89165,2 (ovirt-engine) is fixed and this change is updated to refer to or rebased on the fixed version, or this change is modified to no longer depend on it. For further details about the change see: https://gerrit.ovirt.org/#/c/89171/2 For further details about the change that seems to be the root cause behind the testing failures see: https://gerrit.ovirt.org/#/c/89165/2 For failed test results see: http://jenkins.ovirt.org/job/ovirt-4.2_change-queue-tester/1378/

1 0

[CQ]: 88549, 6 (ovirt-engine) failed "ovirt-master" system tests, but isn't the failure root cause
by oVirt Jenkins 19 Mar '18

19 Mar '18

A system test invoked by the "ovirt-master" change queue including change 88549,6 (ovirt-engine) failed. However, this change seems not to be the root cause for this failure. Change 88777,3 (ovirt-engine) that this change depends on or is based on, was detected as the cause of the testing failures. This change had been removed from the testing queue. Artifacts built from this change will not be released until either change 88777,3 (ovirt-engine) is fixed and this change is updated to refer to or rebased on the fixed version, or this change is modified to no longer depend on it. For further details about the change see: https://gerrit.ovirt.org/#/c/88549/6 For further details about the change that seems to be the root cause behind the testing failures see: https://gerrit.ovirt.org/#/c/88777/3 For failed test results see: http://jenkins.ovirt.org/job/ovirt-master_change-queue-tester/6455/

1 0

[CQ]: 89170, 2 (ovirt-engine) failed "ovirt-4.2" system tests, but isn't the failure root cause
by oVirt Jenkins 19 Mar '18

19 Mar '18

A system test invoked by the "ovirt-4.2" change queue including change 89170,2 (ovirt-engine) failed. However, this change seems not to be the root cause for this failure. Change 89165,2 (ovirt-engine) that this change depends on or is based on, was detected as the cause of the testing failures. This change had been removed from the testing queue. Artifacts built from this change will not be released until either change 89165,2 (ovirt-engine) is fixed and this change is updated to refer to or rebased on the fixed version, or this change is modified to no longer depend on it. For further details about the change see: https://gerrit.ovirt.org/#/c/89170/2 For further details about the change that seems to be the root cause behind the testing failures see: https://gerrit.ovirt.org/#/c/89165/2 For failed test results see: http://jenkins.ovirt.org/job/ovirt-4.2_change-queue-tester/1374/

1 0

[CQ]: 88604, 7 (ovirt-engine) failed "ovirt-master" system tests, but isn't the failure root cause
by oVirt Jenkins 19 Mar '18

19 Mar '18

A system test invoked by the "ovirt-master" change queue including change 88604,7 (ovirt-engine) failed. However, this change seems not to be the root cause for this failure. Change 88777,3 (ovirt-engine) that this change depends on or is based on, was detected as the cause of the testing failures. This change had been removed from the testing queue. Artifacts built from this change will not be released until either change 88777,3 (ovirt-engine) is fixed and this change is updated to refer to or rebased on the fixed version, or this change is modified to no longer depend on it. For further details about the change see: https://gerrit.ovirt.org/#/c/88604/7 For further details about the change that seems to be the root cause behind the testing failures see: https://gerrit.ovirt.org/#/c/88777/3 For failed test results see: http://jenkins.ovirt.org/job/ovirt-master_change-queue-tester/6450/

1 0

[CQ]: 89169, 2 (ovirt-engine) failed "ovirt-4.2" system tests, but isn't the failure root cause
by oVirt Jenkins 19 Mar '18

19 Mar '18

A system test invoked by the "ovirt-4.2" change queue including change 89169,2 (ovirt-engine) failed. However, this change seems not to be the root cause for this failure. Change 89165,2 (ovirt-engine) that this change depends on or is based on, was detected as the cause of the testing failures. This change had been removed from the testing queue. Artifacts built from this change will not be released until either change 89165,2 (ovirt-engine) is fixed and this change is updated to refer to or rebased on the fixed version, or this change is modified to no longer depend on it. For further details about the change see: https://gerrit.ovirt.org/#/c/89169/2 For further details about the change that seems to be the root cause behind the testing failures see: https://gerrit.ovirt.org/#/c/89165/2 For failed test results see: http://jenkins.ovirt.org/job/ovirt-4.2_change-queue-tester/1370/

1 0

[oVirt Jenkins] ovirt-system-tests_hc-basic-suite-master - Build # 234 - Failure!
by jenkins＠jenkins.phx.ovirt.org 19 Mar '18

19 Mar '18

------=_Part_383_725808303.1521469561145 Content-Type: text/plain; charset=UTF-8 Content-Transfer-Encoding: quoted-printable Project: http://jenkins.ovirt.org/job/ovirt-system-tests_hc-basic-suite-mas= ter/=20 Build: http://jenkins.ovirt.org/job/ovirt-system-tests_hc-basic-suite-maste= r/234/ Build Number: 234 Build Status: Failure Triggered By: Started by timer ------------------------------------- Changes Since Last Success: ------------------------------------- Changes for Build #234 [Marcin Mirecki] network-suite-master: Make openstack_client_config fixture= available to [Daniel Belenky] Align standard-stage with normalized DSL output [Daniel Belenky] Add python-pluggy to .packages ----------------- Failed Tests: ----------------- 1 tests failed. FAILED: 002_bootstrap.add_hosts Error Message: The response content type 'text/html; charset=3Diso-8859-1' isn't the expec= ted XML -------------------- >> begin captured logging << -------------------- ovirtlago.testlib: ERROR: * Unhandled exception in <function _host_is_u= p_4 at 0x4da6398> Traceback (most recent call last): File "/usr/lib/python2.7/site-packages/ovirtlago/testlib.py", line 219, i= n assert_equals_within res =3D func() File "/home/jenkins/workspace/ovirt-system-tests_hc-basic-suite-master/ov= irt-system-tests/hc-basic-suite-master/test-scenarios/002_bootstrap.py", li= ne 255, in _host_is_up_4 host_obj =3D host_service.get() File "/usr/lib64/python2.7/site-packages/ovirtsdk4/services.py", line 363= 46, in get return self._internal_get(headers, query, wait) File "/usr/lib64/python2.7/site-packages/ovirtsdk4/service.py", line 211,= in _internal_get return future.wait() if wait else future File "/usr/lib64/python2.7/site-packages/ovirtsdk4/service.py", line 55, = in wait return self._code(response) File "/usr/lib64/python2.7/site-packages/ovirtsdk4/service.py", line 208,= in callback self._check_fault(response) File "/usr/lib64/python2.7/site-packages/ovirtsdk4/service.py", line 130,= in _check_fault body =3D self._internal_read_body(response) File "/usr/lib64/python2.7/site-packages/ovirtsdk4/service.py", line 316,= in _internal_read_body self._connection.check_xml_content_type(response) File "/usr/lib64/python2.7/site-packages/ovirtsdk4/__init__.py", line 849= , in check_xml_content_type response.headers File "/usr/lib64/python2.7/site-packages/ovirtsdk4/__init__.py", line 889= , in _check_content_type raise Error(msg) Error: The response content type 'text/html; charset=3Diso-8859-1' isn't th= e expected XML --------------------- >> end captured logging << --------------------- Stack Trace: File "/usr/lib64/python2.7/unittest/case.py", line 369, in run testMethod() File "/usr/lib/python2.7/site-packages/nose/case.py", line 197, in runTes= t self.test(*self.arg) File "/usr/lib/python2.7/site-packages/ovirtlago/testlib.py", line 129, i= n wrapped_test test() File "/usr/lib/python2.7/site-packages/ovirtlago/testlib.py", line 59, in= wrapper return func(get_test_prefix(), *args, **kwargs) File "/home/jenkins/workspace/ovirt-system-tests_hc-basic-suite-master/ov= irt-system-tests/hc-basic-suite-master/test-scenarios/002_bootstrap.py", li= ne 230, in add_hosts add_hosts_4(prefix) File "/home/jenkins/workspace/ovirt-system-tests_hc-basic-suite-master/ov= irt-system-tests/hc-basic-suite-master/test-scenarios/002_bootstrap.py", li= ne 274, in add_hosts_4 testlib.assert_true_within(_host_is_up_4, timeout=3D15*60) File "/usr/lib/python2.7/site-packages/ovirtlago/testlib.py", line 263, i= n assert_true_within assert_equals_within(func, True, timeout, allowed_exceptions) File "/usr/lib/python2.7/site-packages/ovirtlago/testlib.py", line 219, i= n assert_equals_within res =3D func() File "/home/jenkins/workspace/ovirt-system-tests_hc-basic-suite-master/ov= irt-system-tests/hc-basic-suite-master/test-scenarios/002_bootstrap.py", li= ne 255, in _host_is_up_4 host_obj =3D host_service.get() File "/usr/lib64/python2.7/site-packages/ovirtsdk4/services.py", line 363= 46, in get return self._internal_get(headers, query, wait) File "/usr/lib64/python2.7/site-packages/ovirtsdk4/service.py", line 211,= in _internal_get return future.wait() if wait else future File "/usr/lib64/python2.7/site-packages/ovirtsdk4/service.py", line 55, = in wait return self._code(response) File "/usr/lib64/python2.7/site-packages/ovirtsdk4/service.py", line 208,= in callback self._check_fault(response) File "/usr/lib64/python2.7/site-packages/ovirtsdk4/service.py", line 130,= in _check_fault body =3D self._internal_read_body(response) File "/usr/lib64/python2.7/site-packages/ovirtsdk4/service.py", line 316,= in _internal_read_body self._connection.check_xml_content_type(response) File "/usr/lib64/python2.7/site-packages/ovirtsdk4/__init__.py", line 849= , in check_xml_content_type response.headers File "/usr/lib64/python2.7/site-packages/ovirtsdk4/__init__.py", line 889= , in _check_content_type raise Error(msg) 'The response content type \'text/html; charset=3Diso-8859-1\' isn\'t the e= xpected XML\n-------------------- >> begin captured logging << ------------= --------\novirtlago.testlib: ERROR: * Unhandled exception in <function = _host_is_up_4 at 0x4da6398>\nTraceback (most recent call last):\n File "/u= sr/lib/python2.7/site-packages/ovirtlago/testlib.py", line 219, in assert_e= quals_within\n res =3D func()\n File "/home/jenkins/workspace/ovirt-sys= tem-tests_hc-basic-suite-master/ovirt-system-tests/hc-basic-suite-master/te= st-scenarios/002_bootstrap.py", line 255, in _host_is_up_4\n host_obj = =3D host_service.get()\n File "/usr/lib64/python2.7/site-packages/ovirtsdk= 4/services.py", line 36346, in get\n return self._internal_get(headers, = query, wait)\n File "/usr/lib64/python2.7/site-packages/ovirtsdk4/service.= py", line 211, in _internal_get\n return future.wait() if wait else futu= re\n File "/usr/lib64/python2.7/site-packages/ovirtsdk4/service.py", line = 55, in wait\n return self._code(response)\n File "/usr/lib64/python2.7/= site-packages/ovirtsdk4/service.py", line 208, in callback\n self._check= _fault(response)\n File "/usr/lib64/python2.7/site-packages/ovirtsdk4/serv= ice.py", line 130, in _check_fault\n body =3D self._internal_read_body(r= esponse)\n File "/usr/lib64/python2.7/site-packages/ovirtsdk4/service.py",= line 316, in _internal_read_body\n self._connection.check_xml_content_t= ype(response)\n File "/usr/lib64/python2.7/site-packages/ovirtsdk4/__init_= _.py", line 849, in check_xml_content_type\n response.headers\n File "/= usr/lib64/python2.7/site-packages/ovirtsdk4/__init__.py", line 889, in _che= ck_content_type\n raise Error(msg)\nError: The response content type \'t= ext/html; charset=3Diso-8859-1\' isn\'t the expected XML\n-----------------= ---- >> end captured logging << ---------------------' ------=_Part_383_725808303.1521469561145--

1 1

[CQ]: 89148, 2 (ovirt-engine) failed "ovirt-master" system tests, but isn't the failure root cause
by oVirt Jenkins 19 Mar '18

19 Mar '18

A system test invoked by the "ovirt-master" change queue including change 89148,2 (ovirt-engine) failed. However, this change seems not to be the root cause for this failure. Change 88777,3 (ovirt-engine) that this change depends on or is based on, was detected as the cause of the testing failures. This change had been removed from the testing queue. Artifacts built from this change will not be released until either change 88777,3 (ovirt-engine) is fixed and this change is updated to refer to or rebased on the fixed version, or this change is modified to no longer depend on it. For further details about the change see: https://gerrit.ovirt.org/#/c/89148/2 For further details about the change that seems to be the root cause behind the testing failures see: https://gerrit.ovirt.org/#/c/88777/3 For failed test results see: http://jenkins.ovirt.org/job/ovirt-master_change-queue-tester/6446/

1 0

[JIRA] (OVIRT-1938) [OST] Build fail - mom >= 0.5.12 require, 0.5.11-1 available
by Nir Soffer (oVirt JIRA) 19 Mar '18

19 Mar '18

This is a multi-part message in MIME format... ------------=_1521490145-12803-299 Content-Type: text/plain; charset=UTF-8 Content-Transfer-Encoding: 7bit Nir Soffer created OVIRT-1938: --------------------------------- Summary: [OST] Build fail - mom >= 0.5.12 require, 0.5.11-1 available Key: OVIRT-1938 URL: https://ovirt-jira.atlassian.net/browse/OVIRT-1938 Project: oVirt - virtualization made easy Issue Type: By-EMAIL Reporter: Nir Soffer Assignee: infra OST run with engine 4.2 failed with missing mom. Is this a known issue? *00:04:54.387* - STDERR*00:04:54.387* + yum -y install ovirt-host*00:04:54.387* Error: Package: vdsm-4.20.22-8.git4f691c1.el7.centos.x86_64 (alocalsync)*00:04:54.388* Requires: mom >= 0.5.12*00:04:54.388* Available: mom-0.5.11-1.el7.centos.noarch (alocalsync)*00:04:54.389* mom = 0.5.11-1.el7.centos*00:04:54.390* commit c103010f8b5ae1ccee93f5611a198162f4ccc018 (tag: v4.20.22) Author: Andrej Krejcir <akrejcir(a)redhat.com> AuthorDate: Mon Feb 26 11:37:02 2018 +0100 Commit: Francesco Romani <fromani(a)redhat.com> CommitDate: Wed Mar 14 08:44:04 2018 -0400 mom: Configure mom to use the new json rpc client Also updating used version of mom to 0.5.12 Change-Id: I44a35129dd80f85785625b822947ee138e67da00 Bug-Url: https://bugzilla.redhat.com/1544476 Signed-off-by: Andrej Krejcir <akrejcir(a)redhat.com> (cherry picked from commit b5a036cd628763fb0770eaf4cb6595eca9c64148) Build: http://jenkins.ovirt.org/job/ovirt-system-tests_manual/2412/consoleFull <http://jenkins.ovirt.org/job/ovirt-system-tests_manual/2412/console> Here is another build with same issue: http://jenkins.ovirt.org/job/ovirt-system-tests_manual/2404/console Nir -- This message was sent by Atlassian Jira (v1001.0.0-SNAPSHOT#100082) ------------=_1521490145-12803-299 Content-Type: text/html; charset="UTF-8" Content-Disposition: inline Content-Transfer-Encoding: 7bit <html><body> <h3>Nir Soffer created OVIRT-1938:</h3> <pre> Summary: [OST] Build fail - mom >= 0.5.12 require, 0.5.11-1 available Key: OVIRT-1938 URL: https://ovirt-jira.atlassian.net/browse/OVIRT-1938 Project: oVirt - virtualization made easy Issue Type: By-EMAIL Reporter: Nir Soffer Assignee: infra</pre> <p>OST run with engine 4.2 failed with missing mom. Is this a known issue?</p> <p><strong>00:04:54.387</strong> – STDERR*00:04:54.387* + yum -y install ovirt-host*00:04:54.387* Error: Package: vdsm-4.20.22-8.git4f691c1.el7.centos.x86_64 (alocalsync)*00:04:54.388*</p> <pre>Requires: mom >= 0.5.12*00:04:54.388* Available:</pre> <p>mom-0.5.11-1.el7.centos.noarch (alocalsync)*00:04:54.389*</p> <pre>mom = 0.5.11-1.el7.centos*00:04:54.390*</pre> <p>commit c103010f8b5ae1ccee93f5611a198162f4ccc018 (tag: v4.20.22) Author: Andrej Krejcir <akrejcir(a)redhat.com> AuthorDate: Mon Feb 26 11:37:02 2018 +0100 Commit: Francesco Romani <fromani(a)redhat.com> CommitDate: Wed Mar 14 08:44:04 2018 -0400</p> <pre>mom: Configure mom to use the new json rpc client</pre> <pre>Also updating used version of mom to 0.5.12</pre> <pre>Change-Id: I44a35129dd80f85785625b822947ee138e67da00 Bug-Url: https://bugzilla.redhat.com/1544476 Signed-off-by: Andrej Krejcir <akrejcir(a)redhat.com> (cherry picked from commit b5a036cd628763fb0770eaf4cb6595eca9c64148)</pre> <p>Build: <a href="http://jenkins.ovirt.org/job/ovirt-system-tests_manual/2412/consoleFull">http://jenkins.ovirt.org/job/ovirt-system-tests_manual/2412/consoleFull</a> <<a href="http://jenkins.ovirt.org/job/ovirt-system-tests_manual/2412/console">http://jenkins.ovirt.org/job/ovirt-system-tests_manual/2412/console</a>></p> <p>Here is another build with same issue: <a href="http://jenkins.ovirt.org/job/ovirt-system-tests_manual/2404/console">http://jenkins.ovirt.org/job/ovirt-system-tests_manual/2404/console</a></p> <p>Nir</p> <p>— This message was sent by Atlassian Jira (v1001.0.0-SNAPSHOT#100082)</p> <img src="https://u4043402.ct.sendgrid.net/wf/open?upn=i5TMWGV99amJbNxJpSp2-2BJ33BSM3…" alt="" width="1" height="1" border="0" style="height:1px !important;width:1px !important;border-width:0 !important;margin-top:0 !important;margin-bottom:0 !important;margin-right:0 !important;margin-left:0 !important;padding-top:0 !important;padding-bottom:0 !important;padding-right:0 !important;padding-left:0 !important;"/> </body></html> ------------=_1521490145-12803-299--

1 0

[CQ]: 89166, 2 (ovirt-engine) failed "ovirt-4.2" system tests, but isn't the failure root cause
by oVirt Jenkins 19 Mar '18

19 Mar '18

A system test invoked by the "ovirt-4.2" change queue including change 89166,2 (ovirt-engine) failed. However, this change seems not to be the root cause for this failure. Change 89165,2 (ovirt-engine) that this change depends on or is based on, was detected as the cause of the testing failures. This change had been removed from the testing queue. Artifacts built from this change will not be released until either change 89165,2 (ovirt-engine) is fixed and this change is updated to refer to or rebased on the fixed version, or this change is modified to no longer depend on it. For further details about the change see: https://gerrit.ovirt.org/#/c/89166/2 For further details about the change that seems to be the root cause behind the testing failures see: https://gerrit.ovirt.org/#/c/89165/2 For failed test results see: http://jenkins.ovirt.org/job/ovirt-4.2_change-queue-tester/1365/

1 0

[oVirt Jenkins] ovirt-system-tests_hc-basic-suite-4.2 - Build # 19 - Failure!
by jenkins＠jenkins.phx.ovirt.org 19 Mar '18

19 Mar '18

------=_Part_373_1594740022.1521451551138 Content-Type: text/plain; charset=UTF-8 Content-Transfer-Encoding: quoted-printable Project: http://jenkins.ovirt.org/job/ovirt-system-tests_hc-basic-suite-4.2= /=20 Build: http://jenkins.ovirt.org/job/ovirt-system-tests_hc-basic-suite-4.2/1= 9/ Build Number: 19 Build Status: Failure Triggered By: Started by timer ------------------------------------- Changes Since Last Success: ------------------------------------- Changes for Build #19 [Martin Sivak] Cleanup of test code - next() replaced with any() [Daniel Belenky] Align standard-stage with normalized DSL output [Daniel Belenky] Add python-pluggy to .packages ----------------- Failed Tests: ----------------- 1 tests failed. FAILED: 002_bootstrap.add_hosts Error Message: The response content type 'text/html; charset=3Diso-8859-1' isn't the expec= ted XML -------------------- >> begin captured logging << -------------------- ovirtlago.testlib: ERROR: * Unhandled exception in <function _host_is_u= p_4 at 0x3631ed8> Traceback (most recent call last): File "/usr/lib/python2.7/site-packages/ovirtlago/testlib.py", line 219, i= n assert_equals_within res =3D func() File "/home/jenkins/workspace/ovirt-system-tests_hc-basic-suite-4.2/ovirt= -system-tests/hc-basic-suite-4.2/test-scenarios/002_bootstrap.py", line 254= , in _host_is_up_4 host_obj =3D host_service.get() File "/usr/lib64/python2.7/site-packages/ovirtsdk4/services.py", line 362= 19, in get return self._internal_get(headers, query, wait) File "/usr/lib64/python2.7/site-packages/ovirtsdk4/service.py", line 211,= in _internal_get return future.wait() if wait else future File "/usr/lib64/python2.7/site-packages/ovirtsdk4/service.py", line 55, = in wait return self._code(response) File "/usr/lib64/python2.7/site-packages/ovirtsdk4/service.py", line 208,= in callback self._check_fault(response) File "/usr/lib64/python2.7/site-packages/ovirtsdk4/service.py", line 130,= in _check_fault body =3D self._internal_read_body(response) File "/usr/lib64/python2.7/site-packages/ovirtsdk4/service.py", line 316,= in _internal_read_body self._connection.check_xml_content_type(response) File "/usr/lib64/python2.7/site-packages/ovirtsdk4/__init__.py", line 849= , in check_xml_content_type response.headers File "/usr/lib64/python2.7/site-packages/ovirtsdk4/__init__.py", line 889= , in _check_content_type raise Error(msg) Error: The response content type 'text/html; charset=3Diso-8859-1' isn't th= e expected XML --------------------- >> end captured logging << --------------------- Stack Trace: File "/usr/lib64/python2.7/unittest/case.py", line 369, in run testMethod() File "/usr/lib/python2.7/site-packages/nose/case.py", line 197, in runTes= t self.test(*self.arg) File "/usr/lib/python2.7/site-packages/ovirtlago/testlib.py", line 129, i= n wrapped_test test() File "/usr/lib/python2.7/site-packages/ovirtlago/testlib.py", line 59, in= wrapper return func(get_test_prefix(), *args, **kwargs) File "/home/jenkins/workspace/ovirt-system-tests_hc-basic-suite-4.2/ovirt= -system-tests/hc-basic-suite-4.2/test-scenarios/002_bootstrap.py", line 229= , in add_hosts add_hosts_4(prefix) File "/home/jenkins/workspace/ovirt-system-tests_hc-basic-suite-4.2/ovirt= -system-tests/hc-basic-suite-4.2/test-scenarios/002_bootstrap.py", line 273= , in add_hosts_4 testlib.assert_true_within(_host_is_up_4, timeout=3D15*60) File "/usr/lib/python2.7/site-packages/ovirtlago/testlib.py", line 263, i= n assert_true_within assert_equals_within(func, True, timeout, allowed_exceptions) File "/usr/lib/python2.7/site-packages/ovirtlago/testlib.py", line 219, i= n assert_equals_within res =3D func() File "/home/jenkins/workspace/ovirt-system-tests_hc-basic-suite-4.2/ovirt= -system-tests/hc-basic-suite-4.2/test-scenarios/002_bootstrap.py", line 254= , in _host_is_up_4 host_obj =3D host_service.get() File "/usr/lib64/python2.7/site-packages/ovirtsdk4/services.py", line 362= 19, in get return self._internal_get(headers, query, wait) File "/usr/lib64/python2.7/site-packages/ovirtsdk4/service.py", line 211,= in _internal_get return future.wait() if wait else future File "/usr/lib64/python2.7/site-packages/ovirtsdk4/service.py", line 55, = in wait return self._code(response) File "/usr/lib64/python2.7/site-packages/ovirtsdk4/service.py", line 208,= in callback self._check_fault(response) File "/usr/lib64/python2.7/site-packages/ovirtsdk4/service.py", line 130,= in _check_fault body =3D self._internal_read_body(response) File "/usr/lib64/python2.7/site-packages/ovirtsdk4/service.py", line 316,= in _internal_read_body self._connection.check_xml_content_type(response) File "/usr/lib64/python2.7/site-packages/ovirtsdk4/__init__.py", line 849= , in check_xml_content_type response.headers File "/usr/lib64/python2.7/site-packages/ovirtsdk4/__init__.py", line 889= , in _check_content_type raise Error(msg) 'The response content type \'text/html; charset=3Diso-8859-1\' isn\'t the e= xpected XML\n-------------------- >> begin captured logging << ------------= --------\novirtlago.testlib: ERROR: * Unhandled exception in <function = _host_is_up_4 at 0x3631ed8>\nTraceback (most recent call last):\n File "/u= sr/lib/python2.7/site-packages/ovirtlago/testlib.py", line 219, in assert_e= quals_within\n res =3D func()\n File "/home/jenkins/workspace/ovirt-sys= tem-tests_hc-basic-suite-4.2/ovirt-system-tests/hc-basic-suite-4.2/test-sce= narios/002_bootstrap.py", line 254, in _host_is_up_4\n host_obj =3D host= _service.get()\n File "/usr/lib64/python2.7/site-packages/ovirtsdk4/servic= es.py", line 36219, in get\n return self._internal_get(headers, query, w= ait)\n File "/usr/lib64/python2.7/site-packages/ovirtsdk4/service.py", lin= e 211, in _internal_get\n return future.wait() if wait else future\n Fi= le "/usr/lib64/python2.7/site-packages/ovirtsdk4/service.py", line 55, in w= ait\n return self._code(response)\n File "/usr/lib64/python2.7/site-pac= kages/ovirtsdk4/service.py", line 208, in callback\n self._check_fault(r= esponse)\n File "/usr/lib64/python2.7/site-packages/ovirtsdk4/service.py",= line 130, in _check_fault\n body =3D self._internal_read_body(response)= \n File "/usr/lib64/python2.7/site-packages/ovirtsdk4/service.py", line 31= 6, in _internal_read_body\n self._connection.check_xml_content_type(resp= onse)\n File "/usr/lib64/python2.7/site-packages/ovirtsdk4/__init__.py", l= ine 849, in check_xml_content_type\n response.headers\n File "/usr/lib6= 4/python2.7/site-packages/ovirtsdk4/__init__.py", line 889, in _check_conte= nt_type\n raise Error(msg)\nError: The response content type \'text/html= ; charset=3Diso-8859-1\' isn\'t the expected XML\n--------------------- >> = end captured logging << ---------------------' ------=_Part_373_1594740022.1521451551138--

1 1

[CQ]: 89036, 2 (ovirt-engine) failed "ovirt-master" system tests, but isn't the failure root cause
by oVirt Jenkins 19 Mar '18

19 Mar '18

A system test invoked by the "ovirt-master" change queue including change 89036,2 (ovirt-engine) failed. However, this change seems not to be the root cause for this failure. Change 88777,3 (ovirt-engine) that this change depends on or is based on, was detected as the cause of the testing failures. This change had been removed from the testing queue. Artifacts built from this change will not be released until either change 88777,3 (ovirt-engine) is fixed and this change is updated to refer to or rebased on the fixed version, or this change is modified to no longer depend on it. For further details about the change see: https://gerrit.ovirt.org/#/c/89036/2 For further details about the change that seems to be the root cause behind the testing failures see: https://gerrit.ovirt.org/#/c/88777/3 For failed test results see: http://jenkins.ovirt.org/job/ovirt-master_change-queue-tester/6441/

1 0

[oVirt Jenkins] ovirt-system-tests_network-suite-master - Build # 26 - Failure!
by jenkins＠jenkins.phx.ovirt.org 19 Mar '18

19 Mar '18

------=_Part_244_725589246.1521168930182 Content-Type: text/plain; charset=UTF-8 Content-Transfer-Encoding: 7bit Project: http://jenkins.ovirt.org/job/ovirt-system-tests_network-suite-master/ Build: http://jenkins.ovirt.org/job/ovirt-system-tests_network-suite-master/26/ Build Number: 26 Build Status: Failure Triggered By: Started by timer ------------------------------------- Changes Since Last Success: ------------------------------------- Changes for Build #26 [Michal Skrivanek] master: add USB to the sanity VM [Sandro Bonazzola] mirrors-reposync: add qemu-ev testing repo [Daniel Belenky] Modify mergeoptions in parser utils to merge runif ----------------- Failed Tests: ----------------- No tests ran. ------=_Part_244_725589246.1521168930182--

1 11

[CQ]: 89165,2 (ovirt-engine) failed "ovirt-4.2" system tests
by oVirt Jenkins 19 Mar '18

19 Mar '18

Change 89165,2 (ovirt-engine) is probably the reason behind recent system test failures in the "ovirt-4.2" change queue and needs to be fixed. This change had been removed from the testing queue. Artifacts build from this change will not be released until it is fixed. For further details about the change see: https://gerrit.ovirt.org/#/c/89165/2 For failed test results see: http://jenkins.ovirt.org/job/ovirt-4.2_change-queue-tester/1362/

1 0

[JIRA] (OVIRT-1937) upgrade jenkins.ovirt.org to 2.107.1 LTS
by Evgheni Dereveanchin (oVirt JIRA) 19 Mar '18

19 Mar '18

This is a multi-part message in MIME format... ------------=_1521467493-21745-253 Content-Type: text/plain; charset=UTF-8 Content-Transfer-Encoding: 7bit [ https://ovirt-jira.atlassian.net/browse/OVIRT-1937?page=com.atlassian.jira.… ] Evgheni Dereveanchin reassigned OVIRT-1937: ------------------------------------------- Assignee: Evgheni Dereveanchin (was: infra) > upgrade jenkins.ovirt.org to 2.107.1 LTS > ---------------------------------------- > > Key: OVIRT-1937 > URL: https://ovirt-jira.atlassian.net/browse/OVIRT-1937 > Project: oVirt - virtualization made easy > Issue Type: Task > Reporter: Evgheni Dereveanchin > Assignee: Evgheni Dereveanchin > -- This message was sent by Atlassian Jira (v1001.0.0-SNAPSHOT#100082) ------------=_1521467493-21745-253 Content-Type: text/html; charset="UTF-8" Content-Disposition: inline Content-Transfer-Encoding: 7bit <html><body> <pre>[ https://ovirt-jira.atlassian.net/browse/OVIRT-1937?page=com.atlassian.jira.… ]</pre> <h3>Evgheni Dereveanchin reassigned OVIRT-1937:</h3> <pre>Assignee: Evgheni Dereveanchin (was: infra)</pre> <blockquote><h3>upgrade jenkins.ovirt.org to 2.107.1 LTS</h3> <pre> Key: OVIRT-1937 URL: https://ovirt-jira.atlassian.net/browse/OVIRT-1937 Project: oVirt - virtualization made easy Issue Type: Task Reporter: Evgheni Dereveanchin Assignee: Evgheni Dereveanchin</pre></blockquote> <p>— This message was sent by Atlassian Jira (v1001.0.0-SNAPSHOT#100082)</p> <img src="https://u4043402.ct.sendgrid.net/wf/open?upn=i5TMWGV99amJbNxJpSp2-2BJ33BSM3…" alt="" width="1" height="1" border="0" style="height:1px !important;width:1px !important;border-width:0 !important;margin-top:0 !important;margin-bottom:0 !important;margin-right:0 !important;margin-left:0 !important;padding-top:0 !important;padding-bottom:0 !important;padding-right:0 !important;padding-left:0 !important;"/> </body></html> ------------=_1521467493-21745-253--

1 0

[JIRA] (OVIRT-1937) upgrade jenkins.ovirt.org to 2.107.1 LTS
by Evgheni Dereveanchin (oVirt JIRA) 19 Mar '18

19 Mar '18

This is a multi-part message in MIME format... ------------=_1521467488-11734-171 Content-Type: text/plain; charset=UTF-8 Content-Transfer-Encoding: 7bit Evgheni Dereveanchin created OVIRT-1937: ------------------------------------------- Summary: upgrade jenkins.ovirt.org to 2.107.1 LTS Key: OVIRT-1937 URL: https://ovirt-jira.atlassian.net/browse/OVIRT-1937 Project: oVirt - virtualization made easy Issue Type: Task Reporter: Evgheni Dereveanchin Assignee: infra A new LTS version of Jenkins is out, our instances need to be updated -- This message was sent by Atlassian Jira (v1001.0.0-SNAPSHOT#100082) ------------=_1521467488-11734-171 Content-Type: text/html; charset="UTF-8" Content-Disposition: inline Content-Transfer-Encoding: 7bit <html><body> <h3>Evgheni Dereveanchin created OVIRT-1937:</h3> <pre> Summary: upgrade jenkins.ovirt.org to 2.107.1 LTS Key: OVIRT-1937 URL: https://ovirt-jira.atlassian.net/browse/OVIRT-1937 Project: oVirt - virtualization made easy Issue Type: Task Reporter: Evgheni Dereveanchin Assignee: infra</pre> <p>A new LTS version of Jenkins is out, our instances need to be updated</p> <p>— This message was sent by Atlassian Jira (v1001.0.0-SNAPSHOT#100082)</p> <img src="https://u4043402.ct.sendgrid.net/wf/open?upn=i5TMWGV99amJbNxJpSp2-2BJ33BSM3…" alt="" width="1" height="1" border="0" style="height:1px !important;width:1px !important;border-width:0 !important;margin-top:0 !important;margin-bottom:0 !important;margin-right:0 !important;margin-left:0 !important;padding-top:0 !important;padding-bottom:0 !important;padding-right:0 !important;padding-left:0 !important;"/> </body></html> ------------=_1521467488-11734-171--

1 0

[CQ]: 89107, 2 (ovirt-engine) failed "ovirt-master" system tests, but isn't the failure root cause
by oVirt Jenkins 19 Mar '18

19 Mar '18

A system test invoked by the "ovirt-master" change queue including change 89107,2 (ovirt-engine) failed. However, this change seems not to be the root cause for this failure. Change 88777,3 (ovirt-engine) that this change depends on or is based on, was detected as the cause of the testing failures. This change had been removed from the testing queue. Artifacts built from this change will not be released until either change 88777,3 (ovirt-engine) is fixed and this change is updated to refer to or rebased on the fixed version, or this change is modified to no longer depend on it. For further details about the change see: https://gerrit.ovirt.org/#/c/89107/2 For further details about the change that seems to be the root cause behind the testing failures see: https://gerrit.ovirt.org/#/c/88777/3 For failed test results see: http://jenkins.ovirt.org/job/ovirt-master_change-queue-tester/6437/

1 0

this is the mock failure
by Dafna Ron 19 Mar '18

19 Mar '18

Hi, I can use a hand debugging this failure. from what I can see, mock is failing on upgrade suite: https://rhv-devops-jenkins.rhev-ci-vms.eng.rdu2.redhat.com/job/rhv-4.2_chan… Can you please have a look to see if you can direct me on what might be the issue? Thanks, Dafna

2 1

[CQ]: 88790, 2 (ovirt-engine) failed "ovirt-master" system tests, but isn't the failure root cause
by oVirt Jenkins 19 Mar '18

19 Mar '18

A system test invoked by the "ovirt-master" change queue including change 88790,2 (ovirt-engine) failed. However, this change seems not to be the root cause for this failure. Change 88777,3 (ovirt-engine) that this change depends on or is based on, was detected as the cause of the testing failures. This change had been removed from the testing queue. Artifacts built from this change will not be released until either change 88777,3 (ovirt-engine) is fixed and this change is updated to refer to or rebased on the fixed version, or this change is modified to no longer depend on it. For further details about the change see: https://gerrit.ovirt.org/#/c/88790/2 For further details about the change that seems to be the root cause behind the testing failures see: https://gerrit.ovirt.org/#/c/88777/3 For failed test results see: http://jenkins.ovirt.org/job/ovirt-master_change-queue-tester/6434/

1 0

[oVirt Jenkins] ovirt-system-tests_he-basic-suite-4.1 - Build # 218 - Failure!
by jenkins＠jenkins.phx.ovirt.org 19 Mar '18

19 Mar '18

Project: http://jenkins.ovirt.org/job/ovirt-system-tests_he-basic-suite-4.1/ Build: http://jenkins.ovirt.org/job/ovirt-system-tests_he-basic-suite-4.1/218/ Build Number: 218 Build Status: Failure Triggered By: Started by timer ------------------------------------- Changes Since Last Success: ------------------------------------- Changes for Build #218 [Martin Sivak] Cleanup of test code - next() replaced with any() ----------------- Failed Tests: ----------------- 1 tests failed. FAILED: 002_bootstrap.he_check_ha_agent Error Message: False is not true -------------------- >> begin captured logging << -------------------- lago.ssh: DEBUG: start task:43578f7a-22ad-42d7-9ea9-509c2acda822:Get ssh client for lago-he-basic-suite-4-1-host0: lago.ssh: DEBUG: end task:43578f7a-22ad-42d7-9ea9-509c2acda822:Get ssh client for lago-he-basic-suite-4-1-host0: lago.ssh: DEBUG: Running 6b4bf9ac on lago-he-basic-suite-4-1-host0: grep Permission denied /var/log/ovirt-hosted-engine-ha/agent.log lago.ssh: DEBUG: Command 6b4bf9ac on lago-he-basic-suite-4-1-host0 returned with 2 lago.ssh: DEBUG: Command 6b4bf9ac on lago-he-basic-suite-4-1-host0 output: /var/log/ovirt-hosted-engine-ha/agent.log:IOError: [Errno 13] Permission denied: '/var/run/ovirt-hosted-engine-ha/vm.conf' /var/log/ovirt-hosted-engine-ha/agent.log:IOError: [Errno 13] Permission denied: '/var/run/ovirt-hosted-engine-ha/vm.conf' /var/log/ovirt-hosted-engine-ha/agent.log:IOError: [Errno 13] Permission denied: '/var/run/ovirt-hosted-engine-ha/vm.conf' /var/log/ovirt-hosted-engine-ha/agent.log:IOError: [Errno 13] Permission denied: '/var/run/ovirt-hosted-engine-ha/vm.conf' /var/log/ovirt-hosted-engine-ha/agent.log:IOError: [Errno 13] Permission denied: '/var/run/ovirt-hosted-engine-ha/vm.conf' /var/log/ovirt-hosted-engine-ha/agent.log:IOError: [Errno 13] Permission denied: '/var/run/ovirt-hosted-engine-ha/vm.conf' /var/log/ovirt-hosted-engine-ha/agent.log:IOError: [Errno 13] Permission denied: '/var/run/ovirt-hosted-engine-ha/vm.conf' /var/log/ovirt-hosted-engine-ha/agent.log:IOError: [Errno 13] Permission denied: '/var/run/ovirt-hosted-engine-ha/vm.conf' lago.ssh: DEBUG: Command 6b4bf9ac on lago-he-basic-suite-4-1-host0 errors: grep: denied: No such file or directory --------------------- >> end captured logging << --------------------- Stack Trace: File "/usr/lib64/python2.7/unittest/case.py", line 369, in run testMethod() File "/usr/lib/python2.7/site-packages/nose/case.py", line 197, in runTest self.test(*self.arg) File "/usr/lib/python2.7/site-packages/ovirtlago/testlib.py", line 129, in wrapped_test test() File "/usr/lib/python2.7/site-packages/ovirtlago/testlib.py", line 59, in wrapper return func(get_test_prefix(), *args, **kwargs) File "/home/jenkins/workspace/ovirt-system-tests_he-basic-suite-4.1/ovirt-system-tests/he-basic-suite-4.1/test-scenarios/002_bootstrap.py", line 1036, in he_check_ha_agent 'Permission denied' not in result.out File "/usr/lib64/python2.7/unittest/case.py", line 462, in assertTrue raise self.failureException(msg) "False is not true\n-------------------- >> begin captured logging << --------------------\nlago.ssh: DEBUG: start task:43578f7a-22ad-42d7-9ea9-509c2acda822:Get ssh client for lago-he-basic-suite-4-1-host0:\nlago.ssh: DEBUG: end task:43578f7a-22ad-42d7-9ea9-509c2acda822:Get ssh client for lago-he-basic-suite-4-1-host0:\nlago.ssh: DEBUG: Running 6b4bf9ac on lago-he-basic-suite-4-1-host0: grep Permission denied /var/log/ovirt-hosted-engine-ha/agent.log\nlago.ssh: DEBUG: Command 6b4bf9ac on lago-he-basic-suite-4-1-host0 returned with 2\nlago.ssh: DEBUG: Command 6b4bf9ac on lago-he-basic-suite-4-1-host0 output:\n /var/log/ovirt-hosted-engine-ha/agent.log:IOError: [Errno 13] Permission denied: '/var/run/ovirt-hosted-engine-ha/vm.conf'\n/var/log/ovirt-hosted-engine-ha/agent.log:IOError: [Errno 13] Permission denied: '/var/run/ovirt-hosted-engine-ha/vm.conf'\n/var/log/ovirt-hosted-engine-ha/agent.log:IOError: [Errno 13] Permission denied: '/var/run/ovirt-hosted-engine-ha/vm.conf'\n/var/log/ovirt-hosted-engine-ha/agent.log:IOError: [Errno 13] Permission denied: '/var/run/ovirt-hosted-engine-ha/vm.conf'\n/var/log/ovirt-hosted-engine-ha/agent.log:IOError: [Errno 13] Permission denied: '/var/run/ovirt-hosted-engine-ha/vm.conf'\n/var/log/ovirt-hosted-engine-ha/agent.log:IOError: [Errno 13] Permission denied: '/var/run/ovirt-hosted-engine-ha/vm.conf'\n/var/log/ovirt-hosted-engine-ha/agent.log:IOError: [Errno 13] Permission denied: '/var/run/ovirt-hosted-engine-ha/vm.conf'\n/var/log/ovirt-hosted-engine-ha/agent.log:IOError: [Errno 13] Permission denied: '/var/run/ovirt-hosted-engine-ha/vm.conf'\n\nlago.ssh: DEBUG: Command 6b4bf9ac on lago-he-basic-suite-4-1-host0 errors:\n grep: denied: No such file or directory\n\n--------------------- >> end captured logging << ---------------------"

3 13

[CQ]: 89145, 2 (ovirt-engine) failed "ovirt-master" system tests, but isn't the failure root cause
by oVirt Jenkins 19 Mar '18

19 Mar '18

A system test invoked by the "ovirt-master" change queue including change 89145,2 (ovirt-engine) failed. However, this change seems not to be the root cause for this failure. Change 88777,3 (ovirt-engine) that this change depends on or is based on, was detected as the cause of the testing failures. This change had been removed from the testing queue. Artifacts built from this change will not be released until either change 88777,3 (ovirt-engine) is fixed and this change is updated to refer to or rebased on the fixed version, or this change is modified to no longer depend on it. For further details about the change see: https://gerrit.ovirt.org/#/c/89145/2 For further details about the change that seems to be the root cause behind the testing failures see: https://gerrit.ovirt.org/#/c/88777/3 For failed test results see: http://jenkins.ovirt.org/job/ovirt-master_change-queue-tester/6431/

1 0

add_secondary_storage_domains failed (was: [oVirt Jenkins] ovirt-system-tests_he-basic-suite-4.1 - Build # 223 - Still Failing!)
by Yedidyah Bar David 19 Mar '18

19 Mar '18

On Mon, Mar 19, 2018 at 9:01 AM, Nir Soffer <nsoffer(a)redhat.com> wrote: > > > On Sun, Mar 18, 2018 at 4:21 PM Yedidyah Bar David <didi(a)redhat.com> wrote: >> >> On Sun, Mar 18, 2018 at 2:48 PM, Yedidyah Bar David <didi(a)redhat.com> >> wrote: >> > On Sun, Mar 18, 2018 at 1:45 PM, Yedidyah Bar David <didi(a)redhat.com> >> > wrote: >> >> On Sun, Mar 18, 2018 at 11:20 AM, <jenkins(a)jenkins.phx.ovirt.org> >> >> wrote: >> >>> Project: >> >>> http://jenkins.ovirt.org/job/ovirt-system-tests_he-basic-suite-4.1/ >> >>> Build: >> >>> http://jenkins.ovirt.org/job/ovirt-system-tests_he-basic-suite-4.1/223/ >> >>> Build Number: 223 >> >>> Build Status: Still Failing >> >>> Triggered By: Started by timer >> >> >> >> It was broken by: >> >> >> >> [1] https://gerrit.ovirt.org/88483 >> >> >> >> It should be fixed by: >> >> >> >> [2] https://bugzilla.redhat.com/show_bug.cgi?id=1554283 >> >> >> >> [2] is on modified, no idea about its status other than that. >> >> >> >> I didn't intend to merge [1] before [2] is fixed, not sure if >> >> Sandro didn't notice my comment there, or thought that [2] is fixed. >> >> >> >> If it's annoying we can revert [1] and re-merge when [2] is fixed. >> > >> > Talked with Gal, and pushed this, which should hopefully fix: >> > >> > https://gerrit.ovirt.org/89136 >> >> It indeed seems to fix [1][2]: >> >> 13:13:24 # he_get_shared_config: >> 13:13:26 # he_get_shared_config: Success (in 0:00:01) >> 13:13:26 # sleep: >> 13:15:26 # sleep: Success (in 0:02:00) >> 13:15:26 # add_he_hosts: >> 13:16:18 # add_he_hosts: Success (in 0:00:52) >> 13:16:18 # he_check_ha_agent: >> 13:16:19 # he_check_ha_agent: Success (in 0:00:00) >> >> But later fails: >> >> 13:16:20 # add_secondary_storage_domains: >> 13:19:30 Error while running thread >> 13:19:30 Traceback (most recent call last): >> 13:19:30 File "/usr/lib/python2.7/site-packages/lago/utils.py", line >> 58, in _ret_via_queue >> 13:19:30 queue.put({'return': func()}) >> 13:19:30 File >> >> "/home/jenkins/workspace/ovirt-system-tests_master_check-patch-el7-x86_64/ovirt-system-tests/he-basic-suite-4.1/test-scenarios/002_bootstrap.py", >> line 491, in add_nfs_storage_domain >> 13:19:30 add_generic_nfs_storage_domain(prefix, SD_NFS_NAME, >> SD_NFS_HOST_NAME, SD_NFS_PATH) >> 13:19:30 File >> >> "/home/jenkins/workspace/ovirt-system-tests_master_check-patch-el7-x86_64/ovirt-system-tests/he-basic-suite-4.1/test-scenarios/002_bootstrap.py", >> line 496, in add_generic_nfs_storage_domain >> 13:19:30 add_generic_nfs_storage_domain_4(prefix, sd_nfs_name, >> nfs_host_name, mount_path, sd_format, sd_type, nfs_version) >> 13:19:30 File >> >> "/home/jenkins/workspace/ovirt-system-tests_master_check-patch-el7-x86_64/ovirt-system-tests/he-basic-suite-4.1/test-scenarios/002_bootstrap.py", >> line 552, in add_generic_nfs_storage_domain_4 >> 13:19:30 _add_storage_domain_4(api, p) >> 13:19:30 File >> >> "/home/jenkins/workspace/ovirt-system-tests_master_check-patch-el7-x86_64/ovirt-system-tests/he-basic-suite-4.1/test-scenarios/002_bootstrap.py", >> line 466, in _add_storage_domain_4 >> 13:19:30 id=sd.id, >> 13:19:30 File >> "/usr/lib64/python2.7/site-packages/ovirtsdk4/services.py", line 2219, >> in add >> 13:19:30 return self._internal_add(storage_domain, headers, query, >> wait) >> 13:19:30 File >> "/usr/lib64/python2.7/site-packages/ovirtsdk4/service.py", line 223, >> in _internal_add >> 13:19:30 return future.wait() if wait else future >> 13:19:30 File >> "/usr/lib64/python2.7/site-packages/ovirtsdk4/service.py", line 53, in >> wait >> 13:19:30 return self._code(response) >> 13:19:30 File >> "/usr/lib64/python2.7/site-packages/ovirtsdk4/service.py", line 220, >> in callback >> 13:19:30 self._check_fault(response) >> 13:19:30 File >> "/usr/lib64/python2.7/site-packages/ovirtsdk4/service.py", line 123, >> in _check_fault >> 13:19:30 self._raise_error(response, body) >> 13:19:30 File >> "/usr/lib64/python2.7/site-packages/ovirtsdk4/service.py", line 109, >> in _raise_error >> 13:19:30 raise error >> 13:19:30 Error: Fault reason is "Operation Failed". Fault detail is >> "[Storage domain cannot be reached. Please ensure it is accessible >> from the host(s).]". HTTP response code is 400. >> 13:19:30 Error while running thread >> >> I'll retrigger now, but perhaps someone from storage wants to check. >> vdsm log has[3]: >> >> 2018-03-18 08:59:52,918-0400 INFO (jsonrpc/3) [jsonrpc.JsonRpcServer] >> RPC call Host.getHardwareInfo succeeded in 0.00 seconds (__init__:539) >> 2018-03-18 08:59:52,924-0400 INFO (jsonrpc/4) [vdsm.api] START >> prepareImage(sdUUID=u'424e809d-b7ad-4ed5-b6a1-5426d373a5d2', >> spUUID=u'8509c64f-cdd7-4713-9e09-a79d90ba26ed', >> imgUUID=u'718090ba-b36e-45cc-bcd6-597c16a766b9', >> leafUUID=u'23ee843b-20e0-4afe-98dc-e165334ac710', allowIllegal=False) >> from=::1,36020, task_id=92aeabf2-084a-4c98-a6e5-03efae38e7b3 (api:46) >> 2018-03-18 08:59:52,928-0400 INFO (jsonrpc/4) [vdsm.api] FINISH >> prepareImage error=Volume does not exist: >> (u'23ee843b-20e0-4afe-98dc-e165334ac710',) from=::1,36020, >> task_id=92aeabf2-084a-4c98-a6e5-03efae38e7b3 (api:50) >> 2018-03-18 08:59:52,928-0400 ERROR (jsonrpc/4) >> [storage.TaskManager.Task] >> (Task='92aeabf2-084a-4c98-a6e5-03efae38e7b3') Unexpected error >> (task:872) >> Traceback (most recent call last): >> File "/usr/share/vdsm/storage/task.py", line 879, in _run >> return fn(*args, **kargs) >> File "<string>", line 2, in prepareImage >> File "/usr/lib/python2.7/site-packages/vdsm/common/api.py", line 48, in >> method >> ret = func(*args, **kwargs) >> File "/usr/share/vdsm/storage/hsm.py", line 3137, in prepareImage >> raise se.VolumeDoesNotExist(leafUUID) >> VolumeDoesNotExist: Volume does not exist: >> (u'23ee843b-20e0-4afe-98dc-e165334ac710',) > > > Immediately after this failed prepare, we see: > > 2018-03-18 08:59:52,941-0400 INFO (jsonrpc/7) [vdsm.api] START > createVolume(sdUUID=u'424e809d-b7ad-4ed5-b6a1-5426d373a5d2', > spUUID=u'8509c64f-cdd7-4713-9e09-a79d90ba26ed', > imgUUID=u'718090ba-b36e-45cc-bcd6-597c16a766b9', size=u'1048576', > volFormat=5, preallocate=1, diskType=2, > volUUID=u'23ee843b-20e0-4afe-98dc-e165334ac710', > desc=u'hosted-engine.lockspace', > srcImgUUID=u'00000000-0000-0000-0000-000000000000', > srcVolUUID=u'00000000-0000-0000-0000-000000000000', initialSize=None) > from=::1,36020, task_id=c4c73314-ee10-419a-8265-b8b87763e99f (api:46) > > So that volume really did not exit when it was prepared, which means it is a > client > error, not vdsm error. > > The caller (hosted engine agent?) The caller is ovirt-system-tests. The code for add_secondary_storage_domains is duplicated among most of the suites, didn't check variations. See e.g.: https://github.com/oVirt/ovirt-system-tests/blob/master/basic-suite-master/… It's not calling vdsm directly, but uses the engine api (through the python sdk). Not sure if it's, as you say, "a bug in the client" (the engine), or we use the sdk improperly, or something else. Can someone please advice? Patches to ovirt-system-tests are welcome :-), especially patches to stress-test this, as it seems it's a sensitive timing issue - usually does not fail. The specific suite that failed above is he-basic-suite-4.1 . I think that the code there is identical with basic-suite-4.1. Both are different from basic-suite-master. No idea if the diff is due to diff between the versions, or it's simply a bug fixed in the master suite and not fixed in 4.1 suites. > should use Volume.getInfo to check if a > volume > exits. Non existing volume is considered expected error in this call and > will be > logged as with INFO log level. > > See > https://github.com/oVirt/vdsm/blob/f286cf7825578d8997ca6abc0801a823dee247ef… > >> >> [1] >> http://jenkins.ovirt.org/job/ovirt-system-tests_master_check-patch-el7-x86_… >> [2] >> http://jenkins.ovirt.org/job/ovirt-system-tests_master_check-patch-el7-x86_… >> [3] >> http://jenkins.ovirt.org/job/ovirt-system-tests_master_check-patch-el7-x86_… >> >> > >> >> >> >>> >> >>> ------------------------------------- >> >>> Changes Since Last Success: >> >>> ------------------------------------- >> >>> Changes for Build #218 >> >>> [Martin Sivak] Cleanup of test code - next() replaced with any() >> >>> >> >>> >> >>> Changes for Build #219 >> >>> [Martin Sivak] Cleanup of test code - next() replaced with any() >> >>> >> >>> >> >>> Changes for Build #220 >> >>> [Martin Sivak] Cleanup of test code - next() replaced with any() >> >>> >> >>> >> >>> Changes for Build #221 >> >>> [Martin Sivak] Cleanup of test code - next() replaced with any() >> >>> >> >>> >> >>> Changes for Build #222 >> >>> [Martin Sivak] Cleanup of test code - next() replaced with any() >> >>> >> >>> >> >>> Changes for Build #223 >> >>> [Martin Sivak] Cleanup of test code - next() replaced with any() >> >>> >> >>> >> >>> >> >>> >> >>> ----------------- >> >>> Failed Tests: >> >>> ----------------- >> >>> 1 tests failed. >> >>> FAILED: 002_bootstrap.he_check_ha_agent >> >>> >> >>> Error Message: >> >>> False is not true >> >>> -------------------- >> begin captured logging << -------------------- >> >>> lago.ssh: DEBUG: start task:cb75d7f3-e333-47f1-b3e9-76ff9c0b4b21:Get >> >>> ssh client for lago-he-basic-suite-4-1-host0: >> >>> lago.ssh: DEBUG: end task:cb75d7f3-e333-47f1-b3e9-76ff9c0b4b21:Get ssh >> >>> client for lago-he-basic-suite-4-1-host0: >> >>> lago.ssh: DEBUG: Running 7e67a956 on lago-he-basic-suite-4-1-host0: >> >>> grep Permission denied /var/log/ovirt-hosted-engine-ha/agent.log >> >>> lago.ssh: DEBUG: Command 7e67a956 on lago-he-basic-suite-4-1-host0 >> >>> returned with 2 >> >>> lago.ssh: DEBUG: Command 7e67a956 on lago-he-basic-suite-4-1-host0 >> >>> output: >> >>> /var/log/ovirt-hosted-engine-ha/agent.log:IOError: [Errno 13] >> >>> Permission denied: '/var/run/ovirt-hosted-engine-ha/vm.conf' >> >>> /var/log/ovirt-hosted-engine-ha/agent.log:IOError: [Errno 13] >> >>> Permission denied: '/var/run/ovirt-hosted-engine-ha/vm.conf' >> >>> /var/log/ovirt-hosted-engine-ha/agent.log:IOError: [Errno 13] >> >>> Permission denied: '/var/run/ovirt-hosted-engine-ha/vm.conf' >> >>> /var/log/ovirt-hosted-engine-ha/agent.log:IOError: [Errno 13] >> >>> Permission denied: '/var/run/ovirt-hosted-engine-ha/vm.conf' >> >>> /var/log/ovirt-hosted-engine-ha/agent.log:IOError: [Errno 13] >> >>> Permission denied: '/var/run/ovirt-hosted-engine-ha/vm.conf' >> >>> /var/log/ovirt-hosted-engine-ha/agent.log:IOError: [Errno 13] >> >>> Permission denied: '/var/run/ovirt-hosted-engine-ha/vm.conf' >> >>> /var/log/ovirt-hosted-engine-ha/agent.log:IOError: [Errno 13] >> >>> Permission denied: '/var/run/ovirt-hosted-engine-ha/vm.conf' >> >>> /var/log/ovirt-hosted-engine-ha/agent.log:IOError: [Errno 13] >> >>> Permission denied: '/var/run/ovirt-hosted-engine-ha/vm.conf' >> >>> /var/log/ovirt-hosted-engine-ha/agent.log:IOError: [Errno 13] >> >>> Permission denied: '/var/run/ovirt-hosted-engine-ha/vm.conf' >> >>> >> >>> lago.ssh: DEBUG: Command 7e67a956 on lago-he-basic-suite-4-1-host0 >> >>> errors: >> >>> grep: denied: No such file or directory >> >>> >> >>> --------------------- >> end captured logging << --------------------- >> >>> >> >>> Stack Trace: >> >>> File "/usr/lib64/python2.7/unittest/case.py", line 369, in run >> >>> testMethod() >> >>> File "/usr/lib/python2.7/site-packages/nose/case.py", line 197, in >> >>> runTest >> >>> self.test(*self.arg) >> >>> File "/usr/lib/python2.7/site-packages/ovirtlago/testlib.py", line >> >>> 129, in wrapped_test >> >>> test() >> >>> File "/usr/lib/python2.7/site-packages/ovirtlago/testlib.py", line >> >>> 59, in wrapper >> >>> return func(get_test_prefix(), *args, **kwargs) >> >>> File >> >>> "/home/jenkins/workspace/ovirt-system-tests_he-basic-suite-4.1/ovirt-system-tests/he-basic-suite-4.1/test-scenarios/002_bootstrap.py", >> >>> line 1036, in he_check_ha_agent >> >>> 'Permission denied' not in result.out >> >>> File "/usr/lib64/python2.7/unittest/case.py", line 462, in >> >>> assertTrue >> >>> raise self.failureException(msg) >> >>> "False is not true\n-------------------- >> begin captured logging << >> >>> --------------------\nlago.ssh: DEBUG: start >> >>> task:cb75d7f3-e333-47f1-b3e9-76ff9c0b4b21:Get ssh client for >> >>> lago-he-basic-suite-4-1-host0:\nlago.ssh: DEBUG: end >> >>> task:cb75d7f3-e333-47f1-b3e9-76ff9c0b4b21:Get ssh client for >> >>> lago-he-basic-suite-4-1-host0:\nlago.ssh: DEBUG: Running 7e67a956 on >> >>> lago-he-basic-suite-4-1-host0: grep Permission denied >> >>> /var/log/ovirt-hosted-engine-ha/agent.log\nlago.ssh: DEBUG: Command 7e67a956 >> >>> on lago-he-basic-suite-4-1-host0 returned with 2\nlago.ssh: DEBUG: Command >> >>> 7e67a956 on lago-he-basic-suite-4-1-host0 output:\n >> >>> /var/log/ovirt-hosted-engine-ha/agent.log:IOError: [Errno 13] Permission >> >>> denied: >> >>> '/var/run/ovirt-hosted-engine-ha/vm.conf'\n/var/log/ovirt-hosted-engine-ha/agent.log:IOError: >> >>> [Errno 13] Permission denied: >> >>> '/var/run/ovirt-hosted-engine-ha/vm.conf'\n/var/log/ovirt-hosted-engine-ha/agent.log:IOError: >> >>> [Errno 13] Permission denied: '/var/run/ovirt-hosted-engine-ha/vm.conf >> '\n/var/log/ovirt-hosted-engine-ha/agent.log:IOError: [Errno 13] >> Permission denied: >> '/var/run/ovirt-hosted-engine-ha/vm.conf'\n/var/log/ovirt-hosted-engine-ha/agent.log:IOError: >> [Errno 13] Permission denied: >> '/var/run/ovirt-hosted-engine-ha/vm.conf'\n/var/log/ovirt-hosted-engine-ha/agent.log:IOError: >> [Errno 13] Permission denied: >> '/var/run/ovirt-hosted-engine-ha/vm.conf'\n/var/log/ovirt-hosted-engine-ha/agent.log:IOError: >> [Errno 13] Permission denied: >> '/var/run/ovirt-hosted-engine-ha/vm.conf'\n/var/log/ovirt-hosted-engine-ha/agent.log:IOError: >> [Errno 13] Permission denied: >> '/var/run/ovirt-hosted-engine-ha/vm.conf'\n/var/log/ovirt-hosted-engine-ha/agent.log:IOError: >> [Errno 13] Permission denied: >> '/var/run/ovirt-hosted-engine-ha/vm.conf'\n\nlago.ssh: DEBUG: Command >> 7e67a956 on lago-he-basic-suite-4-1-host0 errors:\n grep: denied: No such >> file or directory\n\n--------------------- >> end captured logging << >> ---------------------" >> >> >> >> >> >> >> >> -- >> >> Didi >> > >> > >> > >> > -- >> > Didi >> >> >> >> -- >> Didi >> _______________________________________________ >> Infra mailing list >> Infra(a)ovirt.org >> http://lists.ovirt.org/mailman/listinfo/infra -- Didi

2 1

[CQ]: 88777,3 (ovirt-engine) failed "ovirt-master" system tests
by oVirt Jenkins 19 Mar '18

19 Mar '18

Change 88777,3 (ovirt-engine) is probably the reason behind recent system test failures in the "ovirt-master" change queue and needs to be fixed. This change had been removed from the testing queue. Artifacts build from this change will not be released until it is fixed. For further details about the change see: https://gerrit.ovirt.org/#/c/88777/3 For failed test results see: http://jenkins.ovirt.org/job/ovirt-master_change-queue-tester/6429/

1 0

[oVirt Jenkins] ovirt-system-tests_hc-basic-suite-4.2 - Build # 16 - Failure!
by jenkins＠jenkins.phx.ovirt.org 18 Mar '18

18 Mar '18

------=_Part_346_190385630.1521367477675 Content-Type: text/plain; charset=UTF-8 Content-Transfer-Encoding: quoted-printable Project: http://jenkins.ovirt.org/job/ovirt-system-tests_hc-basic-suite-4.2= /=20 Build: http://jenkins.ovirt.org/job/ovirt-system-tests_hc-basic-suite-4.2/1= 6/ Build Number: 16 Build Status: Failure Triggered By: Started by timer ------------------------------------- Changes Since Last Success: ------------------------------------- Changes for Build #16 [Martin Sivak] Cleanup of test code - next() replaced with any() ----------------- Failed Tests: ----------------- 1 tests failed. FAILED: 002_bootstrap.add_hosts Error Message: The response content type 'text/html; charset=3Diso-8859-1' isn't the expec= ted XML -------------------- >> begin captured logging << -------------------- ovirtlago.testlib: ERROR: * Unhandled exception in <function _host_is_u= p_4 at 0x3564c08> Traceback (most recent call last): File "/usr/lib/python2.7/site-packages/ovirtlago/testlib.py", line 219, i= n assert_equals_within res =3D func() File "/home/jenkins/workspace/ovirt-system-tests_hc-basic-suite-4.2/ovirt= -system-tests/hc-basic-suite-4.2/test-scenarios/002_bootstrap.py", line 254= , in _host_is_up_4 host_obj =3D host_service.get() File "/usr/lib64/python2.7/site-packages/ovirtsdk4/services.py", line 362= 19, in get return self._internal_get(headers, query, wait) File "/usr/lib64/python2.7/site-packages/ovirtsdk4/service.py", line 211,= in _internal_get return future.wait() if wait else future File "/usr/lib64/python2.7/site-packages/ovirtsdk4/service.py", line 55, = in wait return self._code(response) File "/usr/lib64/python2.7/site-packages/ovirtsdk4/service.py", line 208,= in callback self._check_fault(response) File "/usr/lib64/python2.7/site-packages/ovirtsdk4/service.py", line 130,= in _check_fault body =3D self._internal_read_body(response) File "/usr/lib64/python2.7/site-packages/ovirtsdk4/service.py", line 316,= in _internal_read_body self._connection.check_xml_content_type(response) File "/usr/lib64/python2.7/site-packages/ovirtsdk4/__init__.py", line 849= , in check_xml_content_type response.headers File "/usr/lib64/python2.7/site-packages/ovirtsdk4/__init__.py", line 889= , in _check_content_type raise Error(msg) Error: The response content type 'text/html; charset=3Diso-8859-1' isn't th= e expected XML --------------------- >> end captured logging << --------------------- Stack Trace: File "/usr/lib64/python2.7/unittest/case.py", line 369, in run testMethod() File "/usr/lib/python2.7/site-packages/nose/case.py", line 197, in runTes= t self.test(*self.arg) File "/usr/lib/python2.7/site-packages/ovirtlago/testlib.py", line 129, i= n wrapped_test test() File "/usr/lib/python2.7/site-packages/ovirtlago/testlib.py", line 59, in= wrapper return func(get_test_prefix(), *args, **kwargs) File "/home/jenkins/workspace/ovirt-system-tests_hc-basic-suite-4.2/ovirt= -system-tests/hc-basic-suite-4.2/test-scenarios/002_bootstrap.py", line 229= , in add_hosts add_hosts_4(prefix) File "/home/jenkins/workspace/ovirt-system-tests_hc-basic-suite-4.2/ovirt= -system-tests/hc-basic-suite-4.2/test-scenarios/002_bootstrap.py", line 273= , in add_hosts_4 testlib.assert_true_within(_host_is_up_4, timeout=3D15*60) File "/usr/lib/python2.7/site-packages/ovirtlago/testlib.py", line 263, i= n assert_true_within assert_equals_within(func, True, timeout, allowed_exceptions) File "/usr/lib/python2.7/site-packages/ovirtlago/testlib.py", line 219, i= n assert_equals_within res =3D func() File "/home/jenkins/workspace/ovirt-system-tests_hc-basic-suite-4.2/ovirt= -system-tests/hc-basic-suite-4.2/test-scenarios/002_bootstrap.py", line 254= , in _host_is_up_4 host_obj =3D host_service.get() File "/usr/lib64/python2.7/site-packages/ovirtsdk4/services.py", line 362= 19, in get return self._internal_get(headers, query, wait) File "/usr/lib64/python2.7/site-packages/ovirtsdk4/service.py", line 211,= in _internal_get return future.wait() if wait else future File "/usr/lib64/python2.7/site-packages/ovirtsdk4/service.py", line 55, = in wait return self._code(response) File "/usr/lib64/python2.7/site-packages/ovirtsdk4/service.py", line 208,= in callback self._check_fault(response) File "/usr/lib64/python2.7/site-packages/ovirtsdk4/service.py", line 130,= in _check_fault body =3D self._internal_read_body(response) File "/usr/lib64/python2.7/site-packages/ovirtsdk4/service.py", line 316,= in _internal_read_body self._connection.check_xml_content_type(response) File "/usr/lib64/python2.7/site-packages/ovirtsdk4/__init__.py", line 849= , in check_xml_content_type response.headers File "/usr/lib64/python2.7/site-packages/ovirtsdk4/__init__.py", line 889= , in _check_content_type raise Error(msg) 'The response content type \'text/html; charset=3Diso-8859-1\' isn\'t the e= xpected XML\n-------------------- >> begin captured logging << ------------= --------\novirtlago.testlib: ERROR: * Unhandled exception in <function = _host_is_up_4 at 0x3564c08>\nTraceback (most recent call last):\n File "/u= sr/lib/python2.7/site-packages/ovirtlago/testlib.py", line 219, in assert_e= quals_within\n res =3D func()\n File "/home/jenkins/workspace/ovirt-sys= tem-tests_hc-basic-suite-4.2/ovirt-system-tests/hc-basic-suite-4.2/test-sce= narios/002_bootstrap.py", line 254, in _host_is_up_4\n host_obj =3D host= _service.get()\n File "/usr/lib64/python2.7/site-packages/ovirtsdk4/servic= es.py", line 36219, in get\n return self._internal_get(headers, query, w= ait)\n File "/usr/lib64/python2.7/site-packages/ovirtsdk4/service.py", lin= e 211, in _internal_get\n return future.wait() if wait else future\n Fi= le "/usr/lib64/python2.7/site-packages/ovirtsdk4/service.py", line 55, in w= ait\n return self._code(response)\n File "/usr/lib64/python2.7/site-pac= kages/ovirtsdk4/service.py", line 208, in callback\n self._check_fault(r= esponse)\n File "/usr/lib64/python2.7/site-packages/ovirtsdk4/service.py",= line 130, in _check_fault\n body =3D self._internal_read_body(response)= \n File "/usr/lib64/python2.7/site-packages/ovirtsdk4/service.py", line 31= 6, in _internal_read_body\n self._connection.check_xml_content_type(resp= onse)\n File "/usr/lib64/python2.7/site-packages/ovirtsdk4/__init__.py", l= ine 849, in check_xml_content_type\n response.headers\n File "/usr/lib6= 4/python2.7/site-packages/ovirtsdk4/__init__.py", line 889, in _check_conte= nt_type\n raise Error(msg)\nError: The response content type \'text/html= ; charset=3Diso-8859-1\' isn\'t the expected XML\n--------------------- >> = end captured logging << ---------------------' ------=_Part_346_190385630.1521367477675--

1 1

[JIRA] (OVIRT-1936) (nested)&&fc27 label has no online slaves
by Evgheni Dereveanchin (oVirt JIRA) 18 Mar '18

18 Mar '18

This is a multi-part message in MIME format... ------------=_1521202838-11894-197 Content-Type: text/plain; charset=UTF-8 Content-Transfer-Encoding: 7bit [ https://ovirt-jira.atlassian.net/browse/OVIRT-1936?page=com.atlassian.jira.… ] Evgheni Dereveanchin reassigned OVIRT-1936: ------------------------------------------- Assignee: Evgheni Dereveanchin (was: infra) > (nested)&&fc27 label has no online slaves > ----------------------------------------- > > Key: OVIRT-1936 > URL: https://ovirt-jira.atlassian.net/browse/OVIRT-1936 > Project: oVirt - virtualization made easy > Issue Type: By-EMAIL > Reporter: sbonazzo > Assignee: Evgheni Dereveanchin > -- This message was sent by Atlassian Jira (v1001.0.0-SNAPSHOT#100082) ------------=_1521202838-11894-197 Content-Type: text/html; charset="UTF-8" Content-Disposition: inline Content-Transfer-Encoding: 7bit <html><body> <pre>[ https://ovirt-jira.atlassian.net/browse/OVIRT-1936?page=com.atlassian.jira.… ]</pre> <h3>Evgheni Dereveanchin reassigned OVIRT-1936:</h3> <pre>Assignee: Evgheni Dereveanchin (was: infra)</pre> <blockquote><h3>(nested)&&fc27 label has no online slaves</h3> <pre> Key: OVIRT-1936 URL: https://ovirt-jira.atlassian.net/browse/OVIRT-1936 Project: oVirt - virtualization made easy Issue Type: By-EMAIL Reporter: sbonazzo Assignee: Evgheni Dereveanchin</pre></blockquote> <p>— This message was sent by Atlassian Jira (v1001.0.0-SNAPSHOT#100082)</p> <img src="https://u4043402.ct.sendgrid.net/wf/open?upn=i5TMWGV99amJbNxJpSp2-2BJ33BSM3…" alt="" width="1" height="1" border="0" style="height:1px !important;width:1px !important;border-width:0 !important;margin-top:0 !important;margin-bottom:0 !important;margin-right:0 !important;margin-left:0 !important;padding-top:0 !important;padding-bottom:0 !important;padding-right:0 !important;padding-left:0 !important;"/> </body></html> ------------=_1521202838-11894-197--

2 1

OST Network suite is failing on "OSError: [Errno 28] No space left on device"
by Edward Haas 18 Mar '18

18 Mar '18

Good morning, The network suite is running Ansible in one of its tests and from the weekend it started failing without a good explanation. It raises "OSError: [Errno 28] No space left on device" when trying to take lock on the python multiprocess module. It smells like a problem on the slave, but I am not sure. Any ideas? Thanks, Edy. =================================== FAILURES =================================== ______________________ test_ovn_provider_create_scenario _______________________ os_client_config = None def test_ovn_provider_create_scenario(os_client_config): > _test_ovn_provider('create_scenario.yml') network-suite-master/tests/test_ovn_provider.py:68: _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ network-suite-master/tests/test_ovn_provider.py:78: in _test_ovn_provider playbook.run() network-suite-master/lib/ansiblelib.py:127: in run self._run_playbook_executor() network-suite-master/lib/ansiblelib.py:138: in _run_playbook_executor pbex = PlaybookExecutor(**self._pbex_args) /usr/lib/python2.7/site-packages/ansible/executor/playbook_executor.py:60: in __init__ self._tqm = TaskQueueManager(inventory=inventory, variable_manager=variable_manager, loader=loader, options=options, passwords=self.passwords) /usr/lib/python2.7/site-packages/ansible/executor/task_queue_manager.py:104: in __init__ self._final_q = multiprocessing.Queue() /usr/lib64/python2.7/multiprocessing/__init__.py:218: in Queue return Queue(maxsize) /usr/lib64/python2.7/multiprocessing/queues.py:63: in __init__ self._rlock = Lock() /usr/lib64/python2.7/multiprocessing/synchronize.py:147: in __init__ SemLock.__init__(self, SEMAPHORE, 1, 1) _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ self = <Lock(owner=unknown)>, kind = 1, value = 1, maxvalue = 1 def __init__(self, kind, value, maxvalue): > sl = self._semlock = _multiprocessing.SemLock(kind, value, maxvalue) E OSError: [Errno 28] No space left on device /usr/lib64/python2.7/multiprocessing/synchronize.py:75: OSError

1 0

[oVirt Jenkins] ovirt-system-tests_hc-basic-suite-4.2 - Build # 13 - Failure!
by jenkins＠jenkins.phx.ovirt.org 17 Mar '18

17 Mar '18

------=_Part_316_1234843677.1521279432434 Content-Type: text/plain; charset=UTF-8 Content-Transfer-Encoding: 7bit Project: http://jenkins.ovirt.org/job/ovirt-system-tests_hc-basic-suite-4.2/ Build: http://jenkins.ovirt.org/job/ovirt-system-tests_hc-basic-suite-4.2/13/ Build Number: 13 Build Status: Failure Triggered By: Started by timer ------------------------------------- Changes Since Last Success: ------------------------------------- Changes for Build #13 [Martin Sivak] Cleanup of test code - next() replaced with any() ----------------- Failed Tests: ----------------- All tests passed ------=_Part_316_1234843677.1521279432434--

1 1

[oVirt Jenkins] ovirt-system-tests_he-basic-ansible-suite-master - Build # 78 - Failure!
by jenkins＠jenkins.phx.ovirt.org 17 Mar '18

17 Mar '18

------=_Part_284_2084206695.1521238462287 Content-Type: text/plain; charset=UTF-8 Content-Transfer-Encoding: 7bit Project: http://jenkins.ovirt.org/job/ovirt-system-tests_he-basic-ansible-suite-mast… Build: http://jenkins.ovirt.org/job/ovirt-system-tests_he-basic-ansible-suite-mast… Build Number: 78 Build Status: Failure Triggered By: Started by timer ------------------------------------- Changes Since Last Success: ------------------------------------- Changes for Build #78 [Martin Sivak] Cleanup of test code - next() replaced with any() ----------------- Failed Tests: ----------------- No tests ran. ------=_Part_284_2084206695.1521238462287--

1 4

[oVirt Jenkins] ovirt-system-tests_hc-basic-suite-master - Build # 225 - Failure!
by jenkins＠jenkins.phx.ovirt.org 17 Mar '18

17 Mar '18

------=_Part_263_1135888245.1521212507833 Content-Type: text/plain; charset=UTF-8 Content-Transfer-Encoding: quoted-printable Project: http://jenkins.ovirt.org/job/ovirt-system-tests_hc-basic-suite-mas= ter/=20 Build: http://jenkins.ovirt.org/job/ovirt-system-tests_hc-basic-suite-maste= r/225/ Build Number: 225 Build Status: Failure Triggered By: Started by timer ------------------------------------- Changes Since Last Success: ------------------------------------- Changes for Build #225 [Yedidyah Bar David] HE: Test hosted-engine --get-shared-config ----------------- Failed Tests: ----------------- 1 tests failed. FAILED: 002_bootstrap.add_hosts Error Message: The response content type 'text/html; charset=3Diso-8859-1' isn't the expec= ted XML -------------------- >> begin captured logging << -------------------- ovirtlago.testlib: ERROR: * Unhandled exception in <function _host_is_u= p_4 at 0x3433230> Traceback (most recent call last): File "/usr/lib/python2.7/site-packages/ovirtlago/testlib.py", line 219, i= n assert_equals_within res =3D func() File "/home/jenkins/workspace/ovirt-system-tests_hc-basic-suite-master/ov= irt-system-tests/hc-basic-suite-master/test-scenarios/002_bootstrap.py", li= ne 255, in _host_is_up_4 host_obj =3D host_service.get() File "/usr/lib64/python2.7/site-packages/ovirtsdk4/services.py", line 363= 46, in get return self._internal_get(headers, query, wait) File "/usr/lib64/python2.7/site-packages/ovirtsdk4/service.py", line 211,= in _internal_get return future.wait() if wait else future File "/usr/lib64/python2.7/site-packages/ovirtsdk4/service.py", line 55, = in wait return self._code(response) File "/usr/lib64/python2.7/site-packages/ovirtsdk4/service.py", line 208,= in callback self._check_fault(response) File "/usr/lib64/python2.7/site-packages/ovirtsdk4/service.py", line 130,= in _check_fault body =3D self._internal_read_body(response) File "/usr/lib64/python2.7/site-packages/ovirtsdk4/service.py", line 316,= in _internal_read_body self._connection.check_xml_content_type(response) File "/usr/lib64/python2.7/site-packages/ovirtsdk4/__init__.py", line 849= , in check_xml_content_type response.headers File "/usr/lib64/python2.7/site-packages/ovirtsdk4/__init__.py", line 889= , in _check_content_type raise Error(msg) Error: The response content type 'text/html; charset=3Diso-8859-1' isn't th= e expected XML --------------------- >> end captured logging << --------------------- Stack Trace: File "/usr/lib64/python2.7/unittest/case.py", line 369, in run testMethod() File "/usr/lib/python2.7/site-packages/nose/case.py", line 197, in runTes= t self.test(*self.arg) File "/usr/lib/python2.7/site-packages/ovirtlago/testlib.py", line 129, i= n wrapped_test test() File "/usr/lib/python2.7/site-packages/ovirtlago/testlib.py", line 59, in= wrapper return func(get_test_prefix(), *args, **kwargs) File "/home/jenkins/workspace/ovirt-system-tests_hc-basic-suite-master/ov= irt-system-tests/hc-basic-suite-master/test-scenarios/002_bootstrap.py", li= ne 230, in add_hosts add_hosts_4(prefix) File "/home/jenkins/workspace/ovirt-system-tests_hc-basic-suite-master/ov= irt-system-tests/hc-basic-suite-master/test-scenarios/002_bootstrap.py", li= ne 274, in add_hosts_4 testlib.assert_true_within(_host_is_up_4, timeout=3D15*60) File "/usr/lib/python2.7/site-packages/ovirtlago/testlib.py", line 263, i= n assert_true_within assert_equals_within(func, True, timeout, allowed_exceptions) File "/usr/lib/python2.7/site-packages/ovirtlago/testlib.py", line 219, i= n assert_equals_within res =3D func() File "/home/jenkins/workspace/ovirt-system-tests_hc-basic-suite-master/ov= irt-system-tests/hc-basic-suite-master/test-scenarios/002_bootstrap.py", li= ne 255, in _host_is_up_4 host_obj =3D host_service.get() File "/usr/lib64/python2.7/site-packages/ovirtsdk4/services.py", line 363= 46, in get return self._internal_get(headers, query, wait) File "/usr/lib64/python2.7/site-packages/ovirtsdk4/service.py", line 211,= in _internal_get return future.wait() if wait else future File "/usr/lib64/python2.7/site-packages/ovirtsdk4/service.py", line 55, = in wait return self._code(response) File "/usr/lib64/python2.7/site-packages/ovirtsdk4/service.py", line 208,= in callback self._check_fault(response) File "/usr/lib64/python2.7/site-packages/ovirtsdk4/service.py", line 130,= in _check_fault body =3D self._internal_read_body(response) File "/usr/lib64/python2.7/site-packages/ovirtsdk4/service.py", line 316,= in _internal_read_body self._connection.check_xml_content_type(response) File "/usr/lib64/python2.7/site-packages/ovirtsdk4/__init__.py", line 849= , in check_xml_content_type response.headers File "/usr/lib64/python2.7/site-packages/ovirtsdk4/__init__.py", line 889= , in _check_content_type raise Error(msg) 'The response content type \'text/html; charset=3Diso-8859-1\' isn\'t the e= xpected XML\n-------------------- >> begin captured logging << ------------= --------\novirtlago.testlib: ERROR: * Unhandled exception in <function = _host_is_up_4 at 0x3433230>\nTraceback (most recent call last):\n File "/u= sr/lib/python2.7/site-packages/ovirtlago/testlib.py", line 219, in assert_e= quals_within\n res =3D func()\n File "/home/jenkins/workspace/ovirt-sys= tem-tests_hc-basic-suite-master/ovirt-system-tests/hc-basic-suite-master/te= st-scenarios/002_bootstrap.py", line 255, in _host_is_up_4\n host_obj = =3D host_service.get()\n File "/usr/lib64/python2.7/site-packages/ovirtsdk= 4/services.py", line 36346, in get\n return self._internal_get(headers, = query, wait)\n File "/usr/lib64/python2.7/site-packages/ovirtsdk4/service.= py", line 211, in _internal_get\n return future.wait() if wait else futu= re\n File "/usr/lib64/python2.7/site-packages/ovirtsdk4/service.py", line = 55, in wait\n return self._code(response)\n File "/usr/lib64/python2.7/= site-packages/ovirtsdk4/service.py", line 208, in callback\n self._check= _fault(response)\n File "/usr/lib64/python2.7/site-packages/ovirtsdk4/serv= ice.py", line 130, in _check_fault\n body =3D self._internal_read_body(r= esponse)\n File "/usr/lib64/python2.7/site-packages/ovirtsdk4/service.py",= line 316, in _internal_read_body\n self._connection.check_xml_content_t= ype(response)\n File "/usr/lib64/python2.7/site-packages/ovirtsdk4/__init_= _.py", line 849, in check_xml_content_type\n response.headers\n File "/= usr/lib64/python2.7/site-packages/ovirtsdk4/__init__.py", line 889, in _che= ck_content_type\n raise Error(msg)\nError: The response content type \'t= ext/html; charset=3Diso-8859-1\' isn\'t the expected XML\n-----------------= ---- >> end captured logging << ---------------------' ------=_Part_263_1135888245.1521212507833--

1 2

[oVirt Jenkins] ovirt-system-tests_he-basic-suite-master - Build # 237 - Failure!
by jenkins＠jenkins.phx.ovirt.org 17 Mar '18

17 Mar '18

------=_Part_278_393064355.1521234372792 Content-Type: text/plain; charset=UTF-8 Content-Transfer-Encoding: 7bit Project: http://jenkins.ovirt.org/job/ovirt-system-tests_he-basic-suite-master/ Build: http://jenkins.ovirt.org/job/ovirt-system-tests_he-basic-suite-master/237/ Build Number: 237 Build Status: Failure Triggered By: Started by timer ------------------------------------- Changes Since Last Success: ------------------------------------- Changes for Build #237 [Martin Sivak] Cleanup of test code - next() replaced with any() ----------------- Failed Tests: ----------------- No tests ran. ------=_Part_278_393064355.1521234372792--

1 4

[oVirt Jenkins] ovirt-system-tests_he-basic-iscsi-suite-master - Build # 1 - Failure!
by jenkins＠jenkins.phx.ovirt.org 16 Mar '18

16 Mar '18

------=_Part_265_946388335.1521215444463 Content-Type: text/plain; charset=UTF-8 Content-Transfer-Encoding: 7bit Project: http://jenkins.ovirt.org/job/ovirt-system-tests_he-basic-iscsi-suite-master/ Build: http://jenkins.ovirt.org/job/ovirt-system-tests_he-basic-iscsi-suite-master… Build Number: 1 Build Status: Failure Triggered By: Started by user Sandro Bonazzola ------------------------------------- Changes Since Last Success: ------------------------------------- ----------------- Failed Tests: ----------------- No tests ran. ------=_Part_265_946388335.1521215444463--

1 3

OST Failure - Weekly update [09/03/2018-16/03/2018]
by Dafna Ron 16 Mar '18

16 Mar '18

Hello, I would like to update on this week's failures and OST current status. We had a very eventful week with 2 regressions identified by OST 1. On 11-03-2018 the CI team reported a regression in project ovirt-engine. The change reported was https://gerrit.ovirt.org/#/c/88738/ - db: add func to turn table columns to empty string. This caused a failure in both master and 4.2 in OST but was also identified to be an issue in 4.1. The fix for all 3 versions can be tracked here: https://bugzilla.redhat.com/show_bug.cgi?id=1554377 and was submitted on 15-03-2018. I hope that in future we would resolve such regression much faster as it took us 5 days to resolve this issue but I would also like to thank all that helped to resolve it once the issue was investigated more deeply. 2. on 13-03-2018 we found a regression in otopi: *core: Use python3 when possible- https://gerrit.ovirt.org/#/c/87276/ <https://gerrit.ovirt.org/#/c/87276/>* *The fix was* submitted *on the same day **https://gerrit.ovirt.org/88931 <https://gerrit.ovirt.org/88931> - **spec: Package otopi-functions* We also had 1 configuration related issue which was fixed by the CI team and we had failed build-artifacts, mostly in fc27. this was due to the change in fedora to not allow the use of yum and had to be solved by the projects making minor patch changes. *Below you can see the chart for this week's resolved issues but cause of failure:*Code = regression of working components/functionalities Configurations = package related issues Other = failed build artifacts Infra = infrastructure/OST/Lago related issues *Below is a chart of resolved failures based on ovirt version* *Below is a chart showing failures by suite type: * Thank you, Dafna

1 0

[CQ]: 89114, 1 (ovirt-engine-nodejs-modules) failed "ovirt-4.2" system tests
by oVirt Jenkins 16 Mar '18

16 Mar '18

Change 89114,1 (ovirt-engine-nodejs-modules) is probably the reason behind recent system test failures in the "ovirt-4.2" change queue and needs to be fixed. This change had been removed from the testing queue. Artifacts build from this change will not be released until it is fixed. For further details about the change see: https://gerrit.ovirt.org/#/c/89114/1 For failed test results see: http://jenkins.ovirt.org/job/ovirt-4.2_change-queue-tester/1346/

1 0

[CQ]: 89114, 1 (ovirt-engine-nodejs-modules) failed "ovirt-4.1" system tests
by oVirt Jenkins 16 Mar '18

16 Mar '18

Change 89114,1 (ovirt-engine-nodejs-modules) is probably the reason behind recent system test failures in the "ovirt-4.1" change queue and needs to be fixed. This change had been removed from the testing queue. Artifacts build from this change will not be released until it is fixed. For further details about the change see: https://gerrit.ovirt.org/#/c/89114/1 For failed test results see: http://jenkins.ovirt.org/job/ovirt-4.1_change-queue-tester/1715/

1 0

[CQ]: 89114, 1 (ovirt-engine-nodejs-modules) failed "ovirt-4.2" system tests
by oVirt Jenkins 16 Mar '18

16 Mar '18

Change 89114,1 (ovirt-engine-nodejs-modules) is probably the reason behind recent system test failures in the "ovirt-4.2" change queue and needs to be fixed. This change had been removed from the testing queue. Artifacts build from this change will not be released until it is fixed. For further details about the change see: https://gerrit.ovirt.org/#/c/89114/1 For failed test results see: http://jenkins.ovirt.org/job/ovirt-4.2_change-queue-tester/1344/

1 0

[CQ]: 89114, 1 (ovirt-engine-nodejs-modules) failed "ovirt-4.1" system tests
by oVirt Jenkins 16 Mar '18

16 Mar '18

Change 89114,1 (ovirt-engine-nodejs-modules) is probably the reason behind recent system test failures in the "ovirt-4.1" change queue and needs to be fixed. This change had been removed from the testing queue. Artifacts build from this change will not be released until it is fixed. For further details about the change see: https://gerrit.ovirt.org/#/c/89114/1 For failed test results see: http://jenkins.ovirt.org/job/ovirt-4.1_change-queue-tester/1713/

1 0

[CQ]: 89114, 1 (ovirt-engine-nodejs-modules) failed "ovirt-master" system tests, but isn't the failure root cause
by oVirt Jenkins 16 Mar '18

16 Mar '18

A system test invoked by the "ovirt-master" change queue including change 89114,1 (ovirt-engine-nodejs-modules) failed. However, this change seems not to be the root cause for this failure. Change 89113,2 (ovirt-engine-nodejs- modules) that this change depends on or is based on, was detected as the cause of the testing failures. This change had been removed from the testing queue. Artifacts built from this change will not be released until either change 89113,2 (ovirt-engine-nodejs- modules) is fixed and this change is updated to refer to or rebased on the fixed version, or this change is modified to no longer depend on it. For further details about the change see: https://gerrit.ovirt.org/#/c/89114/1 For further details about the change that seems to be the root cause behind the testing failures see: https://gerrit.ovirt.org/#/c/89113/2 For failed test results see: http://jenkins.ovirt.org/job/ovirt-master_change-queue-tester/6403/

1 0

[CQ]: 89113, 2 (ovirt-engine-nodejs-modules) failed "ovirt-master" system tests
by oVirt Jenkins 16 Mar '18

16 Mar '18

Change 89113,2 (ovirt-engine-nodejs-modules) is probably the reason behind recent system test failures in the "ovirt-master" change queue and needs to be fixed. This change had been removed from the testing queue. Artifacts build from this change will not be released until it is fixed. For further details about the change see: https://gerrit.ovirt.org/#/c/89113/2 For failed test results see: http://jenkins.ovirt.org/job/ovirt-master_change-queue-tester/6401/

1 0

[oVirt Jenkins] ovirt-system-tests_hc-basic-suite-master - Build # 223 - Failure!
by jenkins＠jenkins.phx.ovirt.org 16 Mar '18

16 Mar '18

------=_Part_248_224867011.1521170625791 Content-Type: text/plain; charset=UTF-8 Content-Transfer-Encoding: 7bit Project: http://jenkins.ovirt.org/job/ovirt-system-tests_hc-basic-suite-master/ Build: http://jenkins.ovirt.org/job/ovirt-system-tests_hc-basic-suite-master/223/ Build Number: 223 Build Status: Failure Triggered By: Started by timer ------------------------------------- Changes Since Last Success: ------------------------------------- Changes for Build #223 [Michal Skrivanek] master: add USB to the sanity VM [Sandro Bonazzola] mirrors-reposync: add qemu-ev testing repo [Daniel Belenky] Modify mergeoptions in parser utils to merge runif ----------------- Failed Tests: ----------------- 1 tests failed. FAILED: 002_bootstrap.add_hosts Error Message: None != True after 900 seconds Stack Trace: Traceback (most recent call last): File "/usr/lib64/python2.7/unittest/case.py", line 369, in run testMethod() File "/usr/lib/python2.7/site-packages/nose/case.py", line 197, in runTest self.test(*self.arg) File "/usr/lib/python2.7/site-packages/ovirtlago/testlib.py", line 129, in wrapped_test test() File "/usr/lib/python2.7/site-packages/ovirtlago/testlib.py", line 59, in wrapper return func(get_test_prefix(), *args, **kwargs) File "/home/jenkins/workspace/ovirt-system-tests_hc-basic-suite-master/ovirt-system-tests/hc-basic-suite-master/test-scenarios/002_bootstrap.py", line 230, in add_hosts add_hosts_4(prefix) File "/home/jenkins/workspace/ovirt-system-tests_hc-basic-suite-master/ovirt-system-tests/hc-basic-suite-master/test-scenarios/002_bootstrap.py", line 274, in add_hosts_4 testlib.assert_true_within(_host_is_up_4, timeout=15*60) File "/usr/lib/python2.7/site-packages/ovirtlago/testlib.py", line 263, in assert_true_within assert_equals_within(func, True, timeout, allowed_exceptions) File "/usr/lib/python2.7/site-packages/ovirtlago/testlib.py", line 237, in assert_equals_within '%s != %s after %s seconds' % (res, value, timeout) AssertionError: None != True after 900 seconds ------=_Part_248_224867011.1521170625791--

1 1

[ OST Failure Report ] [ oVirt Master (cockpit-ovirt) ] [ 15-03-2018 ] [ 098_ovirt_provider_ovn.use_ovn_provider ]
by Dafna Ron 16 Mar '18

16 Mar '18

Hi, We have a failure on master for test 098_ovirt_provider_ovn.use_ovn_provider in project cockpit-ovirt. This seems to be a race because object is locked. also, the actual failure is logged as WARN and not ERROR. I don't think the patch is actually related to the failure but I think the test should be fixed. can you please review to make sure we do not have an actual regression and let me know if we need to open a bz to fix the test? *Link and headline of suspected patches: * *https://gerrit.ovirt.org/#/c/89020/2 <https://gerrit.ovirt.org/#/c/89020/2> - * *wizard: Enable scroll on start page for low-res screensLink to Job:http://jenkins.ovirt.org/job/ovirt-master_change-queue-tester/6374 <http://jenkins.ovirt.org/job/ovirt-master_change-queue-tester/6374>Link to all logs:http://jenkins.ovirt.org/job/ovirt-master_change-queue-tester/6374/art… <http://jenkins.ovirt.org/job/ovirt-master_change-queue-tester/6374/artifacts>(Relevant) error snippet from the log: <error>2018-03-15 10:05:00,160-04 DEBUG [org.ovirt.engine.core.sso.servlets.OAuthTokenInfoServlet] (default task-10) [] Sending json response2018-03-15 10:05:00,160-04 DEBUG [org.ovirt.engine.core.sso.utils.TokenCleanupUtility] (default task-10) [] Not cleaning up expired tokens2018-03-15 10:05:00,169-04 INFO [org.ovirt.engine.core.bll.provider.network.SyncNetworkProviderCommand] (EE-ManagedThreadFactory-engineScheduled-Thread-90) [789edb23] Lock Acquired to object 'EngineLock:{exclusiveLocks='[c38a67ec-0b48-4e6f-be85-70c700df5483=PROVIDER]', sharedLocks=''}'2018-03-15 10:05:00,184-04 INFO [org.ovirt.engine.core.bll.provider.network.SyncNetworkProviderCommand] (EE-ManagedThreadFactory-engineScheduled-Thread-90) [789edb23] Running command: SyncNetworkProviderCommand internal: true.2018-03-15 10:05:00,228-04 DEBUG [org.ovirt.engine.core.dal.dbbroker.PostgresDbEngineDialect$PostgresSimpleJdbcCall] (default task-13) [e1328379-17b7-49f8-beb2-cf8331784828] Compiled stored procedure. Call string is [{call getdcidbyexternalnetworkid(?)}]2018-03-15 10:05:00,228-04 DEBUG [org.ovirt.engine.core.dal.dbbroker.PostgresDbEngineDialect$PostgresSimpleJdbcCall] (default task-13) [e1328379-17b7-49f8-beb2-cf8331784828] SqlCall for procedure [GetDcIdByExternalNetworkId] compiled2018-03-15 10:05:00,229-04 DEBUG [org.ovirt.engine.core.common.di.interceptor.DebugLoggingInterceptor] (default task-13) [e1328379-17b7-49f8-beb2-cf8331784828] method: runQuery, params: [GetAllExternalNetworksOnProvider, IdQueryParameters:{refresh='false', filtered='false'}], timeElapsed: 353ms2018-03-15 10:05:00,239-04 INFO [org.ovirt.engine.core.bll.network.dc.AddNetworkCommand] (default task-13) [e1328379-17b7-49f8-beb2-cf8331784828] Failed to Acquire Lock to object 'EngineLock:{exclusiveLocks='[network_1=NETWORK, c38a67ec-0b48-4e6f-be85-70c700df5483=PROVIDER]', sharedLocks=''}'2018-03-15 10:05:00,239-04 WARN [org.ovirt.engine.core.bll.network.dc.AddNetworkCommand] (default task-13) [e1328379-17b7-49f8-beb2-cf8331784828] Validation of action 'AddNetwork' failed for user admin@internal-authz. Reasons: VAR__TYPE__NETWORK,VAR__ACTION__ADD,ACTION_TYPE_FAILED_PROVIDER_LOCKED,$providerId c38a67ec-0b48-4e6f-be85-70c700df54832018-03-15 10:05:00,240-04 DEBUG [org.ovirt.engine.core.common.di.interceptor.DebugLoggingInterceptor] (default task-13) [e1328379-17b7-49f8-beb2-cf8331784828] method: runAction, params: [AddNetwork, AddNetworkStoragePoolParameters:{commandId='61b365ec-27c1-49af-ad72-f907df8befcd', user='null', commandType='Unknown'}], timeElapsed: 10ms2018-03-15 10:05:00,250-04 ERROR [org.ovirt.engine.api.restapi.resource.AbstractBackendResource] (default task-13) [] Operation Failed: [Cannot add Network. Related operation on provider with the id c38a67ec-0b48-4e6f-be85-70c700df5483 is currently in progress. Please try again later.]2018-03-15 10:05:00,254-04 DEBUG [org.ovirt.engine.core.utils.servlet.LocaleFilter] (default task-14) [] Incoming locale 'en-US'. Filter determined locale to be 'en-US'2018-03-15 10:05:00,254-04 DEBUG [org.ovirt.engine.core.sso.servlets.OAuthTokenServlet] (default task-14) [] Entered OAuthTokenServlet Query String: null, Parameters : password = ***, grant_type = password, scope = ovirt-app-api ovirt-ext=token-info:validate, username = admin@internal, </error>*

2 2

Build failed in Jenkins: system-sync_mirrors-centos-updates-el7-x86_64 #1373
by jenkins＠jenkins.phx.ovirt.org 16 Mar '18

16 Mar '18

See <http://jenkins.ovirt.org/job/system-sync_mirrors-centos-updates-el7-x86_64/…> Changes: [Eyal Edri] Run all nightly suites 3 times a day [Sandro Bonazzola] system-tests: add he-basic-iscsi-suite-master ------------------------------------------ Started by timer [EnvInject] - Loading node environment variables. Building remotely on mirrors.phx.ovirt.org (mirrors) in workspace <http://jenkins.ovirt.org/job/system-sync_mirrors-centos-updates-el7-x86_64/…> > git rev-parse --is-inside-work-tree # timeout=10 Fetching changes from the remote Git repository > git config remote.origin.url http://gerrit.ovirt.org/jenkins.git # timeout=10 Cleaning workspace > git rev-parse --verify HEAD # timeout=10 Resetting working tree > git reset --hard # timeout=10 > git clean -fdx # timeout=10 Pruning obsolete local branches Fetching upstream changes from http://gerrit.ovirt.org/jenkins.git > git --version # timeout=10 > git fetch --tags --progress http://gerrit.ovirt.org/jenkins.git +refs/heads/*:refs/remotes/origin/* --prune > git rev-parse origin/master^{commit} # timeout=10 Checking out Revision f395591f18d0a3b6ae3ff8daaafd93321b7d61f3 (origin/master) > git config core.sparsecheckout # timeout=10 > git checkout -f f395591f18d0a3b6ae3ff8daaafd93321b7d61f3 Commit message: "system-tests: add he-basic-iscsi-suite-master" > git rev-list --no-walk 26142503ea0774cd18e740630cd68c7ab86d9f83 # timeout=10 [system-sync_mirrors-centos-updates-el7-x86_64] $ /bin/bash -xe /tmp/jenkins4149197632192240639.sh + jenkins/scripts/mirror_mgr.sh resync_yum_mirror centos-updates-el7 x86_64 jenkins/data/mirrors-reposync.conf Checking if mirror needs a resync Traceback (most recent call last): File "/usr/bin/reposync", line 343, in <module> main() File "/usr/bin/reposync", line 175, in main my.doRepoSetup() File "/usr/lib/python2.7/site-packages/yum/__init__.py", line 681, in doRepoSetup return self._getRepos(thisrepo, True) File "/usr/lib/python2.7/site-packages/yum/__init__.py", line 721, in _getRepos self._repos.doSetup(thisrepo) File "/usr/lib/python2.7/site-packages/yum/repos.py", line 157, in doSetup self.retrieveAllMD() File "/usr/lib/python2.7/site-packages/yum/repos.py", line 88, in retrieveAllMD dl = repo._async and repo._commonLoadRepoXML(repo) File "/usr/lib/python2.7/site-packages/yum/yumRepo.py", line 1474, in _commonLoadRepoXML if self._latestRepoXML(local): File "/usr/lib/python2.7/site-packages/yum/yumRepo.py", line 1443, in _latestRepoXML oxml = self._saveOldRepoXML(local) File "/usr/lib/python2.7/site-packages/yum/yumRepo.py", line 1300, in _saveOldRepoXML shutil.copy2(local, old_local) File "/usr/lib64/python2.7/shutil.py", line 131, in copy2 copystat(src, dst) File "/usr/lib64/python2.7/shutil.py", line 98, in copystat os.utime(dst, (st.st_atime, st.st_mtime)) OSError: [Errno 2] No such file or directory: '/home/jenkins/mirrors_cache/fedora-updates-fc24/repomd.xml.old.tmp' Build step 'Execute shell' marked build as failure

1 1

[oVirt Jenkins] ovirt-system-tests_he-basic-ansible-suite-master - Build # 75 - Failure!
by jenkins＠jenkins.phx.ovirt.org 16 Mar '18

16 Mar '18

------=_Part_242_652355769.1521166632821 Content-Type: text/plain; charset=UTF-8 Content-Transfer-Encoding: 7bit Project: http://jenkins.ovirt.org/job/ovirt-system-tests_he-basic-ansible-suite-mast… Build: http://jenkins.ovirt.org/job/ovirt-system-tests_he-basic-ansible-suite-mast… Build Number: 75 Build Status: Failure Triggered By: Started by timer ------------------------------------- Changes Since Last Success: ------------------------------------- Changes for Build #75 [Michal Skrivanek] master: add USB to the sanity VM [Sandro Bonazzola] mirrors-reposync: add qemu-ev testing repo [Daniel Belenky] Modify mergeoptions in parser utils to merge runif ----------------- Failed Tests: ----------------- No tests ran. ------=_Part_242_652355769.1521166632821--

1 1

[JIRA] (OVIRT-1936) (nested)&&fc27 label has no online slaves
by sbonazzo (oVirt JIRA) 16 Mar '18

16 Mar '18

This is a multi-part message in MIME format... ------------=_1521177247-12355-118 Content-Type: text/plain; charset=UTF-8 Content-Transfer-Encoding: 7bit sbonazzo created OVIRT-1936: ------------------------------- Summary: (nested)&&fc27 label has no online slaves Key: OVIRT-1936 URL: https://ovirt-jira.atlassian.net/browse/OVIRT-1936 Project: oVirt - virtualization made easy Issue Type: By-EMAIL Reporter: sbonazzo Assignee: infra jenkins has pending job waiting for a slave on (nested)&&fc27 label but all slaves matching this label are offline. vm0159.workers-phx.ovirt.org Jan 21, 2018 6:49:39 AM Disconnected by dbelenky : fc27 slave not functioning properly - bringing offline vm0160.workers-phx.ovirt.org Jan 21, 2018 6:49:36 AM Disconnected by dbelenky : fc27 slave not functioning properly - bringing offline vm0161.workers-phx.ovirt.org Jan 21, 2018 6:49:34 AM Disconnected by dbelenky : fc27 slave not functioning properly - bringing offline vm0162.workers-phx.ovirt.org Jan 21, 2018 6:45:53 AM Disconnected by bkorren : slave not functioning properly - bringing offline vm0163.workers-phx.ovirt.org Jan 21, 2018 6:49:51 AM Disconnected by dbelenky : fc27 slave not functioning properly - bringing offline vm0164.workers-phx.ovirt.org Jan 21, 2018 6:49:38 AM Disconnected by dbelenky : fc27 slave not functioning properly - bringing offline vm0165.workers-phx.ovirt.org Jan 21, 2018 6:49:16 AM Disconnected by dbelenky : fc27 slave not functioning properly - bringing offline Please re-provision the slaves and bring them back online If this can't be done in a short loop, in the meanwhile please move back lago_master_github_check-patch-fc27-x86_64 job to (libvirt phx nested fc26) label -- SANDRO BONAZZOLA ASSOCIATE MANAGER, SOFTWARE ENGINEERING, EMEA ENG VIRTUALIZATION R&D Red Hat EMEA <https://www.redhat.com/> sbonazzo(a)redhat.com <https://red.ht/sig> <https://redhat.com/summit> -- This message was sent by Atlassian Jira (v1001.0.0-SNAPSHOT#100082) ------------=_1521177247-12355-118 Content-Type: text/html; charset="UTF-8" Content-Disposition: inline Content-Transfer-Encoding: 7bit <html><body> <h3>sbonazzo created OVIRT-1936:</h3> <pre> Summary: (nested)&&fc27 label has no online slaves Key: OVIRT-1936 URL: https://ovirt-jira.atlassian.net/browse/OVIRT-1936 Project: oVirt - virtualization made easy Issue Type: By-EMAIL Reporter: sbonazzo Assignee: infra</pre> <p>jenkins has pending job waiting for a slave on (nested)&&fc27 label but all slaves matching this label are offline.</p> <p>vm0159.workers-phx.ovirt.org Jan 21, 2018 6:49:39 AM Disconnected by dbelenky : fc27 slave not functioning properly – bringing offline</p> <p>vm0160.workers-phx.ovirt.org Jan 21, 2018 6:49:36 AM Disconnected by dbelenky : fc27 slave not functioning properly – bringing offline</p> <p>vm0161.workers-phx.ovirt.org Jan 21, 2018 6:49:34 AM Disconnected by dbelenky : fc27 slave not functioning properly – bringing offline</p> <p>vm0162.workers-phx.ovirt.org Jan 21, 2018 6:45:53 AM Disconnected by bkorren : slave not functioning properly – bringing offline</p> <p>vm0163.workers-phx.ovirt.org Jan 21, 2018 6:49:51 AM Disconnected by dbelenky : fc27 slave not functioning properly – bringing offline</p> <p>vm0164.workers-phx.ovirt.org Jan 21, 2018 6:49:38 AM Disconnected by dbelenky : fc27 slave not functioning properly – bringing offline</p> <p>vm0165.workers-phx.ovirt.org Jan 21, 2018 6:49:16 AM Disconnected by dbelenky : fc27 slave not functioning properly – bringing offline</p> <p>Please re-provision the slaves and bring them back online If this can't be done in a short loop, in the meanwhile please move back lago_master_github_check-patch-fc27-x86_64 job to (libvirt phx nested fc26) label</p> <p>—</p> <p>SANDRO BONAZZOLA</p> <p>ASSOCIATE MANAGER, SOFTWARE ENGINEERING, EMEA ENG VIRTUALIZATION R&D</p> <p>Red Hat EMEA <<a href="https://www.redhat.com/">https://www.redhat.com/</a>></p> <p>sbonazzo(a)redhat.com <<a href="https://red.ht/sig">https://red.ht/sig</a>> <<a href="https://redhat.com/summit">https://redhat.com/summit</a>></p> <p>— This message was sent by Atlassian Jira (v1001.0.0-SNAPSHOT#100082)</p> <img src="https://u4043402.ct.sendgrid.net/wf/open?upn=i5TMWGV99amJbNxJpSp2-2BJ33BSM3…" alt="" width="1" height="1" border="0" style="height:1px !important;width:1px !important;border-width:0 !important;margin-top:0 !important;margin-bottom:0 !important;margin-right:0 !important;margin-left:0 !important;padding-top:0 !important;padding-bottom:0 !important;padding-right:0 !important;padding-left:0 !important;"/> </body></html> ------------=_1521177247-12355-118--

1 0

[CQ]: af3e0c6 (ovirt-flexdriver) failed "ovirt-master" system tests
by oVirt Jenkins 15 Mar '18

15 Mar '18

Change af3e0c6 (ovirt-flexdriver) is probably the reason behind recent system test failures in the "ovirt-master" change queue and needs to be fixed. This change had been removed from the testing queue. Artifacts build from this change will not be released until it is fixed. For further details about the change see: https://github.com/rgolangh/ovirt-flexdriver/commit/af3e0c65e2a5f3a3ba313c1… For failed test results see: http://jenkins.ovirt.org/job/ovirt-master_change-queue-tester/6389/

1 0

[CQ]: 88690, 7 (ovirt-engine) failed "ovirt-master" system tests, but isn't the failure root cause
by oVirt Jenkins 15 Mar '18

15 Mar '18

A system test invoked by the "ovirt-master" change queue including change 88690,7 (ovirt-engine) failed. However, this change seems not to be the root cause for this failure. Change 88620,4 (ovirt-engine) that this change depends on or is based on, was detected as the cause of the testing failures. This change had been removed from the testing queue. Artifacts built from this change will not be released until either change 88620,4 (ovirt-engine) is fixed and this change is updated to refer to or rebased on the fixed version, or this change is modified to no longer depend on it. For further details about the change see: https://gerrit.ovirt.org/#/c/88690/7 For further details about the change that seems to be the root cause behind the testing failures see: https://gerrit.ovirt.org/#/c/88620/4 For failed test results see: http://jenkins.ovirt.org/job/ovirt-master_change-queue-tester/6384/

1 0

[CQ]: 88920, 8 (ovirt-engine) failed "ovirt-master" system tests, but isn't the failure root cause
by oVirt Jenkins 15 Mar '18

15 Mar '18

A system test invoked by the "ovirt-master" change queue including change 88920,8 (ovirt-engine) failed. However, this change seems not to be the root cause for this failure. Change 88620,4 (ovirt-engine) that this change depends on or is based on, was detected as the cause of the testing failures. This change had been removed from the testing queue. Artifacts built from this change will not be released until either change 88620,4 (ovirt-engine) is fixed and this change is updated to refer to or rebased on the fixed version, or this change is modified to no longer depend on it. For further details about the change see: https://gerrit.ovirt.org/#/c/88920/8 For further details about the change that seems to be the root cause behind the testing failures see: https://gerrit.ovirt.org/#/c/88620/4 For failed test results see: http://jenkins.ovirt.org/job/ovirt-master_change-queue-tester/6379/

1 0

Build failed in Jenkins: system-sync_mirrors-epel-el6-x86_64 #1294
by jenkins＠jenkins.phx.ovirt.org 15 Mar '18

15 Mar '18

See <http://jenkins.ovirt.org/job/system-sync_mirrors-epel-el6-x86_64/1294/displ…> Changes: [Sandro Bonazzola] mirrors-reposync: add qemu-ev testing repo ------------------------------------------ Started by timer [EnvInject] - Loading node environment variables. Building remotely on mirrors.phx.ovirt.org (mirrors) in workspace <http://jenkins.ovirt.org/job/system-sync_mirrors-epel-el6-x86_64/ws/> > git rev-parse --is-inside-work-tree # timeout=10 Fetching changes from the remote Git repository > git config remote.origin.url http://gerrit.ovirt.org/jenkins.git # timeout=10 Cleaning workspace > git rev-parse --verify HEAD # timeout=10 Resetting working tree > git reset --hard # timeout=10 > git clean -fdx # timeout=10 Pruning obsolete local branches Fetching upstream changes from http://gerrit.ovirt.org/jenkins.git > git --version # timeout=10 > git fetch --tags --progress http://gerrit.ovirt.org/jenkins.git +refs/heads/*:refs/remotes/origin/* --prune > git rev-parse origin/master^{commit} # timeout=10 Checking out Revision 69275d191c2de9980fccd25d7f9a6079871d2d81 (origin/master) > git config core.sparsecheckout # timeout=10 > git checkout -f 69275d191c2de9980fccd25d7f9a6079871d2d81 Commit message: "mirrors-reposync: add qemu-ev testing repo" > git rev-list --no-walk da1e4fe5ea4c69de5cefab65baaa9579f4180ca7 # timeout=10 [system-sync_mirrors-epel-el6-x86_64] $ /bin/bash -xe /tmp/jenkins590594148875260902.sh + jenkins/scripts/mirror_mgr.sh resync_yum_mirror epel-el6 x86_64 jenkins/data/mirrors-reposync.conf Checking if mirror needs a resync Traceback (most recent call last): File "/usr/bin/reposync", line 343, in <module> main() File "/usr/bin/reposync", line 175, in main my.doRepoSetup() File "/usr/lib/python2.7/site-packages/yum/__init__.py", line 681, in doRepoSetup return self._getRepos(thisrepo, True) File "/usr/lib/python2.7/site-packages/yum/__init__.py", line 721, in _getRepos self._repos.doSetup(thisrepo) File "/usr/lib/python2.7/site-packages/yum/repos.py", line 157, in doSetup self.retrieveAllMD() File "/usr/lib/python2.7/site-packages/yum/repos.py", line 112, in retrieveAllMD repo._revertOldRepoXML() File "/usr/lib/python2.7/site-packages/yum/yumRepo.py", line 1331, in _revertOldRepoXML os.rename(old_data['old_local'], old_data['local']) OSError: [Errno 2] No such file or directory Build step 'Execute shell' marked build as failure

1 1

[CQ]: 89020,2 (cockpit-ovirt) failed "ovirt-master" system tests
by oVirt Jenkins 15 Mar '18

15 Mar '18

Change 89020,2 (cockpit-ovirt) is probably the reason behind recent system test failures in the "ovirt-master" change queue and needs to be fixed. This change had been removed from the testing queue. Artifacts build from this change will not be released until it is fixed. For further details about the change see: https://gerrit.ovirt.org/#/c/89020/2 For failed test results see: http://jenkins.ovirt.org/job/ovirt-master_change-queue-tester/6374/

1 0