[ovirt-devel] [oVirt Jenkins] ovirt_4.1_hc-system-tests - Build # 76 - Failure!
Sandro Bonazzola
sbonazzo at redhat.com
Mon May 29 11:50:30 UTC 2017
On Mon, May 29, 2017 at 1:27 PM, Sahina Bose <sabose at redhat.com> wrote:
> Yes, it is. See http://jenkins.ovirt.org/job/ovirt_4.1_hc-system-tests/81/
> artifact/exported-artifacts/test_logs/hc-basic-suite-4.1/
> post-002_bootstrap.py/lago-hc-basic-suite-4-1-engine/_var_
> log/ovirt-engine/engine.log
>
>
I see that above jobs includes[1]:
- vdsm-jsonrpc-java-1.3.12-1.20170523144826.git9b8bd15.el7.centos.noarch.rpm
-
ovirt-engine-4.1.3.1-0.0.master.20170526092731.git4b52b4f.el7.centos.noarch.rpm
- ovirt-engine-appliance-4.1-20170524.1.el7.centos.noarch.rpm
Appliance is not updated there since it's currently failing to build[2]
Failing on:
00:03:18.316 OSError: [Errno 30] Read-only file system:
'/var/lib/mock/epel-7-x86_64-4a7b794316995f80fd7e3741db6918a0-35584/root/var/tmp/lorax.imgutils.n_ul7A/EFI/BOOT/BOOTX64.EFI'
The job started failing at build #161[3] with following changes:
Changes
Git
1. add image-ng to list of optional suits (detail
<http://jenkins.ovirt.org/job/ovirt-appliance_ovirt-4.1-snapshot_build-artifacts-el7-x86_64/161/changes#detail>)
<- May 24, 2017 10:40 PM good candidate as breaking patch
2. Use include-raw-escape for STC-CI setup scripts (detail
<http://jenkins.ovirt.org/job/ovirt-appliance_ovirt-4.1-snapshot_build-artifacts-el7-x86_64/161/changes#detail>)
<- May 25, 2017 1:46 PM good candidate as breaking patch
3. Support loader_main method for pipelines (detail
<http://jenkins.ovirt.org/job/ovirt-appliance_ovirt-4.1-snapshot_build-artifacts-el7-x86_64/161/changes#detail>)
<- May 25, 2017 1:46 PM; good candidate as breaking patch
Git
1. engine: add cloud-utils-growpart to appliance (detail
<http://jenkins.ovirt.org/job/ovirt-appliance_ovirt-4.1-snapshot_build-artifacts-el7-x86_64/161/changes#detail>)
<- Applied on March 13th, excluding
[1]
http://jenkins.ovirt.org/job/ovirt_4.1_hc-system-tests/81/artifact/exported-artifacts/test_logs/hc-basic-suite-4.1/post-002_bootstrap.py/lago_logs/lago.log/*view*/
[2]
http://jenkins.ovirt.org/job/ovirt-appliance_ovirt-4.1-snapshot_build-artifacts-el7-x86_64/
[3]
http://jenkins.ovirt.org/job/ovirt-appliance_ovirt-4.1-snapshot_build-artifacts-el7-x86_64/161/
> On Mon, May 29, 2017 at 4:45 PM, Sandro Bonazzola <sbonazzo at redhat.com>
> wrote:
>
>>
>>
>> On Sat, May 27, 2017 at 8:23 PM, Eyal Edri <eedri at redhat.com> wrote:
>>
>>> Since HC tests are using appliance, I think we need to verify the
>>> appliance build is working and includes the latest fix.
>>>
>>
>> Is this still an issue?
>>
>>
>>
>>>
>>> On Fri, May 26, 2017 at 3:39 PM, Sahina Bose <sabose at redhat.com> wrote:
>>>
>>>> I see that the 4.1 hc test is still failing with same error :
>>>>
>>>> Caused by: java.lang.ClassNotFoundException: org.apache.commons.lang.StringUtils from [Module "org.ovirt.vdsm-jsonrpc-java:main" from local module loader @5e91993f (finder: local module finder @1c4af82c (roots: /usr/share/ovirt-engine-wildfly-overlay/modules,/usr/share/ovirt-engine/modules/common,/usr/share/ovirt-engine-extension-aaa-jdbc/modules,/usr/share/ovirt-engine-wildfly/modules,/usr/share/ovirt-engine-wildfly/modules/system/layers/base))]
>>>>
>>>> Does the lago test need to be updated, as this fix has already landed?
>>>>
>>>> On Thu, May 25, 2017 at 2:03 PM, Eyal Edri <eedri at redhat.com> wrote:
>>>>
>>>>> So that was fixed already since hc suite is running once a day, I
>>>>> guess triggering it will fix it.
>>>>>
>>>>> On Thu, May 25, 2017 at 11:31 AM, Piotr Kliczewski <
>>>>> piotr.kliczewski at gmail.com> wrote:
>>>>>
>>>>>> This is the issue I described in the other thread. Engine you are
>>>>>> using do not have a patch [1].
>>>>>>
>>>>>> [1] https://gerrit.ovirt.org/#/c/77203
>>>>>>
>>>>>> On Thu, May 25, 2017 at 10:26 AM, Eyal Edri <eedri at redhat.com> wrote:
>>>>>>
>>>>>>> Adding devel and Piotr.
>>>>>>>
>>>>>>> On Thu, May 25, 2017 at 9:31 AM, Sahina Bose <sabose at redhat.com>
>>>>>>> wrote:
>>>>>>>
>>>>>>>> Connecting to lago-hc-basic-suite-4-1-host0.lago.local/192.168.200.2
>>>>>>>> 2017-05-24 23:31:00,492-04 ERROR [org.ovirt.engine.core.vdsbroker.vdsbroker.PollVDSCommand] (org.ovirt.thread.pool-7-thread-1) [712d05a6] Error: org.ovirt.engine.core.vdsbroker.TransportRunTimeException: Connection issues during send request
>>>>>>>> 2017-05-24 23:31:00,492-04 ERROR [org.ovirt.engine.core.vdsbroker.vdsbroker.PollVDSCommand] (org.ovirt.thread.pool-7-thread-1) [712d05a6] Exception: java.util.concurrent.ExecutionException: org.ovirt.engine.core.vdsbroker.TransportRunTimeException: Connection issues during send request
>>>>>>>> Caused by: java.lang.ClassNotFoundException: org.apache.commons.lang.StringUtils from [Module "org.ovirt.vdsm-jsonrpc-java:main" from local module loader @5e91993f (finder: local module finder @1c4af82c (roots: /usr/share/ovirt-engine-wildfly-overlay/modules,/usr/share/ovirt-engine/modules/common,/usr/share/ovirt-engine-extension-aaa-jdbc/modules,/usr/share/ovirt-engine-wildfly/modules,/usr/share/ovirt-engine-wildfly/modules/system/layers/base))]
>>>>>>>>
>>>>>>>>
>>>>>>>> Known issue?
>>>>>>>>
>>>>>>>>
>>>>>>>> On Thu, May 25, 2017 at 9:16 AM, <jenkins at jenkins.phx.ovirt.org>
>>>>>>>> wrote:
>>>>>>>>
>>>>>>>>> Project: http://jenkins.ovirt.org/job/ovirt_4.1_hc-system-tests/
>>>>>>>>> Build: http://jenkins.ovirt.org/job/ovirt_4.1_hc-system-tests/76/
>>>>>>>>> Build Number: 76
>>>>>>>>> Build Status: Failure
>>>>>>>>> Triggered By: Started by timer
>>>>>>>>>
>>>>>>>>> -------------------------------------
>>>>>>>>> Changes Since Last Success:
>>>>>>>>> -------------------------------------
>>>>>>>>> Changes for Build #76
>>>>>>>>> [Ondřej Svoboda] Use SDKv4 code in add_labeled_network.
>>>>>>>>>
>>>>>>>>> [Barak Korren] Refactor: Use macros in defaults file
>>>>>>>>>
>>>>>>>>> [Barak Korren] Default values for 'arch' and 'distro' in STD-CI
>>>>>>>>>
>>>>>>>>>
>>>>>>>>>
>>>>>>>>>
>>>>>>>>> -----------------
>>>>>>>>> Failed Tests:
>>>>>>>>> -----------------
>>>>>>>>> 1 tests failed.
>>>>>>>>> FAILED: 002_bootstrap.add_hosts
>>>>>>>>>
>>>>>>>>> Error Message:
>>>>>>>>>
>>>>>>>>> status: 409
>>>>>>>>> reason: Conflict
>>>>>>>>> detail: Cannot add Host. There is no available server in the
>>>>>>>>> cluster to probe the new server.
>>>>>>>>> -------------------- >> begin captured logging <<
>>>>>>>>> --------------------
>>>>>>>>> lago.utils: ERROR: Error while running thread
>>>>>>>>> Traceback (most recent call last):
>>>>>>>>> File "/usr/lib/python2.7/site-packages/lago/utils.py", line 58,
>>>>>>>>> in _ret_via_queue
>>>>>>>>> queue.put({'return': func()})
>>>>>>>>> File "/home/jenkins/workspace/ovirt
>>>>>>>>> _4.1_hc-system-tests/ovirt-system-tests/hc-basic-suite-4.1/t
>>>>>>>>> est-scenarios/002_bootstrap.py", line 141, in _add_host
>>>>>>>>> return api.hosts.add(p)
>>>>>>>>> File "/usr/lib/python2.7/site-packa
>>>>>>>>> ges/ovirtsdk/infrastructure/brokers.py", line 18306, in add
>>>>>>>>> headers={"Correlation-Id":correlation_id, "Expect":expect}
>>>>>>>>> File "/usr/lib/python2.7/site-packa
>>>>>>>>> ges/ovirtsdk/infrastructure/proxy.py", line 79, in add
>>>>>>>>> return self.request('POST', url, body, headers, cls=cls)
>>>>>>>>> File "/usr/lib/python2.7/site-packa
>>>>>>>>> ges/ovirtsdk/infrastructure/proxy.py", line 122, in request
>>>>>>>>> persistent_auth=self.__persistent_auth
>>>>>>>>> File "/usr/lib/python2.7/site-packa
>>>>>>>>> ges/ovirtsdk/infrastructure/connectionspool.py", line 79, in
>>>>>>>>> do_request
>>>>>>>>> persistent_auth)
>>>>>>>>> File "/usr/lib/python2.7/site-packa
>>>>>>>>> ges/ovirtsdk/infrastructure/connectionspool.py", line 162, in
>>>>>>>>> __do_request
>>>>>>>>> raise errors.RequestError(response_code, response_reason,
>>>>>>>>> response_body)
>>>>>>>>> RequestError:
>>>>>>>>> status: 409
>>>>>>>>> reason: Conflict
>>>>>>>>> detail: Cannot add Host. There is no available server in the
>>>>>>>>> cluster to probe the new server.
>>>>>>>>> lago.utils: ERROR: Error while running thread
>>>>>>>>> Traceback (most recent call last):
>>>>>>>>> File "/usr/lib/python2.7/site-packages/lago/utils.py", line 58,
>>>>>>>>> in _ret_via_queue
>>>>>>>>> queue.put({'return': func()})
>>>>>>>>> File "/home/jenkins/workspace/ovirt
>>>>>>>>> _4.1_hc-system-tests/ovirt-system-tests/hc-basic-suite-4.1/t
>>>>>>>>> est-scenarios/002_bootstrap.py", line 141, in _add_host
>>>>>>>>> return api.hosts.add(p)
>>>>>>>>> File "/usr/lib/python2.7/site-packa
>>>>>>>>> ges/ovirtsdk/infrastructure/brokers.py", line 18306, in add
>>>>>>>>> headers={"Correlation-Id":correlation_id, "Expect":expect}
>>>>>>>>> File "/usr/lib/python2.7/site-packa
>>>>>>>>> ges/ovirtsdk/infrastructure/proxy.py", line 79, in add
>>>>>>>>> return self.request('POST', url, body, headers, cls=cls)
>>>>>>>>> File "/usr/lib/python2.7/site-packa
>>>>>>>>> ges/ovirtsdk/infrastructure/proxy.py", line 122, in request
>>>>>>>>> persistent_auth=self.__persistent_auth
>>>>>>>>> File "/usr/lib/python2.7/site-packa
>>>>>>>>> ges/ovirtsdk/infrastructure/connectionspool.py", line 79, in
>>>>>>>>> do_request
>>>>>>>>> persistent_auth)
>>>>>>>>> File "/usr/lib/python2.7/site-packa
>>>>>>>>> ges/ovirtsdk/infrastructure/connectionspool.py", line 162, in
>>>>>>>>> __do_request
>>>>>>>>> raise errors.RequestError(response_code, response_reason,
>>>>>>>>> response_body)
>>>>>>>>> RequestError:
>>>>>>>>> status: 409
>>>>>>>>> reason: Conflict
>>>>>>>>> detail: Cannot add Host. There is no available server in the
>>>>>>>>> cluster to probe the new server.
>>>>>>>>> --------------------- >> end captured logging <<
>>>>>>>>> ---------------------
>>>>>>>>>
>>>>>>>>> Stack Trace:
>>>>>>>>> File "/usr/lib64/python2.7/unittest/case.py", line 369, in run
>>>>>>>>> testMethod()
>>>>>>>>> File "/usr/lib/python2.7/site-packages/nose/case.py", line 197,
>>>>>>>>> in runTest
>>>>>>>>> self.test(*self.arg)
>>>>>>>>> File "/usr/lib/python2.7/site-packages/ovirtlago/testlib.py",
>>>>>>>>> line 129, in wrapped_test
>>>>>>>>> test()
>>>>>>>>> File "/usr/lib/python2.7/site-packages/ovirtlago/testlib.py",
>>>>>>>>> line 59, in wrapper
>>>>>>>>> return func(get_test_prefix(), *args, **kwargs)
>>>>>>>>> File "/home/jenkins/workspace/ovirt
>>>>>>>>> _4.1_hc-system-tests/ovirt-system-tests/hc-basic-suite-4.1/t
>>>>>>>>> est-scenarios/002_bootstrap.py", line 160, in add_hosts
>>>>>>>>> nt.assert_true(all(vt.join_all()))
>>>>>>>>> File "/usr/lib/python2.7/site-packages/lago/utils.py", line 58,
>>>>>>>>> in _ret_via_queue
>>>>>>>>> queue.put({'return': func()})
>>>>>>>>> File "/home/jenkins/workspace/ovirt
>>>>>>>>> _4.1_hc-system-tests/ovirt-system-tests/hc-basic-suite-4.1/t
>>>>>>>>> est-scenarios/002_bootstrap.py", line 141, in _add_host
>>>>>>>>> return api.hosts.add(p)
>>>>>>>>> File "/usr/lib/python2.7/site-packa
>>>>>>>>> ges/ovirtsdk/infrastructure/brokers.py", line 18306, in add
>>>>>>>>> headers={"Correlation-Id":correlation_id, "Expect":expect}
>>>>>>>>> File "/usr/lib/python2.7/site-packa
>>>>>>>>> ges/ovirtsdk/infrastructure/proxy.py", line 79, in add
>>>>>>>>> return self.request('POST', url, body, headers, cls=cls)
>>>>>>>>> File "/usr/lib/python2.7/site-packa
>>>>>>>>> ges/ovirtsdk/infrastructure/proxy.py", line 122, in request
>>>>>>>>> persistent_auth=self.__persistent_auth
>>>>>>>>> File "/usr/lib/python2.7/site-packa
>>>>>>>>> ges/ovirtsdk/infrastructure/connectionspool.py", line 79, in
>>>>>>>>> do_request
>>>>>>>>> persistent_auth)
>>>>>>>>> File "/usr/lib/python2.7/site-packa
>>>>>>>>> ges/ovirtsdk/infrastructure/connectionspool.py", line 162, in
>>>>>>>>> __do_request
>>>>>>>>> raise errors.RequestError(response_code, response_reason,
>>>>>>>>> response_body)
>>>>>>>>>
>>>>>>>>> status: 409
>>>>>>>>> reason: Conflict
>>>>>>>>> detail: Cannot add Host. There is no available server in the
>>>>>>>>> cluster to probe the new server.
>>>>>>>>> -------------------- >> begin captured logging <<
>>>>>>>>> --------------------
>>>>>>>>> lago.utils: ERROR: Error while running thread
>>>>>>>>> Traceback (most recent call last):
>>>>>>>>> File "/usr/lib/python2.7/site-packages/lago/utils.py", line 58,
>>>>>>>>> in _ret_via_queue
>>>>>>>>> queue.put({'return': func()})
>>>>>>>>> File "/home/jenkins/workspace/ovirt
>>>>>>>>> _4.1_hc-system-tests/ovirt-system-tests/hc-basic-suite-4.1/t
>>>>>>>>> est-scenarios/002_bootstrap.py", line 141, in _add_host
>>>>>>>>> return api.hosts.add(p)
>>>>>>>>> File "/usr/lib/python2.7/site-packa
>>>>>>>>> ges/ovirtsdk/infrastructure/brokers.py", line 18306, in add
>>>>>>>>> headers={"Correlation-Id":correlation_id, "Expect":expect}
>>>>>>>>> File "/usr/lib/python2.7/site-packa
>>>>>>>>> ges/ovirtsdk/infrastructure/proxy.py", line 79, in add
>>>>>>>>> return self.request('POST', url, body, headers, cls=cls)
>>>>>>>>> File "/usr/lib/python2.7/site-packa
>>>>>>>>> ges/ovirtsdk/infrastructure/proxy.py", line 122, in request
>>>>>>>>> persistent_auth=self.__persistent_auth
>>>>>>>>> File "/usr/lib/python2.7/site-packa
>>>>>>>>> ges/ovirtsdk/infrastructure/connectionspool.py", line 79, in
>>>>>>>>> do_request
>>>>>>>>> persistent_auth)
>>>>>>>>> File "/usr/lib/python2.7/site-packa
>>>>>>>>> ges/ovirtsdk/infrastructure/connectionspool.py", line 162, in
>>>>>>>>> __do_request
>>>>>>>>> raise errors.RequestError(response_code, response_reason,
>>>>>>>>> response_body)
>>>>>>>>> RequestError:
>>>>>>>>> status: 409
>>>>>>>>> reason: Conflict
>>>>>>>>> detail: Cannot add Host. There is no available server in the
>>>>>>>>> cluster to probe the new server.
>>>>>>>>> lago.utils: ERROR: Error while running thread
>>>>>>>>> Traceback (most recent call last):
>>>>>>>>> File "/usr/lib/python2.7/site-packages/lago/utils.py", line 58,
>>>>>>>>> in _ret_via_queue
>>>>>>>>> queue.put({'return': func()})
>>>>>>>>> File "/home/jenkins/workspace/ovirt
>>>>>>>>> _4.1_hc-system-tests/ovirt-system-tests/hc-basic-suite-4.1/t
>>>>>>>>> est-scenarios/002_bootstrap.py", line 141, in _add_host
>>>>>>>>> return api.hosts.add(p)
>>>>>>>>> File "/usr/lib/python2.7/site-packa
>>>>>>>>> ges/ovirtsdk/infrastructure/brokers.py", line 18306, in add
>>>>>>>>> headers={"Correlation-Id":correlation_id, "Expect":expect}
>>>>>>>>> File "/usr/lib/python2.7/site-packa
>>>>>>>>> ges/ovirtsdk/infrastructure/proxy.py", line 79, in add
>>>>>>>>> return self.request('POST', url, body, headers, cls=cls)
>>>>>>>>> File "/usr/lib/python2.7/site-packa
>>>>>>>>> ges/ovirtsdk/infrastructure/proxy.py", line 122, in request
>>>>>>>>> persistent_auth=self.__persistent_auth
>>>>>>>>> File "/usr/lib/python2.7/site-packa
>>>>>>>>> ges/ovirtsdk/infrastructure/connectionspool.py", line 79, in
>>>>>>>>> do_request
>>>>>>>>> persistent_auth)
>>>>>>>>> File "/usr/lib/python2.7/site-packa
>>>>>>>>> ges/ovirtsdk/infrastructure/connectionspool.py", line 162, in
>>>>>>>>> __do_request
>>>>>>>>> raise errors.RequestError(response_code, response_reason,
>>>>>>>>> response_body)
>>>>>>>>> RequestError:
>>>>>>>>> status: 409
>>>>>>>>> reason: Conflict
>>>>>>>>> detail: Cannot add Host. There is no available server in the
>>>>>>>>> cluster to probe the new server.
>>>>>>>>> --------------------- >> end captured logging <<
>>>>>>>>> ---------------------
>>>>>>>>
>>>>>>>>
>>>>>>>>
>>>>>>>> _______________________________________________
>>>>>>>> Infra mailing list
>>>>>>>> Infra at ovirt.org
>>>>>>>> http://lists.ovirt.org/mailman/listinfo/infra
>>>>>>>>
>>>>>>>>
>>>>>>>
>>>>>>>
>>>>>>> --
>>>>>>>
>>>>>>> Eyal edri
>>>>>>>
>>>>>>>
>>>>>>> ASSOCIATE MANAGER
>>>>>>>
>>>>>>> RHV DevOps
>>>>>>>
>>>>>>> EMEA VIRTUALIZATION R&D
>>>>>>>
>>>>>>>
>>>>>>> Red Hat EMEA <https://www.redhat.com/>
>>>>>>> <https://red.ht/sig> TRIED. TESTED. TRUSTED.
>>>>>>> <https://redhat.com/trusted>
>>>>>>> phone: +972-9-7692018 <+972%209-769-2018>
>>>>>>> irc: eedri (on #tlv #rhev-dev #rhev-integ)
>>>>>>>
>>>>>>> _______________________________________________
>>>>>>> Devel mailing list
>>>>>>> Devel at ovirt.org
>>>>>>> http://lists.ovirt.org/mailman/listinfo/devel
>>>>>>>
>>>>>>
>>>>>>
>>>>>
>>>>>
>>>>> --
>>>>>
>>>>> Eyal edri
>>>>>
>>>>>
>>>>> ASSOCIATE MANAGER
>>>>>
>>>>> RHV DevOps
>>>>>
>>>>> EMEA VIRTUALIZATION R&D
>>>>>
>>>>>
>>>>> Red Hat EMEA <https://www.redhat.com/>
>>>>> <https://red.ht/sig> TRIED. TESTED. TRUSTED.
>>>>> <https://redhat.com/trusted>
>>>>> phone: +972-9-7692018 <+972%209-769-2018>
>>>>> irc: eedri (on #tlv #rhev-dev #rhev-integ)
>>>>>
>>>>
>>>>
>>>
>>>
>>> --
>>>
>>> Eyal edri
>>>
>>>
>>> ASSOCIATE MANAGER
>>>
>>> RHV DevOps
>>>
>>> EMEA VIRTUALIZATION R&D
>>>
>>>
>>> Red Hat EMEA <https://www.redhat.com/>
>>> <https://red.ht/sig> TRIED. TESTED. TRUSTED.
>>> <https://redhat.com/trusted>
>>> phone: +972-9-7692018 <+972%209-769-2018>
>>> irc: eedri (on #tlv #rhev-dev #rhev-integ)
>>>
>>
>>
>>
>> --
>>
>> SANDRO BONAZZOLA
>>
>> ASSOCIATE MANAGER, SOFTWARE ENGINEERING, EMEA ENG VIRTUALIZATION R&D
>>
>> Red Hat EMEA <https://www.redhat.com/>
>> <https://red.ht/sig>
>> TRIED. TESTED. TRUSTED. <https://redhat.com/trusted>
>>
>
>
--
SANDRO BONAZZOLA
ASSOCIATE MANAGER, SOFTWARE ENGINEERING, EMEA ENG VIRTUALIZATION R&D
Red Hat EMEA <https://www.redhat.com/>
<https://red.ht/sig>
TRIED. TESTED. TRUSTED. <https://redhat.com/trusted>
-------------- next part --------------
An HTML attachment was scrubbed...
URL: <http://lists.ovirt.org/pipermail/infra/attachments/20170529/11b19ffc/attachment-0001.html>
More information about the Infra
mailing list