Devel
Threads by month
- ----- 2026 -----
- February
- January
- ----- 2025 -----
- December
- November
- October
- September
- August
- July
- June
- May
- April
- March
- February
- January
- ----- 2024 -----
- December
- November
- October
- September
- August
- July
- June
- May
- April
- March
- February
- January
- ----- 2023 -----
- December
- November
- October
- September
- August
- July
- June
- May
- April
- March
- February
- January
- ----- 2022 -----
- December
- November
- October
- September
- August
- July
- June
- May
- April
- March
- February
- January
- ----- 2021 -----
- December
- November
- October
- September
- August
- July
- June
- May
- April
- March
- February
- January
- ----- 2020 -----
- December
- November
- October
- September
- August
- July
- June
- May
- April
- March
- February
- January
- ----- 2019 -----
- December
- November
- October
- September
- August
- July
- June
- May
- April
- March
- February
- January
- ----- 2018 -----
- December
- November
- October
- September
- August
- July
- June
- May
- April
- March
- February
- January
- ----- 2017 -----
- December
- November
- October
- September
- August
- July
- June
- May
- April
- March
- February
- January
- ----- 2016 -----
- December
- November
- October
- September
- August
- July
- June
- May
- April
- March
- February
- January
- ----- 2015 -----
- December
- November
- October
- September
- August
- July
- June
- May
- April
- March
- February
- January
- ----- 2014 -----
- December
- November
- October
- September
- August
- July
- June
- May
- April
- March
- February
- January
- ----- 2013 -----
- December
- November
- October
- September
- August
- July
- June
- May
- April
- March
- February
- January
- ----- 2012 -----
- December
- November
- October
- September
- August
- July
- June
- May
- April
- March
- February
- January
- ----- 2011 -----
- December
- November
- October
November 2017
- 49 participants
- 61 discussions
Hi all,
The default range went through several changes during the lifetime of
the project - from being small and constant until 3.2, to small and
random in 3.3 until 3.5, back to fixed and small in 3.6, slightly
enlarged (to 1024 addresses) in current master.
In all cases, the address range was taken from the IANA-assigned range
001A4A of Qumranet (now Red Hat) [1].
I suggest to:
1. Default to a random range again
- So that two or (a few) more setups using the same network segment
are less likely to use the same range
2. One in the locally-administered address range [2]
- So that we have a larger address range
3. Make it a bit larger, say 65000 addresses
- So that admins do not have to handle this manually, even for large
setups, unless they have specific needs
If there is objection to (2.), we can still have the others - among
the 24 bits in the 001A4A prefix range, choose among the first 8 a
random value during engine-setup, and have the pool of size 16 bits.
Comments/ideas/etc. are welcome.
Best regards,
[1] https://regauth.standards.ieee.org/standards-ra-web/pub/view.html
[2] https://en.wikipedia.org/wiki/MAC_address#Universal_vs._local
--
Didi
2
2
[ OST Failure Report ] [ oVirt 4.2 ] [ 22-11-2017 ] [ 002_bootstrap.add_dc ]
by Dafna Ron 22 Nov '17
by Dafna Ron 22 Nov '17
22 Nov '17
This is a multi-part message in MIME format.
--------------4E0BDA3C10A37D37E4CBA152
Content-Type: text/plain; charset=utf-8
Content-Transfer-Encoding: 7bit
Hi,
I am not sure we should be alerting on 4.2 versions yet but since we are
getting several cq failures for the same reasons and test, I decided
this should be communicated.
The failed test is 002_bootstrap.add_dc in OST 4.2. and all are pointing
to one patch.
Remove mandatory option for cluster in register operation -
https://gerrit.ovirt.org/#/c/84359/*
*
can you please have a look?
I was unable to determine from what log the error was coming from as
well but looking at the failed test and the patch reported there seem to
be a connection.
**
*Link and headline of suspected patches: *
**
Patch that is marked as root cause:
Remove mandatory option for cluster in register operation -
https://gerrit.ovirt.org/#/c/84359/*
*
Patches reported as failed:
1. Add discoveredTargets property to HostService -
https://gerrit.ovirt.org/#/c/84489/
*2. Document the fact that sparse/raw isn't supported on block -
https://gerrit.ovirt.org/#/c/84490/*
3. Update CHANGES.adoc for release 4.2.25 -
https://gerrit.ovirt.org/#/c/84499/*
*
4. [maven-release-plugin] prepare release 4.2.26 -
https://gerrit.ovirt.org/#/c/84500/*
*
**
*
Link to Job:
http://jenkins.ovirt.org/job/ovirt-4.2_change-queue-tester/*
*
*
*Link to all logs:*
*http://jenkins.ovirt.org/job/ovirt-4.2_change-queue-tester/75/artifact/*
*
*
*(Relevant) error snippet from the log: *
*
<error>
*
Traceback (most recent call last):
File "/usr/lib64/python2.7/unittest/case.py", line 369, in run
testMethod()
File "/usr/lib/python2.7/site-packages/nose/case.py", line 197, in runTest
self.test(*self.arg)
File "/usr/lib/python2.7/site-packages/ovirtlago/testlib.py", line 129, in wrapped_test
test()
File "/usr/lib/python2.7/site-packages/ovirtlago/testlib.py", line 59, in wrapper
return func(get_test_prefix(), *args, **kwargs)
File "/home/jenkins/workspace/ovirt-4.2_change-queue-tester/ovirt-system-tests/basic-suite-4.2/test-scenarios/002_bootstrap.py", line 109, in add_dc
api = prefix.virt_env.engine_vm().get_api(api_ver=4)
File "/usr/lib/python2.7/site-packages/ovirtlago/virt.py", line 329, in get_api
return self.get_api_v4()
File "/usr/lib/python2.7/site-packages/ovirtlago/virt.py", line 338, in get_api_v4
self._api_v4 = self._get_api(api_ver=4)
File "/usr/lib/python2.7/site-packages/ovirtlago/virt.py", line 316, in _get_api
while not testapi.test():
File "/usr/lib64/python2.7/site-packages/ovirtsdk4/__init__.py", line 744, in test
six.reraise(*sys.exc_info())
File "/usr/lib64/python2.7/site-packages/ovirtsdk4/__init__.py", line 741, in test
self.system_service().get()
File "/usr/lib64/python2.7/site-packages/ovirtsdk4/services.py", line 21865, in get
return self._internal_get(headers, query, wait)
File "/usr/lib64/python2.7/site-packages/ovirtsdk4/service.py", line 202, in _internal_get
context = self._connection.send(request)
File "/usr/lib64/python2.7/site-packages/ovirtsdk4/__init__.py", line 370, in send
return self.__send(request)
File "/usr/lib64/python2.7/site-packages/ovirtsdk4/__init__.py", line 388, in __send
self.authenticate()
File "/usr/lib64/python2.7/site-packages/ovirtsdk4/__init__.py", line 381, in authenticate
self._sso_token = self._get_access_token()
File "/usr/lib64/python2.7/site-packages/ovirtsdk4/__init__.py", line 617, in _get_access_token
sso_response = self._get_sso_response(self._sso_url, post_data)
File "/usr/lib64/python2.7/site-packages/ovirtsdk4/__init__.py", line 702, in _get_sso_response
self._check_content_type(self.__JSON_CONTENT_TYPE_RE, 'JSON', header_lines)
File "/usr/lib64/python2.7/site-packages/ovirtsdk4/__init__.py", line 887, in _check_content_type
raise Error(msg)
Error: The response content type 'text/html; charset=UTF-8' isn't the expected JSON
**
*</error>*
**
--------------4E0BDA3C10A37D37E4CBA152
Content-Type: text/html; charset=utf-8
Content-Transfer-Encoding: 7bit
<html>
<head>
<meta http-equiv="content-type" content="text/html; charset=utf-8">
</head>
<body text="#000000" bgcolor="#FFFFFF">
<p>Hi, <br>
</p>
<p>I am not sure we should be alerting on 4.2 versions yet but since
we are getting several cq failures for the same reasons and test,
I decided this should be communicated. <br>
</p>
<p>The failed test is 002_bootstrap.add_dc in OST 4.2. and all are
pointing to one patch. <br>
</p>
<p><span style="font-weight:normal;">Remove mandatory option for
cluster in register operation -
<a class="moz-txt-link-freetext" href="https://gerrit.ovirt.org/#/c/84359/">https://gerrit.ovirt.org/#/c/84359/</a></span><b
style="font-weight:normal;"
id="docs-internal-guid-5859b7a1-e37b-8f20-ebe3-4aa825e533e3"><br>
</b></p>
<p>can you please have a look? <br>
</p>
<p>I was unable to determine from what log the error was coming from
as well but looking at the failed test and the patch reported
there seem to be a connection. <br>
</p>
<p><b style="font-weight:normal;"
id="docs-internal-guid-5859b7a1-e37b-8f20-ebe3-4aa825e533e3">
<p dir="ltr"
style="line-height:1.38;margin-top:0pt;margin-bottom:0pt;"><span style="font-size:11pt;font-family:Arial;color:#000000;background-color:transparent;font-weight:400;font-style:normal;font-variant:normal;text-decoration:none;vertical-align:baseline;white-space:pre-wrap;">Link and headline of suspected patches:
</span></p>
</b></p>
<p><span style="font-weight:normal;">Patch that is marked as root
cause: <br>
</span></p>
<p><span style="font-weight:normal;">Remove mandatory option for
cluster in register operation -
<a class="moz-txt-link-freetext" href="https://gerrit.ovirt.org/#/c/84359/">https://gerrit.ovirt.org/#/c/84359/</a></span><b
style="font-weight:normal;"
id="docs-internal-guid-5859b7a1-e37b-8f20-ebe3-4aa825e533e3"><br>
</b></p>
Patches reported as failed: <br>
1. Add discoveredTargets property to HostService -
<a class="moz-txt-link-freetext" href="https://gerrit.ovirt.org/#/c/84489/">https://gerrit.ovirt.org/#/c/84489/</a><br>
<p><b style="font-weight:normal;"
id="docs-internal-guid-5859b7a1-e37b-8f20-ebe3-4aa825e533e3">2.
Document the fact that sparse/raw isn't supported on block -
<a class="moz-txt-link-freetext" href="https://gerrit.ovirt.org/#/c/84490/">https://gerrit.ovirt.org/#/c/84490/</a></b></p>
<p><span style="font-weight:normal;">3. Update CHANGES.adoc for
release 4.2.25 - <a class="moz-txt-link-freetext" href="https://gerrit.ovirt.org/#/c/84499/">https://gerrit.ovirt.org/#/c/84499/</a></span><b
style="font-weight:normal;"
id="docs-internal-guid-5859b7a1-e37b-8f20-ebe3-4aa825e533e3"><br>
</b></p>
<p><span style="font-weight:normal;">4. [maven-release-plugin]
prepare release 4.2.26 - <a class="moz-txt-link-freetext" href="https://gerrit.ovirt.org/#/c/84500/">https://gerrit.ovirt.org/#/c/84500/</a></span><b
style="font-weight:normal;"
id="docs-internal-guid-5859b7a1-e37b-8f20-ebe3-4aa825e533e3"><br>
<p dir="ltr"
style="line-height:1.38;margin-top:0pt;margin-bottom:0pt;"><span style="font-size:11pt;font-family:Arial;color:#000000;background-color:transparent;font-weight:400;font-style:normal;font-variant:normal;text-decoration:none;vertical-align:baseline;white-space:pre-wrap;">
</span></p>
<p dir="ltr"
style="line-height:1.38;margin-top:0pt;margin-bottom:0pt;"><span style="font-size:11pt;font-family:Arial;color:#000000;background-color:transparent;font-weight:400;font-style:normal;font-variant:normal;text-decoration:none;vertical-align:baseline;white-space:pre-wrap;">Link to Job:</span></p>
<a class="moz-txt-link-freetext" href="http://jenkins.ovirt.org/job/ovirt-4.2_change-queue-tester/">http://jenkins.ovirt.org/job/ovirt-4.2_change-queue-tester/</a></b></p>
<p><b style="font-weight:normal;"
id="docs-internal-guid-5859b7a1-e37b-8f20-ebe3-4aa825e533e3"><br>
<p dir="ltr"
style="line-height:1.38;margin-top:0pt;margin-bottom:0pt;"><span style="font-size:11pt;font-family:Arial;color:#000000;background-color:transparent;font-weight:400;font-style:normal;font-variant:normal;text-decoration:none;vertical-align:baseline;white-space:pre-wrap;">Link to all logs:</span></p>
<a class="moz-txt-link-freetext" href="http://jenkins.ovirt.org/job/ovirt-4.2_change-queue-tester/75/artifact/">http://jenkins.ovirt.org/job/ovirt-4.2_change-queue-tester/75/artifact/</a></b></p>
<p><b style="font-weight:normal;"
id="docs-internal-guid-5859b7a1-e37b-8f20-ebe3-4aa825e533e3"><br>
<p dir="ltr"
style="line-height:1.38;margin-top:0pt;margin-bottom:0pt;"><span style="font-size:11pt;font-family:Arial;color:#000000;background-color:transparent;font-weight:400;font-style:normal;font-variant:normal;text-decoration:none;vertical-align:baseline;white-space:pre-wrap;">(Relevant) error snippet from the log: </span></p>
<p dir="ltr"
style="line-height:1.38;margin-top:0pt;margin-bottom:0pt;"><span style="font-size:11pt;font-family:Arial;color:#000000;background-color:transparent;font-weight:400;font-style:normal;font-variant:normal;text-decoration:none;vertical-align:baseline;white-space:pre-wrap;"><error></span></p>
<br>
<br>
</b><br>
</p>
<pre style="box-sizing: border-box; white-space: pre-wrap; word-wrap: break-word; margin: 0px; color: rgb(51, 51, 51); font-size: 14px; font-style: normal; font-variant-ligatures: normal; font-variant-caps: normal; font-weight: normal; letter-spacing: normal; orphans: 2; text-align: start; text-indent: 0px; text-transform: none; widows: 2; word-spacing: 0px; -webkit-text-stroke-width: 0px; text-decoration-style: initial; text-decoration-color: initial;">Traceback (most recent call last):
File "/usr/lib64/python2.7/unittest/case.py", line 369, in run
testMethod()
File "/usr/lib/python2.7/site-packages/nose/case.py", line 197, in runTest
self.test(*self.arg)
File "/usr/lib/python2.7/site-packages/ovirtlago/testlib.py", line 129, in wrapped_test
test()
File "/usr/lib/python2.7/site-packages/ovirtlago/testlib.py", line 59, in wrapper
return func(get_test_prefix(), *args, **kwargs)
File "/home/jenkins/workspace/ovirt-4.2_change-queue-tester/ovirt-system-tests/basic-suite-4.2/test-scenarios/002_bootstrap.py", line 109, in add_dc
api = prefix.virt_env.engine_vm().get_api(api_ver=4)
File "/usr/lib/python2.7/site-packages/ovirtlago/virt.py", line 329, in get_api
return self.get_api_v4()
File "/usr/lib/python2.7/site-packages/ovirtlago/virt.py", line 338, in get_api_v4
self._api_v4 = self._get_api(api_ver=4)
File "/usr/lib/python2.7/site-packages/ovirtlago/virt.py", line 316, in _get_api
while not testapi.test():
File "/usr/lib64/python2.7/site-packages/ovirtsdk4/__init__.py", line 744, in test
six.reraise(*sys.exc_info())
File "/usr/lib64/python2.7/site-packages/ovirtsdk4/__init__.py", line 741, in test
self.system_service().get()
File "/usr/lib64/python2.7/site-packages/ovirtsdk4/services.py", line 21865, in get
return self._internal_get(headers, query, wait)
File "/usr/lib64/python2.7/site-packages/ovirtsdk4/service.py", line 202, in _internal_get
context = self._connection.send(request)
File "/usr/lib64/python2.7/site-packages/ovirtsdk4/__init__.py", line 370, in send
return self.__send(request)
File "/usr/lib64/python2.7/site-packages/ovirtsdk4/__init__.py", line 388, in __send
self.authenticate()
File "/usr/lib64/python2.7/site-packages/ovirtsdk4/__init__.py", line 381, in authenticate
self._sso_token = self._get_access_token()
File "/usr/lib64/python2.7/site-packages/ovirtsdk4/__init__.py", line 617, in _get_access_token
sso_response = self._get_sso_response(self._sso_url, post_data)
File "/usr/lib64/python2.7/site-packages/ovirtsdk4/__init__.py", line 702, in _get_sso_response
self._check_content_type(self.__JSON_CONTENT_TYPE_RE, 'JSON', header_lines)
File "/usr/lib64/python2.7/site-packages/ovirtsdk4/__init__.py", line 887, in _check_content_type
raise Error(msg)
Error: The response content type 'text/html; charset=UTF-8' isn't the expected JSON</pre>
<p><b style="font-weight:normal;"
id="docs-internal-guid-5859b7a1-e37b-8f20-ebe3-4aa825e533e3">
<p dir="ltr"
style="line-height:1.38;margin-top:0pt;margin-bottom:0pt;"><span style="font-size:11pt;font-family:Arial;color:#000000;background-color:transparent;font-weight:400;font-style:normal;font-variant:normal;text-decoration:none;vertical-align:baseline;white-space:pre-wrap;"></error></span></p>
</b><br class="Apple-interchange-newline">
</p>
<p><br>
</p>
</body>
</html>
--------------4E0BDA3C10A37D37E4CBA152--
3
2
Re: [ovirt-devel] Question about availability of image-io in ovirt master repo
by Nir Soffer 21 Nov '17
by Nir Soffer 21 Nov '17
21 Nov '17
On Tue, Nov 21, 2017 at 9:38 PM Idan Shaby <ishaby(a)redhat.com> wrote:
> So that what it gives me:
> - el6 - nothing.
> - el7:
> ovirt-imageio-common-0:1.2.0-0.201711211045.gitd4425c0.el7.centos.noarch
> ovirt-imageio-daemon-0:1.2.0-0.201711211045.gitd4425c0.el7.centos.noarch
> ovirt-imageio-proxy-0:1.2.0-0.201711211045.gitd4425c0.el7.centos.noarch
>
> ovirt-imageio-proxy-setup-0:1.2.0-0.201711211045.gitd4425c0.el7.centos.noarch
> - fc27 - nothing.
> - fc26:
> ovirt-imageio-common-0:1.2.0-0.201711211044.gitd4425c0.fc26.noarch
> ovirt-imageio-daemon-0:1.2.0-0.201711211044.gitd4425c0.fc26.noarch
> ovirt-imageio-proxy-0:1.2.0-0.201711211044.gitd4425c0.fc26.noarch
> ovirt-imageio-proxy-setup-0:1.2.0-0.201711211044.gitd4425c0.fc26.noarch
> - fc25:
> ovirt-imageio-common-0:1.2.0-0.201711211044.gitd4425c0.fc25.noarch
> ovirt-imageio-daemon-0:1.2.0-0.201711211044.gitd4425c0.fc25.noarch
> ovirt-imageio-proxy-0:1.2.0-0.201711211044.gitd4425c0.fc25.noarch
> ovirt-imageio-proxy-setup-0:1.2.0-0.201711211044.gitd4425c0.fc25.noarch
> fc24 - nothing.
>
> So I guess it means we don't deliver imageio1.2.0 on el6, fc27 and fc24.
> fc27 should be a problem, right?
>
I hope that tomorrow we will have also fc27 and fcraw, but these are not
supported platforms yet.
Looks like we can require 1.2 in vdsm.
>
> Regards,
> Idan
>
> ---------- Forwarded message ----------
> From: Barak Korren <bkorren(a)redhat.com>
> Date: Tue, Nov 21, 2017 at 9:27 PM
> Subject: Re: Question about availability of image-io in ovirt master repo
> To: Idan Shaby <ishaby(a)redhat.com>
> Cc: infra <infra(a)ovirt.org>
>
>
> On 21 November 2017 at 21:04, Idan Shaby <ishaby(a)redhat.com> wrote:
> > Hi,
> >
> > How do I check if our ovirt master repos contain version 1.2.0 of
> > ovirt-imageio-common and ovirt-imageio-daemon?
> > I want to add these requirements to vdsm and I am not sure how to verify
> it.
>
> This searches for a certain package in a certain repo:
>
> repoquery --repofrompath=r,
> http://resources.ovirt.org/repos/ovirt/tested/master/rpm/el7
> --repoid=r
> <http://resources.ovirt.org/repos/ovirt/tested/master/rpm/el7%0D--repoid=r>
> list '*imageio*'
>
Adding devel, may be useful for people adding dependencies.
>
> You can change the url slightly to check for other distros and oVirt
> versions.
>
> --
> Barak Korren
> RHV DevOps team , RHCE, RHCi
> Red Hat EMEA
> redhat.com | TRIED. TESTED. TRUSTED. | redhat.com/trusted
>
>
1
0
[ OST Failure Report ] [ oVirt master ] [ 20-11-2017 ] [004_basic_sanity.vm_run ]
by Dafna Ron 20 Nov '17
by Dafna Ron 20 Nov '17
20 Nov '17
This is a multi-part message in MIME format.
--------------2EEBD942435F80A2268AD3F9
Content-Type: text/plain; charset=utf-8
Content-Transfer-Encoding: 7bit
Hi,
We have a failure in OST on test 004_basic_sanity.vm_run.
it seems to be an error in vm type which is related to the patch reported.
**
*Link to suspected patches: https://gerrit.ovirt.org/#/c/84343/*
*
Link to Job:
http://jenkins.ovirt.org/job/ovirt-master_change-queue-tester/3922
Link to all logs:
http://jenkins.ovirt.org/job/ovirt-master_change-queue-tester/3922/artifact
(Relevant) error snippet from the log:
<error>
vdsm log:
*
*2017-11-20 07:40:12,779-0500 ERROR (jsonrpc/2) [jsonrpc.JsonRpcServer]
Internal server error (__init__:611)
Traceback (most recent call last):
File "/usr/lib/python2.7/site-packages/yajsonrpc/__init__.py", line
606, in _handle_request
res = method(**params)
File "/usr/lib/python2.7/site-packages/vdsm/rpc/Bridge.py", line 201,
in _dynamicMethod
result = fn(*methodArgs)
File "<string>", line 2, in getAllVmStats
File "/usr/lib/python2.7/site-packages/vdsm/common/api.py", line 48,
in method
ret = func(*args, **kwargs)
File "/usr/lib/python2.7/site-packages/vdsm/API.py", line 1341, in
getAllVmStats
statsList = self._cif.getAllVmStats()
File "/usr/lib/python2.7/site-packages/vdsm/clientIF.py", line 508, in
getAllVmStats
return [v.getStats() for v in self.vmContainer.values()]
File "/usr/lib/python2.7/site-packages/vdsm/virt/vm.py", line 1664, in
getStats
stats.update(self._getConfigVmStats())
File "/usr/lib/python2.7/site-packages/vdsm/virt/vm.py", line 1703, in
_getConfigVmStats
'vmType': self.conf['vmType'],
KeyError: 'vmType'
*
*
*
engine log:
***2017-11-20 07:43:07,675-05 DEBUG
[org.ovirt.vdsm.jsonrpc.client.internal.ResponseWorker] (ResponseWorker)
[] Message received: {"jsonrpc": "2.0", "id":
"5bf12e5a-4a09-4999-a6ce-a7dd639d3833", "error": {"message": "Internal
JSON-RPC error:
{'reason': \"'vmType'\"}", "code": -32603}}
2017-11-20 07:43:07,676-05 WARN
[org.ovirt.engine.core.vdsbroker.vdsbroker.GetAllVmStatsVDSCommand]
(EE-ManagedThreadFactory-engineScheduled-Thread-70) [] Unexpected return
value: Status [code=-32603, message=Internal JSON-RPC error: {'r
eason': "'vmType'"}]
2017-11-20 07:43:07,676-05 ERROR
[org.ovirt.engine.core.vdsbroker.vdsbroker.GetAllVmStatsVDSCommand]
(EE-ManagedThreadFactory-engineScheduled-Thread-70) [] Failed in
'GetAllVmStatsVDS' method
2017-11-20 07:43:07,676-05 ERROR
[org.ovirt.engine.core.vdsbroker.vdsbroker.GetAllVmStatsVDSCommand]
(EE-ManagedThreadFactory-engineScheduled-Thread-70) [] Command
'GetAllVmStatsVDSCommand(HostName = lago-basic-suite-master-host-0, VdsIdV
DSCommandParametersBase:{hostId='1af28f2c-79db-4069-aa53-5bb46528c5e9'})'
execution failed: VDSGenericException: VDSErrorException: Failed to
GetAllVmStatsVDS, error = Internal JSON-RPC error: {'reason':
"'vmType'"}, code = -32603
2017-11-20 07:43:07,676-05 DEBUG
[org.ovirt.engine.core.vdsbroker.vdsbroker.GetAllVmStatsVDSCommand]
(EE-ManagedThreadFactory-engineScheduled-Thread-70) [] Exception:
org.ovirt.engine.core.vdsbroker.vdsbroker.VDSErrorException: VDSGeneric
Exception: VDSErrorException: Failed to GetAllVmStatsVDS, error =
Internal JSON-RPC error: {'reason': "'vmType'"}, code = -32603
at
org.ovirt.engine.core.vdsbroker.vdsbroker.VdsBrokerCommand.createDefaultConcreteException(VdsBrokerCommand.java:81)
[vdsbroker.jar:]
at
org.ovirt.engine.core.vdsbroker.vdsbroker.BrokerCommandBase.createException(BrokerCommandBase.java:223)
[vdsbroker.jar:]
at
org.ovirt.engine.core.vdsbroker.vdsbroker.BrokerCommandBase.proceedProxyReturnValue(BrokerCommandBase.java:193)
[vdsbroker.jar:]
at
org.ovirt.engine.core.vdsbroker.vdsbroker.GetAllVmStatsVDSCommand.executeVdsBrokerCommand(GetAllVmStatsVDSCommand.java:23)
[vdsbroker.jar:]
at
org.ovirt.engine.core.vdsbroker.vdsbroker.VdsBrokerCommand.executeVDSCommand(VdsBrokerCommand.java:112)
[vdsbroker.jar:]
at
org.ovirt.engine.core.vdsbroker.VDSCommandBase.executeCommand(VDSCommandBase.java:73)
[vdsbroker.jar:]
at
org.ovirt.engine.core.dal.VdcCommandBase.execute(VdcCommandBase.java:33)
[dal.jar:]
at
org.ovirt.engine.core.vdsbroker.vdsbroker.DefaultVdsCommandExecutor.execute(DefaultVdsCommandExecutor.java:14)
[vdsbroker.jar:]
at
org.ovirt.engine.core.vdsbroker.ResourceManager.runVdsCommand(ResourceManager.java:387)
[vdsbroker.jar:]
at
org.ovirt.engine.core.vdsbroker.ResourceManager$Proxy$_$$_WeldSubclass.runVdsCommand$$super(Unknown
Source) [vdsbroker.jar:]
at sun.reflect.GeneratedMethodAccessor247.invoke(Unknown Source)
[:1.8.0_151]
at
sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:43)
[rt.jar:1.8.0_151]
at java.lang.reflect.Method.invoke(Method.java:498)
[rt.jar:1.8.0_151]
at
org.jboss.weld.interceptor.proxy.TerminalAroundInvokeInvocationContext.proceedInternal(TerminalAroundInvokeInvocationContext.java:49)
[weld-core-impl-2.4.3.Final.jar:2.4.3.Final]
at
org.jboss.weld.interceptor.proxy.AroundInvokeInvocationContext.proceed(AroundInvokeInvocationContext.java:77)
[weld-core-impl-2.4.3.Final.jar:2.4.3.Final]
at
org.ovirt.engine.core.common.di.interceptor.LoggingInterceptor.apply(LoggingInterceptor.java:12)
[common.jar:]
at sun.reflect.GeneratedMethodAccessor69.invoke(Unknown Source)
[:1.8.0_151]
at
sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:43)
[rt.jar:1.8.0_151]
at java.lang.reflect.Method.invoke(Method.java:498)
[rt.jar:1.8.0_151]
at
org.jboss.weld.interceptor.reader.SimpleInterceptorInvocation$SimpleMethodInvocation.invoke(SimpleInterceptorInvocation.java:73)
[weld-core-impl-2.4.3.Final.jar:2.4.3.Final]
at
org.jboss.weld.interceptor.proxy.InterceptorMethodHandler.executeAroundInvoke(InterceptorMethodHandler.java:84)
[weld-core-impl-2.4.3.Final.jar:2.4.3.Final]
at
org.jboss.weld.interceptor.proxy.InterceptorMethodHandler.executeInterception(InterceptorMethodHandler.java:72)
[weld-core-impl-2.4.3.Final.jar:2.4.3.Final]
at
org.jboss.weld.interceptor.proxy.InterceptorMethodHandler.invoke(InterceptorMethodHandler.java:56)
[weld-core-impl-2.4.3.Final.jar:2.4.3.Final]
at
org.jboss.weld.bean.proxy.CombinedInterceptorAndDecoratorStackMethodHandler.invoke(CombinedInterceptorAndDecoratorStackMethodHandler.java:79)
[weld-core-impl-2.4.3.Final.jar:2.4.3.Final]
at
org.jboss.weld.bean.proxy.CombinedInterceptorAndDecoratorStackMethodHandler.invoke(CombinedInterceptorAndDecoratorStackMethodHandler.java:68)
[weld-core-impl-2.4.3.Final.jar:2.4.3.Final]
at
org.ovirt.engine.core.vdsbroker.ResourceManager$Proxy$_$$_WeldSubclass.runVdsCommand(Unknown
Source) [vdsbroker.jar:]
at
org.ovirt.engine.core.vdsbroker.monitoring.VmsStatisticsFetcher.poll(VmsStatisticsFetcher.java:29)
[vdsbroker.jar:]
at
org.ovirt.engine.core.vdsbroker.monitoring.VmsListFetcher.fetch(VmsListFetcher.java:57)
[vdsbroker.jar:]
at
org.ovirt.engine.core.vdsbroker.monitoring.PollVmStatsRefresher.poll(PollVmStatsRefresher.java:42)
[vdsbroker.jar:]
at
java.util.concurrent.Executors$RunnableAdapter.call(Executors.java:511)
[rt.jar:1.8.0_151]
at
java.util.concurrent.FutureTask.runAndReset(FutureTask.java:308)
[rt.jar:1.8.0_151]
at
org.glassfish.enterprise.concurrent.internal.ManagedScheduledThreadPoolExecutor$ManagedScheduledFutureTask.access$201(ManagedScheduledThreadPoolExecutor.java:383)
[javax.enterprise.concurrent-1.0.jar:]
at
org.glassfish.enterprise.concurrent.internal.ManagedScheduledThreadPoolExecutor$ManagedScheduledFutureTask.run(ManagedScheduledThreadPoolExecutor.java:534)
[javax.enterprise.concurrent-1.0.jar:]
at
java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1149)
[rt.jar:1.8.0_151]
at
java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:624)
[rt.jar:1.8.0_151]
at java.lang.Thread.run(Thread.java:748) [rt.jar:1.8.0_151]
at
org.glassfish.enterprise.concurrent.ManagedThreadFactoryImpl$ManagedThread.run(ManagedThreadFactoryImpl.java:250)
[javax.enterprise.concurrent-1.0.jar:]
at
org.jboss.as.ee.concurrent.service.ElytronManagedThreadFactory$ElytronManagedThread.run(ElytronManagedThreadFactory.java:78)
*
*</error>*
**
--------------2EEBD942435F80A2268AD3F9
Content-Type: text/html; charset=utf-8
Content-Transfer-Encoding: 8bit
<html>
<head>
<meta http-equiv="content-type" content="text/html; charset=utf-8">
</head>
<body text="#000000" bgcolor="#FFFFFF">
<p>Hi, <br>
</p>
<p>We have a failure in OST on test 004_basic_sanity.vm_run. <br>
</p>
<p>it seems to be an error in vm type which is related to the patch
reported. <br>
</p>
<p><br>
</p>
<p><b style="font-weight:normal;"
id="docs-internal-guid-5859b7a1-d9a9-4242-5122-975989463d16">
<p dir="ltr"
style="line-height:1.38;margin-top:0pt;margin-bottom:0pt;"><span style="font-size:11pt;font-family:Arial;color:#000000;background-color:transparent;font-weight:400;font-style:normal;font-variant:normal;text-decoration:none;vertical-align:baseline;white-space:pre-wrap;">Link to suspected patches: <a class="moz-txt-link-freetext" href="https://gerrit.ovirt.org/#/c/84343/">https://gerrit.ovirt.org/#/c/84343/</a></span></p>
<br>
<p dir="ltr"
style="line-height:1.38;margin-top:0pt;margin-bottom:0pt;"><span style="font-size:11pt;font-family:Arial;color:#000000;background-color:transparent;font-weight:400;font-style:normal;font-variant:normal;text-decoration:none;vertical-align:baseline;white-space:pre-wrap;">Link to Job: <a class="moz-txt-link-freetext" href="http://jenkins.ovirt.org/job/ovirt-master_change-queue-tester/3922">http://jenkins.ovirt.org/job/ovirt-master_change-queue-tester/3922</a></span></p>
<br>
<p dir="ltr"
style="line-height:1.38;margin-top:0pt;margin-bottom:0pt;"><span style="font-size:11pt;font-family:Arial;color:#000000;background-color:transparent;font-weight:400;font-style:normal;font-variant:normal;text-decoration:none;vertical-align:baseline;white-space:pre-wrap;">Link to all logs: <a class="moz-txt-link-freetext" href="http://jenkins.ovirt.org/job/ovirt-master_change-queue-tester/3922/artifact">http://jenkins.ovirt.org/job/ovirt-master_change-queue-tester/3922/artifact</a></span></p>
<br>
<p dir="ltr"
style="line-height:1.38;margin-top:0pt;margin-bottom:0pt;"><span style="font-size:11pt;font-family:Arial;color:#000000;background-color:transparent;font-weight:400;font-style:normal;font-variant:normal;text-decoration:none;vertical-align:baseline;white-space:pre-wrap;">(Relevant) error snippet from the log: </span></p>
<p dir="ltr"
style="line-height:1.38;margin-top:0pt;margin-bottom:0pt;"><span style="font-size:11pt;font-family:Arial;color:#000000;background-color:transparent;font-weight:400;font-style:normal;font-variant:normal;text-decoration:none;vertical-align:baseline;white-space:pre-wrap;"><error></span></p>
<br>
vdsm log: <br>
</b></p>
<p><b style="font-weight:normal;"
id="docs-internal-guid-5859b7a1-d9a9-4242-5122-975989463d16">2017-11-20
07:40:12,779-0500 ERROR (jsonrpc/2) [jsonrpc.JsonRpcServer]
Internal server error (__init__:611)<br>
Traceback (most recent call last):<br>
 File "/usr/lib/python2.7/site-packages/yajsonrpc/__init__.py",
line 606, in _handle_request<br>
   res = method(**params)<br>
 File "/usr/lib/python2.7/site-packages/vdsm/rpc/Bridge.py",
line 201, in _dynamicMethod<br>
   result = fn(*methodArgs)<br>
 File "<string>", line 2, in getAllVmStats<br>
 File "/usr/lib/python2.7/site-packages/vdsm/common/api.py",
line 48, in method<br>
   ret = func(*args, **kwargs)<br>
 File "/usr/lib/python2.7/site-packages/vdsm/API.py", line
1341, in getAllVmStats<br>
   statsList = self._cif.getAllVmStats()<br>
 File "/usr/lib/python2.7/site-packages/vdsm/clientIF.py", line
508, in getAllVmStats<br>
   return [v.getStats() for v in self.vmContainer.values()]<br>
 File "/usr/lib/python2.7/site-packages/vdsm/virt/vm.py", line
1664, in getStats<br>
   stats.update(self._getConfigVmStats())<br>
 File "/usr/lib/python2.7/site-packages/vdsm/virt/vm.py", line
1703, in _getConfigVmStats<br>
   'vmType': self.conf['vmType'],<br>
KeyError: 'vmType'<br>
<br>
</b></p>
<p><b style="font-weight:normal;"
id="docs-internal-guid-5859b7a1-d9a9-4242-5122-975989463d16"><br>
</b></p>
<p><span style="font-weight:normal;">engine log:</span></p>
<p><b style="font-weight:normal;"
id="docs-internal-guid-5859b7a1-d9a9-4242-5122-975989463d16"><b></b>2017-11-20
07:43:07,675-05 DEBUG
[org.ovirt.vdsm.jsonrpc.client.internal.ResponseWorker]
(ResponseWorker) [] Message received: {"jsonrpc": "2.0", "id":
"5bf12e5a-4a09-4999-a6ce-a7dd639d3833", "error": {"message":
"Internal JSON-RPC error:<br>
 {'reason': \"'vmType'\"}", "code": -32603}}<br>
2017-11-20 07:43:07,676-05 WARNÂ
[org.ovirt.engine.core.vdsbroker.vdsbroker.GetAllVmStatsVDSCommand]
(EE-ManagedThreadFactory-engineScheduled-Thread-70) []
Unexpected return value: Status [code=-32603, message=Internal
JSON-RPC error: {'r<br>
eason': "'vmType'"}]<br>
2017-11-20 07:43:07,676-05 ERROR
[org.ovirt.engine.core.vdsbroker.vdsbroker.GetAllVmStatsVDSCommand]
(EE-ManagedThreadFactory-engineScheduled-Thread-70) [] Failed in
'GetAllVmStatsVDS' method<br>
2017-11-20 07:43:07,676-05 ERROR
[org.ovirt.engine.core.vdsbroker.vdsbroker.GetAllVmStatsVDSCommand]
(EE-ManagedThreadFactory-engineScheduled-Thread-70) [] Command
'GetAllVmStatsVDSCommand(HostName =
lago-basic-suite-master-host-0, VdsIdV<br>
DSCommandParametersBase:{hostId='1af28f2c-79db-4069-aa53-5bb46528c5e9'})'
execution failed: VDSGenericException: VDSErrorException: Failed
to GetAllVmStatsVDS, error = Internal JSON-RPC error: {'reason':
"'vmType'"}, code = -32603<br>
2017-11-20 07:43:07,676-05 DEBUG
[org.ovirt.engine.core.vdsbroker.vdsbroker.GetAllVmStatsVDSCommand]
(EE-ManagedThreadFactory-engineScheduled-Thread-70) []
Exception:
org.ovirt.engine.core.vdsbroker.vdsbroker.VDSErrorException:
VDSGeneric<br>
Exception: VDSErrorException: Failed to GetAllVmStatsVDS, error
= Internal JSON-RPC error: {'reason': "'vmType'"}, code = -32603<br>
       at
org.ovirt.engine.core.vdsbroker.vdsbroker.VdsBrokerCommand.createDefaultConcreteException(VdsBrokerCommand.java:81)
[vdsbroker.jar:]<br>
       at
org.ovirt.engine.core.vdsbroker.vdsbroker.BrokerCommandBase.createException(BrokerCommandBase.java:223)
[vdsbroker.jar:]<br>
       at
org.ovirt.engine.core.vdsbroker.vdsbroker.BrokerCommandBase.proceedProxyReturnValue(BrokerCommandBase.java:193)
[vdsbroker.jar:]<br>
       at
org.ovirt.engine.core.vdsbroker.vdsbroker.GetAllVmStatsVDSCommand.executeVdsBrokerCommand(GetAllVmStatsVDSCommand.java:23)
[vdsbroker.jar:]<br>
       at
org.ovirt.engine.core.vdsbroker.vdsbroker.VdsBrokerCommand.executeVDSCommand(VdsBrokerCommand.java:112)
[vdsbroker.jar:]<br>
       at
org.ovirt.engine.core.vdsbroker.VDSCommandBase.executeCommand(VDSCommandBase.java:73)
[vdsbroker.jar:]<br>
       at
org.ovirt.engine.core.dal.VdcCommandBase.execute(VdcCommandBase.java:33)
[dal.jar:]<br>
       at
org.ovirt.engine.core.vdsbroker.vdsbroker.DefaultVdsCommandExecutor.execute(DefaultVdsCommandExecutor.java:14)
[vdsbroker.jar:]<br>
       at
org.ovirt.engine.core.vdsbroker.ResourceManager.runVdsCommand(ResourceManager.java:387)
[vdsbroker.jar:]<br>
       at
org.ovirt.engine.core.vdsbroker.ResourceManager$Proxy$_$$_WeldSubclass.runVdsCommand$$super(Unknown
Source) [vdsbroker.jar:]<br>
       at sun.reflect.GeneratedMethodAccessor247.invoke(Unknown
Source) [:1.8.0_151]<br>
       at
sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:43)
[rt.jar:1.8.0_151]<br>
       at java.lang.reflect.Method.invoke(Method.java:498)
[rt.jar:1.8.0_151]<br>
       at
org.jboss.weld.interceptor.proxy.TerminalAroundInvokeInvocationContext.proceedInternal(TerminalAroundInvokeInvocationContext.java:49)
[weld-core-impl-2.4.3.Final.jar:2.4.3.Final]<br>
       at
org.jboss.weld.interceptor.proxy.AroundInvokeInvocationContext.proceed(AroundInvokeInvocationContext.java:77)
[weld-core-impl-2.4.3.Final.jar:2.4.3.Final]<br>
       at
org.ovirt.engine.core.common.di.interceptor.LoggingInterceptor.apply(LoggingInterceptor.java:12)
[common.jar:]<br>
       at sun.reflect.GeneratedMethodAccessor69.invoke(Unknown
Source) [:1.8.0_151]<br>
       at
sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:43)
[rt.jar:1.8.0_151]<br>
       at java.lang.reflect.Method.invoke(Method.java:498)
[rt.jar:1.8.0_151]<br>
       at
org.jboss.weld.interceptor.reader.SimpleInterceptorInvocation$SimpleMethodInvocation.invoke(SimpleInterceptorInvocation.java:73)
[weld-core-impl-2.4.3.Final.jar:2.4.3.Final]<br>
       at
org.jboss.weld.interceptor.proxy.InterceptorMethodHandler.executeAroundInvoke(InterceptorMethodHandler.java:84)
[weld-core-impl-2.4.3.Final.jar:2.4.3.Final]<br>
       at
org.jboss.weld.interceptor.proxy.InterceptorMethodHandler.executeInterception(InterceptorMethodHandler.java:72)
[weld-core-impl-2.4.3.Final.jar:2.4.3.Final]<br>
       at
org.jboss.weld.interceptor.proxy.InterceptorMethodHandler.invoke(InterceptorMethodHandler.java:56)
[weld-core-impl-2.4.3.Final.jar:2.4.3.Final]<br>
       at
org.jboss.weld.bean.proxy.CombinedInterceptorAndDecoratorStackMethodHandler.invoke(CombinedInterceptorAndDecoratorStackMethodHandler.java:79)
[weld-core-impl-2.4.3.Final.jar:2.4.3.Final]<br>
       at
org.jboss.weld.bean.proxy.CombinedInterceptorAndDecoratorStackMethodHandler.invoke(CombinedInterceptorAndDecoratorStackMethodHandler.java:68)
[weld-core-impl-2.4.3.Final.jar:2.4.3.Final]<br>
       at
org.ovirt.engine.core.vdsbroker.ResourceManager$Proxy$_$$_WeldSubclass.runVdsCommand(Unknown
Source) [vdsbroker.jar:]<br>
       at
org.ovirt.engine.core.vdsbroker.monitoring.VmsStatisticsFetcher.poll(VmsStatisticsFetcher.java:29)
[vdsbroker.jar:]<br>
       at
org.ovirt.engine.core.vdsbroker.monitoring.VmsListFetcher.fetch(VmsListFetcher.java:57)
[vdsbroker.jar:]<br>
       at
org.ovirt.engine.core.vdsbroker.monitoring.PollVmStatsRefresher.poll(PollVmStatsRefresher.java:42)
[vdsbroker.jar:]<br>
       at
java.util.concurrent.Executors$RunnableAdapter.call(Executors.java:511)
[rt.jar:1.8.0_151]<br>
       at
java.util.concurrent.FutureTask.runAndReset(FutureTask.java:308)
[rt.jar:1.8.0_151]<br>
       at
org.glassfish.enterprise.concurrent.internal.ManagedScheduledThreadPoolExecutor$ManagedScheduledFutureTask.access$201(ManagedScheduledThreadPoolExecutor.java:383)
[javax.enterprise.concurrent-1.0.jar:]<br>
       at
org.glassfish.enterprise.concurrent.internal.ManagedScheduledThreadPoolExecutor$ManagedScheduledFutureTask.run(ManagedScheduledThreadPoolExecutor.java:534)
[javax.enterprise.concurrent-1.0.jar:]<br>
       at
java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1149)
[rt.jar:1.8.0_151]<br>
       at
java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:624)
[rt.jar:1.8.0_151]<br>
       at java.lang.Thread.run(Thread.java:748)
[rt.jar:1.8.0_151]<br>
       at
org.glassfish.enterprise.concurrent.ManagedThreadFactoryImpl$ManagedThread.run(ManagedThreadFactoryImpl.java:250)
[javax.enterprise.concurrent-1.0.jar:]<br>
       at
org.jboss.as.ee.concurrent.service.ElytronManagedThreadFactory$ElytronManagedThread.run(ElytronManagedThreadFactory.java:78)<br>
<br>
<p dir="ltr"
style="line-height:1.38;margin-top:0pt;margin-bottom:0pt;"><span style="font-size:11pt;font-family:Arial;color:#000000;background-color:transparent;font-weight:400;font-style:normal;font-variant:normal;text-decoration:none;vertical-align:baseline;white-space:pre-wrap;"></error></span></p>
</b><br class="Apple-interchange-newline">
</p>
</body>
</html>
--------------2EEBD942435F80A2268AD3F9--
2
1
[ OST Failure Report ] [ oVirt master ] [ 20-11-1017 ] [ 002_bootstrap.verify_add_all_hosts ]
by Dafna Ron 20 Nov '17
by Dafna Ron 20 Nov '17
20 Nov '17
This is a multi-part message in MIME format.
--------------FBD48B98231B5BEAA6AAA899
Content-Type: text/plain; charset=utf-8
Content-Transfer-Encoding: 7bit
Hi,
We had a failure in OST for test 002_bootstrap.verify_add_all_hosts.
>From the logs I can see that vdsm on host0 was reporting that it cannot
find the physical volume but eventually the storage was created and is
reported as responsive.
However, Host1 is reported to became non-operational with storage domain
does not exist error and I think that there is a race.
I think that we create the storage domain while host1 is being installed
and if the domain is not created and reported as activated in time,
host1 will become nonOperational.
are we starting installation of host1 before host0 and storage are active?
**
*Link to suspected patches: I do not think that the patch reported is
related to the error*
*
**
**
https://gerrit.ovirt.org/#/c/84133/**
Link to Job:
**
http://jenkins.ovirt.org/job/ovirt-master_change-queue-tester/3902/
Link to all logs:
*
*http://jenkins.ovirt.org/job/ovirt-master_change-queue-tester/3902/artifact/
*
*
*
*(Relevant) error snippet from the log: *
*
<error>
Lago log:
*
2017-11-18 11:15:25,472::log_utils.py::end_log_task::670::nose::INFO::
# add_master_storage_domain: ESC[32mSuccessESC[0m (in 0:01:09)
2017-11-18
11:15:25,472::log_utils.py::start_log_task::655::nose::INFO:: #
add_secondary_storage_domains: ESC[0mESC[0m
2017-11-18 11:16:47,455::log_utils.py::end_log_task::670::nose::INFO::
# add_secondary_storage_domains: ESC[32mSuccessESC[0m (in 0:01:21)
2017-11-18
11:16:47,456::log_utils.py::start_log_task::655::nose::INFO:: #
import_templates: ESC[0mESC[0m
2017-11-18 11:16:47,513::testlib.py::stopTest::198::nose::INFO:: *
SKIPPED: Exported domain generation not supported yet
2017-11-18 11:16:47,514::log_utils.py::end_log_task::670::nose::INFO::
# import_templates: ESC[32mSuccessESC[0m (in 0:00:00)
2017-11-18
11:16:47,514::log_utils.py::start_log_task::655::nose::INFO:: #
verify_add_all_hosts: ESC[0mESC[0m
2017-11-18
11:16:47,719::testlib.py::assert_equals_within::227::ovirtlago.testlib::ERROR::
* Unhandled exception in <function <lambda> at 0x2909230>
Traceback (most recent call last):
File "/usr/lib/python2.7/site-packages/ovirtlago/testlib.py", line
219, in assert_equals_within
res = func()
File
"/home/jenkins/workspace/ovirt-master_change-queue-tester/ovirt-system-tests/basic-suite-master/test-scenarios/002_bootstrap.py",
line 430, in <lambda>
lambda: _all_hosts_up(hosts_service, total_hosts)
File
"/home/jenkins/workspace/ovirt-master_change-queue-tester/ovirt-system-tests/basic-suite-master/test-scenarios/002_bootstrap.py",
line 129, in _all_hosts_up
_check_problematic_hosts(hosts_service)
File
"/home/jenkins/workspace/ovirt-master_change-queue-tester/ovirt-system-tests/basic-suite-master/test-scenarios/002_bootstrap.py",
line 149, in _check_problematic_hosts
raise RuntimeError(dump_hosts)
RuntimeError: 1 hosts failed installation:
lago-basic-suite-master-host-1: non_operational
2017-11-18
11:16:47,722::utils.py::wrapper::480::lago.utils::DEBUG::Looking for a
workdir
2017-11-18
11:16:47,722::workdir.py::resolve_workdir_path::361::lago.workdir::DEBUG::Checking
if /dev/shm/ost/deployment-basic-suite-master is a workdir
2017-11-18
11:16:47,724::log_utils.py::__enter__::600::lago.prefix::INFO:: *
Collect artifacts: ESC[0mESC[0m
2017-11-18
11:16:47,724::log_utils.py::__enter__::600::lago.prefix::INFO:: *
Collect artifacts: ESC[0mESC[0m
vdsm host0:
2017-11-18 06:14:23,980-0500 INFO (jsonrpc/0) [vdsm.api] START
getDeviceList(storageType=3,
guids=[u'360014059618895272774e97a2aaf5dd6'], checkStatus=False,
options={}) from=::ffff:192.168.201.4,45636,
flow_id=ed8310a1-a7af-4a67-b351-8ff
364766b8a, task_id=6ced0092-34cd-49f0-aa0f-6aae498af37f (api:46)
2017-11-18 06:14:24,353-0500 WARN (jsonrpc/0) [storage.LVM] lvm pvs
failed: 5 [] [' Failed to find physical volume
"/dev/mapper/360014059618895272774e97a2aaf5dd6".'] (lvm:322)
2017-11-18 06:14:24,353-0500 WARN (jsonrpc/0) [storage.HSM] getPV
failed for guid: 360014059618895272774e97a2aaf5dd6 (hsm:1973)
Traceback (most recent call last):
File "/usr/lib/python2.7/site-packages/vdsm/storage/hsm.py", line
1970, in _getDeviceList
pv = lvm.getPV(guid)
File "/usr/lib/python2.7/site-packages/vdsm/storage/lvm.py", line 852,
in getPV
raise se.InaccessiblePhysDev((pvName,))
InaccessiblePhysDev: Multipath cannot access physical device(s):
"devices=(u'360014059618895272774e97a2aaf5dd6',)"
2017-11-18 06:14:24,389-0500 INFO (jsonrpc/0) [vdsm.api] FINISH
getDeviceList return={'devList': [{'status': 'unknown', 'vendorID':
'LIO-ORG', 'capacity': '21474836480', 'fwrev': '4.0',
'discard_zeroes_data': 0, 'vgUUID': '', 'pvsize': '', 'pathlist':
[{'initiatorname': u'default', 'connection': u'192.168.200.4', 'iqn':
u'iqn.2014-07.org.ovirt:storage', 'portal': '1', 'user': u'username',
'password': '********', 'port': '3260'}, {'initiatorname': u'default',
'connection': u'192.168.201.4', 'iqn': u'iqn.2014-07.org.ovirt:storage',
'portal': '1', 'user': u'username', 'password': '********', 'port':
'3260'}], 'logicalblocksize': '512', 'discard_max_bytes': 1073741824,
'pathstatus': [{'type': 'iSCSI', 'physdev': 'sda', 'capacity':
'21474836480', 'state': 'active', 'lun': '0'}, {'type': 'iSCSI',
'physdev': 'sdf', 'capacity': '21474836480', 'state': 'active', 'lun':
'0'}], 'devtype': 'iSCSI', 'physicalblocksize': '512', 'pvUUID': '',
'serial': 'SLIO-ORG_lun0_bdev_96188952-7277-4e97-a2aa-f5dd6aad6fc2',
'GUID': '360014059618895272774e97a2aaf5dd6', 'productID': 'lun0_bdev'}]}
from=::ffff:192.168.201.4,45636,
flow_id=ed8310a1-a7af-4a67-b351-8ff364766b8a,
task_id=6ced0092-34cd-49f0-aa0f-6aae498af37f (api:52)
2017-11-18 06:14:31,788-0500 INFO (jsonrpc/0) [vdsm.api] FINISH
getStorageDomainInfo return={'info': {'uuid':
'cc61e074-a3b6-4371-9185-66079a39f123', 'vgMetadataDevice':
'360014059618895272774e97a2aaf5dd6', 'vguuid': '7ifbmt-0elj-uWZZ-zS
LG-plA8-8hd3-JG298b', 'metadataDevice':
'360014059618895272774e97a2aaf5dd6', 'state': 'OK', 'version': '4',
'role': 'Regular', 'type': 'ISCSI', 'class': 'Data', 'pool': [], 'name':
'iscsi'}} from=::ffff:192.168.201.4,45636, flow_id=2c1876
99, task_id=c2080b61-d4a5-4bdb-9d75-f81580a8257a (api:
vdsm host1:
2017-11-18 06:16:34,315-0500 ERROR (monitor/c65437c) [storage.Monitor]
Setting up monitor for c65437ce-339f-4b01-aeb5-45c1d486bf49 failed
(monitor:329)
Traceback (most recent call last):
File "/usr/lib/python2.7/site-packages/vdsm/storage/monitor.py", line
326, in _setupLoop
self._setupMonitor()
File "/usr/lib/python2.7/site-packages/vdsm/storage/monitor.py", line
348, in _setupMonitor
self._produceDomain()
File "/usr/lib/python2.7/site-packages/vdsm/utils.py", line 177, in
wrapper
value = meth(self, *a, **kw)
File "/usr/lib/python2.7/site-packages/vdsm/storage/monitor.py", line
366, in _produceDomain
self.domain = sdCache.produce(self.sdUUID)
File "/usr/lib/python2.7/site-packages/vdsm/storage/sdc.py", line 110,
in produce
domain.getRealDomain()
File "/usr/lib/python2.7/site-packages/vdsm/storage/sdc.py", line 51,
in getRealDomain
return self._cache._realProduce(self._sdUUID)
File "/usr/lib/python2.7/site-packages/vdsm/storage/sdc.py", line 134,
in _realProduce
domain = self._findDomain(sdUUID)
File "/usr/lib/python2.7/site-packages/vdsm/storage/sdc.py", line 151,
in _findDomain
return findMethod(sdUUID)
File "/usr/lib/python2.7/site-packages/vdsm/storage/sdc.py", line 176,
in _findUnfetchedDomain
raise se.StorageDomainDoesNotExist(sdUUID)
StorageDomainDoesNotExist: Storage domain does not exist:
(u'c65437ce-339f-4b01-aeb5-45c1d486bf49',)
2017-11-18 06:16:40,377-0500 INFO (jsonrpc/7) [api.host] START
getStats() from=::ffff:192.168.201.4,58722 (api:46)
2017-11-18 06:16:40,378-0500 INFO (jsonrpc/7) [vdsm.api] START
repoStats(domains=()) from=::ffff:192.168.201.4,58722,
task_id=8fb74944-08c0-491e-ad55-a7a9f0a11ef8 (api:46)
2017-11-18 06:16:40,379-0500 INFO (jsonrpc/7) [vdsm.api] FINISH
repoStats return={u'c65437ce-339f-4b01-aeb5-45c1d486bf49': {'code': 358,
'actual': True, 'version': -1, 'acquired': False, 'delay': '0',
'lastCheck': '6.1', 'valid': False},
u'cc61e074-a3b6-4371-9185-66079a39f123': {'code': 0, 'actual': True,
'version': 4, 'acquired': True, 'delay': '0.00103987', 'lastCheck':
'6.5', 'valid': True}} from=::ffff:192.168.201.4,58722,
task_id=8fb74944-08c0-491e-ad55-a7a9f0a11ef8
(api:52)
engine log:
2017-11-18 06:15:54,040-05 ERROR
[org.ovirt.engine.core.vdsbroker.irsbroker.IrsProxy]
(EE-ManagedThreadFactory-engine-Thread-29) [4ce8aff3] Domain
'c65437ce-339f-4b01-aeb5-45c1d486bf49:nfs' was reported with error code
'358'
2017-11-18 06:15:54,041-05 ERROR
[org.ovirt.engine.core.bll.InitVdsOnUpCommand]
(EE-ManagedThreadFactory-engine-Thread-29) [4ce8aff3] Storage Domain
'nfs' of pool 'test-dc' is in problem in host
'lago-basic-suite-master-host-1'
2017-11-18 06:15:54,045-05 ERROR
[org.ovirt.engine.core.dal.dbbroker.auditloghandling.AuditLogDirector]
(EE-ManagedThreadFactory-engine-Thread-29) [4ce8aff3] EVENT_ID:
VDS_STORAGE_VDS_STATS_FAILED(189), Host lago-basic-suite-master-host-1
reports about one of the Active Storage Domains as Problematic.**
**
***
*
*</error>*
**
--------------FBD48B98231B5BEAA6AAA899
Content-Type: text/html; charset=utf-8
Content-Transfer-Encoding: 8bit
<html>
<head>
<meta http-equiv="content-type" content="text/html; charset=utf-8">
</head>
<body text="#000000" bgcolor="#FFFFFF">
<p>Hi, <br>
</p>
<p>We had a failure in OST for test
002_bootstrap.verify_add_all_hosts. <br>
</p>
<p>From the logs I can see that vdsm on host0 was reporting that it
cannot find the physical volume but eventually the storage was
created and is reported as responsive. <br>
</p>
<p>However, Host1 is reported to became non-operational with storage
domain does not exist error and I think that there is a race. <br>
</p>
<p>I think that we create the storage domain while host1 is being
installed and if the domain is not created and reported as
activated in time, host1 will become nonOperational. <br>
</p>
<p>are we starting installation of host1 before host0 and storage
are active? <br>
</p>
<p><b style="font-weight:normal;"
id="docs-internal-guid-5859b7a1-d974-a2c9-3d0d-cb5378c92f81">
<p dir="ltr"
style="line-height:1.38;margin-top:0pt;margin-bottom:0pt;"><span style="font-size:11pt;font-family:Arial;color:#000000;background-color:transparent;font-weight:400;font-style:normal;font-variant:normal;text-decoration:none;vertical-align:baseline;white-space:pre-wrap;">Link to suspected patches: I do not think that the patch reported is related to the error</span></p>
<p dir="ltr"
style="line-height:1.38;margin-top:0pt;margin-bottom:0pt;"><span style="font-size:11pt;font-family:Arial;color:#000000;background-color:transparent;font-weight:400;font-style:normal;font-variant:normal;text-decoration:none;vertical-align:baseline;white-space:pre-wrap;"><b>
</b></span></p>
</b><b style="font-weight:normal;"
id="docs-internal-guid-5859b7a1-d974-a2c9-3d0d-cb5378c92f81">
<p dir="ltr"
style="line-height:1.38;margin-top:0pt;margin-bottom:0pt;"><span style="font-size:11pt;font-family:Arial;color:#000000;background-color:transparent;font-weight:400;font-style:normal;font-variant:normal;text-decoration:none;vertical-align:baseline;white-space:pre-wrap;"><a class="moz-txt-link-freetext" href="https://gerrit.ovirt.org/#/c/84133/">https://gerrit.ovirt.org/#/c/84133/</a><b>
</b></span></p>
<br>
<p dir="ltr"
style="line-height:1.38;margin-top:0pt;margin-bottom:0pt;"><span style="font-size:11pt;font-family:Arial;color:#000000;background-color:transparent;font-weight:400;font-style:normal;font-variant:normal;text-decoration:none;vertical-align:baseline;white-space:pre-wrap;">Link to Job:</span></p>
<p dir="ltr"
style="line-height:1.38;margin-top:0pt;margin-bottom:0pt;"><span style="font-size:11pt;font-family:Arial;color:#000000;background-color:transparent;font-weight:400;font-style:normal;font-variant:normal;text-decoration:none;vertical-align:baseline;white-space:pre-wrap;">
</span></p>
</b><b style="font-weight:normal;"
id="docs-internal-guid-5859b7a1-d974-a2c9-3d0d-cb5378c92f81">
<p dir="ltr"
style="line-height:1.38;margin-top:0pt;margin-bottom:0pt;"><span style="font-size:11pt;font-family:Arial;color:#000000;background-color:transparent;font-weight:400;font-style:normal;font-variant:normal;text-decoration:none;vertical-align:baseline;white-space:pre-wrap;"><a class="moz-txt-link-freetext" href="http://jenkins.ovirt.org/job/ovirt-master_change-queue-tester/3902/">http://jenkins.ovirt.org/job/ovirt-master_change-queue-tester/3902/</a>
</span></p>
<br>
<p dir="ltr"
style="line-height:1.38;margin-top:0pt;margin-bottom:0pt;"><span style="font-size:11pt;font-family:Arial;color:#000000;background-color:transparent;font-weight:400;font-style:normal;font-variant:normal;text-decoration:none;vertical-align:baseline;white-space:pre-wrap;">Link to all logs:</span></p>
<br>
</b></p>
<p><b style="font-weight:normal;"
id="docs-internal-guid-5859b7a1-d974-a2c9-3d0d-cb5378c92f81"><a class="moz-txt-link-freetext" href="http://jenkins.ovirt.org/job/ovirt-master_change-queue-tester/3902/artifact/">http://jenkins.ovirt.org/job/ovirt-master_change-queue-tester/3902/artifact/</a><br>
</b></p>
<p><b style="font-weight:normal;"
id="docs-internal-guid-5859b7a1-d974-a2c9-3d0d-cb5378c92f81"><br>
<p dir="ltr"
style="line-height:1.38;margin-top:0pt;margin-bottom:0pt;"><span style="font-size:11pt;font-family:Arial;color:#000000;background-color:transparent;font-weight:400;font-style:normal;font-variant:normal;text-decoration:none;vertical-align:baseline;white-space:pre-wrap;">(Relevant) error snippet from the log: </span></p>
<p dir="ltr"
style="line-height:1.38;margin-top:0pt;margin-bottom:0pt;"><span style="font-size:11pt;font-family:Arial;color:#000000;background-color:transparent;font-weight:400;font-style:normal;font-variant:normal;text-decoration:none;vertical-align:baseline;white-space:pre-wrap;"><error></span></p>
<br>
Lago log: <br>
</b></p>
<p><span style="font-weight:normal;">2017-11-18
11:15:25,472::log_utils.py::end_log_task::670::nose::INFO::Â #
add_master_storage_domain: ESC[32mSuccessESC[0m (in 0:01:09)<br>
2017-11-18
11:15:25,472::log_utils.py::start_log_task::655::nose::INFO::Â #
add_secondary_storage_domains: ESC[0mESC[0m<br>
2017-11-18
11:16:47,455::log_utils.py::end_log_task::670::nose::INFO::Â #
add_secondary_storage_domains: ESC[32mSuccessESC[0m (in 0:01:21)<br>
2017-11-18
11:16:47,456::log_utils.py::start_log_task::655::nose::INFO::Â #
import_templates: ESC[0mESC[0m<br>
2017-11-18
11:16:47,513::testlib.py::stopTest::198::nose::INFO::Â Â Â *
SKIPPED: Exported domain generation not supported yet<br>
2017-11-18
11:16:47,514::log_utils.py::end_log_task::670::nose::INFO::Â #
import_templates: ESC[32mSuccessESC[0m (in 0:00:00)<br>
2017-11-18
11:16:47,514::log_utils.py::start_log_task::655::nose::INFO::Â #
verify_add_all_hosts: ESC[0mESC[0m<br>
2017-11-18
11:16:47,719::testlib.py::assert_equals_within::227::ovirtlago.testlib::ERROR::Â Â Â
* Unhandled exception in <function <lambda> at
0x2909230><br>
Traceback (most recent call last):<br>
 File "/usr/lib/python2.7/site-packages/ovirtlago/testlib.py",
line 219, in assert_equals_within<br>
   res = func()<br>
 File
"/home/jenkins/workspace/ovirt-master_change-queue-tester/ovirt-system-tests/basic-suite-master/test-scenarios/002_bootstrap.py",
line 430, in <lambda><br>
   lambda: _all_hosts_up(hosts_service, total_hosts)<br>
 File
"/home/jenkins/workspace/ovirt-master_change-queue-tester/ovirt-system-tests/basic-suite-master/test-scenarios/002_bootstrap.py",
line 129, in _all_hosts_up<br>
   _check_problematic_hosts(hosts_service)<br>
 File
"/home/jenkins/workspace/ovirt-master_change-queue-tester/ovirt-system-tests/basic-suite-master/test-scenarios/002_bootstrap.py",
line 149, in _check_problematic_hosts<br>
   raise RuntimeError(dump_hosts)<br>
RuntimeError: 1 hosts failed installation:<br>
lago-basic-suite-master-host-1: non_operational<br>
<br>
2017-11-18
11:16:47,722::utils.py::wrapper::480::lago.utils::DEBUG::Looking
for a workdir<br>
2017-11-18
11:16:47,722::workdir.py::resolve_workdir_path::361::lago.workdir::DEBUG::Checking
if /dev/shm/ost/deployment-basic-suite-master is a workdir<br>
2017-11-18
11:16:47,724::log_utils.py::__enter__::600::lago.prefix::INFO::Â Â Â
* Collect artifacts: ESC[0mESC[0m<br>
2017-11-18
11:16:47,724::log_utils.py::__enter__::600::lago.prefix::INFO::Â Â Â
* Collect artifacts: ESC[0mESC[0m<br>
</span></p>
<p><span style="font-weight:normal;">vdsm host0: <br>
</span></p>
<p><span style="font-weight:normal;">2017-11-18 06:14:23,980-0500
INFOÂ (jsonrpc/0) [vdsm.api] START getDeviceList(storageType=3,
guids=[u'360014059618895272774e97a2aaf5dd6'], checkStatus=False,
options={}) from=::ffff:192.168.201.4,45636,
flow_id=ed8310a1-a7af-4a67-b351-8ff<br>
364766b8a, task_id=6ced0092-34cd-49f0-aa0f-6aae498af37f (api:46)<br>
2017-11-18 06:14:24,353-0500 WARNÂ (jsonrpc/0) [storage.LVM] lvm
pvs failed: 5 [] ['Â Failed to find physical volume
"/dev/mapper/360014059618895272774e97a2aaf5dd6".'] (lvm:322)<br>
2017-11-18 06:14:24,353-0500 WARNÂ (jsonrpc/0) [storage.HSM]
getPV failed for guid: 360014059618895272774e97a2aaf5dd6
(hsm:1973)<br>
Traceback (most recent call last):<br>
 File "/usr/lib/python2.7/site-packages/vdsm/storage/hsm.py",
line 1970, in _getDeviceList<br>
   pv = lvm.getPV(guid)<br>
 File "/usr/lib/python2.7/site-packages/vdsm/storage/lvm.py",
line 852, in getPV<br>
   raise se.InaccessiblePhysDev((pvName,))<br>
InaccessiblePhysDev: Multipath cannot access physical device(s):
"devices=(u'360014059618895272774e97a2aaf5dd6',)"<br>
2017-11-18 06:14:24,389-0500 INFOÂ (jsonrpc/0) [vdsm.api] FINISH
getDeviceList return={'devList': [{'status': 'unknown',
'vendorID': 'LIO-ORG', 'capacity': '21474836480', 'fwrev':
'4.0', 'discard_zeroes_data': 0, 'vgUUID': '', 'pvsize': '',
'pathlist': [{'initiatorname': u'default', 'connection':
u'192.168.200.4', 'iqn': u'iqn.2014-07.org.ovirt:storage',
'portal': '1', 'user': u'username', 'password': '********',
'port': '3260'}, {'initiatorname': u'default', 'connection':
u'192.168.201.4', 'iqn': u'iqn.2014-07.org.ovirt:storage',
'portal': '1', 'user': u'username', 'password': '********',
'port': '3260'}], 'logicalblocksize': '512',
'discard_max_bytes': 1073741824, 'pathstatus': [{'type':
'iSCSI', 'physdev': 'sda', 'capacity': '21474836480', 'state':
'active', 'lun': '0'}, {'type': 'iSCSI', 'physdev': 'sdf',
'capacity': '21474836480', 'state': 'active', 'lun': '0'}],
'devtype': 'iSCSI', 'physicalblocksize': '512', 'pvUUID': '',
'serial':
'SLIO-ORG_lun0_bdev_96188952-7277-4e97-a2aa-f5dd6aad6fc2',
'GUID': '360014059618895272774e97a2aaf5dd6', 'productID':
'lun0_bdev'}]} from=::ffff:192.168.201.4,45636,
flow_id=ed8310a1-a7af-4a67-b351-8ff364766b8a,
task_id=6ced0092-34cd-49f0-aa0f-6aae498af37f (api:52)</span></p>
<p><span style="font-weight:normal;"><br>
</span></p>
<p><span style="font-weight:normal;">2017-11-18 06:14:31,788-0500
INFOÂ (jsonrpc/0) [vdsm.api] FINISH getStorageDomainInfo
return={'info': {'uuid': 'cc61e074-a3b6-4371-9185-66079a39f123',
'vgMetadataDevice': '360014059618895272774e97a2aaf5dd6',
'vguuid': '7ifbmt-0elj-uWZZ-zS<br>
LG-plA8-8hd3-JG298b', 'metadataDevice':
'360014059618895272774e97a2aaf5dd6', 'state': 'OK', 'version':
'4', 'role': 'Regular', 'type': 'ISCSI', 'class': 'Data',
'pool': [], 'name': 'iscsi'}} from=::ffff:192.168.201.4,45636,
flow_id=2c1876<br>
99, task_id=c2080b61-d4a5-4bdb-9d75-f81580a8257a (api:<br>
</span></p>
<p><span style="font-weight:normal;">vdsm host1:</span></p>
<p><span style="font-weight:normal;">2017-11-18 06:16:34,315-0500
ERROR (monitor/c65437c) [storage.Monitor] Setting up monitor for
c65437ce-339f-4b01-aeb5-45c1d486bf49 failed (monitor:329)<br>
Traceback (most recent call last):<br>
 File
"/usr/lib/python2.7/site-packages/vdsm/storage/monitor.py", line
326, in _setupLoop<br>
   self._setupMonitor()<br>
 File
"/usr/lib/python2.7/site-packages/vdsm/storage/monitor.py", line
348, in _setupMonitor<br>
   self._produceDomain()<br>
 File "/usr/lib/python2.7/site-packages/vdsm/utils.py", line
177, in wrapper<br>
   value = meth(self, *a, **kw)<br>
 File
"/usr/lib/python2.7/site-packages/vdsm/storage/monitor.py", line
366, in _produceDomain<br>
   self.domain = sdCache.produce(self.sdUUID)<br>
 File "/usr/lib/python2.7/site-packages/vdsm/storage/sdc.py",
line 110, in produce<br>
   domain.getRealDomain()<br>
 File "/usr/lib/python2.7/site-packages/vdsm/storage/sdc.py",
line 51, in getRealDomain<br>
   return self._cache._realProduce(self._sdUUID)<br>
 File "/usr/lib/python2.7/site-packages/vdsm/storage/sdc.py",
line 134, in _realProduce<br>
   domain = self._findDomain(sdUUID)<br>
 File "/usr/lib/python2.7/site-packages/vdsm/storage/sdc.py",
line 151, in _findDomain<br>
   return findMethod(sdUUID)<br>
 File "/usr/lib/python2.7/site-packages/vdsm/storage/sdc.py",
line 176, in _findUnfetchedDomain<br>
   raise se.StorageDomainDoesNotExist(sdUUID)<br>
StorageDomainDoesNotExist: Storage domain does not exist:
(u'c65437ce-339f-4b01-aeb5-45c1d486bf49',)<br>
2017-11-18 06:16:40,377-0500 INFOÂ (jsonrpc/7) [api.host] START
getStats() from=::ffff:192.168.201.4,58722 (api:46)<br>
2017-11-18 06:16:40,378-0500 INFOÂ (jsonrpc/7) [vdsm.api] START
repoStats(domains=()) from=::ffff:192.168.201.4,58722,
task_id=8fb74944-08c0-491e-ad55-a7a9f0a11ef8 (api:46)<br>
2017-11-18 06:16:40,379-0500 INFOÂ (jsonrpc/7) [vdsm.api] FINISH
repoStats return={u'c65437ce-339f-4b01-aeb5-45c1d486bf49':
{'code': 358, 'actual': True, 'version': -1, 'acquired': False,
'delay': '0', 'lastCheck': '6.1', 'valid': False},<br>
 u'cc61e074-a3b6-4371-9185-66079a39f123': {'code': 0, 'actual':
True, 'version': 4, 'acquired': True, 'delay': '0.00103987',
'lastCheck': '6.5', 'valid': True}}
from=::ffff:192.168.201.4,58722,
task_id=8fb74944-08c0-491e-ad55-a7a9f0a11ef8<br>
 (api:52)<br>
</span></p>
<p><span style="font-weight:normal;">engine log: <br>
</span></p>
<p><span style="font-weight:normal;">2017-11-18 06:15:54,040-05
ERROR [org.ovirt.engine.core.vdsbroker.irsbroker.IrsProxy]
(EE-ManagedThreadFactory-engine-Thread-29) [4ce8aff3] Domain
'c65437ce-339f-4b01-aeb5-45c1d486bf49:nfs' was reported with
error code '358'<br>
2017-11-18 06:15:54,041-05 ERROR
[org.ovirt.engine.core.bll.InitVdsOnUpCommand]
(EE-ManagedThreadFactory-engine-Thread-29) [4ce8aff3] Storage
Domain 'nfs' of pool 'test-dc' is in problem in host
'lago-basic-suite-master-host-1'<br>
2017-11-18 06:15:54,045-05 ERROR
[org.ovirt.engine.core.dal.dbbroker.auditloghandling.AuditLogDirector]
(EE-ManagedThreadFactory-engine-Thread-29) [4ce8aff3] EVENT_ID:
VDS_STORAGE_VDS_STATS_FAILED(189), Host
lago-basic-suite-master-host-1<br>
 reports about one of the Active Storage Domains as Problematic.</span><b
style="font-weight:normal;"
id="docs-internal-guid-5859b7a1-d974-a2c9-3d0d-cb5378c92f81"><b><br>
</b></b></p>
<p><b style="font-weight:normal;"
id="docs-internal-guid-5859b7a1-d974-a2c9-3d0d-cb5378c92f81"><b></b><br>
<p dir="ltr"
style="line-height:1.38;margin-top:0pt;margin-bottom:0pt;"><span style="font-size:11pt;font-family:Arial;color:#000000;background-color:transparent;font-weight:400;font-style:normal;font-variant:normal;text-decoration:none;vertical-align:baseline;white-space:pre-wrap;"></error></span></p>
</b><br class="Apple-interchange-newline">
</p>
<p><br>
</p>
</body>
</html>
--------------FBD48B98231B5BEAA6AAA899--
2
2
I'm trying to update ioprocess to use subprocess32 instead of cpopen.
The build fail on el7-*
http://jenkins.ovirt.org/job/ioprocess_master_check-patch-el7-x86_64/262/co…
http://jenkins.ovirt.org/job/ioprocess_master_check-patch-el7-ppc64le/32/
I think I'm missing a secret repository.
Who can tell me where subprocess32 is hiding?
Nir
3
4
Hi,
sos 3.5 has been released upstream.
I packaged it for Fedora >= 26 (and waiting karma[fc26][fc27]) being 25
almost dead.
Is there interest in packaging it for CentOS Virt SIG considering CentOS
7.4 is still shipping sos 3.4?
Among the improvements in sos 3.5 there's support for PostgreSQL from SCL,
the same we are using in oVirt 4.2.
Or are we fine waiting until sos 3.5 will be packaged and shipped within
RHEL?
[fc26] https://bodhi.fedoraproject.org/updates/FEDORA-2017-bef483c3a0
[fc27] https://bodhi.fedoraproject.org/updates/FEDORA-2017-4bd9528c50
Thanks,
--
SANDRO BONAZZOLA
ASSOCIATE MANAGER, SOFTWARE ENGINEERING, EMEA ENG VIRTUALIZATION R&D
Red Hat EMEA <https://www.redhat.com/>
<https://red.ht/sig>
TRIED. TESTED. TRUSTED. <https://redhat.com/trusted>
<http://www.teraplan.it/redhat-osd-2017/>
1
0
Hi,
Thanks to everyone who helped with the feature categorization effort
on the ovirt
feature page <https://ovirt.org/develop/release-management/features/>.
We've made significant progress, and thanks also to Eldan's UX work, the
page is looking much better organized.
There are a *few simple things* you can do to help keep the page organized,
and to improve it.
1. Adding a new feature to ovirt.org:
Please add your feature to a suitable category:
- Gluster (The location in the repository is:
/develop/release-management/features/gluster)
- Infra (Same pattern as above)
- Integration
- Metrics
- Network
- Node
- SLA
- Storage
- UX
- VDSM
- Virt
2. ‎To help us all find which release each feature is associated with,
please add a link to your feature under *Features by Release
<https://ovirt.org/develop/release-management/features/#features-by-release>. *
The folder location in the source repository is under:
source/develop/release-management/releases/x.x/feature.html.md
x.x = release version
3. We want to continue to list all *existing features* under the correct
category and under Features by Release
<https://ovirt.org/develop/release-management/features/#features-by-release>
.
You can help by visiting this spreadsheet and completing the missing
feature category and release version info:
https://docs.google.com/spreadsheets/d/1c7Di6xz0N0j0rPFRzogBhErJj7rnTCatAZe…
*Legend*
Green*: *The feature is listed under the correct category and release
*Brown**: *The feature needs to be moved to a different category / link
needs to be added to the feature under the relevant release.
Let me know if you have any questions.
Thanks!
John
--
John Marks
Community Manager & Technical Writer, oVirt
redhat Israel
Cell: +972 52 8644 491
4
6
[ OST Failure Report ] [ oVirt master ] [ 16-11-2017 ] [ 002_bootstrap.add_dc ]
by Dafna Ron 16 Nov '17
by Dafna Ron 16 Nov '17
16 Nov '17
This is a multi-part message in MIME format.
--------------D85A420919595D580C076AD7
Content-Type: text/plain; charset=utf-8
Content-Transfer-Encoding: 7bit
Hi,
we have a CQ failure in add_dc test. I can see at engine long that we
have an issue with 'extension.aaa.jdbc.binding.api.AuthnExtension'
I don't think that the patch reported was the actual cause of the
failure. Yesterday we were failing on missing package ovirt-cockpit-sso
which we added to our repos. and I think perhaps we now exposed an issue
with sso support which was undiscovered before.
**
*Link to suspected patches: *
**
These are the reported patches:
https://gerrit.ovirt.org/#/c/83790/
https://gerrit.ovirt.org/#/c/83779/
*
**
Here is the original patch for the sso support:
https://gerrit.ovirt.org/#/c/82997/
Link to Job:
*http://jenkins.ovirt.org/job/ovirt-master_change-queue-tester/3877/
http://jenkins.ovirt.org/job/ovirt-master_change-queue-tester/3878/
**
*Link to all logs:*
***
http://jenkins.ovirt.org/job/ovirt-master_change-queue-tester/3877/artifact/
http://jenkins.ovirt.org/job/ovirt-master_change-queue-tester/3878/artifact/
(Relevant) error snippet from the log:
<error>
*
2017-11-15 16:50:07,115-05 INFO [org.ovirt.engine.core.extensions.mgr.ExtensionsManager] (ServerService Thread Pool -- 45) [] Loading extension 'internal-authn'
2017-11-15 16:50:07,118-05 INFO [org.ovirt.engine.core.extensions.mgr.ExtensionsManager] (ServerService Thread Pool -- 45) [] Extension 'internal-authn' loaded
2017-11-15 16:50:07,118-05 DEBUG [org.ovirt.engine.core.extensions.mgr.ExtensionsManager.trace."ovirt-engine-extension-aaa-jdbc".authn.internal-authn] (ServerService Thread Pool -- 45) [] Config BEGIN
2017-11-15 16:50:07,118-05 DEBUG [org.ovirt.engine.core.extensions.mgr.ExtensionsManager.trace."ovirt-engine-extension-aaa-jdbc".authn.internal-authn] (ServerService Thread Pool -- 45) [] ovirt.engine.aaa.authn.profile.name: internal
2017-11-15 16:50:07,118-05 DEBUG [org.ovirt.engine.core.extensions.mgr.ExtensionsManager.trace."ovirt-engine-extension-aaa-jdbc".authn.internal-authn] (ServerService Thread Pool -- 45) [] ovirt.engine.extension.provides: org.ovirt.engine.api.extensions.aaa.Authn
2017-11-15 16:50:07,118-05 DEBUG [org.ovirt.engine.core.extensions.mgr.ExtensionsManager.trace."ovirt-engine-extension-aaa-jdbc".authn.internal-authn] (ServerService Thread Pool -- 45) [] ovirt.engine.extension.binding.jbossmodule.class: org.ovirt.engine.extension.aaa.jdbc.binding.api.AuthnExtension
2017-11-15 16:50:07,118-05 DEBUG [org.ovirt.engine.core.extensions.mgr.ExtensionsManager.trace."ovirt-engine-extension-aaa-jdbc".authn.internal-authn] (ServerService Thread Pool -- 45) [] ovirt.engine.aaa.authn.authz.plugin: internal-authz
2017-11-15 16:50:07,118-05 DEBUG [org.ovirt.engine.core.extensions.mgr.ExtensionsManager.trace."ovirt-engine-extension-aaa-jdbc".authn.internal-authn] (ServerService Thread Pool -- 45) [] ovirt.engine.extension.bindings.method: jbossmodule
2017-11-15 16:50:07,118-05 DEBUG [org.ovirt.engine.core.extensions.mgr.ExtensionsManager.trace."ovirt-engine-extension-aaa-jdbc".authn.internal-authn] (ServerService Thread Pool -- 45) [] config.datasource.file: /etc/ovirt-engine/aaa/internal.properties
2017-11-15 16:50:07,118-05 DEBUG [org.ovirt.engine.core.extensions.mgr.ExtensionsManager.trace."ovirt-engine-extension-aaa-jdbc".authn.internal-authn] (ServerService Thread Pool -- 45) [] ovirt.engine.extension.name: internal-authn
2017-11-15 16:50:07,118-05 DEBUG [org.ovirt.engine.core.extensions.mgr.ExtensionsManager.trace."ovirt-engine-extension-aaa-jdbc".authn.internal-authn] (ServerService Thread Pool -- 45) [] ovirt.engine.extension.binding.jbossmodule.module: org.ovirt.engine.extension.aaa.jdbc
2017-11-15 16:50:07,118-05 DEBUG [org.ovirt.engine.core.extensions.mgr.ExtensionsManager.trace."ovirt-engine-extension-aaa-jdbc".authn.internal-authn] (ServerService Thread Pool -- 45) [] Config END
2017-11-15 16:50:07,119-05 INFO [org.ovirt.engine.core.extensions.mgr.ExtensionsManager] (ServerService Thread Pool -- 45) [] Loading extension 'internal-authz'
2017-11-15 16:50:07,120-05 INFO [org.ovirt.engine.core.extensions.mgr.ExtensionsManager] (ServerService Thread Pool -- 45) [] Extension 'internal-authz' loaded
2017-11-15 16:50:07,120-05 DEBUG [org.ovirt.engine.core.extensions.mgr.ExtensionsManager.trace."ovirt-engine-extension-aaa-jdbc".authz.internal-authz] (ServerService Thread Pool -- 45) [] Config BEGIN
2017-11-15 16:50:07,120-05 DEBUG [org.ovirt.engine.core.extensions.mgr.ExtensionsManager.trace."ovirt-engine-extension-aaa-jdbc".authz.internal-authz] (ServerService Thread Pool -- 45) [] ovirt.engine.extension.provides: org.ovirt.engine.api.extensions.aaa.Authz
2017-11-15 16:50:07,120-05 DEBUG [org.ovirt.engine.core.extensions.mgr.ExtensionsManager.trace."ovirt-engine-extension-aaa-jdbc".authz.internal-authz] (ServerService Thread Pool -- 45) [] ovirt.engine.extension.binding.jbossmodule.class: org.ovirt.engine.extension.aaa.jdbc.binding.api.AuthzExtension
2017-11-15 16:50:07,120-05 DEBUG [org.ovirt.engine.core.extensions.mgr.ExtensionsManager.trace."ovirt-engine-extension-aaa-jdbc".authz.internal-authz] (ServerService Thread Pool -- 45) [] ovirt.engine.extension.bindings.method: jbossmodule
2017-11-15 16:50:07,120-05 DEBUG [org.ovirt.engine.core.extensions.mgr.ExtensionsManager.trace."ovirt-engine-extension-aaa-jdbc".authz.internal-authz] (ServerService Thread Pool -- 45) [] config.datasource.file: /etc/ovirt-engine/aaa/internal.properties
2017-11-15 16:50:07,120-05 DEBUG [org.ovirt.engine.core.extensions.mgr.ExtensionsManager.trace."ovirt-engine-extension-aaa-jdbc".authz.internal-authz] (ServerService Thread Pool -- 45) [] ovirt.engine.extension.name: internal-authz
2017-11-15 16:50:07,120-05 DEBUG [org.ovirt.engine.core.extensions.mgr.ExtensionsManager.trace."ovirt-engine-extension-aaa-jdbc".authz.internal-authz] (ServerService Thread Pool -- 45) [] ovirt.engine.extension.binding.jbossmodule.module: org.ovirt.engine.extension.aaa.jdbc
2017-11-15 16:50:07,120-05 DEBUG [org.ovirt.engine.core.extensions.mgr.ExtensionsManager.trace."ovirt-engine-extension-aaa-jdbc".authz.internal-authz] (ServerService Thread Pool -- 45) [] Config END
2017-11-15 16:50:07,121-05 INFO [org.ovirt.engine.core.extensions.mgr.ExtensionsManager] (ServerService Thread Pool -- 45) [] Initializing extension 'internal-authn'
2017-11-15 16:50:07,182-05 ERROR [org.ovirt.engine.extension.aaa.jdbc.binding.api.AuthnExtension] (ServerService Thread Pool -- 45) [] Unexpected Exception invoking: EXTENSION_INITIALIZE[e5ae1b7f-9104-4f23-a444-7b9175ff68d2]
2017-11-15 16:50:07,182-05 DEBUG [org.ovirt.engine.extension.aaa.jdbc.binding.api.AuthnExtension] (ServerService Thread Pool -- 45) [] Exception:: org.postgresql.util.PSQLException: FATAL: the database system is shutting down
at org.postgresql.core.v3.ConnectionFactoryImpl.doAuthentication(ConnectionFactoryImpl.java:398) [postgresql.jar:]
at org.postgresql.core.v3.ConnectionFactoryImpl.openConnectionImpl(ConnectionFactoryImpl.java:173) [postgresql.jar:]
at org.postgresql.core.ConnectionFactory.openConnection(ConnectionFactory.java:64) [postgresql.jar:]
at org.postgresql.jdbc2.AbstractJdbc2Connection.<init>(AbstractJdbc2Connection.java:136) [postgresql.jar:]
at org.postgresql.jdbc3.AbstractJdbc3Connection.<init>(AbstractJdbc3Connection.java:29) [postgresql.jar:]
at org.postgresql.jdbc3g.AbstractJdbc3gConnection.<init>(AbstractJdbc3gConnection.java:21) [postgresql.jar:]
at org.postgresql.jdbc4.AbstractJdbc4Connection.<init>(AbstractJdbc4Connection.java:31) [postgresql.jar:]
at org.postgresql.jdbc4.Jdbc4Connection.<init>(Jdbc4Connection.java:24) [postgresql.jar:]
at org.postgresql.Driver.makeConnection(Driver.java:393) [postgresql.jar:]
at org.postgresql.Driver.connect(Driver.java:267) [postgresql.jar:]
at java.sql.DriverManager.getConnection(DriverManager.java:664) [rt.jar:1.8.0_151]
at java.sql.DriverManager.getConnection(DriverManager.java:247) [rt.jar:1.8.0_151]
at org.ovirt.engine.extension.aaa.jdbc.core.datasource.DataSourceProvider$1.invoke(DataSourceProvider.java:86) [ovirt-engine-extension-aaa-jdbc.jar:]
at com.sun.proxy.$Proxy177.getConnection(Unknown Source)
at org.ovirt.engine.extension.aaa.jdbc.core.datasource.SchemaAwareDataSource.getConnection(SchemaAwareDataSource.java:41) [ovirt-engine-extension-aaa-jdbc.jar:]
at org.ovirt.engine.extension.aaa.jdbc.binding.api.ExtensionUtils.checkDbVersion(ExtensionUtils.java:146) [ovirt-engine-extension-aaa-jdbc.jar:]
at org.ovirt.engine.extension.aaa.jdbc.binding.api.AuthnExtension.doInit(AuthnExtension.java:114) [ovirt-engine-extension-aaa-jdbc.jar:]
at org.ovirt.engine.extension.aaa.jdbc.binding.api.AuthnExtension.invoke(AuthnExtension.java:56) [ovirt-engine-extension-aaa-jdbc.jar:]
at org.ovirt.engine.core.extensions.mgr.ExtensionProxy.invoke(ExtensionProxy.java:49)
at org.ovirt.engine.core.extensions.mgr.ExtensionProxy.invoke(ExtensionProxy.java:73)
at org.ovirt.engine.core.extensions.mgr.ExtensionProxy.invoke(ExtensionProxy.java:109)
at org.ovirt.engine.core.extensions.mgr.ExtensionsManager.initialize(ExtensionsManager.java:315)
at org.ovirt.engine.core.sso.utils.SsoExtensionsManager.initialize(SsoExtensionsManager.java:91)
at org.ovirt.engine.core.sso.utils.SsoExtensionsManager.<init>(SsoExtensionsManager.java:28)
at org.ovirt.engine.core.sso.context.SsoContextListener.contextInitialized(SsoContextListener.java:28)
at io.undertow.servlet.core.ApplicationListeners.contextInitialized(ApplicationListeners.java:187)
at io.undertow.servlet.core.DeploymentManagerImpl$1.call(DeploymentManagerImpl.java:205)
at io.undertow.servlet.core.DeploymentManagerImpl$1.call(DeploymentManagerImpl.java:174)
at io.undertow.servlet.core.ServletRequestContextThreadSetupAction$1.call(ServletRequestContextThreadSetupAction.java:42)
at io.undertow.servlet.core.ContextClassLoaderSetupAction$1.call(ContextClassLoaderSetupAction.java:43)
at org.wildfly.extension.undertow.security.SecurityContextThreadSetupAction.lambda$create$0(SecurityContextThreadSetupAction.java:105)
at org.wildfly.extension.undertow.deployment.UndertowDeploymentInfoService$UndertowThreadSetupAction.lambda$create$0(UndertowDeploymentInfoService.java:1508)
at org.wildfly.extension.undertow.deployment.UndertowDeploymentInfoService$UndertowThreadSetupAction.lambda$create$0(UndertowDeploymentInfoService.java:1508)
at org.wildfly.extension.undertow.deployment.UndertowDeploymentInfoService$UndertowThreadSetupAction.lambda$create$0(UndertowDeploymentInfoService.java:1508)
at io.undertow.servlet.core.DeploymentManagerImpl.deploy(DeploymentManagerImpl.java:239)
at org.wildfly.extension.undertow.deployment.UndertowDeploymentService.startContext(UndertowDeploymentService.java:99)
at org.wildfly.extension.undertow.deployment.UndertowDeploymentService$1.run(UndertowDeploymentService.java:81)
at java.util.concurrent.Executors$RunnableAdapter.call(Executors.java:511) [rt.jar:1.8.0_151]
at java.util.concurrent.FutureTask.run(FutureTask.java:266) [rt.jar:1.8.0_151]
at java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1149) [rt.jar:1.8.0_151]
at java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:624) [rt.jar:1.8.0_151]
at java.lang.Thread.run(Thread.java:748) [rt.jar:1.8.0_151]
at org.jboss.threads.JBossThread.run(JBossThread.java:320)
2017-11-15 16:50:07,185-05 ERROR [org.ovirt.engine.core.extensions.mgr.ExtensionsManager] (ServerService Thread Pool -- 45) [] Error in activating extension 'internal-authn': FATAL: the database system is shutting down
2017-11-15 16:50:07,185-05 DEBUG [org.ovirt.engine.core.extensions.mgr.ExtensionsManager] (ServerService Thread Pool -- 45) [] Class: class org.ovirt.engine.core.extensions.mgr.ExtensionInvokeCommandFailedException
**
*</error>*
**
--------------D85A420919595D580C076AD7
Content-Type: text/html; charset=utf-8
Content-Transfer-Encoding: 7bit
<html>
<head>
<meta http-equiv="content-type" content="text/html; charset=utf-8">
</head>
<body text="#000000" bgcolor="#FFFFFF">
<p>Hi, <br>
</p>
<p>we have a CQ failure in add_dc test. I can see at engine long
that we have an issue with
'extension.aaa.jdbc.binding.api.AuthnExtension'</p>
<p>I don't think that the patch reported was the actual cause of the
failure. Yesterday we were failing on missing package
ovirt-cockpit-sso which we added to our repos. and I think perhaps
we now exposed an issue with sso support which was undiscovered
before. <br>
</p>
<p><br>
</p>
<p><b style="font-weight:normal;"
id="docs-internal-guid-5859b7a1-c45f-74ff-f3ff-ccb59b69f04d">
<p dir="ltr"
style="line-height:1.38;margin-top:0pt;margin-bottom:0pt;"><span style="font-size:11pt;font-family:Arial;color:#000000;background-color:transparent;font-weight:400;font-style:normal;font-variant:normal;text-decoration:none;vertical-align:baseline;white-space:pre-wrap;">Link to suspected patches:
</span></p>
</b><span style="font-weight:normal;">
<p dir="ltr"
style="line-height:1.38;margin-top:0pt;margin-bottom:0pt;"><span style="font-size:11pt;font-family:Arial;color:#000000;background-color:transparent;font-weight:400;font-style:normal;font-variant:normal;text-decoration:none;vertical-align:baseline;white-space:pre-wrap;">These are the reported patches:
</span></p>
<p dir="ltr"
style="line-height:1.38;margin-top:0pt;margin-bottom:0pt;"><span style="font-size:11pt;font-family:Arial;color:#000000;background-color:transparent;font-weight:400;font-style:normal;font-variant:normal;text-decoration:none;vertical-align:baseline;white-space:pre-wrap;"><a class="moz-txt-link-freetext" href="https://gerrit.ovirt.org/#/c/83790/">https://gerrit.ovirt.org/#/c/83790/</a></span></p>
<p dir="ltr"
style="line-height:1.38;margin-top:0pt;margin-bottom:0pt;"><span style="font-size:11pt;font-family:Arial;color:#000000;background-color:transparent;font-weight:400;font-style:normal;font-variant:normal;text-decoration:none;vertical-align:baseline;white-space:pre-wrap;"><a class="moz-txt-link-freetext" href="https://gerrit.ovirt.org/#/c/83779/">https://gerrit.ovirt.org/#/c/83779/</a></span></p>
</span><b style="font-weight:normal;"
id="docs-internal-guid-5859b7a1-c45f-74ff-f3ff-ccb59b69f04d">
<p dir="ltr"
style="line-height:1.38;margin-top:0pt;margin-bottom:0pt;"><span style="font-size:11pt;font-family:Arial;color:#000000;background-color:transparent;font-weight:400;font-style:normal;font-variant:normal;text-decoration:none;vertical-align:baseline;white-space:pre-wrap;"><b>
</b></span></p>
<p dir="ltr"
style="line-height:1.38;margin-top:0pt;margin-bottom:0pt;"><font
face="Arial">Here is the original patch for the sso support:
<br>
</font></p>
<p dir="ltr"
style="line-height:1.38;margin-top:0pt;margin-bottom:0pt;"><font
face="Arial"><a class="moz-txt-link-freetext" href="https://gerrit.ovirt.org/#/c/82997/">https://gerrit.ovirt.org/#/c/82997/</a></font></p>
<br>
<p dir="ltr"
style="line-height:1.38;margin-top:0pt;margin-bottom:0pt;"><span style="font-size:11pt;font-family:Arial;color:#000000;background-color:transparent;font-weight:400;font-style:normal;font-variant:normal;text-decoration:none;vertical-align:baseline;white-space:pre-wrap;">Link to Job:</span></p>
</b><a class="moz-txt-link-freetext"
href="http://jenkins.ovirt.org/job/ovirt-master_change-queue-tester/3877/">http://jenkins.ovirt.org/job/ovirt-master_change-queue-tester/3877/</a><a
class="moz-txt-link-freetext"
href="http://jenkins.ovirt.org/job/ovirt-master_change-queue-tester/3878/"><br>
</a></p>
<p><a class="moz-txt-link-freetext"
href="http://jenkins.ovirt.org/job/ovirt-master_change-queue-tester/3878/">http://jenkins.ovirt.org/job/ovirt-master_change-queue-tester/3878/</a></p>
<p><b style="font-weight:normal;"
id="docs-internal-guid-5859b7a1-c45f-74ff-f3ff-ccb59b69f04d">
<p dir="ltr"
style="line-height:1.38;margin-top:0pt;margin-bottom:0pt;"><span style="font-size:11pt;font-family:Arial;color:#000000;background-color:transparent;font-weight:400;font-style:normal;font-variant:normal;text-decoration:none;vertical-align:baseline;white-space:pre-wrap;">Link to all logs:</span></p>
</b><b style="font-weight:normal;"
id="docs-internal-guid-5859b7a1-c45f-74ff-f3ff-ccb59b69f04d">
<p dir="ltr"
style="line-height:1.38;margin-top:0pt;margin-bottom:0pt;"><span style="font-size:11pt;font-family:Arial;color:#000000;background-color:transparent;font-weight:400;font-style:normal;font-variant:normal;text-decoration:none;vertical-align:baseline;white-space:pre-wrap;"><a class="moz-txt-link-freetext" href="http://jenkins.ovirt.org/job/ovirt-master_change-queue-tester/3877/artifact/">http://jenkins.ovirt.org/job/ovirt-master_change-queue-tester/3877/artifact/</a></span></p>
<p dir="ltr"
style="line-height:1.38;margin-top:0pt;margin-bottom:0pt;"><span style="font-size:11pt;font-family:Arial;color:#000000;background-color:transparent;font-weight:400;font-style:normal;font-variant:normal;text-decoration:none;vertical-align:baseline;white-space:pre-wrap;"><a class="moz-txt-link-freetext" href="http://jenkins.ovirt.org/job/ovirt-master_change-queue-tester/3878/artifact/">http://jenkins.ovirt.org/job/ovirt-master_change-queue-tester/3878/artifact/</a>
</span></p>
<br>
<p dir="ltr"
style="line-height:1.38;margin-top:0pt;margin-bottom:0pt;"><span style="font-size:11pt;font-family:Arial;color:#000000;background-color:transparent;font-weight:400;font-style:normal;font-variant:normal;text-decoration:none;vertical-align:baseline;white-space:pre-wrap;">(Relevant) error snippet from the log: </span></p>
<p dir="ltr"
style="line-height:1.38;margin-top:0pt;margin-bottom:0pt;"><span style="font-size:11pt;font-family:Arial;color:#000000;background-color:transparent;font-weight:400;font-style:normal;font-variant:normal;text-decoration:none;vertical-align:baseline;white-space:pre-wrap;"><error></span></p>
</b></p>
<p><br>
</p>
<pre style="color: rgb(0, 0, 0); font-style: normal; font-variant-ligatures: normal; font-variant-caps: normal; font-weight: normal; letter-spacing: normal; orphans: 2; text-align: start; text-indent: 0px; text-transform: none; widows: 2; word-spacing: 0px; -webkit-text-stroke-width: 0px; text-decoration-style: initial; text-decoration-color: initial;">2017-11-15 16:50:07,115-05 INFO [org.ovirt.engine.core.extensions.mgr.ExtensionsManager] (ServerService Thread Pool -- 45) [] Loading extension 'internal-authn'
2017-11-15 16:50:07,118-05 INFO [org.ovirt.engine.core.extensions.mgr.ExtensionsManager] (ServerService Thread Pool -- 45) [] Extension 'internal-authn' loaded
2017-11-15 16:50:07,118-05 DEBUG [org.ovirt.engine.core.extensions.mgr.ExtensionsManager.trace."ovirt-engine-extension-aaa-jdbc".authn.internal-authn] (ServerService Thread Pool -- 45) [] Config BEGIN
2017-11-15 16:50:07,118-05 DEBUG [org.ovirt.engine.core.extensions.mgr.ExtensionsManager.trace."ovirt-engine-extension-aaa-jdbc".authn.internal-authn] (ServerService Thread Pool -- 45) [] ovirt.engine.aaa.authn.profile.name: internal
2017-11-15 16:50:07,118-05 DEBUG [org.ovirt.engine.core.extensions.mgr.ExtensionsManager.trace."ovirt-engine-extension-aaa-jdbc".authn.internal-authn] (ServerService Thread Pool -- 45) [] ovirt.engine.extension.provides: org.ovirt.engine.api.extensions.aaa.Authn
2017-11-15 16:50:07,118-05 DEBUG [org.ovirt.engine.core.extensions.mgr.ExtensionsManager.trace."ovirt-engine-extension-aaa-jdbc".authn.internal-authn] (ServerService Thread Pool -- 45) [] ovirt.engine.extension.binding.jbossmodule.class: org.ovirt.engine.extension.aaa.jdbc.binding.api.AuthnExtension
2017-11-15 16:50:07,118-05 DEBUG [org.ovirt.engine.core.extensions.mgr.ExtensionsManager.trace."ovirt-engine-extension-aaa-jdbc".authn.internal-authn] (ServerService Thread Pool -- 45) [] ovirt.engine.aaa.authn.authz.plugin: internal-authz
2017-11-15 16:50:07,118-05 DEBUG [org.ovirt.engine.core.extensions.mgr.ExtensionsManager.trace."ovirt-engine-extension-aaa-jdbc".authn.internal-authn] (ServerService Thread Pool -- 45) [] ovirt.engine.extension.bindings.method: jbossmodule
2017-11-15 16:50:07,118-05 DEBUG [org.ovirt.engine.core.extensions.mgr.ExtensionsManager.trace."ovirt-engine-extension-aaa-jdbc".authn.internal-authn] (ServerService Thread Pool -- 45) [] config.datasource.file: /etc/ovirt-engine/aaa/internal.properties
2017-11-15 16:50:07,118-05 DEBUG [org.ovirt.engine.core.extensions.mgr.ExtensionsManager.trace."ovirt-engine-extension-aaa-jdbc".authn.internal-authn] (ServerService Thread Pool -- 45) [] ovirt.engine.extension.name: internal-authn
2017-11-15 16:50:07,118-05 DEBUG [org.ovirt.engine.core.extensions.mgr.ExtensionsManager.trace."ovirt-engine-extension-aaa-jdbc".authn.internal-authn] (ServerService Thread Pool -- 45) [] ovirt.engine.extension.binding.jbossmodule.module: org.ovirt.engine.extension.aaa.jdbc
2017-11-15 16:50:07,118-05 DEBUG [org.ovirt.engine.core.extensions.mgr.ExtensionsManager.trace."ovirt-engine-extension-aaa-jdbc".authn.internal-authn] (ServerService Thread Pool -- 45) [] Config END
2017-11-15 16:50:07,119-05 INFO [org.ovirt.engine.core.extensions.mgr.ExtensionsManager] (ServerService Thread Pool -- 45) [] Loading extension 'internal-authz'
2017-11-15 16:50:07,120-05 INFO [org.ovirt.engine.core.extensions.mgr.ExtensionsManager] (ServerService Thread Pool -- 45) [] Extension 'internal-authz' loaded
2017-11-15 16:50:07,120-05 DEBUG [org.ovirt.engine.core.extensions.mgr.ExtensionsManager.trace."ovirt-engine-extension-aaa-jdbc".authz.internal-authz] (ServerService Thread Pool -- 45) [] Config BEGIN
2017-11-15 16:50:07,120-05 DEBUG [org.ovirt.engine.core.extensions.mgr.ExtensionsManager.trace."ovirt-engine-extension-aaa-jdbc".authz.internal-authz] (ServerService Thread Pool -- 45) [] ovirt.engine.extension.provides: org.ovirt.engine.api.extensions.aaa.Authz
2017-11-15 16:50:07,120-05 DEBUG [org.ovirt.engine.core.extensions.mgr.ExtensionsManager.trace."ovirt-engine-extension-aaa-jdbc".authz.internal-authz] (ServerService Thread Pool -- 45) [] ovirt.engine.extension.binding.jbossmodule.class: org.ovirt.engine.extension.aaa.jdbc.binding.api.AuthzExtension
2017-11-15 16:50:07,120-05 DEBUG [org.ovirt.engine.core.extensions.mgr.ExtensionsManager.trace."ovirt-engine-extension-aaa-jdbc".authz.internal-authz] (ServerService Thread Pool -- 45) [] ovirt.engine.extension.bindings.method: jbossmodule
2017-11-15 16:50:07,120-05 DEBUG [org.ovirt.engine.core.extensions.mgr.ExtensionsManager.trace."ovirt-engine-extension-aaa-jdbc".authz.internal-authz] (ServerService Thread Pool -- 45) [] config.datasource.file: /etc/ovirt-engine/aaa/internal.properties
2017-11-15 16:50:07,120-05 DEBUG [org.ovirt.engine.core.extensions.mgr.ExtensionsManager.trace."ovirt-engine-extension-aaa-jdbc".authz.internal-authz] (ServerService Thread Pool -- 45) [] ovirt.engine.extension.name: internal-authz
2017-11-15 16:50:07,120-05 DEBUG [org.ovirt.engine.core.extensions.mgr.ExtensionsManager.trace."ovirt-engine-extension-aaa-jdbc".authz.internal-authz] (ServerService Thread Pool -- 45) [] ovirt.engine.extension.binding.jbossmodule.module: org.ovirt.engine.extension.aaa.jdbc
2017-11-15 16:50:07,120-05 DEBUG [org.ovirt.engine.core.extensions.mgr.ExtensionsManager.trace."ovirt-engine-extension-aaa-jdbc".authz.internal-authz] (ServerService Thread Pool -- 45) [] Config END
2017-11-15 16:50:07,121-05 INFO [org.ovirt.engine.core.extensions.mgr.ExtensionsManager] (ServerService Thread Pool -- 45) [] Initializing extension 'internal-authn'
2017-11-15 16:50:07,182-05 ERROR [org.ovirt.engine.extension.aaa.jdbc.binding.api.AuthnExtension] (ServerService Thread Pool -- 45) [] Unexpected Exception invoking: EXTENSION_INITIALIZE[e5ae1b7f-9104-4f23-a444-7b9175ff68d2]
2017-11-15 16:50:07,182-05 DEBUG [org.ovirt.engine.extension.aaa.jdbc.binding.api.AuthnExtension] (ServerService Thread Pool -- 45) [] Exception:: org.postgresql.util.PSQLException: FATAL: the database system is shutting down
at org.postgresql.core.v3.ConnectionFactoryImpl.doAuthentication(ConnectionFactoryImpl.java:398) [postgresql.jar:]
at org.postgresql.core.v3.ConnectionFactoryImpl.openConnectionImpl(ConnectionFactoryImpl.java:173) [postgresql.jar:]
at org.postgresql.core.ConnectionFactory.openConnection(ConnectionFactory.java:64) [postgresql.jar:]
at org.postgresql.jdbc2.AbstractJdbc2Connection.<init>(AbstractJdbc2Connection.java:136) [postgresql.jar:]
at org.postgresql.jdbc3.AbstractJdbc3Connection.<init>(AbstractJdbc3Connection.java:29) [postgresql.jar:]
at org.postgresql.jdbc3g.AbstractJdbc3gConnection.<init>(AbstractJdbc3gConnection.java:21) [postgresql.jar:]
at org.postgresql.jdbc4.AbstractJdbc4Connection.<init>(AbstractJdbc4Connection.java:31) [postgresql.jar:]
at org.postgresql.jdbc4.Jdbc4Connection.<init>(Jdbc4Connection.java:24) [postgresql.jar:]
at org.postgresql.Driver.makeConnection(Driver.java:393) [postgresql.jar:]
at org.postgresql.Driver.connect(Driver.java:267) [postgresql.jar:]
at java.sql.DriverManager.getConnection(DriverManager.java:664) [rt.jar:1.8.0_151]
at java.sql.DriverManager.getConnection(DriverManager.java:247) [rt.jar:1.8.0_151]
at org.ovirt.engine.extension.aaa.jdbc.core.datasource.DataSourceProvider$1.invoke(DataSourceProvider.java:86) [ovirt-engine-extension-aaa-jdbc.jar:]
at com.sun.proxy.$Proxy177.getConnection(Unknown Source)
at org.ovirt.engine.extension.aaa.jdbc.core.datasource.SchemaAwareDataSource.getConnection(SchemaAwareDataSource.java:41) [ovirt-engine-extension-aaa-jdbc.jar:]
at org.ovirt.engine.extension.aaa.jdbc.binding.api.ExtensionUtils.checkDbVersion(ExtensionUtils.java:146) [ovirt-engine-extension-aaa-jdbc.jar:]
at org.ovirt.engine.extension.aaa.jdbc.binding.api.AuthnExtension.doInit(AuthnExtension.java:114) [ovirt-engine-extension-aaa-jdbc.jar:]
at org.ovirt.engine.extension.aaa.jdbc.binding.api.AuthnExtension.invoke(AuthnExtension.java:56) [ovirt-engine-extension-aaa-jdbc.jar:]
at org.ovirt.engine.core.extensions.mgr.ExtensionProxy.invoke(ExtensionProxy.java:49)
at org.ovirt.engine.core.extensions.mgr.ExtensionProxy.invoke(ExtensionProxy.java:73)
at org.ovirt.engine.core.extensions.mgr.ExtensionProxy.invoke(ExtensionProxy.java:109)
at org.ovirt.engine.core.extensions.mgr.ExtensionsManager.initialize(ExtensionsManager.java:315)
at org.ovirt.engine.core.sso.utils.SsoExtensionsManager.initialize(SsoExtensionsManager.java:91)
at org.ovirt.engine.core.sso.utils.SsoExtensionsManager.<init>(SsoExtensionsManager.java:28)
at org.ovirt.engine.core.sso.context.SsoContextListener.contextInitialized(SsoContextListener.java:28)
at io.undertow.servlet.core.ApplicationListeners.contextInitialized(ApplicationListeners.java:187)
at io.undertow.servlet.core.DeploymentManagerImpl$1.call(DeploymentManagerImpl.java:205)
at io.undertow.servlet.core.DeploymentManagerImpl$1.call(DeploymentManagerImpl.java:174)
at io.undertow.servlet.core.ServletRequestContextThreadSetupAction$1.call(ServletRequestContextThreadSetupAction.java:42)
at io.undertow.servlet.core.ContextClassLoaderSetupAction$1.call(ContextClassLoaderSetupAction.java:43)
at org.wildfly.extension.undertow.security.SecurityContextThreadSetupAction.lambda$create$0(SecurityContextThreadSetupAction.java:105)
at org.wildfly.extension.undertow.deployment.UndertowDeploymentInfoService$UndertowThreadSetupAction.lambda$create$0(UndertowDeploymentInfoService.java:1508)
at org.wildfly.extension.undertow.deployment.UndertowDeploymentInfoService$UndertowThreadSetupAction.lambda$create$0(UndertowDeploymentInfoService.java:1508)
at org.wildfly.extension.undertow.deployment.UndertowDeploymentInfoService$UndertowThreadSetupAction.lambda$create$0(UndertowDeploymentInfoService.java:1508)
at io.undertow.servlet.core.DeploymentManagerImpl.deploy(DeploymentManagerImpl.java:239)
at org.wildfly.extension.undertow.deployment.UndertowDeploymentService.startContext(UndertowDeploymentService.java:99)
at org.wildfly.extension.undertow.deployment.UndertowDeploymentService$1.run(UndertowDeploymentService.java:81)
at java.util.concurrent.Executors$RunnableAdapter.call(Executors.java:511) [rt.jar:1.8.0_151]
at java.util.concurrent.FutureTask.run(FutureTask.java:266) [rt.jar:1.8.0_151]
at java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1149) [rt.jar:1.8.0_151]
at java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:624) [rt.jar:1.8.0_151]
at java.lang.Thread.run(Thread.java:748) [rt.jar:1.8.0_151]
at org.jboss.threads.JBossThread.run(JBossThread.java:320)
2017-11-15 16:50:07,185-05 ERROR [org.ovirt.engine.core.extensions.mgr.ExtensionsManager] (ServerService Thread Pool -- 45) [] Error in activating extension 'internal-authn': FATAL: the database system is shutting down
2017-11-15 16:50:07,185-05 DEBUG [org.ovirt.engine.core.extensions.mgr.ExtensionsManager] (ServerService Thread Pool -- 45) [] Class: class org.ovirt.engine.core.extensions.mgr.ExtensionInvokeCommandFailedException</pre>
<p><b style="font-weight:normal;"
id="docs-internal-guid-5859b7a1-c45f-74ff-f3ff-ccb59b69f04d">
<p dir="ltr"
style="line-height:1.38;margin-top:0pt;margin-bottom:0pt;"><span style="font-size:11pt;font-family:Arial;color:#000000;background-color:transparent;font-weight:400;font-style:normal;font-variant:normal;text-decoration:none;vertical-align:baseline;white-space:pre-wrap;"></error></span></p>
</b><br class="Apple-interchange-newline">
</p>
</body>
</html>
--------------D85A420919595D580C076AD7--
4
4
[ OST Failure Report ] [ oVirt master ] [ 14-11-2017 ] [ 002_bootstrap.add_hosts ]
by Dafna Ron 15 Nov '17
by Dafna Ron 15 Nov '17
15 Nov '17
This is a multi-part message in MIME format.
--------------5185435FD38913B821CED967
Content-Type: text/plain; charset=utf-8
Content-Transfer-Encoding: 7bit
Hi,
We had a failure in upgrade suite for 002_bootstrap.add_hosts. I am not
seeing any error that can suggest on an issue in engine.
I can see in the host messages host that we have stopped writing to the
log for 15 minutes and it may suggest that there is something that is
keeping the host from starting which causes us to fail the test on timeout.
However, i can use some help in determining the cause for this failure
and weather its connected to the bootstrap_add_host test in upgrade.
**
*Link to suspected patches: As I said,**I do not think its related, but
this is the patch that was reported. *
*
https://gerrit.ovirt.org/#/c/83854/
Link to Job:
http://jenkins.ovirt.org/job/ovirt-master_change-queue-tester/3795/*
*
*
*Link to all logs:*
*http://jenkins.ovirt.org/job/ovirt-master_change-queue-tester/3795/artifact/*
*
*
*(Relevant) error snippet from the log: *
*
<error>
**
_*Test error: *_**
**
*
Error Message
False != True after 900 seconds
Stacktrace
Traceback (most recent call last):
File "/usr/lib64/python2.7/unittest/case.py", line 369, in run
testMethod()
File "/usr/lib/python2.7/site-packages/nose/case.py", line 197, in runTest
self.test(*self.arg)
File "/usr/lib/python2.7/site-packages/ovirtlago/testlib.py", line 129, in wrapped_test
test()
File "/usr/lib/python2.7/site-packages/ovirtlago/testlib.py", line 59, in wrapper
return func(get_test_prefix(), *args, **kwargs)
File "/home/jenkins/workspace/ovirt-master_change-queue-tester/ovirt-system-tests/upgrade-from-release-suite-master/test-scenarios-after-upgrade/002_bootstrap.py", line 187, in add_hosts
testlib.assert_true_within(_host_is_up_4, timeout=15*60)
File "/usr/lib/python2.7/site-packages/ovirtlago/testlib.py", line 263, in assert_true_within
assert_equals_within(func, True, timeout, allowed_exceptions)
File "/usr/lib/python2.7/site-packages/ovirtlago/testlib.py", line 237, in assert_equals_within
'%s != %s after %s seconds' % (res, value, timeout)
AssertionError: False != True after 900 seconds
**
**
**
**_lago log: _*
*
*2017-11-13
15:31:23,212::log_utils.py::__enter__::600::lago.prefix::INFO::ESC[0mESC[0m
2017-11-13
15:31:23,213::log_utils.py::__enter__::600::lago.ssh::DEBUG::start
task:0468ed2f-b174-4d94-bc66-2b6e08087a86:Get ssh client for
lago-upgrade-from-release-suite-master-host0:
2017-11-13
15:31:23,213::log_utils.py::__enter__::600::lago.ssh::DEBUG::start
task:91372413-f7fd-4b72-85b9-9f5216ca7ae9:Get ssh client for
lago-upgrade-from-release-suite-master-engine:
2017-11-13
15:31:23,213::ssh.py::get_ssh_client::339::lago.ssh::DEBUG::Still got 1
tries for lago-upgrade-from-release-suite-master-host0
2017-11-13
15:31:23,213::ssh.py::get_ssh_client::339::lago.ssh::DEBUG::Still got 1
tries for lago-upgrade-from-release-suite-master-engine
2017-11-13
15:31:26,220::ssh.py::get_ssh_client::354::lago.ssh::DEBUG::Socket error
connecting to lago-upgrade-from-release-suite-master-engine: [Errno
None] Unable to connect to port 22 on 192.168.200.3
2017-11-13
15:31:26,221::ssh.py::get_ssh_client::354::lago.ssh::DEBUG::Socket error
connecting to lago-upgrade-from-release-suite-master-host0: [Errno None]
Unable to connect to port 22 on 192.168.200.2
2017-11-13
15:31:27,222::log_utils.py::__exit__::611::lago.ssh::DEBUG::end
task:91372413-f7fd-4b72-85b9-9f5216ca7ae9:Get ssh client for
lago-upgrade-from-release-suite-master-engine:
2017-11-13
15:31:27,222::log_utils.py::__exit__::611::lago.ssh::DEBUG::end
task:0468ed2f-b174-4d94-bc66-2b6e08087a86:Get ssh client for
lago-upgrade-from-release-suite-master-host0:
2017-11-13 15:31:27,222::ssh.py::wait_for_ssh::129::lago.ssh::DEBUG::Got
exception while sshing to lago-upgrade-from-release-suite-master-engine:
Timed out (in 4 s) trying to ssh to
lago-upgrade-from-release-suite-master-engine
2017-11-13 15:31:27,222::ssh.py::wait_for_ssh::129::lago.ssh::DEBUG::Got
exception while sshing to lago-upgrade-from-release-suite-master-host0:
Timed out (in 4 s) trying to ssh to
lago-upgrade-from-release-suite-master-host0
2017-11-13
15:31:28,224::log_utils.py::__enter__::600::lago.ssh::DEBUG::start
task:fc88f641-b012-4636-a471-9ccaaf361a53:Get ssh client for
lago-upgrade-from-release-suite-master-engine:
2017-11-13
15:31:28,224::log_utils.py::__enter__::600::lago.ssh::DEBUG::start
task:afb01a46-6338-407f-b7c9-9d5c6b91404d:Get ssh client for
lago-upgrade-from-release-suite-master-host0:
2017-11-13
15:31:28,224::ssh.py::get_ssh_client::339::lago.ssh::DEBUG::Still got 1
tries for lago-upgrade-from-release-suite-master-engine
2017-11-13
15:31:28,225::ssh.py::get_ssh_client::339::lago.ssh::DEBUG::Still got 1
tries for lago-upgrade-from-release-suite-master-host0
2017-11-13
15:31:28,226::ssh.py::get_ssh_client::354::lago.ssh::DEBUG::Socket error
connecting to lago-upgrade-from-release-suite-master-host0: [Errno None]
Unable to connect to port 22 on 192.168.200.2
2017-11-13
15:31:29,228::log_utils.py::__exit__::611::lago.ssh::DEBUG::end
task:afb01a46-6338-407f-b7c9-9d5c6b91404d:Get ssh client for
lago-upgrade-from-release-suite-master-host0:
2017-11-13 15:31:29,228::ssh.py::wait_for_ssh::129::lago.ssh::DEBUG::Got
exception while sshing to lago-upgrade-from-release-suite-master-host0:
Timed out (in 1 s) trying to ssh to
lago-upgrade-from-release-suite-master-host0
2017-11-13
15:31:29,229::ssh.py::get_ssh_client::354::lago.ssh::DEBUG::Socket error
connecting to lago-upgrade-from-release-suite-master-engine: [Errno
None] Unable to connect to port 22 on 192.168.200.3
2017-11-13
15:31:30,229::log_utils.py::__exit__::611::lago.ssh::DEBUG::end
task:fc88f641-b012-4636-a471-9ccaaf361a53:Get ssh client for
lago-upgrade-from-release-suite-master-engine:
2017-11-13
15:31:30,230::log_utils.py::__enter__::600::lago.ssh::DEBUG::start
task:9df5ddca-bbc4-485a-9f3a-b0b9a5bb5990:Get ssh client for
lago-upgrade-from-release-suite-master-host0:
2017-11-13 15:31:30,230::ssh.py::wait_for_ssh::129::lago.ssh::DEBUG::Got
exception while sshing to lago-upgrade-from-release-suite-master-engine:
Timed out (in 2 s) trying to ssh to
lago-upgrade-from-release-suite-master-engine
2017-11-13
15:31:30,230::ssh.py::get_ssh_client::339::lago.ssh::DEBUG::Still got 1
tries for lago-upgrade-from-release-suite-master-host0
2017-11-13
15:31:30,231::ssh.py::get_ssh_client::354::lago.ssh::DEBUG::Socket error
connecting to lago-upgrade-from-release-suite-master-host0: [Errno None]
Unable to connect to port 22 on 192.168.200.2
2017-11-13
15:31:31,231::log_utils.py::__enter__::600::lago.ssh::DEBUG::start
task:53fe67da-a632-49fe-b697-e8a2c4cb7d23:Get ssh client for
lago-upgrade-from-release-suite-master-engine:
2017-11-13
15:31:31,232::ssh.py::get_ssh_client::339::lago.ssh::DEBUG::Still got 1
tries for lago-upgrade-from-release-suite-master-engine
2017-11-13
15:31:31,232::ssh.py::get_ssh_client::354::lago.ssh::DEBUG::Socket error
connecting to lago-upgrade-from-release-suite-master-engine: [Errno
None] Unable to connect to port 22 on 192.168.200.3
2017-11-13
15:31:31,232::log_utils.py::__exit__::611::lago.ssh::DEBUG::end
task:9df5ddca-bbc4-485a-9f3a-b0b9a5bb5990:Get ssh client for
lago-upgrade-from-release-suite-master-host0:
2017-11-13 15:31:31,233::ssh.py::wait_for_ssh::129::lago.ssh::DEBUG::Got
exception while sshing to lago-upgrade-from-release-suite-master-host0:
Timed out (in 1 s) trying to ssh to
lago-upgrade-from-release-suite-master-host0
2017-11-13
15:31:32,234::log_utils.py::__exit__::611::lago.ssh::DEBUG::end
task:53fe67da-a632-49fe-b697-e8a2c4cb7d23:Get ssh client for
lago-upgrade-from-release-suite-master-engine:
2017-11-13 15:31:32,234::ssh.py::wait_for_ssh::129::lago.ssh::DEBUG::Got
exception while sshing to lago-upgrade-from-release-suite-master-engine:
Timed out (in 1 s) trying to ssh to
lago-upgrade-from-release-suite-master-engine
2017-11-13
15:31:32,234::log_utils.py::__enter__::600::lago.ssh::DEBUG::start
task:1969bf5b-4e66-43d7-91c6-a28949e98fe8:Get ssh client for
lago-upgrade-from-release-suite-master-host0:
2017-11-13
15:31:32,234::ssh.py::get_ssh_client::339::lago.ssh::DEBUG::Still got 1
tries for lago-upgrade-from-release-suite-master-host0
2017-11-13
15:31:32,235::ssh.py::get_ssh_client::354::lago.ssh::DEBUG::Socket error
connecting to lago-upgrade-from-release-suite-master-host0: [Errno None]
Unable to connect to port 22 on 192.168.200.2
2017-11-13
15:31:33,235::log_utils.py::__enter__::600::lago.ssh::DEBUG::start
task:5fd0e3c3-d83c-46f9-9c88-bc739aa9c430:Get ssh client for
lago-upgrade-from-release-suite-master-engine:
2017-11-13
15:31:33,235::ssh.py::get_ssh_client::339::lago.ssh::DEBUG::Still got 1
tries for lago-upgrade-from-release-suite-master-engine
2017-11-13
15:31:33,236::log_utils.py::__exit__::611::lago.ssh::DEBUG::end
task:1969bf5b-4e66-43d7-91c6-a28949e98fe8:Get ssh client for
lago-upgrade-from-release-suite-master-host0:
2017-11-13
15:31:33,236::ssh.py::get_ssh_client::354::lago.ssh::DEBUG::Socket error
connecting to lago-upgrade-from-release-suite-master-engine: [Errno
None] Unable to connect to port 22 on 192.168.200.3
2017-11-13 15:31:33,237::ssh.py::wait_for_ssh::129::lago.ssh::DEBUG::Got
exception while sshing to lago-upgrade-from-release-suite-master-host0:
Timed out (in 1 s) trying to ssh to
lago-upgrade-from-release-suite-master-host0
2017-11-13
15:31:34,238::log_utils.py::__exit__::611::lago.ssh::DEBUG::end
task:5fd0e3c3-d83c-46f9-9c88-bc739aa9c430:Get ssh client for
lago-upgrade-from-release-suite-master-engine:
2017-11-13 15:31:34,238::ssh.py::wait_for_ssh::129::lago.ssh::DEBUG::Got
exception while sshing to lago-upgrade-from-release-suite-master-engine:
Timed out (in 1 s) trying to ssh to
lago-upgrade-from-release-suite-master-engine
2017-11-13
15:31:34,238::log_utils.py::__enter__::600::lago.ssh::DEBUG::start
task:22550d6e-c397-43dc-9691-c8d9debfe416:Get ssh client for
lago-upgrade-from-release-suite-master-host0:
2017-11-13
15:31:34,239::ssh.py::get_ssh_client::339::lago.ssh::DEBUG::Still got 1
tries for lago-upgrade-from-release-suite-master-host0
2017-11-13
15:31:34,407::log_utils.py::__exit__::611::lago.ssh::DEBUG::end
task:22550d6e-c397-43dc-9691-c8d9debfe416:Get ssh client for
lago-upgrade-from-release-suite-master-host0:
2017-11-13 15:31:34,639::ssh.py::ssh::58::lago.ssh::DEBUG::Running
bb49adfa on lago-upgrade-from-release-suite-master-host0: true
2017-11-13 15:31:34,665::ssh.py::ssh::81::lago.ssh::DEBUG::Command
bb49adfa on lago-upgrade-from-release-suite-master-host0 returned with 0
2017-11-13
15:31:34,665::ssh.py::wait_for_ssh::153::lago.ssh::DEBUG::Wait succeeded
for ssh to lago-upgrade-from-release-suite-master-host0
2017-11-13
15:31:34,665::log_utils.py::__exit__::611::lago.prefix::INFO::ESC[32mSuccessESC[0m
(in 0:00:11)
2017-11-13
15:31:34,666::log_utils.py::__enter__::600::lago.prefix::INFO::ESC[0mESC[0m
2017-11-13
15:31:34,666::log_utils.py::__enter__::600::lago.ssh::DEBUG::start
task:4e2a7421-28f7-4679-955f-7261fe377939:Get ssh client for
lago-upgrade-from-release-suite-master-host0:
2017-11-13
15:31:34,667::ssh.py::get_ssh_client::339::lago.ssh::DEBUG::Still got
100 tries for lago-upgrade-from-release-suite-master-host0
2017-11-13
15:31:34,805::log_utils.py::__exit__::611::lago.ssh::DEBUG::end
task:4e2a7421-28f7-4679-955f-7261fe377939:Get ssh client for
lago-upgrade-from-release-suite-master-host0:
2017-11-13 15:31:34,906::ssh.py::ssh::58::lago.ssh::DEBUG::Running
bb724bf2 on lago-upgrade-from-release-suite-master-host0: bash -s < "set -xe
DIST=$(uname -r | sed -r 's/^.*\.([^\.]+)\.[^\.]+$/\1/')
*
*
*
**_from host: _*
*
ions. Proceeding anyway.
Nov 13 10:31:34 lago-upgrade-from-release-suite-master-host0 systemd:
Configuration file /usr/lib/systemd/system/ebtables.service is marked
executable. Please remove executable permission bits. Proceeding anyway.
Nov 13 10:31:34 lago-upgrade-from-release-suite-master-host0 systemd:
Configuration file /usr/lib/systemd/system/wpa_supplicant.service is
marked executable. Please remove executable permission bits. Proceeding
anyway.
Nov 13 10:31:55 lago-upgrade-from-release-suite-master-host0 yum[1294]:
Updated: iptables-1.4.21-18.2.el7_4.x86_64
Nov 13 10:31:56 lago-upgrade-from-release-suite-master-host0
systemd-logind: Removed session 3.
Nov 13 10:31:56 lago-upgrade-from-release-suite-master-host0 systemd:
Removed slice user-0.slice.
Nov 13 10:31:56 lago-upgrade-from-release-suite-master-host0 systemd:
Stopping user-0.slice.
Nov 13 10:44:00 lago-upgrade-from-release-suite-master-host0 systemd:
Created slice user-0.slice.
Nov 13 10:44:00 lago-upgrade-from-release-suite-master-host0 systemd:
Starting user-0.slice.
Nov 13 10:44:00 lago-upgrade-from-release-suite-master-host0 systemd:
Started Session 4 of user root.
Nov 13 10:44:00 lago-upgrade-from-release-suite-master-host0 systemd:
Starting Session 4 of user root.
Nov 13 10:44:00 lago-upgrade-from-release-suite-master-host0
systemd-logind: New session 4 of user root.
Nov 13 10:44:00 lago-upgrade-from-release-suite-master-host0
systemd-logind: Removed session 4.
Nov 13 10:44:00 lago-upgrade-from-release-suite-master-host0 systemd:
Removed slice user-0.slice.
Nov 13 10:44:00 lago-upgrade-from-release-suite-master-host0 systemd:
Stopping user-0.slice.
Nov 13 10:44:00 lago-upgrade-from-release-suite-master-host0 systemd:
Created slice user-0.slice.
Nov 13 10:44:00 lago-upgrade-from-release-suite-master-host0 systemd:
Starting user-0.slice.
Nov 13 10:44:00 lago-upgrade-from-release-suite-master-host0 systemd:
Started Session 5 of user root.
Nov 13 10:44:00 lago-upgrade-from-release-suite-master-host0 systemd:
Starting Session 5 of user root.
Nov 13 10:44:00 lago-upgrade-from-release-suite-master-host0
systemd-logind: New session 5 of user root.
Nov 13 10:44:00 lago-upgrade-from-release-suite-master-host0
systemd-logind: Removed session 5.
Nov 13 10:44:01 lago-upgrade-from-release-suite-master-host0 systemd:
Removed slice user-0.slice.
Nov 13 10:44:01 lago-upgrade-from-release-suite-master-host0 systemd:
Stopping user-0.slice.
Nov 13 10:46:27 lago-upgrade-from-release-suite-master-host0 systemd:
Starting Cleanup of Temporary Directories...
Nov 13 10:46:27 lago-upgrade-from-release-suite-master-host0 systemd:
Started Cleanup of Temporary Directories.
Nov 13 10:54:36 lago-upgrade-from-release-suite-master-host0
dhclient[906]: DHCPREQUEST on eth0 to 192.168.200.1 port 67 (xid=0x564629cb)
Nov 13 10:54:36 lago-upgrade-from-release-suite-master-host0
dhclient[906]: DHCPACK from 192.168.200.1 (xid=0x564629cb)**
*
*
*</error>*
**
--------------5185435FD38913B821CED967
Content-Type: text/html; charset=utf-8
Content-Transfer-Encoding: 8bit
<html>
<head>
<meta http-equiv="content-type" content="text/html; charset=utf-8">
</head>
<body text="#000000" bgcolor="#FFFFFF">
<p>Hi, <br>
</p>
<p>We had a failure in upgrade suite for 002_bootstrap.add_hosts. I
am not seeing any error that can suggest on an issue in engine. <br>
</p>
<p>I can see in the host messages host that we have stopped writing
to the log for 15 minutes and it may suggest that there is
something that is keeping the host from starting which causes us
to fail the test on timeout. <br>
</p>
<p>However, i can use some help in determining the cause for this
failure and weather its connected to the bootstrap_add_host test
in upgrade.<br>
</p>
<p><b style="font-weight:normal;"
id="docs-internal-guid-5859b7a1-ba60-843c-cfdd-7366571bcb57">
<p dir="ltr"
style="line-height:1.38;margin-top:0pt;margin-bottom:0pt;"><span style="font-size:11pt;font-family:Arial;color:#000000;background-color:transparent;font-weight:400;font-style:normal;font-variant:normal;text-decoration:none;vertical-align:baseline;white-space:pre-wrap;">Link to suspected patches: As I said,<b> </b>I do not think its related, but this is the patch that was reported.
</span></p>
<p dir="ltr"
style="line-height:1.38;margin-top:0pt;margin-bottom:0pt;"><span style="font-size:11pt;font-family:Arial;color:#000000;background-color:transparent;font-weight:400;font-style:normal;font-variant:normal;text-decoration:none;vertical-align:baseline;white-space:pre-wrap;">
</span></p>
<p dir="ltr"
style="line-height:1.38;margin-top:0pt;margin-bottom:0pt;"><span style="font-size:11pt;font-family:Arial;color:#000000;background-color:transparent;font-weight:400;font-style:normal;font-variant:normal;text-decoration:none;vertical-align:baseline;white-space:pre-wrap;"><a class="moz-txt-link-freetext" href="https://gerrit.ovirt.org/#/c/83854/">https://gerrit.ovirt.org/#/c/83854/</a></span></p>
<br>
<p dir="ltr"
style="line-height:1.38;margin-top:0pt;margin-bottom:0pt;"><span style="font-size:11pt;font-family:Arial;color:#000000;background-color:transparent;font-weight:400;font-style:normal;font-variant:normal;text-decoration:none;vertical-align:baseline;white-space:pre-wrap;">Link to Job:</span></p>
<a class="moz-txt-link-freetext" href="http://jenkins.ovirt.org/job/ovirt-master_change-queue-tester/3795/">http://jenkins.ovirt.org/job/ovirt-master_change-queue-tester/3795/</a></b></p>
<p><b style="font-weight:normal;"
id="docs-internal-guid-5859b7a1-ba60-843c-cfdd-7366571bcb57"><br>
<p dir="ltr"
style="line-height:1.38;margin-top:0pt;margin-bottom:0pt;"><span style="font-size:11pt;font-family:Arial;color:#000000;background-color:transparent;font-weight:400;font-style:normal;font-variant:normal;text-decoration:none;vertical-align:baseline;white-space:pre-wrap;">Link to all logs:</span></p>
<a class="moz-txt-link-freetext" href="http://jenkins.ovirt.org/job/ovirt-master_change-queue-tester/3795/artifact/">http://jenkins.ovirt.org/job/ovirt-master_change-queue-tester/3795/artifact/</a></b></p>
<p><b style="font-weight:normal;"
id="docs-internal-guid-5859b7a1-ba60-843c-cfdd-7366571bcb57"><br>
<p dir="ltr"
style="line-height:1.38;margin-top:0pt;margin-bottom:0pt;"><span style="font-size:11pt;font-family:Arial;color:#000000;background-color:transparent;font-weight:400;font-style:normal;font-variant:normal;text-decoration:none;vertical-align:baseline;white-space:pre-wrap;">(Relevant) error snippet from the log: </span></p>
<p dir="ltr"
style="line-height:1.38;margin-top:0pt;margin-bottom:0pt;"><span style="font-size:11pt;font-family:Arial;color:#000000;background-color:transparent;font-weight:400;font-style:normal;font-variant:normal;text-decoration:none;vertical-align:baseline;white-space:pre-wrap;"><error></span></p>
<p dir="ltr"
style="line-height:1.38;margin-top:0pt;margin-bottom:0pt;"><span style="font-size:11pt;font-family:Arial;color:#000000;background-color:transparent;font-weight:400;font-style:normal;font-variant:normal;text-decoration:none;vertical-align:baseline;white-space:pre-wrap;">
</span></p>
<p dir="ltr"
style="line-height:1.38;margin-top:0pt;margin-bottom:0pt;"><span style="font-size:11pt;font-family:Arial;color:#000000;background-color:transparent;font-weight:400;font-style:normal;font-variant:normal;text-decoration:none;vertical-align:baseline;white-space:pre-wrap;">
</span></p>
</b><b style="font-weight:normal;"
id="docs-internal-guid-5859b7a1-ba60-843c-cfdd-7366571bcb57">
<p dir="ltr"
style="line-height:1.38;margin-top:0pt;margin-bottom:0pt;"><span style="font-size:11pt;font-family:Arial;color:#000000;background-color:transparent;font-weight:400;font-style:normal;font-variant:normal;text-decoration:none;vertical-align:baseline;white-space:pre-wrap;"><u><font size="+1"><b>Test error: </b></font></u><b>
</b></span></p>
<p dir="ltr"
style="line-height:1.38;margin-top:0pt;margin-bottom:0pt;"><span style="font-size:11pt;font-family:Arial;color:#000000;background-color:transparent;font-weight:400;font-style:normal;font-variant:normal;text-decoration:none;vertical-align:baseline;white-space:pre-wrap;"><b>
</b></span></p>
</b></p>
<h3 style="box-sizing: border-box; color: rgb(51, 51, 51);
font-family: Helvetica, Arial, sans-serif; font-style: normal;
font-variant-ligatures: normal; font-variant-caps: normal;
letter-spacing: normal; orphans: 2; text-align: start;
text-indent: 0px; text-transform: none; white-space: normal;
widows: 2; word-spacing: 0px; -webkit-text-stroke-width: 0px;
text-decoration-style: initial; text-decoration-color: initial;">Error
Message</h3>
<pre style="box-sizing: border-box; white-space: pre-wrap; word-wrap: break-word; margin: 0px; color: rgb(51, 51, 51); font-size: 14px; font-style: normal; font-variant-ligatures: normal; font-variant-caps: normal; font-weight: normal; letter-spacing: normal; orphans: 2; text-align: start; text-indent: 0px; text-transform: none; widows: 2; word-spacing: 0px; -webkit-text-stroke-width: 0px; text-decoration-style: initial; text-decoration-color: initial;">False != True after 900 seconds</pre>
<h3 style="box-sizing: border-box; color: rgb(51, 51, 51);
font-family: Helvetica, Arial, sans-serif; font-style: normal;
font-variant-ligatures: normal; font-variant-caps: normal;
letter-spacing: normal; orphans: 2; text-align: start;
text-indent: 0px; text-transform: none; white-space: normal;
widows: 2; word-spacing: 0px; -webkit-text-stroke-width: 0px;
text-decoration-style: initial; text-decoration-color: initial;">Stacktrace</h3>
<pre style="box-sizing: border-box; white-space: pre-wrap; word-wrap: break-word; margin: 0px; color: rgb(51, 51, 51); font-size: 14px; font-style: normal; font-variant-ligatures: normal; font-variant-caps: normal; font-weight: normal; letter-spacing: normal; orphans: 2; text-align: start; text-indent: 0px; text-transform: none; widows: 2; word-spacing: 0px; -webkit-text-stroke-width: 0px; text-decoration-style: initial; text-decoration-color: initial;">Traceback (most recent call last):
File "/usr/lib64/python2.7/unittest/case.py", line 369, in run
testMethod()
File "/usr/lib/python2.7/site-packages/nose/case.py", line 197, in runTest
self.test(*self.arg)
File "/usr/lib/python2.7/site-packages/ovirtlago/testlib.py", line 129, in wrapped_test
test()
File "/usr/lib/python2.7/site-packages/ovirtlago/testlib.py", line 59, in wrapper
return func(get_test_prefix(), *args, **kwargs)
File "/home/jenkins/workspace/ovirt-master_change-queue-tester/ovirt-system-tests/upgrade-from-release-suite-master/test-scenarios-after-upgrade/002_bootstrap.py", line 187, in add_hosts
testlib.assert_true_within(_host_is_up_4, timeout=15*60)
File "/usr/lib/python2.7/site-packages/ovirtlago/testlib.py", line 263, in assert_true_within
assert_equals_within(func, True, timeout, allowed_exceptions)
File "/usr/lib/python2.7/site-packages/ovirtlago/testlib.py", line 237, in assert_equals_within
'%s != %s after %s seconds' % (res, value, timeout)
AssertionError: False != True after 900 seconds</pre>
<p><b style="font-weight:normal;"
id="docs-internal-guid-5859b7a1-ba60-843c-cfdd-7366571bcb57">
<p dir="ltr"
style="line-height:1.38;margin-top:0pt;margin-bottom:0pt;"><span style="font-size:11pt;font-family:Arial;color:#000000;background-color:transparent;font-weight:400;font-style:normal;font-variant:normal;text-decoration:none;vertical-align:baseline;white-space:pre-wrap;"></span></p>
</b></p>
<p><b style="font-weight:normal;"
id="docs-internal-guid-5859b7a1-ba60-843c-cfdd-7366571bcb57"><b><u><font
size="+1">lago log: </font></u></b><br>
</b></p>
<p><b style="font-weight:normal;"
id="docs-internal-guid-5859b7a1-ba60-843c-cfdd-7366571bcb57">2017-11-13
15:31:23,212::log_utils.py::__enter__::600::lago.prefix::<a class="moz-txt-link-freetext" href="INFO::ESC">INFO::ESC</a>[0mESC[0m<br>
2017-11-13
15:31:23,213::log_utils.py::__enter__::600::lago.ssh::DEBUG::start
task:0468ed2f-b174-4d94-bc66-2b6e08087a86:Get ssh client for
lago-upgrade-from-release-suite-master-host0:<br>
2017-11-13
15:31:23,213::log_utils.py::__enter__::600::lago.ssh::DEBUG::start
task:91372413-f7fd-4b72-85b9-9f5216ca7ae9:Get ssh client for
lago-upgrade-from-release-suite-master-engine:<br>
2017-11-13
15:31:23,213::ssh.py::get_ssh_client::339::lago.ssh::DEBUG::Still
got 1 tries for lago-upgrade-from-release-suite-master-host0<br>
2017-11-13
15:31:23,213::ssh.py::get_ssh_client::339::lago.ssh::DEBUG::Still
got 1 tries for lago-upgrade-from-release-suite-master-engine<br>
2017-11-13
15:31:26,220::ssh.py::get_ssh_client::354::lago.ssh::DEBUG::Socket
error connecting to
lago-upgrade-from-release-suite-master-engine: [Errno None]
Unable to connect to port 22 on 192.168.200.3<br>
2017-11-13
15:31:26,221::ssh.py::get_ssh_client::354::lago.ssh::DEBUG::Socket
error connecting to
lago-upgrade-from-release-suite-master-host0: [Errno None]
Unable to connect to port 22 on 192.168.200.2<br>
2017-11-13
15:31:27,222::log_utils.py::__exit__::611::lago.ssh::DEBUG::end
task:91372413-f7fd-4b72-85b9-9f5216ca7ae9:Get ssh client for
lago-upgrade-from-release-suite-master-engine:<br>
2017-11-13
15:31:27,222::log_utils.py::__exit__::611::lago.ssh::DEBUG::end
task:0468ed2f-b174-4d94-bc66-2b6e08087a86:Get ssh client for
lago-upgrade-from-release-suite-master-host0:<br>
2017-11-13
15:31:27,222::ssh.py::wait_for_<a class="moz-txt-link-freetext" href="ssh::129::lago.ssh::DEBUG::Got">ssh::129::lago.ssh::DEBUG::Got</a>
exception while sshing to
lago-upgrade-from-release-suite-master-engine: Timed out (in 4
s) trying to ssh to
lago-upgrade-from-release-suite-master-engine<br>
2017-11-13
15:31:27,222::ssh.py::wait_for_<a class="moz-txt-link-freetext" href="ssh::129::lago.ssh::DEBUG::Got">ssh::129::lago.ssh::DEBUG::Got</a>
exception while sshing to
lago-upgrade-from-release-suite-master-host0: Timed out (in 4 s)
trying to ssh to lago-upgrade-from-release-suite-master-host0<br>
2017-11-13
15:31:28,224::log_utils.py::__enter__::600::lago.ssh::DEBUG::start
task:fc88f641-b012-4636-a471-9ccaaf361a53:Get ssh client for
lago-upgrade-from-release-suite-master-engine:<br>
2017-11-13
15:31:28,224::log_utils.py::__enter__::600::lago.ssh::DEBUG::start
task:afb01a46-6338-407f-b7c9-9d5c6b91404d:Get ssh client for
lago-upgrade-from-release-suite-master-host0:<br>
2017-11-13
15:31:28,224::ssh.py::get_ssh_client::339::lago.ssh::DEBUG::Still
got 1 tries for lago-upgrade-from-release-suite-master-engine<br>
2017-11-13
15:31:28,225::ssh.py::get_ssh_client::339::lago.ssh::DEBUG::Still
got 1 tries for lago-upgrade-from-release-suite-master-host0<br>
2017-11-13
15:31:28,226::ssh.py::get_ssh_client::354::lago.ssh::DEBUG::Socket
error connecting to
lago-upgrade-from-release-suite-master-host0: [Errno None]
Unable to connect to port 22 on 192.168.200.2<br>
2017-11-13
15:31:29,228::log_utils.py::__exit__::611::lago.ssh::DEBUG::end
task:afb01a46-6338-407f-b7c9-9d5c6b91404d:Get ssh client for
lago-upgrade-from-release-suite-master-host0:<br>
2017-11-13
15:31:29,228::ssh.py::wait_for_<a class="moz-txt-link-freetext" href="ssh::129::lago.ssh::DEBUG::Got">ssh::129::lago.ssh::DEBUG::Got</a>
exception while sshing to
lago-upgrade-from-release-suite-master-host0: Timed out (in 1 s)
trying to ssh to lago-upgrade-from-release-suite-master-host0<br>
2017-11-13
15:31:29,229::ssh.py::get_ssh_client::354::lago.ssh::DEBUG::Socket
error connecting to
lago-upgrade-from-release-suite-master-engine: [Errno None]
Unable to connect to port 22 on 192.168.200.3<br>
2017-11-13
15:31:30,229::log_utils.py::__exit__::611::lago.ssh::DEBUG::end
task:fc88f641-b012-4636-a471-9ccaaf361a53:Get ssh client for
lago-upgrade-from-release-suite-master-engine:<br>
2017-11-13
15:31:30,230::log_utils.py::__enter__::600::lago.ssh::DEBUG::start
task:9df5ddca-bbc4-485a-9f3a-b0b9a5bb5990:Get ssh client for
lago-upgrade-from-release-suite-master-host0:<br>
2017-11-13
15:31:30,230::ssh.py::wait_for_<a class="moz-txt-link-freetext" href="ssh::129::lago.ssh::DEBUG::Got">ssh::129::lago.ssh::DEBUG::Got</a>
exception while sshing to
lago-upgrade-from-release-suite-master-engine: Timed out (in 2
s) trying to ssh to
lago-upgrade-from-release-suite-master-engine<br>
2017-11-13
15:31:30,230::ssh.py::get_ssh_client::339::lago.ssh::DEBUG::Still
got 1 tries for lago-upgrade-from-release-suite-master-host0<br>
2017-11-13
15:31:30,231::ssh.py::get_ssh_client::354::lago.ssh::DEBUG::Socket
error connecting to
lago-upgrade-from-release-suite-master-host0: [Errno None]
Unable to connect to port 22 on 192.168.200.2<br>
2017-11-13
15:31:31,231::log_utils.py::__enter__::600::lago.ssh::DEBUG::start
task:53fe67da-a632-49fe-b697-e8a2c4cb7d23:Get ssh client for
lago-upgrade-from-release-suite-master-engine:<br>
2017-11-13
15:31:31,232::ssh.py::get_ssh_client::339::lago.ssh::DEBUG::Still
got 1 tries for lago-upgrade-from-release-suite-master-engine<br>
2017-11-13
15:31:31,232::ssh.py::get_ssh_client::354::lago.ssh::DEBUG::Socket
error connecting to
lago-upgrade-from-release-suite-master-engine: [Errno None]
Unable to connect to port 22 on 192.168.200.3<br>
2017-11-13
15:31:31,232::log_utils.py::__exit__::611::lago.ssh::DEBUG::end
task:9df5ddca-bbc4-485a-9f3a-b0b9a5bb5990:Get ssh client for
lago-upgrade-from-release-suite-master-host0:<br>
2017-11-13
15:31:31,233::ssh.py::wait_for_<a class="moz-txt-link-freetext" href="ssh::129::lago.ssh::DEBUG::Got">ssh::129::lago.ssh::DEBUG::Got</a>
exception while sshing to
lago-upgrade-from-release-suite-master-host0: Timed out (in 1 s)
trying to ssh to lago-upgrade-from-release-suite-master-host0<br>
2017-11-13
15:31:32,234::log_utils.py::__exit__::611::lago.ssh::DEBUG::end
task:53fe67da-a632-49fe-b697-e8a2c4cb7d23:Get ssh client for
lago-upgrade-from-release-suite-master-engine:<br>
2017-11-13
15:31:32,234::ssh.py::wait_for_<a class="moz-txt-link-freetext" href="ssh::129::lago.ssh::DEBUG::Got">ssh::129::lago.ssh::DEBUG::Got</a>
exception while sshing to
lago-upgrade-from-release-suite-master-engine: Timed out (in 1
s) trying to ssh to
lago-upgrade-from-release-suite-master-engine<br>
2017-11-13
15:31:32,234::log_utils.py::__enter__::600::lago.ssh::DEBUG::start
task:1969bf5b-4e66-43d7-91c6-a28949e98fe8:Get ssh client for
lago-upgrade-from-release-suite-master-host0:<br>
2017-11-13
15:31:32,234::ssh.py::get_ssh_client::339::lago.ssh::DEBUG::Still
got 1 tries for lago-upgrade-from-release-suite-master-host0<br>
2017-11-13
15:31:32,235::ssh.py::get_ssh_client::354::lago.ssh::DEBUG::Socket
error connecting to
lago-upgrade-from-release-suite-master-host0: [Errno None]
Unable to connect to port 22 on 192.168.200.2<br>
2017-11-13
15:31:33,235::log_utils.py::__enter__::600::lago.ssh::DEBUG::start
task:5fd0e3c3-d83c-46f9-9c88-bc739aa9c430:Get ssh client for
lago-upgrade-from-release-suite-master-engine:<br>
2017-11-13
15:31:33,235::ssh.py::get_ssh_client::339::lago.ssh::DEBUG::Still
got 1 tries for lago-upgrade-from-release-suite-master-engine<br>
2017-11-13
15:31:33,236::log_utils.py::__exit__::611::lago.ssh::DEBUG::end
task:1969bf5b-4e66-43d7-91c6-a28949e98fe8:Get ssh client for
lago-upgrade-from-release-suite-master-host0:<br>
2017-11-13
15:31:33,236::ssh.py::get_ssh_client::354::lago.ssh::DEBUG::Socket
error connecting to
lago-upgrade-from-release-suite-master-engine: [Errno None]
Unable to connect to port 22 on 192.168.200.3<br>
2017-11-13
15:31:33,237::ssh.py::wait_for_<a class="moz-txt-link-freetext" href="ssh::129::lago.ssh::DEBUG::Got">ssh::129::lago.ssh::DEBUG::Got</a>
exception while sshing to
lago-upgrade-from-release-suite-master-host0: Timed out (in 1 s)
trying to ssh to lago-upgrade-from-release-suite-master-host0<br>
2017-11-13
15:31:34,238::log_utils.py::__exit__::611::lago.ssh::DEBUG::end
task:5fd0e3c3-d83c-46f9-9c88-bc739aa9c430:Get ssh client for
lago-upgrade-from-release-suite-master-engine:<br>
2017-11-13
15:31:34,238::ssh.py::wait_for_<a class="moz-txt-link-freetext" href="ssh::129::lago.ssh::DEBUG::Got">ssh::129::lago.ssh::DEBUG::Got</a>
exception while sshing to
lago-upgrade-from-release-suite-master-engine: Timed out (in 1
s) trying to ssh to
lago-upgrade-from-release-suite-master-engine<br>
2017-11-13
15:31:34,238::log_utils.py::__enter__::600::lago.ssh::DEBUG::start
task:22550d6e-c397-43dc-9691-c8d9debfe416:Get ssh client for
lago-upgrade-from-release-suite-master-host0:<br>
2017-11-13
15:31:34,239::ssh.py::get_ssh_client::339::lago.ssh::DEBUG::Still
got 1 tries for lago-upgrade-from-release-suite-master-host0<br>
2017-11-13
15:31:34,407::log_utils.py::__exit__::611::lago.ssh::DEBUG::end
task:22550d6e-c397-43dc-9691-c8d9debfe416:Get ssh client for
lago-upgrade-from-release-suite-master-host0:<br>
2017-11-13
15:31:34,639::ssh.py::<a class="moz-txt-link-freetext" href="ssh::58::lago.ssh::DEBUG::Running">ssh::58::lago.ssh::DEBUG::Running</a> bb49adfa
on lago-upgrade-from-release-suite-master-host0: true<br>
2017-11-13
15:31:34,665::ssh.py::<a class="moz-txt-link-freetext" href="ssh::81::lago.ssh::DEBUG::Command">ssh::81::lago.ssh::DEBUG::Command</a> bb49adfa
on lago-upgrade-from-release-suite-master-host0 returned with 0<br>
2017-11-13
15:31:34,665::ssh.py::wait_for_<a class="moz-txt-link-freetext" href="ssh::153::lago.ssh::DEBUG::Wait">ssh::153::lago.ssh::DEBUG::Wait</a>
succeeded for ssh to
lago-upgrade-from-release-suite-master-host0<br>
2017-11-13
15:31:34,665::log_utils.py::__exit__::611::lago.prefix::<a class="moz-txt-link-freetext" href="INFO::ESC">INFO::ESC</a>[32mSuccessESC[0m
(in 0:00:11)<br>
2017-11-13
15:31:34,666::log_utils.py::__enter__::600::lago.prefix::<a class="moz-txt-link-freetext" href="INFO::ESC">INFO::ESC</a>[0mESC[0m<br>
2017-11-13
15:31:34,666::log_utils.py::__enter__::600::lago.ssh::DEBUG::start
task:4e2a7421-28f7-4679-955f-7261fe377939:Get ssh client for
lago-upgrade-from-release-suite-master-host0:<br>
2017-11-13
15:31:34,667::ssh.py::get_ssh_client::339::lago.ssh::DEBUG::Still
got 100 tries for lago-upgrade-from-release-suite-master-host0<br>
2017-11-13
15:31:34,805::log_utils.py::__exit__::611::lago.ssh::DEBUG::end
task:4e2a7421-28f7-4679-955f-7261fe377939:Get ssh client for
lago-upgrade-from-release-suite-master-host0:<br>
2017-11-13
15:31:34,906::ssh.py::<a class="moz-txt-link-freetext" href="ssh::58::lago.ssh::DEBUG::Running">ssh::58::lago.ssh::DEBUG::Running</a> bb724bf2
on lago-upgrade-from-release-suite-master-host0: bash -s <
"set -xe<br>
DIST=$(uname -r | sed -r 's/^.*\.([^\.]+)\.[^\.]+$/\1/')<br>
<br>
</b></p>
<p><b style="font-weight:normal;"
id="docs-internal-guid-5859b7a1-ba60-843c-cfdd-7366571bcb57"><br>
</b></p>
<p><b style="font-weight:normal;"
id="docs-internal-guid-5859b7a1-ba60-843c-cfdd-7366571bcb57"><b><u><font
size="+1">from host: </font></u></b><br>
</b></p>
<p><span style="font-weight:normal;">ions. Proceeding anyway.<br>
Nov 13 10:31:34 lago-upgrade-from-release-suite-master-host0
systemd: Configuration file
/usr/lib/systemd/system/ebtables.service is marked executable.
Please remove executable permission bits. Proceeding anyway.<br>
Nov 13 10:31:34 lago-upgrade-from-release-suite-master-host0
systemd: Configuration file
/usr/lib/systemd/system/wpa_supplicant.service is marked
executable. Please remove executable permission bits. Proceeding
anyway.<br>
Nov 13 10:31:55 lago-upgrade-from-release-suite-master-host0
yum[1294]: Updated: iptables-1.4.21-18.2.el7_4.x86_64<br>
Nov 13 10:31:56 lago-upgrade-from-release-suite-master-host0
systemd-logind: Removed session 3.<br>
Nov 13 10:31:56 lago-upgrade-from-release-suite-master-host0
systemd: Removed slice user-0.slice.<br>
Nov 13 10:31:56 lago-upgrade-from-release-suite-master-host0
systemd: Stopping user-0.slice.<br>
Nov 13 10:44:00 lago-upgrade-from-release-suite-master-host0
systemd: Created slice user-0.slice.<br>
Nov 13 10:44:00 lago-upgrade-from-release-suite-master-host0
systemd: Starting user-0.slice.<br>
Nov 13 10:44:00 lago-upgrade-from-release-suite-master-host0
systemd: Started Session 4 of user root.<br>
Nov 13 10:44:00 lago-upgrade-from-release-suite-master-host0
systemd: Starting Session 4 of user root.<br>
Nov 13 10:44:00 lago-upgrade-from-release-suite-master-host0
systemd-logind: New session 4 of user root.<br>
Nov 13 10:44:00 lago-upgrade-from-release-suite-master-host0
systemd-logind: Removed session 4.<br>
Nov 13 10:44:00 lago-upgrade-from-release-suite-master-host0
systemd: Removed slice user-0.slice.<br>
Nov 13 10:44:00 lago-upgrade-from-release-suite-master-host0
systemd: Stopping user-0.slice.<br>
Nov 13 10:44:00 lago-upgrade-from-release-suite-master-host0
systemd: Created slice user-0.slice.<br>
Nov 13 10:44:00 lago-upgrade-from-release-suite-master-host0
systemd: Starting user-0.slice.<br>
Nov 13 10:44:00 lago-upgrade-from-release-suite-master-host0
systemd: Started Session 5 of user root.<br>
Nov 13 10:44:00 lago-upgrade-from-release-suite-master-host0
systemd: Starting Session 5 of user root.<br>
Nov 13 10:44:00 lago-upgrade-from-release-suite-master-host0
systemd-logind: New session 5 of user root.<br>
Nov 13 10:44:00 lago-upgrade-from-release-suite-master-host0
systemd-logind: Removed session 5.<br>
Nov 13 10:44:01 lago-upgrade-from-release-suite-master-host0
systemd: Removed slice user-0.slice.<br>
Nov 13 10:44:01 lago-upgrade-from-release-suite-master-host0
systemd: Stopping user-0.slice.<br>
Nov 13 10:46:27 lago-upgrade-from-release-suite-master-host0
systemd: Starting Cleanup of Temporary Directories...<br>
Nov 13 10:46:27 lago-upgrade-from-release-suite-master-host0
systemd: Started Cleanup of Temporary Directories.<br>
Nov 13 10:54:36 lago-upgrade-from-release-suite-master-host0
dhclient[906]: DHCPREQUEST on eth0 to 192.168.200.1 port 67
(xid=0x564629cb)<br>
Nov 13 10:54:36 lago-upgrade-from-release-suite-master-host0
dhclient[906]: DHCPACK from 192.168.200.1 (xid=0x564629cb)</span><b
style="font-weight:normal;"
id="docs-internal-guid-5859b7a1-ba60-843c-cfdd-7366571bcb57"><b><br>
</b><br>
<p dir="ltr"
style="line-height:1.38;margin-top:0pt;margin-bottom:0pt;"><span style="font-size:11pt;font-family:Arial;color:#000000;background-color:transparent;font-weight:400;font-style:normal;font-variant:normal;text-decoration:none;vertical-align:baseline;white-space:pre-wrap;"></error></span></p>
</b><br class="Apple-interchange-newline">
</p>
</body>
</html>
--------------5185435FD38913B821CED967--
3
4
Hi,
I made some changes [1][2] to ovirt-engine that make it now more
aggressively (and correctly) cache all static assets. We were previously
caching some things, but not all. (Thanks to ykaul for noticing.)
The upside is reduced server hits and faster page loads.
The downside is you'll have to clear your browser cache more often as you
are developing. (You can use a browser plugin, incognito mode, F12 >
disable cache, etc. if you don't want to keep opening the dialog.)
[1] https://gerrit.ovirt.org/#/c/83879/
[2] https://gerrit.ovirt.org/#/c/83871/
Best wishes,
Greg
--
GREG SHEREMETA
SENIOR SOFTWARE ENGINEER - TEAM LEAD - RHV UX
Red Hat NA
<https://www.redhat.com/>
gshereme(a)redhat.com IRC: gshereme
<https://red.ht/sig>
2
2
[ OST Failure Report ] [ oVirt master ] [ 15-11-2017 ] [ 001_upgrade_engine.test_initialize_engine ]
by Dafna Ron 15 Nov '17
by Dafna Ron 15 Nov '17
15 Nov '17
This is a multi-part message in MIME format.
--------------1EB3285B75C820E3A1990192
Content-Type: text/plain; charset=utf-8
Content-Transfer-Encoding: 7bit
Hi,
We have a few alerts from cq all pointing to the same patch.
Adding Sandro since he's working on fixing the packages issue and
resolve this.
**
*Link to suspected patches: **https://gerrit.ovirt.org/#/c/83852/*
**
**
*Link to Job:*
***
*
*http://jenkins.ovirt.org/job/ovirt-master_change-queue-tester/3824*
http://jenkins.ovirt.org/job/ovirt-master_change-queue-tester/3829/testRepo…
http://jenkins.ovirt.org/job/ovirt-master_change-queue-tester/3833/testRepo…
http://jenkins.ovirt.org/job/ovirt-master_change-queue-tester/3839/testRepo…
http://jenkins.ovirt.org/job/ovirt-master_change-queue-tester/3844/testRepo…
http://jenkins.ovirt.org/job/ovirt-master_change-queue-tester/3850/*
*
*
*
*Link to all logs:*
*
**
http://jenkins.ovirt.org/job/ovirt-master_change-queue-tester/3824/artifact/
http://jenkins.ovirt.org/job/ovirt-master_change-queue-tester/3829/artifact/
http://jenkins.ovirt.org/job/ovirt-master_change-queue-tester/3833/artifact/
http://jenkins.ovirt.org/job/ovirt-master_change-queue-tester/3839/artifact/
http://jenkins.ovirt.org/job/ovirt-master_change-queue-tester/3844/artifact/
http://jenkins.ovirt.org/job/ovirt-master_change-queue-tester/3850/artifact/
(Relevant) error snippet from the log:
<error>
*
Removing obsolete
ovirt-engine-setup-plugin-websocket-proxy-4.1.7.7-0.0.master.20171114102253.git9267e4c.el7.centos.noarch.rpm
cockpit-ovirt-dashboard-0.10.9 FAILED
(1/145): cockpit-ovirt-das 0% [ ] 0.0 B/s | 0 B --:-- ETA
cockpit-ovirt-uiplugin-0.10.9- FAILED
(1/145): cockpit-ovirt-uip 0% [ ] 0.0 B/s | 0 B --:-- ETA
Public key for cockpit-ovirt-uiplugin-0.10.9-1.el7.centos.noarch.rpm is not installed
ovirt-ansible-roles-1.0.4-1.el FAILED
ovirt-engine-setup-plugin-live FAILED
ovirt-engine-yarn-0.19.1-2.el7 FAILED
**
*</error>*
**
--------------1EB3285B75C820E3A1990192
Content-Type: text/html; charset=utf-8
Content-Transfer-Encoding: 7bit
<html>
<head>
<meta http-equiv="content-type" content="text/html; charset=utf-8">
</head>
<body text="#000000" bgcolor="#FFFFFF">
<p>Hi, <br>
</p>
<p>We have a few alerts from cq all pointing to the same patch. <br>
</p>
<p>Adding Sandro since he's working on fixing the packages issue and
resolve this. <br>
</p>
<p><br>
</p>
<p><b style="font-weight:normal;"
id="docs-internal-guid-5859b7a1-bf27-32f5-1d02-0e995510a2c2">
<p dir="ltr"
style="line-height:1.38;margin-top:0pt;margin-bottom:0pt;"><span style="font-size:11pt;font-family:Arial;color:#000000;background-color:transparent;font-weight:400;font-style:normal;font-variant:normal;text-decoration:none;vertical-align:baseline;white-space:pre-wrap;">Link to suspected patches: <b> </b><a class="moz-txt-link-freetext" href="https://gerrit.ovirt.org/#/c/83852/">https://gerrit.ovirt.org/#/c/83852/</a></span></p>
</b></p>
<p><b style="font-weight:normal;"
id="docs-internal-guid-5859b7a1-bf27-32f5-1d02-0e995510a2c2">
<p dir="ltr"
style="line-height:1.38;margin-top:0pt;margin-bottom:0pt;"><span style="font-size:11pt;font-family:Arial;color:#000000;background-color:transparent;font-weight:400;font-style:normal;font-variant:normal;text-decoration:none;vertical-align:baseline;white-space:pre-wrap;">Link to Job:</span></p>
</b><b style="font-weight:normal;"
id="docs-internal-guid-5859b7a1-bf27-32f5-1d02-0e995510a2c2"><br>
</b></p>
<p><b style="font-weight:normal;"
id="docs-internal-guid-5859b7a1-bf27-32f5-1d02-0e995510a2c2"><a class="moz-txt-link-freetext" href="http://jenkins.ovirt.org/job/ovirt-master_change-queue-tester/3824">http://jenkins.ovirt.org/job/ovirt-master_change-queue-tester/3824</a></b></p>
<p><span style="font-weight:normal;"><a class="moz-txt-link-freetext" href="http://jenkins.ovirt.org/job/ovirt-master_change-queue-tester/3829/testRepo…">http://jenkins.ovirt.org/job/ovirt-master_change-queue-tester/3829/testRepo…</a></span></p>
<p><span style="font-weight:normal;"><a class="moz-txt-link-freetext" href="http://jenkins.ovirt.org/job/ovirt-master_change-queue-tester/3833/testRepo…">http://jenkins.ovirt.org/job/ovirt-master_change-queue-tester/3833/testRepo…</a></span></p>
<p><span style="font-weight:normal;"><a class="moz-txt-link-freetext" href="http://jenkins.ovirt.org/job/ovirt-master_change-queue-tester/3839/testRepo…">http://jenkins.ovirt.org/job/ovirt-master_change-queue-tester/3839/testRepo…</a></span></p>
<p><span style="font-weight:normal;"><a class="moz-txt-link-freetext" href="http://jenkins.ovirt.org/job/ovirt-master_change-queue-tester/3844/testRepo…">http://jenkins.ovirt.org/job/ovirt-master_change-queue-tester/3844/testRepo…</a></span></p>
<p><span style="font-weight:normal;"><a class="moz-txt-link-freetext" href="http://jenkins.ovirt.org/job/ovirt-master_change-queue-tester/3850/">http://jenkins.ovirt.org/job/ovirt-master_change-queue-tester/3850/</a></span><b
style="font-weight:normal;"
id="docs-internal-guid-5859b7a1-bf27-32f5-1d02-0e995510a2c2"><br>
</b></p>
<p><b style="font-weight:normal;"
id="docs-internal-guid-5859b7a1-bf27-32f5-1d02-0e995510a2c2"><br>
<p dir="ltr"
style="line-height:1.38;margin-top:0pt;margin-bottom:0pt;"><span style="font-size:11pt;font-family:Arial;color:#000000;background-color:transparent;font-weight:400;font-style:normal;font-variant:normal;text-decoration:none;vertical-align:baseline;white-space:pre-wrap;">Link to all logs:</span></p>
<p dir="ltr"
style="line-height:1.38;margin-top:0pt;margin-bottom:0pt;"><span style="font-size:11pt;font-family:Arial;color:#000000;background-color:transparent;font-weight:400;font-style:normal;font-variant:normal;text-decoration:none;vertical-align:baseline;white-space:pre-wrap;">
</span></p>
</b><b style="font-weight:normal;"
id="docs-internal-guid-5859b7a1-bf27-32f5-1d02-0e995510a2c2">
<p dir="ltr"
style="line-height:1.38;margin-top:0pt;margin-bottom:0pt;"><span style="font-size:11pt;font-family:Arial;color:#000000;background-color:transparent;font-weight:400;font-style:normal;font-variant:normal;text-decoration:none;vertical-align:baseline;white-space:pre-wrap;"><a class="moz-txt-link-freetext" href="http://jenkins.ovirt.org/job/ovirt-master_change-queue-tester/3824/artifact/">http://jenkins.ovirt.org/job/ovirt-master_change-queue-tester/3824/artifact/</a></span></p>
<p dir="ltr"
style="line-height:1.38;margin-top:0pt;margin-bottom:0pt;"><span style="font-size:11pt;font-family:Arial;color:#000000;background-color:transparent;font-weight:400;font-style:normal;font-variant:normal;text-decoration:none;vertical-align:baseline;white-space:pre-wrap;"><a class="moz-txt-link-freetext" href="http://jenkins.ovirt.org/job/ovirt-master_change-queue-tester/3829/artifact/">http://jenkins.ovirt.org/job/ovirt-master_change-queue-tester/3829/artifact/</a></span></p>
<p dir="ltr"
style="line-height:1.38;margin-top:0pt;margin-bottom:0pt;"><span style="font-size:11pt;font-family:Arial;color:#000000;background-color:transparent;font-weight:400;font-style:normal;font-variant:normal;text-decoration:none;vertical-align:baseline;white-space:pre-wrap;"><a class="moz-txt-link-freetext" href="http://jenkins.ovirt.org/job/ovirt-master_change-queue-tester/3833/artifact/">http://jenkins.ovirt.org/job/ovirt-master_change-queue-tester/3833/artifact/</a></span></p>
<p dir="ltr"
style="line-height:1.38;margin-top:0pt;margin-bottom:0pt;"><span style="font-size:11pt;font-family:Arial;color:#000000;background-color:transparent;font-weight:400;font-style:normal;font-variant:normal;text-decoration:none;vertical-align:baseline;white-space:pre-wrap;"><a class="moz-txt-link-freetext" href="http://jenkins.ovirt.org/job/ovirt-master_change-queue-tester/3839/artifact/">http://jenkins.ovirt.org/job/ovirt-master_change-queue-tester/3839/artifact/</a></span></p>
<p dir="ltr"
style="line-height:1.38;margin-top:0pt;margin-bottom:0pt;"><span style="font-size:11pt;font-family:Arial;color:#000000;background-color:transparent;font-weight:400;font-style:normal;font-variant:normal;text-decoration:none;vertical-align:baseline;white-space:pre-wrap;"><a class="moz-txt-link-freetext" href="http://jenkins.ovirt.org/job/ovirt-master_change-queue-tester/3844/artifact/">http://jenkins.ovirt.org/job/ovirt-master_change-queue-tester/3844/artifact/</a></span></p>
<p dir="ltr"
style="line-height:1.38;margin-top:0pt;margin-bottom:0pt;"><span style="font-size:11pt;font-family:Arial;color:#000000;background-color:transparent;font-weight:400;font-style:normal;font-variant:normal;text-decoration:none;vertical-align:baseline;white-space:pre-wrap;"><a class="moz-txt-link-freetext" href="http://jenkins.ovirt.org/job/ovirt-master_change-queue-tester/3850/artifact/">http://jenkins.ovirt.org/job/ovirt-master_change-queue-tester/3850/artifact/</a>
</span></p>
<br>
<p dir="ltr"
style="line-height:1.38;margin-top:0pt;margin-bottom:0pt;"><span style="font-size:11pt;font-family:Arial;color:#000000;background-color:transparent;font-weight:400;font-style:normal;font-variant:normal;text-decoration:none;vertical-align:baseline;white-space:pre-wrap;">(Relevant) error snippet from the log: </span></p>
<p dir="ltr"
style="line-height:1.38;margin-top:0pt;margin-bottom:0pt;"><span style="font-size:11pt;font-family:Arial;color:#000000;background-color:transparent;font-weight:400;font-style:normal;font-variant:normal;text-decoration:none;vertical-align:baseline;white-space:pre-wrap;"><error></span></p>
</b></p>
Removing obsolete
ovirt-engine-setup-plugin-websocket-proxy-4.1.7.7-0.0.master.20171114102253.git9267e4c.el7.centos.noarch.rpm
<pre style="color: rgb(0, 0, 0); font-style: normal; font-variant-ligatures: normal; font-variant-caps: normal; font-weight: normal; letter-spacing: normal; orphans: 2; text-align: start; text-indent: 0px; text-transform: none; widows: 2; word-spacing: 0px; -webkit-text-stroke-width: 0px; text-decoration-style: initial; text-decoration-color: initial;">
cockpit-ovirt-dashboard-0.10.9 FAILED
(1/145): cockpit-ovirt-das 0% [ ] 0.0 B/s | 0 B --:-- ETA
cockpit-ovirt-uiplugin-0.10.9- FAILED
(1/145): cockpit-ovirt-uip 0% [ ] 0.0 B/s | 0 B --:-- ETA
Public key for cockpit-ovirt-uiplugin-0.10.9-1.el7.centos.noarch.rpm is not installed
ovirt-ansible-roles-1.0.4-1.el FAILED
ovirt-engine-setup-plugin-live FAILED
ovirt-engine-yarn-0.19.1-2.el7 FAILED
</pre>
<p><b style="font-weight:normal;"
id="docs-internal-guid-5859b7a1-bf27-32f5-1d02-0e995510a2c2">
<p dir="ltr"
style="line-height:1.38;margin-top:0pt;margin-bottom:0pt;"><span style="font-size:11pt;font-family:Arial;color:#000000;background-color:transparent;font-weight:400;font-style:normal;font-variant:normal;text-decoration:none;vertical-align:baseline;white-space:pre-wrap;"></error></span></p>
</b><br class="Apple-interchange-newline">
</p>
</body>
</html>
--------------1EB3285B75C820E3A1990192--
2
1
Hi,
I've added a SKIP_CHECKS flag to the ovirt-engine build here [1]. Thanks to
Roy for the idea [2].
TL;DR: you'll save roughly 2 minutes 10 seconds per full compile with
SKIP_CHECKS. You'll save more (3 min 10 sec) if you don't need a 'clean'.
You'll save the most (about 3 min 20 sec) if you can skip rebuilding GWT.
Here are some statistics (3 runs per command, for averages sake)
My test machine is a
Intel(R) Core(TM) i5-3570 CPU @ 3.40GHz
256GB Samsung SSD (old, using SATA)
32GB DDR3 @ 1333 MT/s
---
fresh master checkout
no clean, no gwt, skip checks:
make install-dev PREFIX=/home/greg/ovirt-engine BUILD_GWT=0 SKIP_CHECKS=1
1:04.38 total
1:06.04 total
1:04.41 total
same as above, but now change one non-GWT class in the utils project
make install-dev PREFIX=/home/greg/ovirt-engine BUILD_GWT=0 SKIP_CHECKS=1
1:05.49 total
Ok, pretend we recompiled and our utils class change had an error.
Same as above, change the one non-GWT class in the utils project again, and
use the resume (-pl) functionality:
make install-dev PREFIX=/home/greg/ovirt-engine BUILD_GWT=0 SKIP_CHECKS=1
EXTRA_BUILD_FLAGS="-pl org.ovirt.engine.core:utils"
17.958 total (yes, 18 seconds, obviously a world record for any engine
build)
[The resume piece is EXTRA_BUILD_FLAGS="-pl org.ovirt.engine.ui:webadmin"
-- I personally need to use this more often!]
---
no clean, no gwt, run checks (but skip unit tests):
make install-dev PREFIX=/home/greg/ovirt-engine BUILD_GWT=0 BUILD_UT=0
3:07.66 total
3:16.69 total
3:14.28 total
clean, no gwt, skip checks:
make clean install-dev PREFIX=/home/greg/ovirt-engine BUILD_GWT=0
SKIP_CHECKS=1
4:17.74 total
4:15.76 total
4:13.90 total
no clean, gwt (1 permutation for chrome), skip checks:
make install-dev PREFIX=/home/greg/ovirt-engine BUILD_GWT=1
SKIP_CHECKS=1 DEV_EXTRA_BUILD_FLAGS_GWT_DEFAULTS="-Dgwt.userAgent=safari"
4:25.63 total
4:26.71 total
4:30.69 total
no clean, gwt (1 permutation for chrome), run checks but skip unit tests:
make install-dev PREFIX=/home/greg/ovirt-engine BUILD_GWT=1 BUILD_UT=0
DEV_EXTRA_BUILD_FLAGS_GWT_DEFAULTS="-Dgwt.userAgent=safari"
6:44.23 total
And the whole enchilada [3] ... clean, gwt (1 permutation for chrome), run
checks, run unit tests:
11:52.25 total ... wow :)
Hope this helps you have a faster change-compile-check cycle :)
Best wishes,
Greg
[1] https://gerrit.ovirt.org/#/c/83961/
[2] https://www.mail-archive.com/devel@ovirt.org/msg10377.html
[3] https://idioms.thefreedictionary.com/whole+enchilada
--
GREG SHEREMETA
SENIOR SOFTWARE ENGINEER - TEAM LEAD - RHV UX
Red Hat NA
<https://www.redhat.com/>
gshereme(a)redhat.com IRC: gshereme
<https://red.ht/sig>
3
2
[ OST Failure Report ] [ oVirt master ] [ 2017-11-13 ] [add_master_storage_domain]
by Barak Korren 13 Nov '17
by Barak Korren 13 Nov '17
13 Nov '17
Test failed: [ add_master_storage_domain ]
Link to suspected patches:
- https://gerrit.ovirt.org/#/c/83849/4
This seems to be a fairly consistent regression as all patches that
follow the patch above exhibit the same issue when tested, this list
of similarly failing patches includes:
- https://gerrit.ovirt.org/#/c/83940/2
- https://gerrit.ovirt.org/#/c/83608/7
- https://gerrit.ovirt.org/#/c/83815/3
The following patch is an exception because it simply failed to build:
- https://gerrit.ovirt.org/#/c/83370/9
Link to Job:
http://jenkins.ovirt.org/job/ovirt-master_change-queue-tester/3770/
Link to all logs:
http://jenkins.ovirt.org/job/ovirt-master_change-queue-tester/3770/artifact…
Error snippet from log:
<error>
Fault reason is "Operation Failed". Fault detail is "[Failed to attach
Storage due to an error on the Data Center master Storage Domain.
-Please activate the master Storage Domain first.]". HTTP response code is 409.
</error>
--
Barak Korren
RHV DevOps team , RHCE, RHCi
Red Hat EMEA
redhat.com | TRIED. TESTED. TRUSTED. | redhat.com/trusted
3
8
Hi,
I'm seeing a weird issue in ovirt-engine check-patch.
There's no way this failure is related to my patch [
https://gerrit.ovirt.org/#/c/83950/] which is a 1 character typo fix in
the constants file.
Anyone have any ideas or also seeing this? It seems random, too --
sometimes 1 or 2 of the jobs will succeed [el7, fc25] but the other will
fail. Etc.
*14:48:08* + automation/packaging-setup-tests.sh*14:48:08* + trap popd
0*14:48:08* +++ readlink -f
automation/packaging-setup-tests.sh*14:48:08* ++ dirname
/home/jenkins/workspace/ovirt-engine_master_check-patch-fc26-x86_64/ovirt-engine/automation/packaging-setup-tests.sh*14:48:08*
+ pushd /home/jenkins/workspace/ovirt-engine_master_check-patch-fc26-x86_64/ovirt-engine/automation/..*14:48:08*
~ ~*14:48:08* + export
PYTHONPATH=packaging/pythonlib:packaging/setup:*14:48:08* +
PYTHONPATH=packaging/pythonlib:packaging/setup:*14:48:08* + python -m
pytest packaging/setup*14:48:08* ============================= test
session starts ==============================*14:48:08* platform
linux2 -- Python 2.7.14, pytest-3.2.3, py-1.4.34,
pluggy-0.4.0*14:48:08* rootdir:
/home/jenkins/workspace/ovirt-engine_master_check-patch-fc26-x86_64/ovirt-engine,
inifile:*14:48:08* collected 0 items / 1 errors*14:48:08* *14:48:08*
==================================== ERRORS
====================================*14:48:08* ERROR collecting
packaging/setup/tests/ovirt_engine_setup/engine_common/test_database.py
*14:48:08* ImportError while importing test module
'/home/jenkins/workspace/ovirt-engine_master_check-patch-fc26-x86_64/ovirt-engine/packaging/setup/tests/ovirt_engine_setup/engine_common/test_database.py'.*14:48:08*
Hint: make sure your test modules/packages have valid Python
names.*14:48:08* Traceback:*14:48:08*
packaging/setup/tests/ovirt_engine_setup/engine_common/test_database.py:19:
in <module>*14:48:08* import
ovirt_engine_setup.engine_common.database as under_test # isort:skip
# noqa: E402*14:48:08*
packaging/setup/ovirt_engine_setup/engine_common/database.py:30: in
<module>*14:48:08* from otopi import base*14:48:08* E
ImportError: No module named otopi*14:48:08* !!!!!!!!!!!!!!!!!!!
Interrupted: 1 errors during collection !!!!!!!!!!!!!!!!!!!!*14:48:08*
=========================== 1 error in 0.33 seconds
============================*14:48:08* + popd*14:48:08* ~
---------- Forwarded message ----------
From: Code Review <gerrit(a)ovirt.org>
Date: Sun, Nov 12, 2017 at 10:31 AM
Subject: Change in ovirt-engine[master]: engine: typo fix snaphot ->
snapshot
To: Greg Sheremeta <gshereme(a)redhat.com>
Jenkins CI *posted comments* on this change.
View Change <https://gerrit.ovirt.org/83950>
Patch set 1:Continuous-Integration -1
Build Failed
http://jenkins.ovirt.org/job/ovirt-engine_master_check-
patch-fc25-x86_64/17231/ : FAILURE
http://jenkins.ovirt.org/job/ovirt-engine_master_check-
patch-fc26-x86_64/157/ : FAILURE
http://jenkins.ovirt.org/job/ovirt-engine_master_check-
patch-el7-x86_64/33193/ : SUCCESS
To view, visit change 83950 <https://gerrit.ovirt.org/83950>. To
unsubscribe, visit settings <https://gerrit.ovirt.org/settings>.
Gerrit-Project: ovirt-engine
Gerrit-Branch: master
Gerrit-MessageType: comment
Gerrit-Change-Id: I88b4d75855c0abe2f0eddd8b282c172205db2e42
Gerrit-Change-Number: 83950
Gerrit-PatchSet: 1
Gerrit-Owner: Greg Sheremeta <gshereme(a)redhat.com>
Gerrit-Reviewer: Alexander Wels <awels(a)redhat.com>
Gerrit-Reviewer: Greg Sheremeta <gshereme(a)redhat.com>
Gerrit-Reviewer: Jenkins CI
Gerrit-Reviewer: Tal Nisan <tnisan(a)redhat.com>
Gerrit-Reviewer: Vojtech Szocs <vszocs(a)redhat.com>
Gerrit-Reviewer: gerrit-hooks <automation(a)ovirt.org>
Gerrit-Comment-Date: Sun, 12 Nov 2017 15:31:27 +0000
Gerrit-HasComments: No
--
GREG SHEREMETA
SENIOR SOFTWARE ENGINEER - TEAM LEAD - RHV UX
Red Hat NA
<https://www.redhat.com/>
gshereme(a)redhat.com IRC: gshereme
<https://red.ht/sig>
3
2
Hi,
we are now targeting oVirt 4.2.0 GA around December 12th and oVirt 4.1.8
around December 5th.
Against tradition that would have set 4.1.8 as last 4.1 release, this round
we'll release also oVIrt 4.1.9 being it already in development just before
4.2.0 GA.
We suggest to be prepared to upgrade to 4.2 since 4.1 won't receive any
further update after 4.1.9 release.
--
SANDRO BONAZZOLA
ASSOCIATE MANAGER, SOFTWARE ENGINEERING, EMEA ENG VIRTUALIZATION R&D
Red Hat EMEA <https://www.redhat.com/>
<https://red.ht/sig>
TRIED. TESTED. TRUSTED. <https://redhat.com/trusted>
<http://www.teraplan.it/redhat-osd-2017/>
1
0
Guys, I'm not sure who did that, but somebody forked the core
'ansible' repo in to the ovirt organisation on GitHub.
IMO this does not look good.
Why was this done?
--
Barak Korren
RHV DevOps team , RHCE, RHCi
Red Hat EMEA
redhat.com | TRIED. TESTED. TRUSTED. | redhat.com/trusted
1
0
12 Nov '17
Hi all,
Some of you might have encountered an error while trying to download the
latest ovirt release (4.1/4.1/master) from resources.ovirt.org/yum-repo/
During the weekend we had an issue with our nightly publisher's job. Those
jobs are responsible to move artifacts from Jenkins to the matching
snapshot repo in resources.ovirt.org.
Since the publishers failed, ovirt-release-master.rpm and
ovirt-release-4.x-snapshot.rpms (which are symlinks to
ovirt-${version}-snapshot) were unavailable.
The nightly publisher is fixed now and, the rpms were properly deployed to
the repo.
Sorry for any inconvenience,
--
DANIEL BELENKY
RHV DEVOPS
EMEA VIRTUALIZATION R&D
<https://red.ht/sig>
3
2
[ OST Failure Report ] [ oVirt 4.1 ] [ 09-11-2017 ] [ 001_upgrade_engine.test_initialize_engine ]
by Dafna Ron 09 Nov '17
by Dafna Ron 09 Nov '17
09 Nov '17
This is a multi-part message in MIME format.
--------------1E382631B1CFE8CFC947FDB5
Content-Type: text/plain; charset=utf-8
Content-Transfer-Encoding: 7bit
Hi,
We had a failure in installing engine with a dependency error between
ovirt-engine-backend and vdsm-jsonrpc-java
Please note that we have other issues which are not related to this
patch so it might be hard to find this issue.
I also added the lago log where you can find the package dependency error.
**
*Link to suspected patches: https://gerrit.ovirt.org/#/c/82995/*
*
Link to Job:http://jenkins.ovirt.org/job/ovirt-4.1_change-queue-tester/1291
*
http://jenkins.ovirt.org/job/ovirt-4.1_change-queue-tester/1291/artifact/ex…
*
*
*Link to all logs:
http://jenkins.ovirt.org/job/ovirt-4.1_change-queue-tester/1291/artifact*
*
(Relevant) error snippet from the log:
<error>
*
[ INFO ] Checking for an update for Setup...
[ ERROR ] Yum [u'ovirt-engine-backend-4.1.7.7-0.0.master.20171109084643.gitcd8d74c.el7.centos.noarch requires vdsm-jsonrpc-java >= 1.3.14']
[ INFO ] Yum Performing yum transaction rollback
[ ERROR ] Failed to execute stage 'Environment customization': [u'ovirt-engine-backend-4.1.7.7-0.0.master.20171109084643.gitcd8d74c.el7.centos.noarch requires vdsm-jsonrpc-java >= 1.3.14']
[ INFO ] Stage: Clean up
Log file is located at /var/log/ovirt-engine/setup/ovirt-engine-setup-20171109042111-tpncix.log
[ INFO ] Generating answer file '/var/lib/ovirt-engine/setup/answers/20171109042115-setup.conf'
[ INFO ] Stage: Pre-termination
[ INFO ] Stage: Termination
[ ERROR ] Execution of setup failed
**
*</error>*
**
--------------1E382631B1CFE8CFC947FDB5
Content-Type: text/html; charset=utf-8
Content-Transfer-Encoding: 7bit
<html>
<head>
<meta http-equiv="content-type" content="text/html; charset=utf-8">
</head>
<body text="#000000" bgcolor="#FFFFFF">
<p>Hi, <br>
</p>
<p>We had a failure in installing engine with a dependency error
between ovirt-engine-backend and vdsm-jsonrpc-java</p>
<p>Please note that we have other issues which are not related to
this patch so it might be hard to find this issue. <br>
</p>
<p>I also added the lago log where you can find the package
dependency error. <br>
</p>
<p><b style="font-weight:normal;"
id="docs-internal-guid-5859b7a1-a107-cb96-32a9-233fa9d2441d">
<p dir="ltr"
style="line-height:1.38;margin-top:0pt;margin-bottom:0pt;"><span style="font-size:11pt;font-family:Arial;color:#000000;background-color:transparent;font-weight:400;font-style:normal;font-variant:normal;text-decoration:none;vertical-align:baseline;white-space:pre-wrap;">Link to suspected patches: <a class="moz-txt-link-freetext" href="https://gerrit.ovirt.org/#/c/82995/">https://gerrit.ovirt.org/#/c/82995/</a></span></p>
<br>
<p dir="ltr"
style="line-height:1.38;margin-top:0pt;margin-bottom:0pt;"><span style="font-size:11pt;font-family:Arial;color:#000000;background-color:transparent;font-weight:400;font-style:normal;font-variant:normal;text-decoration:none;vertical-align:baseline;white-space:pre-wrap;">Link to Job:<a class="moz-txt-link-freetext" href="http://jenkins.ovirt.org/job/ovirt-4.1_change-queue-tester/1291">http://jenkins.ovirt.org/job/ovirt-4.1_change-queue-tester/1291</a></span></p>
<br>
</b></p>
<p><span style="font-weight:normal;"><a class="moz-txt-link-freetext" href="http://jenkins.ovirt.org/job/ovirt-4.1_change-queue-tester/1291/artifact/ex…">http://jenkins.ovirt.org/job/ovirt-4.1_change-queue-tester/1291/artifact/ex…</a></span></p>
<p><b style="font-weight:normal;"
id="docs-internal-guid-5859b7a1-a107-cb96-32a9-233fa9d2441d"><br>
<p dir="ltr"
style="line-height:1.38;margin-top:0pt;margin-bottom:0pt;"><span style="font-size:11pt;font-family:Arial;color:#000000;background-color:transparent;font-weight:400;font-style:normal;font-variant:normal;text-decoration:none;vertical-align:baseline;white-space:pre-wrap;">Link to all logs: <a class="moz-txt-link-freetext" href="http://jenkins.ovirt.org/job/ovirt-4.1_change-queue-tester/1291/artifact">http://jenkins.ovirt.org/job/ovirt-4.1_change-queue-tester/1291/artifact</a></span></p>
<br>
<p dir="ltr"
style="line-height:1.38;margin-top:0pt;margin-bottom:0pt;"><span style="font-size:11pt;font-family:Arial;color:#000000;background-color:transparent;font-weight:400;font-style:normal;font-variant:normal;text-decoration:none;vertical-align:baseline;white-space:pre-wrap;">(Relevant) error snippet from the log: </span></p>
<p dir="ltr"
style="line-height:1.38;margin-top:0pt;margin-bottom:0pt;"><span style="font-size:11pt;font-family:Arial;color:#000000;background-color:transparent;font-weight:400;font-style:normal;font-variant:normal;text-decoration:none;vertical-align:baseline;white-space:pre-wrap;"><error></span></p>
</b></p>
<p><br>
</p>
<pre style="color: rgb(0, 0, 0); font-style: normal; font-variant-ligatures: normal; font-variant-caps: normal; font-weight: normal; letter-spacing: normal; orphans: 2; text-align: start; text-indent: 0px; text-transform: none; widows: 2; word-spacing: 0px; -webkit-text-stroke-width: 0px; text-decoration-style: initial; text-decoration-color: initial;">[ INFO ] Checking for an update for Setup...
[ ERROR ] Yum [u'ovirt-engine-backend-4.1.7.7-0.0.master.20171109084643.gitcd8d74c.el7.centos.noarch requires vdsm-jsonrpc-java >= 1.3.14']
[ INFO ] Yum Performing yum transaction rollback
[ ERROR ] Failed to execute stage 'Environment customization': [u'ovirt-engine-backend-4.1.7.7-0.0.master.20171109084643.gitcd8d74c.el7.centos.noarch requires vdsm-jsonrpc-java >= 1.3.14']
[ INFO ] Stage: Clean up
Log file is located at /var/log/ovirt-engine/setup/ovirt-engine-setup-20171109042111-tpncix.log
[ INFO ] Generating answer file '/var/lib/ovirt-engine/setup/answers/20171109042115-setup.conf'
[ INFO ] Stage: Pre-termination
[ INFO ] Stage: Termination
[ ERROR ] Execution of setup failed
</pre>
<br class="Apple-interchange-newline">
<p><b style="font-weight:normal;"
id="docs-internal-guid-5859b7a1-a107-cb96-32a9-233fa9d2441d">
<p dir="ltr"
style="line-height:1.38;margin-top:0pt;margin-bottom:0pt;"><span style="font-size:11pt;font-family:Arial;color:#000000;background-color:transparent;font-weight:400;font-style:normal;font-variant:normal;text-decoration:none;vertical-align:baseline;white-space:pre-wrap;"></error></span></p>
</b><br class="Apple-interchange-newline">
</p>
</body>
</html>
--------------1E382631B1CFE8CFC947FDB5--
3
3
Hi,
I ran into a problem after building ovirt-engine from current master to
verify my last outstanding patch: but now I can't start my VM anymore,
because the <disk> element is omitted from the final libvirt XML created
by vdsm.
The xml argument sent by the engine for the VM.create call contains a
valid disk definition, but it disappears from the final libvirt XML.
The preceding prepareImage call returns a seemingly valid path info...
I am currently at loss and would appreciate any hints.
--
Mit freundlichen Grüßen/Kind Regards
Viktor Mihajlovski
1
1
This is a multi-part message in MIME format.
--------------C2C1B6C457A9CCD4A0E14F7B
Content-Type: text/plain; charset=utf-8
Content-Transfer-Encoding: 7bit
Hi,
We had some confusion last week when I sent a message about an
infrastructure related issue we have found and people thought I was
reporting an error on OST so I decided that perhaps it would be a good
idea to create a general message that would alert people on an on-going
infra event and not just OST related issues.
iBelow is the format I was thinking of using:
##############################################
Subject: Ovirt CI - [Infrastructure Incident] [$Date] [$TIME]
Hello,
We encountered an incident that can suggest there is an ongoing
infrastructure incident.
Our team is currently investigating the issue and we will send a mail
once we have more information or the issue is resolved.
Issue summary:
################################################
Please let me know what you're ideas on this are and if the message is
at all lacking in some way.
Thanks.
Dafna
--------------C2C1B6C457A9CCD4A0E14F7B
Content-Type: text/html; charset=utf-8
Content-Transfer-Encoding: 7bit
<html>
<head>
<meta http-equiv="content-type" content="text/html; charset=utf-8">
</head>
<body text="#000000" bgcolor="#FFFFFF">
<p>Hi, <br>
</p>
<p>We had some confusion last week when I sent a message about an
infrastructure related issue we have found and people thought I
was reporting an error on OST so I decided that perhaps it would
be a good idea to create a general message that would alert people
on an on-going infra event and not just OST related issues. <br>
</p>
<p>iBelow is the format I was thinking of using:<br>
</p>
<p><span style="font-weight:normal;">
<p dir="ltr"
style="line-height:1.38;margin-top:0pt;margin-bottom:0pt;"><span style="font-size:11pt;font-family:Arial;color:#000000;background-color:transparent;font-weight:400;font-style:normal;font-variant:normal;text-decoration:none;vertical-align:baseline;white-space:pre-wrap;">##############################################
</span></p>
<p dir="ltr"
style="line-height:1.38;margin-top:0pt;margin-bottom:0pt;"><span style="font-size:11pt;font-family:Arial;color:#000000;background-color:transparent;font-weight:400;font-style:normal;font-variant:normal;text-decoration:none;vertical-align:baseline;white-space:pre-wrap;">Subject: Ovirt CI - [Infrastructure Incident] [$Date] [$TIME] </span></p>
<br>
<p dir="ltr"
style="line-height:1.38;margin-top:0pt;margin-bottom:0pt;"><span style="font-size:11pt;font-family:Arial;color:#000000;background-color:transparent;font-weight:400;font-style:normal;font-variant:normal;text-decoration:none;vertical-align:baseline;white-space:pre-wrap;">Hello, </span></p>
<br>
<p dir="ltr"
style="line-height:1.38;margin-top:0pt;margin-bottom:0pt;"><span style="font-size:11pt;font-family:Arial;color:#000000;background-color:transparent;font-weight:400;font-style:normal;font-variant:normal;text-decoration:none;vertical-align:baseline;white-space:pre-wrap;">We encountered an incident that can suggest there is an ongoing infrastructure incident. </span></p>
<p dir="ltr"
style="line-height:1.38;margin-top:0pt;margin-bottom:0pt;"><span style="font-size:11pt;font-family:Arial;color:#000000;background-color:transparent;font-weight:400;font-style:normal;font-variant:normal;text-decoration:none;vertical-align:baseline;white-space:pre-wrap;">Our team is currently investigating the issue and we will send a mail once we have more information or the issue is resolved. </span></p>
<br>
<p dir="ltr"
style="line-height:1.38;margin-top:0pt;margin-bottom:0pt;"><span style="font-size:11pt;font-family:Arial;color:#000000;background-color:transparent;font-weight:400;font-style:normal;font-variant:normal;text-decoration:none;vertical-align:baseline;white-space:pre-wrap;">Issue </span><span style="font-size:11pt;font-family:Arial;color:#000000;background-color:transparent;font-weight:400;font-style:normal;font-variant:normal;text-decoration:none;vertical-align:baseline;white-space:pre-wrap;">summary</span><span style="font-size:11pt;font-family:Arial;color:#000000;background-color:transparent;font-weight:400;font-style:normal;font-variant:normal;text-decoration:none;vertical-align:baseline;white-space:pre-wrap;">:
</span></p>
<p dir="ltr"
style="line-height:1.38;margin-top:0pt;margin-bottom:0pt;"><span style="font-size:11pt;font-family:Arial;color:#000000;background-color:transparent;font-weight:400;font-style:normal;font-variant:normal;text-decoration:none;vertical-align:baseline;white-space:pre-wrap;">
</span></p>
<p dir="ltr"
style="line-height:1.38;margin-top:0pt;margin-bottom:0pt;"><span style="font-size:11pt;font-family:Arial;color:#000000;background-color:transparent;font-weight:400;font-style:normal;font-variant:normal;text-decoration:none;vertical-align:baseline;white-space:pre-wrap;">################################################
</span></p>
</span></p>
<p>Please let me know what you're ideas on this are and if the
message is at all lacking in some way. <br>
</p>
<p>Thanks. <br>
</p>
<p>Dafna</p>
<p><br class="Apple-interchange-newline">
</p>
</body>
</html>
--------------C2C1B6C457A9CCD4A0E14F7B--
2
1
[ OST Failure Report ] [ oVirt master ] [ 07-11-2017 ] [ 004_basic_sanity.disk_operations ]
by Dafna Ron 08 Nov '17
by Dafna Ron 08 Nov '17
08 Nov '17
This is a multi-part message in MIME format.
--------------106EB129C1BC1AEAACE8C19F
Content-Type: text/plain; charset=utf-8
Content-Transfer-Encoding: 7bit
Hi,
We have a failure in test 004_basic_sanity.disk_operations.
>From the log I can see that vdsm has a KeyError: 'diskReplicate' and
than reports the drive not found.
Can you please check the issue?
**
*Link to suspected patches: https://gerrit.ovirt.org/#/c/83641/*
*
Link to Job:
http://jenkins.ovirt.org/job/ovirt-master_change-queue-tester/3653
Link to all logs:
http://jenkins.ovirt.org/job/ovirt-master_change-queue-tester/3653/artifact/
engine log:
http://jenkins.ovirt.org/job/ovirt-master_change-queue-tester/3653/artifact…
vdsm log:
**
http://jenkins.ovirt.org/job/ovirt-master_change-queue-tester/3653/artifact…
(Relevant) error snippet from the log:
<error>
>From engine log:
*
2017-11-07 09:20:42,396-05 ERROR
[org.ovirt.engine.core.vdsbroker.vdsbroker.VmReplicateDiskFinishVDSCommand]
(EE-ManagedThreadFactory-engineScheduled-Thread-72) [4bcdd94e] Failed in
'VmReplicateDiskFinishVDS' method
2017-11-07 09:20:42,399-05 ERROR
[org.ovirt.engine.core.dal.dbbroker.auditloghandling.AuditLogDirector]
(EE-ManagedThreadFactory-engineScheduled-Thread-72) [4bcdd94e] EVENT_ID:
VDS_BROKER_COMMAND_FAILURE(10,802), VDSM lago-basic-suite-mas
ter-host-0 command VmReplicateDiskFinishVDS failed: Drive image file
could not be found
2017-11-07 09:20:42,399-05 INFO
[org.ovirt.engine.core.vdsbroker.vdsbroker.VmReplicateDiskFinishVDSCommand]
(EE-ManagedThreadFactory-engineScheduled-Thread-72) [4bcdd94e] Command
'org.ovirt.engine.core.vdsbroker.vdsbroker.VmReplicateDisk
FinishVDSCommand' return value 'StatusOnlyReturn [status=Status
[code=13, message=Drive image file could not be found]]'
2017-11-07 09:20:42,399-05 INFO
[org.ovirt.engine.core.vdsbroker.vdsbroker.VmReplicateDiskFinishVDSCommand]
(EE-ManagedThreadFactory-engineScheduled-Thread-72) [4bcdd94e] HostName
= lago-basic-suite-master-host-0
2017-11-07 09:20:42,399-05 ERROR
[org.ovirt.engine.core.vdsbroker.vdsbroker.VmReplicateDiskFinishVDSCommand]
(EE-ManagedThreadFactory-engineScheduled-Thread-72) [4bcdd94e] Command
'VmReplicateDiskFinishVDSCommand(HostName = lago-basic-sui
te-master-host-0,
VmReplicateDiskParameters:{hostId='038b0c9a-f64c-4aa7-8e36-72e435771dab',
vmId='61d328bc-c920-4691-997f-09bbecb5de10',
storagePoolId='b20f0eb7-6bea-4994-b87b-e1080c4fd9e5',
srcStorageDomainId='4a088fb2-5497-4b4d-adec-4c9
1d998c56e',
targetStorageDomainId='4a088fb2-5497-4b4d-adec-4c91d998c56e',
imageGroupId='36837a9b-6396-4d65-a354-df97ce808f01',
imageId='8a1660d7-f153-4aca-9f25-c1184c10841a'})' execution failed:
VDSGenericException: VDSErrorException: Fai
led to VmReplicateDiskFinishVDS, error = Drive image file could not be
found, code = 13
2017-11-07 09:20:42,399-05 DEBUG
[org.ovirt.engine.core.vdsbroker.vdsbroker.VmReplicateDiskFinishVDSCommand]
(EE-ManagedThreadFactory-engineScheduled-Thread-72) [4bcdd94e]
Exception: org.ovirt.engine.core.vdsbroker.vdsbroker.VDSErrorExcep
tion: VDSGenericException: VDSErrorException: Failed to
VmReplicateDiskFinishVDS, error = Drive image file could not be found,
code = 13
at
org.ovirt.engine.core.vdsbroker.vdsbroker.VdsBrokerCommand.createDefaultConcreteException(VdsBrokerCommand.java:81)
[vdsbroker.jar:]
at
org.ovirt.engine.core.vdsbroker.vdsbroker.BrokerCommandBase.createException(BrokerCommandBase.java:223)
[vdsbroker.jar:]
at
org.ovirt.engine.core.vdsbroker.vdsbroker.BrokerCommandBase.proceedProxyReturnValue(BrokerCommandBase.java:193)
[vdsbroker.jar:]
at
org.ovirt.engine.core.vdsbroker.vdsbroker.VmReplicateDiskFinishVDSCommand.executeVdsBrokerCommand(VmReplicateDiskFinishVDSCommand.java:15)
[vdsbroker.jar:]
at
org.ovirt.engine.core.vdsbroker.vdsbroker.VdsBrokerCommand.executeVDSCommand(VdsBrokerCommand.java:112)
[vdsbroker.jar:]
at
org.ovirt.engine.core.vdsbroker.VDSCommandBase.executeCommand(VDSCommandBase.java:73)
[vdsbroker.jar:]
at
org.ovirt.engine.core.dal.VdcCommandBase.execute(VdcCommandBase.java:33)
[dal.jar:]
at
org.ovirt.engine.core.vdsbroker.vdsbroker.DefaultVdsCommandExecutor.execute(DefaultVdsCommandExecutor.java:14)
[vdsbroker.jar:]
at
org.ovirt.engine.core.vdsbroker.ResourceManager.runVdsCommand(ResourceManager.java:387)
[vdsbroker.jar:]
at
org.ovirt.engine.core.vdsbroker.ResourceManager$Proxy$_$$_WeldSubclass.runVdsCommand$$super(Unknown
Source) [vdsbroker.jar:]
at sun.reflect.GeneratedMethodAccessor244.invoke(Unknown Source)
[:1.8.0_151]
at
sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:43)
[rt.jar:1.8.0_151]
at java.lang.reflect.Method.invoke(Method.java:498)
[rt.jar:1.8.0_151]
at
org.jboss.weld.interceptor.proxy.TerminalAroundInvokeInvocationContext.proceedInternal(TerminalAroundInvokeInvocationContext.java:49)
[weld-core-impl-2.4.3.Final.jar:2.4.3.Final]
at
org.jboss.weld.interceptor.proxy.AroundInvokeInvocationContext.proceed(AroundInvokeInvocationContext.java:77)
[weld-core-impl-2.4.3.Final.jar:2.4.3.Final]
at
org.ovirt.engine.core.common.di.interceptor.LoggingInterceptor.apply(LoggingInterceptor.java:12)
[common.jar:]
at sun.reflect.GeneratedMethodAccessor69.invoke(Unknown Source)
[:1.8.0_151]
at
sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:43)
[rt.jar:1.8.0_151]
at java.lang.reflect.Method.invoke(Method.java:498)
[rt.jar:1.8.0_151]
at
org.jboss.weld.interceptor.reader.SimpleInterceptorInvocation$SimpleMethodInvocation.invoke(SimpleInterceptorInvocation.java:73)
[weld-core-impl-2.4.3.Final.jar:2.4.3.Final]
at
org.jboss.weld.interceptor.proxy.InterceptorMethodHandler.executeAroundInvoke(InterceptorMethodHandler.java:84)
[weld-core-impl-2.4.3.Final.jar:2.4.3.Final]
at
org.jboss.weld.interceptor.proxy.InterceptorMethodHandler.executeInterception(InterceptorMethodHandler.java:72)
[weld-core-impl-2.4.3.Final.jar:2.4.3.Final]
at
org.jboss.weld.interceptor.proxy.InterceptorMethodHandler.invoke(InterceptorMethodHandler.java:56)
[weld-core-impl-2.4.3.Final.jar:2.4.3.Final]
at
org.jboss.weld.bean.proxy.CombinedInterceptorAndDecoratorStackMethodHandler.invoke(CombinedInterceptorAndDecoratorStackMethodHandler.java:79)
[weld-core-impl-2.4.3.Final.jar:2.4.3.Final]
at
org.jboss.weld.bean.proxy.CombinedInterceptorAndDecoratorStackMethodHandler.invoke(CombinedInterceptorAndDecoratorStackMethodHandler.java:68)
[weld-core-impl-2.4.3.Final.jar:2.4.3.Final]
at
org.ovirt.engine.core.vdsbroker.ResourceManager$Proxy$_$$_WeldSubclass.runVdsCommand(Unknown
Source) [vdsbroker.jar:]
at
org.ovirt.engine.core.bll.storage.lsm.LiveMigrateDiskCommand.replicateDiskFinish(LiveMigrateDiskCommand.java:374)
[bll.jar:]
at
org.ovirt.engine.core.bll.storage.lsm.LiveMigrateDiskCommand.handleDestDisk(LiveMigrateDiskCommand.java:681)
[bll.jar:]
at
org.ovirt.engine.core.bll.storage.lsm.LiveMigrateDiskCommand.endWithFailure(LiveMigrateDiskCommand.java:337)
[bll.jar:]
at
org.ovirt.engine.core.bll.CommandBase.internalEndWithFailure(CommandBase.java:783)
[bll.jar:]
at
org.ovirt.engine.core.bll.CommandBase.endActionInTransactionScope(CommandBase.java:712)
[bll.jar:]
at
org.ovirt.engine.core.bll.CommandBase.runInTransaction(CommandBase.java:1989)
[bll.jar:]
at
org.ovirt.engine.core.utils.transaction.TransactionSupport.executeInNewTransaction(TransactionSupport.java:202)
[utils.jar:]
at
org.ovirt.engine.core.utils.transaction.TransactionSupport.executeInRequired(TransactionSupport.java:137)
[utils.jar:]
at
org.ovirt.engine.core.utils.transaction.TransactionSupport.executeInScope(TransactionSupport.java:105)
[utils.jar:]
at
org.ovirt.engine.core.bll.CommandBase.endAction(CommandBase.java:574)
[bll.jar:]
at
org.ovirt.engine.core.bll.ChildCommandsCallbackBase.endAction(ChildCommandsCallbackBase.java:125)
[bll.jar:]
at
org.ovirt.engine.core.bll.ChildCommandsCallbackBase.onFailed(ChildCommandsCallbackBase.java:156)
[bll.jar:]
at
org.ovirt.engine.core.bll.tasks.CommandCallbacksPoller.endCallback(CommandCallbacksPoller.java:65)
[bll.jar:]
at
org.ovirt.engine.core.bll.tasks.CommandCallbacksPoller.invokeCallbackMethodsImpl(CommandCallbacksPoller.java:141)
[bll.jar:]
at
org.ovirt.engine.core.bll.tasks.CommandCallbacksPoller.invokeCallbackMethods(CommandCallbacksPoller.java:106)
[bll.jar:]
at
java.util.concurrent.Executors$RunnableAdapter.call(Executors.java:511)
[rt.jar:1.8.0_151]
at
java.util.concurrent.FutureTask.runAndReset(FutureTask.java:308)
[rt.jar:1.8.0_151]
at
org.glassfish.enterprise.concurrent.internal.ManagedScheduledThreadPoolExecutor$ManagedScheduledFutureTask.access$201(ManagedScheduledThreadPoolExecutor.java:383)
[javax.enterprise.concurrent-1.0.jar:]
at
org.glassfish.enterprise.concurrent.internal.ManagedScheduledThreadPoolExecutor$ManagedScheduledFutureTask.run(ManagedScheduledThreadPoolExecutor.java:534)
[javax.enterprise.concurrent-1.0.jar:]
at
java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1149)
[rt.jar:1.8.0_151]
:
at
java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:624)
[rt.jar:1.8.0_151]
at java.lang.Thread.run(Thread.java:748) [rt.jar:1.8.0_151]
at
org.glassfish.enterprise.concurrent.ManagedThreadFactoryImpl$ManagedThread.run(ManagedThreadFactoryImpl.java:250)
[javax.enterprise.concurrent-1.0.jar:]
at
org.jboss.as.ee.concurrent.service.ElytronManagedThreadFactory$ElytronManagedThread.run(ElytronManagedThreadFactory.java:78)
2017-11-07 09:20:42,400-05 INFO
[org.ovirt.engine.core.vdsbroker.vdsbroker.VmReplicateDiskFinishVDSCommand]
(EE-ManagedThreadFactory-engineScheduled-Thread-72) [4bcdd94e] FINISH,
VmReplicateDiskFinishVDSCommand, log id: 32c5a56e
2017-11-07 09:20:42,400-05 DEBUG
[org.ovirt.engine.core.common.di.interceptor.DebugLoggingInterceptor]
(EE-ManagedThreadFactory-engineScheduled-Thread-72) [4bcdd94e] method:
runVdsCommand, params: [VmReplicateDiskFinish,
VmReplicateDiskParameters:{hostId='038b0c9a-f64c-4aa7-8e36-72e435771dab',
vmId='61d328bc-c920-4691-997f-09bbecb5de10',
storagePoolId='b20f0eb7-6bea-4994-b87b-e1080c4fd9e5',
srcStorageDomainId='4a088fb2-5497-4b4d-adec-4c91d998c56e',
targetStorageDomainId='4a088fb2-5497-4b4d-adec-4c91d998c56e',
imageGroupId='36837a9b-6396-4d65-a354-df97ce808f01',
imageId='8a1660d7-f153-4aca-9f25-c1184c10841a'}], timeElapsed: 12ms
2017-11-07 09:20:42,400-05 ERROR
[org.ovirt.engine.core.bll.storage.lsm.LiveMigrateDiskCommand]
(EE-ManagedThreadFactory-engineScheduled-Thread-72) [4bcdd94e]
Replication end of disk '36837a9b-6396-4d65-a354-df97ce808f01' in vm
'61d328bc-c920-4691-997f-09bbecb5de10' back to the source failed,
skipping deletion of the target disk
2017-11-07 09:20:42,408-05 WARN
[org.ovirt.engine.core.bll.lock.InMemoryLockManager]
(EE-ManagedThreadFactory-engineScheduled-Thread-72) [4bcdd94e] Trying to
release exclusive lock which does not exist, lock key:
'61d328bc-c920-4691-997f-09bbecb5de10LIVE_STORAGE_MIGRATION'
2017-11-07 09:20:42,408-05 INFO
[org.ovirt.engine.core.bll.storage.lsm.LiveMigrateDiskCommand]
(EE-ManagedThreadFactory-engineScheduled-Thread-72) [4bcdd94e] Lock
freed to object
'EngineLock:{exclusiveLocks='[61d328bc-c920-4691-997f-09bbecb5de10=LIVE_STORAGE_MIGRATION]',
sharedLocks=''}'
2017-11-07 09:20:42,428-05 DEBUG
[org.ovirt.engine.core.common.di.interceptor.DebugLoggingInterceptor]
(EE-ManagedThreadFactory-engineScheduled-Thread-72) [4bcdd94e] method:
get, params: [038b0c9a-f64c-4aa7-8e36-72e435771dab], timeElapsed: 2ms
2017-11-07 09:20:42,430-05 ERROR
[org.ovirt.engine.core.dal.dbbroker.auditloghandling.AuditLogDirector]
(EE-ManagedThreadFactory-engineScheduled-Thread-72) [4bcdd94e] EVENT_ID:
USER_MOVED_DISK_FINISHED_FAILURE(2,011), User admin@internal-authz have
failed to move disk vm0_disk0 to domain iscsi.**
**
*
*
*
*
vdsm log:
2017-11-07 09:20:41,178-0500 INFO (jsonrpc/6) [vdsm.api] FINISH
getVolumeSize return={'truesize': '2415919104', 'apparentsize':
'2415919104'} from=::ffff:192.168.204.4,49358, flow_id=4bcdd94e,
task_id=042481eb-74ac-42a8-9d1e-8fb1c84db234
(api:52)
2017-11-07 09:20:41,193-0500 ERROR (jsonrpc/6) [api] FINISH
diskReplicateFinish error='diskReplicate' (api:127)
Traceback (most recent call last):
File "/usr/lib/python2.7/site-packages/vdsm/common/api.py", line 117,
in method
ret = func(*args, **kwargs)
File "/usr/lib/python2.7/site-packages/vdsm/API.py", line 558, in
diskReplicateFinish
return self.vm.diskReplicateFinish(srcDisk, dstDisk)
File "/usr/lib/python2.7/site-packages/vdsm/virt/vm.py", line 4477, in
diskReplicateFinish
self._delDiskReplica(drive)
File "/usr/lib/python2.7/site-packages/vdsm/virt/vm.py", line 4528, in
_delDiskReplica
del dev['diskReplicate']
KeyError: 'diskReplicate'
2017-11-07 09:20:41,195-0500 INFO (jsonrpc/6) [api.virt] FINISH
diskReplicateFinish return={'status': {'message': 'General Exception:
("\'diskReplicate\'",)', 'code': 100}} from=::ffff:192.168.204.4,49358,
flow_id=4bcdd94e (api:52)
2017-11-07 09:20:41,195-0500 INFO (jsonrpc/6) [jsonrpc.JsonRpcServer]
RPC call VM.diskReplicateFinish failed (error 100) in 0.79 seconds
(__init__:630)
2017-11-07 09:20:41,253-0500 INFO (jsonrpc/2) [vdsm.api] START
finalizeMerge(spUUID=u'b20f0eb7-6bea-4994-b87b-e1080c4fd9e5',
subchainInfo={u'img_id': u'8401cd0f-419c-417c-8da3-52da285fa47c',
u'sd_id': u'4a088fb2-5497-4b4d-adec-4c91d998c5
6e', u'top_id': u'7a295270-fdf5-4561-b688-033c5d312d1a', u'base_id':
u'e77f8714-9e0f-4f39-8437-e75a3fa44c74'})
from=::ffff:192.168.204.4,49380,
flow_id=56c9f051-c091-49cd-9fd5-cc98e92ef05f,
task_id=9ce348bc-e3fb-4f4e-aea0-f8c79e7ae204 (ap
i:46)
2017-11-07 09:20:41,267-0500 INFO (jsonrpc/2) [vdsm.api] FINISH
finalizeMerge return=None from=::ffff:192.168.204.4,49380,
flow_id=56c9f051-c091-49cd-9fd5-cc98e92ef05f,
task_id=9ce348bc-e3fb-4f4e-aea0-f8c79e7ae204 (api:52)
2017-11-07 09:20:41,285-0500 INFO (jsonrpc/2) [jsonrpc.JsonRpcServer]
RPC call StoragePool.finalizeMerge succeeded in 0.03 seconds (__init__:630)
2017-11-07 09:20:41,285-0500 INFO (tasks/8)
[storage.ThreadPool.WorkerThread] START task
9ce348bc-e3fb-4f4e-aea0-f8c79e7ae204 (cmd=<bound method Task.commit of
<vdsm.storage.task.Task instance at 0x2f0a560>>, args=None) (threadPool:208)
2017-11-07 09:20:41,295-0500 INFO (tasks/8) [storage.merge] Finalizing
subchain after merge: <SubchainInfo
sd_id=4a088fb2-5497-4b4d-adec-4c91d998c56e,
img_id=8401cd0f-419c-417c-8da3-52da285fa47c,
top_id=7a295270-fdf5-4561-b688-033c5d312d
1a, base_id=e77f8714-9e0f-4f39-8437-e75a3fa44c74 base_generation=None at
0x30a7990> (merge:254)
2017-11-07 09:20:41,332-0500 INFO (tasks/8) [storage.SANLock] Acquiring
Lease(name='e77f8714-9e0f-4f39-8437-e75a3fa44c74',
path=u'/rhev/data-center/mnt/192.168.204.4:_exports_nfs_share1/4a088fb2-5497-4b4d-adec-4c91d998c56e/images/8401cd0
f-419c-417c-8da3-52da285fa47c/e77f8714-9e0f-4f39-8437-e75a3fa44c74.lease',
offset=0) for host id 1 (clusterlock:377)
2017-11-07 09:20:41,345-0500 INFO (tasks/8) [storage.SANLock]
Successfully acquired Lease(name='e77f8714-9e0f-4f39-8437-e75a3fa44c74',
path=u'/rhev/data-center/mnt/192.168.204.4:_exports_nfs_share1/4a088fb2-5497-4b4d-adec-4c91d998c56e/im
ages/8401cd0f-419c-417c-8da3-52da285fa47c/e77f8714-9e0f-4f39-8437-e75a3fa44c74.lease',
offset=0) for host id 1 (clusterlock:415)
2017-11-07 09:20:41,372-0500 INFO (tasks/8) [storage.VolumeManifest]
Volume: preparing volume
4a088fb2-5497-4b4d-adec-4c91d998c56e/e77f8714-9e0f-4f39-8437-e75a3fa44c74
(volume:558)
2017-11-07 09:20:41,377-0500 INFO (tasks/8) [storage.VolumeManifest]
Volume: preparing volume
4a088fb2-5497-4b4d-adec-4c91d998c56e/7a295270-fdf5-4561-b688-033c5d312d1a
(volume:558)
2017-11-07 09:20:41,395-0500 INFO (tasks/8) [storage.VolumeManifest]
Volume: preparing volume
4a088fb2-5497-4b4d-adec-4c91d998c56e/a6cb7a60-e500-4a75-9da8-9e7cce3e21ac
(volume:558)
2017-11-07 09:20:41,399-0500 INFO (tasks/8) [storage.VolumeManifest]
sdUUID=4a088fb2-5497-4b4d-adec-4c91d998c56e
imgUUID=8401cd0f-419c-417c-8da3-52da285fa47c volUUID =
7a295270-fdf5-4561-b688-033c5d312d1a legality = ILLEGAL (volume:399)
2017-11-07 09:20:41,667-0500 INFO (tasks/8) [storage.merge] Updating
Vdsm metadata, syncing new chain:
['e77f8714-9e0f-4f39-8437-e75a3fa44c74',
u'a6cb7a60-e500-4a75-9da8-9e7cce3e21ac'] (merge:311)
2017-11-07 09:20:41,690-0500 INFO (tasks/8) [storage.Image] Current
chain=e77f8714-9e0f-4f39-8437-e75a3fa44c74 <
7a295270-fdf5-4561-b688-033c5d312d1a <
a6cb7a60-e500-4a75-9da8-9e7cce3e21ac (top) (image:1162)
2017-11-07 09:20:41,690-0500 INFO (tasks/8) [storage.Image] Unlinking
subchain: ['7a295270-fdf5-4561-b688-033c5d312d1a'] (image:1172)
2017-11-07 09:20:41,709-0500 INFO (tasks/8) [storage.Image] Setting
parent of volume a6cb7a60-e500-4a75-9da8-9e7cce3e21ac to
e77f8714-9e0f-4f39-8437-e75a3fa44c74 (image:1185)
2017-11-07 09:20:41,824-0500 INFO (tasks/8) [storage.SANLock] Releasing
Lease(name='e77f8714-9e0f-4f39-8437-e75a3fa44c74',
path=u'/rhev/data-center/mnt/192.168.204.4:_exports_nfs_share1/4a088fb2-5497-4b4d-adec-4c91d998c56e/images/8401cd0
f-419c-417c-8da3-52da285fa47c/e77f8714-9e0f-4f39-8437-e75a3fa44c74.lease',
offset=0) (clusterlock:435)
2017-11-07 09:20:41,828-0500 INFO (tasks/8) [storage.SANLock]
Successfully released Lease(name='e77f8714-9e0f-4f39-8437-e75a3fa44c74',
path=u'/rhev/data-center/mnt/192.168.204.4:_exports_nfs_share1/4a088fb2-5497-4b4d-adec-4c91d998c56e/im
ages/8401cd0f-419c-417c-8da3-52da285fa47c/e77f8714-9e0f-4f39-8437-e75a3fa44c74.lease',
offset=0) (clusterlock:444)
2017-11-07 09:20:41,842-0500 INFO (tasks/8)
[storage.ThreadPool.WorkerThread] FINISH task
9ce348bc-e3fb-4f4e-aea0-f8c79e7ae204 (threadPool:210)
2017-11-07 09:20:42,388-0500 INFO (jsonrpc/4) [api.virt] START
diskReplicateFinish(srcDisk={u'device': u'disk', u'poolID':
u'b20f0eb7-6bea-4994-b87b-e1080c4fd9e5', u'volumeID':
u'8a1660d7-f153-4aca-9f25-c1184c10841a', u'domainID': u'4a08
8fb2-5497-4b4d-adec-4c91d998c56e', u'imageID':
u'36837a9b-6396-4d65-a354-df97ce808f01'}, dstDisk={u'device': u'disk',
u'poolID': u'b20f0eb7-6bea-4994-b87b-e1080c4fd9e5', u'volumeID':
u'8a1660d7-f153-4aca-9f25-c1184c10841a', u'domainID': u
'4a088fb2-5497-4b4d-adec-4c91d998c56e', u'imageID':
u'36837a9b-6396-4d65-a354-df97ce808f01'})
from=::ffff:192.168.204.4,49358, flow_id=4bcdd94e (api:46)
2017-11-07 09:20:42,388-0500 ERROR (jsonrpc/4) [virt.vm]
(vmId='61d328bc-c920-4691-997f-09bbecb5de10') Drive not found (srcDisk:
{u'device': u'disk', u'poolID': u'b20f0eb7-6bea-4994-b87b-e1080c4fd9e5',
u'volumeID': u'8a1660d7-f153-4aca-9f
25-c1184c10841a', u'domainID': u'4a088fb2-5497-4b4d-adec-4c91d998c56e',
u'imageID': u'36837a9b-6396-4d65-a354-df97ce808f01'}) (vm:4386)
2017-11-07 09:20:42,389-0500 INFO (jsonrpc/4) [api.virt] FINISH
diskReplicateFinish return={'status': {'message': 'Drive image file
could not be found', 'code': 13}} from=::ffff:192.168.204.4,49358,
flow_id=4bcdd94e (api:52)
2017-11-07 09:20:42,389-0500 INFO (jsonrpc/4) [jsonrpc.JsonRpcServer]
RPC call VM.diskReplicateFinish failed (error 13) in 0.01 seconds
(__init__:630)**
:
*
*
*</error>*
**
--------------106EB129C1BC1AEAACE8C19F
Content-Type: text/html; charset=utf-8
Content-Transfer-Encoding: 8bit
<html>
<head>
<meta http-equiv="content-type" content="text/html; charset=utf-8">
</head>
<body text="#000000" bgcolor="#FFFFFF">
<p>Hi, <br>
</p>
<p>We have a failure in test 004_basic_sanity.disk_operations. <br>
</p>
<p>From the log I can see that vdsm has a KeyError: 'diskReplicate'
and than reports the drive not found. <br>
</p>
<p>Can you please check the issue? <br>
</p>
<p><br>
</p>
<p><b style="font-weight:normal;"
id="docs-internal-guid-5859b7a1-974d-e74a-41ed-aa8671a5b48d">
<p dir="ltr"
style="line-height:1.38;margin-top:0pt;margin-bottom:0pt;"><span style="font-size:11pt;font-family:Arial;color:#000000;background-color:transparent;font-weight:400;font-style:normal;font-variant:normal;text-decoration:none;vertical-align:baseline;white-space:pre-wrap;">Link to suspected patches: <a class="moz-txt-link-freetext" href="https://gerrit.ovirt.org/#/c/83641/">https://gerrit.ovirt.org/#/c/83641/</a></span></p>
<br>
<p dir="ltr"
style="line-height:1.38;margin-top:0pt;margin-bottom:0pt;"><span style="font-size:11pt;font-family:Arial;color:#000000;background-color:transparent;font-weight:400;font-style:normal;font-variant:normal;text-decoration:none;vertical-align:baseline;white-space:pre-wrap;">Link to Job: <a class="moz-txt-link-freetext" href="http://jenkins.ovirt.org/job/ovirt-master_change-queue-tester/3653">http://jenkins.ovirt.org/job/ovirt-master_change-queue-tester/3653</a></span></p>
<br>
<p dir="ltr"
style="line-height:1.38;margin-top:0pt;margin-bottom:0pt;"><span style="font-size:11pt;font-family:Arial;color:#000000;background-color:transparent;font-weight:400;font-style:normal;font-variant:normal;text-decoration:none;vertical-align:baseline;white-space:pre-wrap;">Link to all logs: <a class="moz-txt-link-freetext" href="http://jenkins.ovirt.org/job/ovirt-master_change-queue-tester/3653/artifact/">http://jenkins.ovirt.org/job/ovirt-master_change-queue-tester/3653/artifact/</a></span></p>
<p dir="ltr"
style="line-height:1.38;margin-top:0pt;margin-bottom:0pt;"><span style="font-size:11pt;font-family:Arial;color:#000000;background-color:transparent;font-weight:400;font-style:normal;font-variant:normal;text-decoration:none;vertical-align:baseline;white-space:pre-wrap;">
</span></p>
<p dir="ltr"
style="line-height:1.38;margin-top:0pt;margin-bottom:0pt;"><span style="font-size:11pt;font-family:Arial;color:#000000;background-color:transparent;font-weight:400;font-style:normal;font-variant:normal;text-decoration:none;vertical-align:baseline;white-space:pre-wrap;">engine log:
</span></p>
<p dir="ltr"
style="line-height:1.38;margin-top:0pt;margin-bottom:0pt;"><span style="font-size:11pt;font-family:Arial;color:#000000;background-color:transparent;font-weight:400;font-style:normal;font-variant:normal;text-decoration:none;vertical-align:baseline;white-space:pre-wrap;"><a class="moz-txt-link-freetext" href="http://jenkins.ovirt.org/job/ovirt-master_change-queue-tester/3653/artifact…">http://jenkins.ovirt.org/job/ovirt-master_change-queue-tester/3653/artifact…</a></span></p>
<p dir="ltr"
style="line-height:1.38;margin-top:0pt;margin-bottom:0pt;"><span style="font-size:11pt;font-family:Arial;color:#000000;background-color:transparent;font-weight:400;font-style:normal;font-variant:normal;text-decoration:none;vertical-align:baseline;white-space:pre-wrap;">
</span></p>
<p dir="ltr"
style="line-height:1.38;margin-top:0pt;margin-bottom:0pt;"><span style="font-size:11pt;font-family:Arial;color:#000000;background-color:transparent;font-weight:400;font-style:normal;font-variant:normal;text-decoration:none;vertical-align:baseline;white-space:pre-wrap;">vdsm log:
</span></p>
</b><b style="font-weight:normal;"
id="docs-internal-guid-5859b7a1-974d-e74a-41ed-aa8671a5b48d">
<p dir="ltr"
style="line-height:1.38;margin-top:0pt;margin-bottom:0pt;"><span style="font-size:11pt;font-family:Arial;color:#000000;background-color:transparent;font-weight:400;font-style:normal;font-variant:normal;text-decoration:none;vertical-align:baseline;white-space:pre-wrap;"><a class="moz-txt-link-freetext" href="http://jenkins.ovirt.org/job/ovirt-master_change-queue-tester/3653/artifact…">http://jenkins.ovirt.org/job/ovirt-master_change-queue-tester/3653/artifact…</a>
</span></p>
<br>
<p dir="ltr"
style="line-height:1.38;margin-top:0pt;margin-bottom:0pt;"><span style="font-size:11pt;font-family:Arial;color:#000000;background-color:transparent;font-weight:400;font-style:normal;font-variant:normal;text-decoration:none;vertical-align:baseline;white-space:pre-wrap;">(Relevant) error snippet from the log: </span></p>
<p dir="ltr"
style="line-height:1.38;margin-top:0pt;margin-bottom:0pt;"><span style="font-size:11pt;font-family:Arial;color:#000000;background-color:transparent;font-weight:400;font-style:normal;font-variant:normal;text-decoration:none;vertical-align:baseline;white-space:pre-wrap;"><error></span></p>
<br>
From engine log: <br>
</b></p>
<p><span style="font-weight:normal;">2017-11-07 09:20:42,396-05
ERROR
[org.ovirt.engine.core.vdsbroker.vdsbroker.VmReplicateDiskFinishVDSCommand]
(EE-ManagedThreadFactory-engineScheduled-Thread-72) [4bcdd94e]
Failed in 'VmReplicateDiskFinishVDS' method<br>
2017-11-07 09:20:42,399-05 ERROR
[org.ovirt.engine.core.dal.dbbroker.auditloghandling.AuditLogDirector]
(EE-ManagedThreadFactory-engineScheduled-Thread-72) [4bcdd94e]
EVENT_ID: VDS_BROKER_COMMAND_FAILURE(10,802), VDSM
lago-basic-suite-mas<br>
ter-host-0 command VmReplicateDiskFinishVDS failed: Drive image
file could not be found<br>
2017-11-07 09:20:42,399-05 INFOÂ
[org.ovirt.engine.core.vdsbroker.vdsbroker.VmReplicateDiskFinishVDSCommand]
(EE-ManagedThreadFactory-engineScheduled-Thread-72) [4bcdd94e]
Command
'org.ovirt.engine.core.vdsbroker.vdsbroker.VmReplicateDisk<br>
FinishVDSCommand' return value 'StatusOnlyReturn [status=Status
[code=13, message=Drive image file could not be found]]'<br>
2017-11-07 09:20:42,399-05 INFOÂ
[org.ovirt.engine.core.vdsbroker.vdsbroker.VmReplicateDiskFinishVDSCommand]
(EE-ManagedThreadFactory-engineScheduled-Thread-72) [4bcdd94e]
HostName = lago-basic-suite-master-host-0<br>
2017-11-07 09:20:42,399-05 ERROR
[org.ovirt.engine.core.vdsbroker.vdsbroker.VmReplicateDiskFinishVDSCommand]
(EE-ManagedThreadFactory-engineScheduled-Thread-72) [4bcdd94e]
Command 'VmReplicateDiskFinishVDSCommand(HostName =
lago-basic-sui<br>
te-master-host-0,
VmReplicateDiskParameters:{hostId='038b0c9a-f64c-4aa7-8e36-72e435771dab',
vmId='61d328bc-c920-4691-997f-09bbecb5de10',
storagePoolId='b20f0eb7-6bea-4994-b87b-e1080c4fd9e5',
srcStorageDomainId='4a088fb2-5497-4b4d-adec-4c9<br>
1d998c56e',
targetStorageDomainId='4a088fb2-5497-4b4d-adec-4c91d998c56e',
imageGroupId='36837a9b-6396-4d65-a354-df97ce808f01',
imageId='8a1660d7-f153-4aca-9f25-c1184c10841a'})' execution
failed: VDSGenericException: VDSErrorException: Fai<br>
led to VmReplicateDiskFinishVDS, error = Drive image file could
not be found, code = 13<br>
2017-11-07 09:20:42,399-05 DEBUG
[org.ovirt.engine.core.vdsbroker.vdsbroker.VmReplicateDiskFinishVDSCommand]
(EE-ManagedThreadFactory-engineScheduled-Thread-72) [4bcdd94e]
Exception:
org.ovirt.engine.core.vdsbroker.vdsbroker.VDSErrorExcep<br>
tion: VDSGenericException: VDSErrorException: Failed to
VmReplicateDiskFinishVDS, error = Drive image file could not be
found, code = 13<br>
       at
org.ovirt.engine.core.vdsbroker.vdsbroker.VdsBrokerCommand.createDefaultConcreteException(VdsBrokerCommand.java:81)
[vdsbroker.jar:]<br>
       at
org.ovirt.engine.core.vdsbroker.vdsbroker.BrokerCommandBase.createException(BrokerCommandBase.java:223)
[vdsbroker.jar:]<br>
       at
org.ovirt.engine.core.vdsbroker.vdsbroker.BrokerCommandBase.proceedProxyReturnValue(BrokerCommandBase.java:193)
[vdsbroker.jar:]<br>
       at
org.ovirt.engine.core.vdsbroker.vdsbroker.VmReplicateDiskFinishVDSCommand.executeVdsBrokerCommand(VmReplicateDiskFinishVDSCommand.java:15)
[vdsbroker.jar:]<br>
       at
org.ovirt.engine.core.vdsbroker.vdsbroker.VdsBrokerCommand.executeVDSCommand(VdsBrokerCommand.java:112)
[vdsbroker.jar:]<br>
       at
org.ovirt.engine.core.vdsbroker.VDSCommandBase.executeCommand(VDSCommandBase.java:73)
[vdsbroker.jar:]<br>
       at
org.ovirt.engine.core.dal.VdcCommandBase.execute(VdcCommandBase.java:33)
[dal.jar:]<br>
       at
org.ovirt.engine.core.vdsbroker.vdsbroker.DefaultVdsCommandExecutor.execute(DefaultVdsCommandExecutor.java:14)
[vdsbroker.jar:]<br>
       at
org.ovirt.engine.core.vdsbroker.ResourceManager.runVdsCommand(ResourceManager.java:387)
[vdsbroker.jar:]<br>
       at
org.ovirt.engine.core.vdsbroker.ResourceManager$Proxy$_$$_WeldSubclass.runVdsCommand$$super(Unknown
Source) [vdsbroker.jar:]<br>
       at sun.reflect.GeneratedMethodAccessor244.invoke(Unknown
Source) [:1.8.0_151]<br>
       at
sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:43)
[rt.jar:1.8.0_151]<br>
       at java.lang.reflect.Method.invoke(Method.java:498)
[rt.jar:1.8.0_151]<br>
       at
org.jboss.weld.interceptor.proxy.TerminalAroundInvokeInvocationContext.proceedInternal(TerminalAroundInvokeInvocationContext.java:49)
[weld-core-impl-2.4.3.Final.jar:2.4.3.Final]<br>
       at
org.jboss.weld.interceptor.proxy.AroundInvokeInvocationContext.proceed(AroundInvokeInvocationContext.java:77)
[weld-core-impl-2.4.3.Final.jar:2.4.3.Final]<br>
       at
org.ovirt.engine.core.common.di.interceptor.LoggingInterceptor.apply(LoggingInterceptor.java:12)
[common.jar:]<br>
       at sun.reflect.GeneratedMethodAccessor69.invoke(Unknown
Source) [:1.8.0_151]<br>
       at
sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:43)
[rt.jar:1.8.0_151]<br>
       at java.lang.reflect.Method.invoke(Method.java:498)
[rt.jar:1.8.0_151]<br>
       at
org.jboss.weld.interceptor.reader.SimpleInterceptorInvocation$SimpleMethodInvocation.invoke(SimpleInterceptorInvocation.java:73)
[weld-core-impl-2.4.3.Final.jar:2.4.3.Final]<br>
       at
org.jboss.weld.interceptor.proxy.InterceptorMethodHandler.executeAroundInvoke(InterceptorMethodHandler.java:84)
[weld-core-impl-2.4.3.Final.jar:2.4.3.Final]<br>
       at
org.jboss.weld.interceptor.proxy.InterceptorMethodHandler.executeInterception(InterceptorMethodHandler.java:72)
[weld-core-impl-2.4.3.Final.jar:2.4.3.Final]<br>
       at
org.jboss.weld.interceptor.proxy.InterceptorMethodHandler.invoke(InterceptorMethodHandler.java:56)
[weld-core-impl-2.4.3.Final.jar:2.4.3.Final]<br>
       at
org.jboss.weld.bean.proxy.CombinedInterceptorAndDecoratorStackMethodHandler.invoke(CombinedInterceptorAndDecoratorStackMethodHandler.java:79)
[weld-core-impl-2.4.3.Final.jar:2.4.3.Final]<br>
       at
org.jboss.weld.bean.proxy.CombinedInterceptorAndDecoratorStackMethodHandler.invoke(CombinedInterceptorAndDecoratorStackMethodHandler.java:68)
[weld-core-impl-2.4.3.Final.jar:2.4.3.Final]<br>
       at
org.ovirt.engine.core.vdsbroker.ResourceManager$Proxy$_$$_WeldSubclass.runVdsCommand(Unknown
Source) [vdsbroker.jar:]<br>
       at
org.ovirt.engine.core.bll.storage.lsm.LiveMigrateDiskCommand.replicateDiskFinish(LiveMigrateDiskCommand.java:374)
[bll.jar:]<br>
       at
org.ovirt.engine.core.bll.storage.lsm.LiveMigrateDiskCommand.handleDestDisk(LiveMigrateDiskCommand.java:681)
[bll.jar:]<br>
       at
org.ovirt.engine.core.bll.storage.lsm.LiveMigrateDiskCommand.endWithFailure(LiveMigrateDiskCommand.java:337)
[bll.jar:]<br>
       at
org.ovirt.engine.core.bll.CommandBase.internalEndWithFailure(CommandBase.java:783)
[bll.jar:]<br>
       at
org.ovirt.engine.core.bll.CommandBase.endActionInTransactionScope(CommandBase.java:712)
[bll.jar:]<br>
       at
org.ovirt.engine.core.bll.CommandBase.runInTransaction(CommandBase.java:1989)
[bll.jar:]<br>
       at
org.ovirt.engine.core.utils.transaction.TransactionSupport.executeInNewTransaction(TransactionSupport.java:202)
[utils.jar:]<br>
       at
org.ovirt.engine.core.utils.transaction.TransactionSupport.executeInRequired(TransactionSupport.java:137)
[utils.jar:]<br>
       at
org.ovirt.engine.core.utils.transaction.TransactionSupport.executeInScope(TransactionSupport.java:105)
[utils.jar:]<br>
       at
org.ovirt.engine.core.bll.CommandBase.endAction(CommandBase.java:574)
[bll.jar:]<br>
       at
org.ovirt.engine.core.bll.ChildCommandsCallbackBase.endAction(ChildCommandsCallbackBase.java:125)
[bll.jar:]<br>
       at
org.ovirt.engine.core.bll.ChildCommandsCallbackBase.onFailed(ChildCommandsCallbackBase.java:156)
[bll.jar:]<br>
       at
org.ovirt.engine.core.bll.tasks.CommandCallbacksPoller.endCallback(CommandCallbacksPoller.java:65)
[bll.jar:]<br>
       at
org.ovirt.engine.core.bll.tasks.CommandCallbacksPoller.invokeCallbackMethodsImpl(CommandCallbacksPoller.java:141)
[bll.jar:]<br>
       at
org.ovirt.engine.core.bll.tasks.CommandCallbacksPoller.invokeCallbackMethods(CommandCallbacksPoller.java:106)
[bll.jar:]<br>
       at
java.util.concurrent.Executors$RunnableAdapter.call(Executors.java:511)
[rt.jar:1.8.0_151]<br>
       at
java.util.concurrent.FutureTask.runAndReset(FutureTask.java:308)
[rt.jar:1.8.0_151]<br>
       at
org.glassfish.enterprise.concurrent.internal.ManagedScheduledThreadPoolExecutor$ManagedScheduledFutureTask.access$201(ManagedScheduledThreadPoolExecutor.java:383)
[javax.enterprise.concurrent-1.0.jar:]<br>
       at
org.glassfish.enterprise.concurrent.internal.ManagedScheduledThreadPoolExecutor$ManagedScheduledFutureTask.run(ManagedScheduledThreadPoolExecutor.java:534)
[javax.enterprise.concurrent-1.0.jar:]<br>
       at
java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1149)
[rt.jar:1.8.0_151]<br>
:<br>
at
java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:624)
[rt.jar:1.8.0_151]<br>
       at java.lang.Thread.run(Thread.java:748)
[rt.jar:1.8.0_151]<br>
       at
org.glassfish.enterprise.concurrent.ManagedThreadFactoryImpl$ManagedThread.run(ManagedThreadFactoryImpl.java:250)
[javax.enterprise.concurrent-1.0.jar:]<br>
       at
org.jboss.as.ee.concurrent.service.ElytronManagedThreadFactory$ElytronManagedThread.run(ElytronManagedThreadFactory.java:78)<br>
<br>
2017-11-07 09:20:42,400-05 INFOÂ
[org.ovirt.engine.core.vdsbroker.vdsbroker.VmReplicateDiskFinishVDSCommand]
(EE-ManagedThreadFactory-engineScheduled-Thread-72) [4bcdd94e]
FINISH, VmReplicateDiskFinishVDSCommand, log id: 32c5a56e<br>
2017-11-07 09:20:42,400-05 DEBUG
[org.ovirt.engine.core.common.di.interceptor.DebugLoggingInterceptor]
(EE-ManagedThreadFactory-engineScheduled-Thread-72) [4bcdd94e]
method: runVdsCommand, params: [VmReplicateDiskFinish,
VmReplicateDiskParameters:{hostId='038b0c9a-f64c-4aa7-8e36-72e435771dab',
vmId='61d328bc-c920-4691-997f-09bbecb5de10',
storagePoolId='b20f0eb7-6bea-4994-b87b-e1080c4fd9e5',
srcStorageDomainId='4a088fb2-5497-4b4d-adec-4c91d998c56e',
targetStorageDomainId='4a088fb2-5497-4b4d-adec-4c91d998c56e',
imageGroupId='36837a9b-6396-4d65-a354-df97ce808f01',
imageId='8a1660d7-f153-4aca-9f25-c1184c10841a'}], timeElapsed:
12ms<br>
2017-11-07 09:20:42,400-05 ERROR
[org.ovirt.engine.core.bll.storage.lsm.LiveMigrateDiskCommand]
(EE-ManagedThreadFactory-engineScheduled-Thread-72) [4bcdd94e]
Replication end of disk '36837a9b-6396-4d65-a354-df97ce808f01'
in vm '61d328bc-c920-4691-997f-09bbecb5de10' back to the source
failed, skipping deletion of the target disk<br>
2017-11-07 09:20:42,408-05 WARNÂ
[org.ovirt.engine.core.bll.lock.InMemoryLockManager]
(EE-ManagedThreadFactory-engineScheduled-Thread-72) [4bcdd94e]
Trying to release exclusive lock which does not exist, lock key:
'61d328bc-c920-4691-997f-09bbecb5de10LIVE_STORAGE_MIGRATION'<br>
2017-11-07 09:20:42,408-05 INFOÂ
[org.ovirt.engine.core.bll.storage.lsm.LiveMigrateDiskCommand]
(EE-ManagedThreadFactory-engineScheduled-Thread-72) [4bcdd94e]
Lock freed to object
'EngineLock:{exclusiveLocks='[61d328bc-c920-4691-997f-09bbecb5de10=LIVE_STORAGE_MIGRATION]',
sharedLocks=''}'<br>
2017-11-07 09:20:42,428-05 DEBUG
[org.ovirt.engine.core.common.di.interceptor.DebugLoggingInterceptor]
(EE-ManagedThreadFactory-engineScheduled-Thread-72) [4bcdd94e]
method: get, params: [038b0c9a-f64c-4aa7-8e36-72e435771dab],
timeElapsed: 2ms<br>
2017-11-07 09:20:42,430-05 ERROR
[org.ovirt.engine.core.dal.dbbroker.auditloghandling.AuditLogDirector]
(EE-ManagedThreadFactory-engineScheduled-Thread-72) [4bcdd94e]
EVENT_ID: USER_MOVED_DISK_FINISHED_FAILURE(2,011), User
admin@internal-authz have failed to move disk vm0_disk0 to
domain iscsi.</span><b style="font-weight:normal;"
id="docs-internal-guid-5859b7a1-974d-e74a-41ed-aa8671a5b48d"><b><br>
</b></b></p>
<p><b style="font-weight:normal;"
id="docs-internal-guid-5859b7a1-974d-e74a-41ed-aa8671a5b48d"><br>
</b></p>
<p><b style="font-weight:normal;"
id="docs-internal-guid-5859b7a1-974d-e74a-41ed-aa8671a5b48d"><br>
</b></p>
<p><span style="font-weight:normal;">vdsm log: <br>
</span></p>
<p><span style="font-weight:normal;"><br>
</span></p>
<p><span style="font-weight:normal;">2017-11-07 09:20:41,178-0500
INFOÂ (jsonrpc/6) [vdsm.api] FINISH getVolumeSize
return={'truesize': '2415919104', 'apparentsize': '2415919104'}
from=::ffff:192.168.204.4,49358, flow_id=4bcdd94e,
task_id=042481eb-74ac-42a8-9d1e-8fb1c84db234<br>
 (api:52)<br>
2017-11-07 09:20:41,193-0500 ERROR (jsonrpc/6) [api] FINISH
diskReplicateFinish error='diskReplicate' (api:127)<br>
Traceback (most recent call last):<br>
 File "/usr/lib/python2.7/site-packages/vdsm/common/api.py",
line 117, in method<br>
   ret = func(*args, **kwargs)<br>
 File "/usr/lib/python2.7/site-packages/vdsm/API.py", line 558,
in diskReplicateFinish<br>
   return self.vm.diskReplicateFinish(srcDisk, dstDisk)<br>
 File "/usr/lib/python2.7/site-packages/vdsm/virt/vm.py", line
4477, in diskReplicateFinish<br>
   self._delDiskReplica(drive)<br>
 File "/usr/lib/python2.7/site-packages/vdsm/virt/vm.py", line
4528, in _delDiskReplica<br>
   del dev['diskReplicate']<br>
KeyError: 'diskReplicate'<br>
2017-11-07 09:20:41,195-0500 INFOÂ (jsonrpc/6) [api.virt] FINISH
diskReplicateFinish return={'status': {'message': 'General
Exception: ("\'diskReplicate\'",)', 'code': 100}}
from=::ffff:192.168.204.4,49358, flow_id=4bcdd94e (api:52)<br>
2017-11-07 09:20:41,195-0500 INFOÂ (jsonrpc/6)
[jsonrpc.JsonRpcServer] RPC call VM.diskReplicateFinish failed
(error 100) in 0.79 seconds (__init__:630)<br>
2017-11-07 09:20:41,253-0500 INFOÂ (jsonrpc/2) [vdsm.api] START
finalizeMerge(spUUID=u'b20f0eb7-6bea-4994-b87b-e1080c4fd9e5',
subchainInfo={u'img_id':
u'8401cd0f-419c-417c-8da3-52da285fa47c', u'sd_id':
u'4a088fb2-5497-4b4d-adec-4c91d998c5<br>
6e', u'top_id': u'7a295270-fdf5-4561-b688-033c5d312d1a',
u'base_id': u'e77f8714-9e0f-4f39-8437-e75a3fa44c74'})
from=::ffff:192.168.204.4,49380,
flow_id=56c9f051-c091-49cd-9fd5-cc98e92ef05f,
task_id=9ce348bc-e3fb-4f4e-aea0-f8c79e7ae204 (ap<br>
i:46)<br>
2017-11-07 09:20:41,267-0500 INFOÂ (jsonrpc/2) [vdsm.api] FINISH
finalizeMerge return=None from=::ffff:192.168.204.4,49380,
flow_id=56c9f051-c091-49cd-9fd5-cc98e92ef05f,
task_id=9ce348bc-e3fb-4f4e-aea0-f8c79e7ae204 (api:52)<br>
2017-11-07 09:20:41,285-0500 INFOÂ (jsonrpc/2)
[jsonrpc.JsonRpcServer] RPC call StoragePool.finalizeMerge
succeeded in 0.03 seconds (__init__:630)<br>
2017-11-07 09:20:41,285-0500 INFOÂ (tasks/8)
[storage.ThreadPool.WorkerThread] START task
9ce348bc-e3fb-4f4e-aea0-f8c79e7ae204 (cmd=<bound method
Task.commit of <vdsm.storage.task.Task instance at
0x2f0a560>>, args=None) (threadPool:208)<br>
2017-11-07 09:20:41,295-0500 INFOÂ (tasks/8) [storage.merge]
Finalizing subchain after merge: <SubchainInfo
sd_id=4a088fb2-5497-4b4d-adec-4c91d998c56e,
img_id=8401cd0f-419c-417c-8da3-52da285fa47c,
top_id=7a295270-fdf5-4561-b688-033c5d312d<br>
1a, base_id=e77f8714-9e0f-4f39-8437-e75a3fa44c74
base_generation=None at 0x30a7990> (merge:254)<br>
2017-11-07 09:20:41,332-0500 INFOÂ (tasks/8) [storage.SANLock]
Acquiring Lease(name='e77f8714-9e0f-4f39-8437-e75a3fa44c74',
path=u'/rhev/data-center/mnt/192.168.204.4:_exports_nfs_share1/4a088fb2-5497-4b4d-adec-4c91d998c56e/images/8401cd0<br>
f-419c-417c-8da3-52da285fa47c/e77f8714-9e0f-4f39-8437-e75a3fa44c74.lease',
offset=0) for host id 1 (clusterlock:377)<br>
2017-11-07 09:20:41,345-0500 INFOÂ (tasks/8) [storage.SANLock]
Successfully acquired
Lease(name='e77f8714-9e0f-4f39-8437-e75a3fa44c74',
path=u'/rhev/data-center/mnt/192.168.204.4:_exports_nfs_share1/4a088fb2-5497-4b4d-adec-4c91d998c56e/im<br>
ages/8401cd0f-419c-417c-8da3-52da285fa47c/e77f8714-9e0f-4f39-8437-e75a3fa44c74.lease',
offset=0) for host id 1 (clusterlock:415)<br>
2017-11-07 09:20:41,372-0500 INFOÂ (tasks/8)
[storage.VolumeManifest] Volume: preparing volume
4a088fb2-5497-4b4d-adec-4c91d998c56e/e77f8714-9e0f-4f39-8437-e75a3fa44c74
(volume:558)<br>
2017-11-07 09:20:41,377-0500 INFOÂ (tasks/8)
[storage.VolumeManifest] Volume: preparing volume
4a088fb2-5497-4b4d-adec-4c91d998c56e/7a295270-fdf5-4561-b688-033c5d312d1a
(volume:558)<br>
2017-11-07 09:20:41,395-0500 INFOÂ (tasks/8)
[storage.VolumeManifest] Volume: preparing volume
4a088fb2-5497-4b4d-adec-4c91d998c56e/a6cb7a60-e500-4a75-9da8-9e7cce3e21ac
(volume:558)<br>
2017-11-07 09:20:41,399-0500 INFOÂ (tasks/8)
[storage.VolumeManifest]
sdUUID=4a088fb2-5497-4b4d-adec-4c91d998c56e
imgUUID=8401cd0f-419c-417c-8da3-52da285fa47c volUUID =
7a295270-fdf5-4561-b688-033c5d312d1a legality = ILLEGALÂ
(volume:399)<br>
2017-11-07 09:20:41,667-0500 INFOÂ (tasks/8) [storage.merge]
Updating Vdsm metadata, syncing new chain:
['e77f8714-9e0f-4f39-8437-e75a3fa44c74',
u'a6cb7a60-e500-4a75-9da8-9e7cce3e21ac'] (merge:311)<br>
2017-11-07 09:20:41,690-0500 INFOÂ (tasks/8) [storage.Image]
Current chain=e77f8714-9e0f-4f39-8437-e75a3fa44c74 <
7a295270-fdf5-4561-b688-033c5d312d1a <
a6cb7a60-e500-4a75-9da8-9e7cce3e21ac (top)Â (image:1162)<br>
2017-11-07 09:20:41,690-0500 INFOÂ (tasks/8) [storage.Image]
Unlinking subchain: ['7a295270-fdf5-4561-b688-033c5d312d1a']
(image:1172)<br>
2017-11-07 09:20:41,709-0500 INFOÂ (tasks/8) [storage.Image]
Setting parent of volume a6cb7a60-e500-4a75-9da8-9e7cce3e21ac to
e77f8714-9e0f-4f39-8437-e75a3fa44c74 (image:1185)<br>
2017-11-07 09:20:41,824-0500 INFOÂ (tasks/8) [storage.SANLock]
Releasing Lease(name='e77f8714-9e0f-4f39-8437-e75a3fa44c74',
path=u'/rhev/data-center/mnt/192.168.204.4:_exports_nfs_share1/4a088fb2-5497-4b4d-adec-4c91d998c56e/images/8401cd0<br>
f-419c-417c-8da3-52da285fa47c/e77f8714-9e0f-4f39-8437-e75a3fa44c74.lease',
offset=0) (clusterlock:435)<br>
2017-11-07 09:20:41,828-0500 INFOÂ (tasks/8) [storage.SANLock]
Successfully released
Lease(name='e77f8714-9e0f-4f39-8437-e75a3fa44c74',
path=u'/rhev/data-center/mnt/192.168.204.4:_exports_nfs_share1/4a088fb2-5497-4b4d-adec-4c91d998c56e/im<br>
ages/8401cd0f-419c-417c-8da3-52da285fa47c/e77f8714-9e0f-4f39-8437-e75a3fa44c74.lease',
offset=0) (clusterlock:444)<br>
2017-11-07 09:20:41,842-0500 INFOÂ (tasks/8)
[storage.ThreadPool.WorkerThread] FINISH task
9ce348bc-e3fb-4f4e-aea0-f8c79e7ae204 (threadPool:210)<br>
2017-11-07 09:20:42,388-0500 INFOÂ (jsonrpc/4) [api.virt] START
diskReplicateFinish(srcDisk={u'device': u'disk', u'poolID':
u'b20f0eb7-6bea-4994-b87b-e1080c4fd9e5', u'volumeID':
u'8a1660d7-f153-4aca-9f25-c1184c10841a', u'domainID': u'4a08<br>
8fb2-5497-4b4d-adec-4c91d998c56e', u'imageID':
u'36837a9b-6396-4d65-a354-df97ce808f01'}, dstDisk={u'device':
u'disk', u'poolID': u'b20f0eb7-6bea-4994-b87b-e1080c4fd9e5',
u'volumeID': u'8a1660d7-f153-4aca-9f25-c1184c10841a',
u'domainID': u<br>
'4a088fb2-5497-4b4d-adec-4c91d998c56e', u'imageID':
u'36837a9b-6396-4d65-a354-df97ce808f01'})
from=::ffff:192.168.204.4,49358, flow_id=4bcdd94e (api:46)<br>
2017-11-07 09:20:42,388-0500 ERROR (jsonrpc/4) [virt.vm]
(vmId='61d328bc-c920-4691-997f-09bbecb5de10') Drive not found
(srcDisk: {u'device': u'disk', u'poolID':
u'b20f0eb7-6bea-4994-b87b-e1080c4fd9e5', u'volumeID':
u'8a1660d7-f153-4aca-9f<br>
25-c1184c10841a', u'domainID':
u'4a088fb2-5497-4b4d-adec-4c91d998c56e', u'imageID':
u'36837a9b-6396-4d65-a354-df97ce808f01'}) (vm:4386)<br>
2017-11-07 09:20:42,389-0500 INFOÂ (jsonrpc/4) [api.virt] FINISH
diskReplicateFinish return={'status': {'message': 'Drive image
file could not be found', 'code': 13}}
from=::ffff:192.168.204.4,49358, flow_id=4bcdd94e (api:52)<br>
2017-11-07 09:20:42,389-0500 INFOÂ (jsonrpc/4)
[jsonrpc.JsonRpcServer] RPC call VM.diskReplicateFinish failed
(error 13) in 0.01 seconds (__init__:630)</span><b
style="font-weight:normal;"
id="docs-internal-guid-5859b7a1-974d-e74a-41ed-aa8671a5b48d"><b><br>
:<br>
</b><br>
<p dir="ltr"
style="line-height:1.38;margin-top:0pt;margin-bottom:0pt;"><span style="font-size:11pt;font-family:Arial;color:#000000;background-color:transparent;font-weight:400;font-style:normal;font-variant:normal;text-decoration:none;vertical-align:baseline;white-space:pre-wrap;"></error></span></p>
</b><br class="Apple-interchange-newline">
</p>
</body>
</html>
--------------106EB129C1BC1AEAACE8C19F--
2
2
[ OST Failure Report ] [ oVirt Master ] [ 06-11-2017 ] [ 002_bootstrap.verify_add_hosts ]
by Dafna Ron 07 Nov '17
by Dafna Ron 07 Nov '17
07 Nov '17
This is a multi-part message in MIME format.
--------------88DD9FCD3B3DDEA34D4EAEF4
Content-Type: text/plain; charset=utf-8
Content-Transfer-Encoding: 7bit
Hi,
We failed test 002_bootstrap.verify_add_hosts
I can see we only tried to install one of the hosts (host-0) and failed.
the second host has no log which means we did not try to deploy it.
The error suggests that we ovirt-imageio-daemon failed to start.
However, there is another message that I think should be addressed about
conflicting vdsm and libvirt configurations.
**
*Link to suspected patches: https://gerrit.ovirt.org/#/c/83612/*
*
Link to Job:
http://jenkins.ovirt.org/job/ovirt-master_change-queue-tester/3626/
Link to all logs:
http://jenkins.ovirt.org/job/ovirt-master_change-queue-tester/3626/artifact/
http://jenkins.ovirt.org/job/ovirt-master_change-queue-tester/3626/artifact…
*
*
*(Relevant) error snippet from the log: *
*
<error>
\*
2017-11-06 02:56:46,526-0500 DEBUG otopi.plugins.ovirt_host_deploy.vdsm.packages plugin.execute:921 execute-output: ('/usr/bin/vdsm-tool', 'configure', '--force') stdout:
Checking configuration status...
abrt is not configured for vdsm
WARNING: LVM local configuration: /etc/lvm/lvmlocal.conf is not based on vdsm configuration
lvm requires configuration
libvirt is not configured for vdsm yet
FAILED: conflicting vdsm and libvirt-qemu tls configuration.
vdsm.conf with ssl=True requires the following changes:
libvirtd.conf: listen_tcp=0, auth_tcp="sasl", listen_tls=1
qemu.conf: spice_tls=1.
multipath requires configuration
2017-11-06 02:56:47,551-0500 DEBUG otopi.plugins.otopi.services.systemd plugin.execute:926 execute-output: ('/usr/bin/systemctl', 'start', 'ovirt-imageio-daemon.service') stderr:
Job for ovirt-imageio-daemon.service failed because the control process exited with error code. See "systemctl status ovirt-imageio-daemon.service" and "journalctl -xe" for details.
2017-11-06 02:56:47,552-0500 DEBUG otopi.context context._executeMethod:143 method exception
Traceback (most recent call last):
File "/tmp/ovirt-R4R8gZhaQI/pythonlib/otopi/context.py", line 133, in _executeMethod
method['method']()
File "/tmp/ovirt-R4R8gZhaQI/otopi-plugins/ovirt-host-deploy/vdsm/packages.py", line 179, in _start
self.services.state('ovirt-imageio-daemon', True)
File "/tmp/ovirt-R4R8gZhaQI/otopi-plugins/otopi/services/systemd.py", line 141, in state
service=name,
RuntimeError: Failed to start service 'ovirt-imageio-daemon'
2017-11-06 02:56:47,553-0500 ERROR otopi.context context._executeMethod:152 Failed to execute stage 'Closing up': Failed to start service 'ovirt-imageio-daemon'
**
*</error>*
*
*
--------------88DD9FCD3B3DDEA34D4EAEF4
Content-Type: text/html; charset=utf-8
Content-Transfer-Encoding: 7bit
<html>
<head>
<meta http-equiv="content-type" content="text/html; charset=utf-8">
</head>
<body text="#000000" bgcolor="#FFFFFF">
<p>Hi, <br>
</p>
<p>We failed test 002_bootstrap.verify_add_hosts</p>
<p>I can see we only tried to install one of the hosts (host-0) and
failed. the second host has no log which means we did not try to
deploy it. <br>
</p>
<p>The error suggests that we ovirt-imageio-daemon failed to start.
However, there is another message that I think should be addressed
about conflicting vdsm and libvirt configurations. <br>
</p>
<p><b style="font-weight:normal;"
id="docs-internal-guid-5859b7a1-911e-5616-3cbc-97286587db85">
<p dir="ltr"
style="line-height:1.38;margin-top:0pt;margin-bottom:0pt;"><span style="font-size:11pt;font-family:Arial;color:#000000;background-color:transparent;font-weight:400;font-style:normal;font-variant:normal;text-decoration:none;vertical-align:baseline;white-space:pre-wrap;">Link to suspected patches: <a class="moz-txt-link-freetext" href="https://gerrit.ovirt.org/#/c/83612/">https://gerrit.ovirt.org/#/c/83612/</a></span></p>
<br>
<p dir="ltr"
style="line-height:1.38;margin-top:0pt;margin-bottom:0pt;"><span style="font-size:11pt;font-family:Arial;color:#000000;background-color:transparent;font-weight:400;font-style:normal;font-variant:normal;text-decoration:none;vertical-align:baseline;white-space:pre-wrap;">Link to Job: <a class="moz-txt-link-freetext" href="http://jenkins.ovirt.org/job/ovirt-master_change-queue-tester/3626/">http://jenkins.ovirt.org/job/ovirt-master_change-queue-tester/3626/</a></span></p>
<br>
<p dir="ltr"
style="line-height:1.38;margin-top:0pt;margin-bottom:0pt;"><span style="font-size:11pt;font-family:Arial;color:#000000;background-color:transparent;font-weight:400;font-style:normal;font-variant:normal;text-decoration:none;vertical-align:baseline;white-space:pre-wrap;">Link to all logs: <a class="moz-txt-link-freetext" href="http://jenkins.ovirt.org/job/ovirt-master_change-queue-tester/3626/artifact/">http://jenkins.ovirt.org/job/ovirt-master_change-queue-tester/3626/artifact/</a></span></p>
<p dir="ltr"
style="line-height:1.38;margin-top:0pt;margin-bottom:0pt;"><span style="font-size:11pt;font-family:Arial;color:#000000;background-color:transparent;font-weight:400;font-style:normal;font-variant:normal;text-decoration:none;vertical-align:baseline;white-space:pre-wrap;">
</span></p>
<a class="moz-txt-link-freetext" href="http://jenkins.ovirt.org/job/ovirt-master_change-queue-tester/3626/artifact…">http://jenkins.ovirt.org/job/ovirt-master_change-queue-tester/3626/artifact…</a></b></p>
<p><b style="font-weight:normal;"
id="docs-internal-guid-5859b7a1-911e-5616-3cbc-97286587db85"><br>
<p dir="ltr"
style="line-height:1.38;margin-top:0pt;margin-bottom:0pt;"><span style="font-size:11pt;font-family:Arial;color:#000000;background-color:transparent;font-weight:400;font-style:normal;font-variant:normal;text-decoration:none;vertical-align:baseline;white-space:pre-wrap;">(Relevant) error snippet from the log: </span></p>
<p dir="ltr"
style="line-height:1.38;margin-top:0pt;margin-bottom:0pt;"><span style="font-size:11pt;font-family:Arial;color:#000000;background-color:transparent;font-weight:400;font-style:normal;font-variant:normal;text-decoration:none;vertical-align:baseline;white-space:pre-wrap;"><error></span></p>
\</b><br>
</p>
<pre style="color: rgb(0, 0, 0); font-style: normal; font-variant-ligatures: normal; font-variant-caps: normal; font-weight: normal; letter-spacing: normal; orphans: 2; text-align: start; text-indent: 0px; text-transform: none; widows: 2; word-spacing: 0px; -webkit-text-stroke-width: 0px; text-decoration-style: initial; text-decoration-color: initial;">2017-11-06 02:56:46,526-0500 DEBUG otopi.plugins.ovirt_host_deploy.vdsm.packages plugin.execute:921 execute-output: ('/usr/bin/vdsm-tool', 'configure', '--force') stdout:
Checking configuration status...
abrt is not configured for vdsm
WARNING: LVM local configuration: /etc/lvm/lvmlocal.conf is not based on vdsm configuration
lvm requires configuration
libvirt is not configured for vdsm yet
FAILED: conflicting vdsm and libvirt-qemu tls configuration.
vdsm.conf with ssl=True requires the following changes:
libvirtd.conf: listen_tcp=0, auth_tcp="sasl", listen_tls=1
qemu.conf: spice_tls=1.
multipath requires configuration
</pre>
<br class="Apple-interchange-newline">
<pre style="color: rgb(0, 0, 0); font-style: normal; font-variant-ligatures: normal; font-variant-caps: normal; font-weight: normal; letter-spacing: normal; orphans: 2; text-align: start; text-indent: 0px; text-transform: none; widows: 2; word-spacing: 0px; -webkit-text-stroke-width: 0px; text-decoration-style: initial; text-decoration-color: initial;">2017-11-06 02:56:47,551-0500 DEBUG otopi.plugins.otopi.services.systemd plugin.execute:926 execute-output: ('/usr/bin/systemctl', 'start', 'ovirt-imageio-daemon.service') stderr:
Job for ovirt-imageio-daemon.service failed because the control process exited with error code. See "systemctl status ovirt-imageio-daemon.service" and "journalctl -xe" for details.
2017-11-06 02:56:47,552-0500 DEBUG otopi.context context._executeMethod:143 method exception
Traceback (most recent call last):
File "/tmp/ovirt-R4R8gZhaQI/pythonlib/otopi/context.py", line 133, in _executeMethod
method['method']()
File "/tmp/ovirt-R4R8gZhaQI/otopi-plugins/ovirt-host-deploy/vdsm/packages.py", line 179, in _start
self.services.state('ovirt-imageio-daemon', True)
File "/tmp/ovirt-R4R8gZhaQI/otopi-plugins/otopi/services/systemd.py", line 141, in state
service=name,
RuntimeError: Failed to start service 'ovirt-imageio-daemon'
2017-11-06 02:56:47,553-0500 ERROR otopi.context context._executeMethod:152 Failed to execute stage 'Closing up': Failed to start service 'ovirt-imageio-daemon'</pre>
<p><b style="font-weight:normal;"
id="docs-internal-guid-5859b7a1-911e-5616-3cbc-97286587db85">
<p dir="ltr"
style="line-height:1.38;margin-top:0pt;margin-bottom:0pt;"><span style="font-size:11pt;font-family:Arial;color:#000000;background-color:transparent;font-weight:400;font-style:normal;font-variant:normal;text-decoration:none;vertical-align:baseline;white-space:pre-wrap;"></error></span></p>
<br>
</b></p>
</body>
</html>
--------------88DD9FCD3B3DDEA34D4EAEF4--
5
7
[ OST Failure Report ] [ oVirt master ] [ 06-11-2017] [ 002_bootstrap.add_hosts ]
by Dafna Ron 07 Nov '17
by Dafna Ron 07 Nov '17
07 Nov '17
This is a multi-part message in MIME format.
--------------F7FAC86DB10D28BA8B32AB1D
Content-Type: text/plain; charset=utf-8
Content-Transfer-Encoding: 7bit
Hi,
We have a failure in test 002_bootstrap.add_hosts
I can see that the hosts are added but there are errors in engine.log
that suggest there is a problem with crypto.
**
*Link to suspected patches: https://gerrit.ovirt.org/#/c/83367/*
*
Link to Job:
http://jenkins.ovirt.org/job/ovirt-master_change-queue-tester/3636/
Link to all logs:
http://jenkins.ovirt.org/job/ovirt-master_change-queue-tester/3636/artifact/
**
http://jenkins.ovirt.org/job/ovirt-master_change-queue-tester/3636/artifact…
(Relevant) error snippet from the log:
<error>
*
2017-11-06 10:29:55,367-05 DEBUG [org.ovirt.engine.core.dal.dbbroker.DbFacadeUtils] (default task-7) [d5f76e37-ef60-4195-a735-400c1ed293be] Failed to decrypt password: javax.crypto.BadPaddingException: Decryption error
at sun.security.rsa.RSAPadding.unpadV15(RSAPadding.java:380) [rt.jar:1.8.0_151]
at sun.security.rsa.RSAPadding.unpad(RSAPadding.java:291) [rt.jar:1.8.0_151]
at com.sun.crypto.provider.RSACipher.doFinal(RSACipher.java:363) [sunjce_provider.jar:1.8.0_151]
at com.sun.crypto.provider.RSACipher.engineDoFinal(RSACipher.java:389) [sunjce_provider.jar:1.8.0_151]
at javax.crypto.Cipher.doFinal(Cipher.java:2165) [jce.jar:1.8.0_151]
at org.ovirt.engine.core.utils.crypt.EngineEncryptionUtils.decrypt(EngineEncryptionUtils.java:179) [utils.jar:]
at org.ovirt.engine.core.dal.dbbroker.DbFacadeUtils.decryptPassword(DbFacadeUtils.java:35) [dal.jar:]
at org.ovirt.engine.core.dao.provider.ProviderDaoImpl$ProviderRowMapper.mapRow(ProviderDaoImpl.java:137) [dal.jar:]
at org.ovirt.engine.core.dao.provider.ProviderDaoImpl$ProviderRowMapper.mapRow(ProviderDaoImpl.java:119) [dal.jar:]
at org.springframework.jdbc.core.RowMapperResultSetExtractor.extractData(RowMapperResultSetExtractor.java:93) [spring-jdbc.jar:4.3.9.RELEASE]
at org.springframework.jdbc.core.RowMapperResultSetExtractor.extractData(RowMapperResultSetExtractor.java:60) [spring-jdbc.jar:4.3.9.RELEASE]
at org.springframework.jdbc.core.JdbcTemplate$1QueryStatementCallback.doInStatement(JdbcTemplate.java:463) [spring-jdbc.jar:4.3.9.RELEASE]
at org.springframework.jdbc.core.JdbcTemplate.execute(JdbcTemplate.java:408) [spring-jdbc.jar:4.3.9.RELEASE]
at org.springframework.jdbc.core.JdbcTemplate.query(JdbcTemplate.java:474) [spring-jdbc.jar:4.3.9.RELEASE]
at org.springframework.jdbc.core.JdbcTemplate.query(JdbcTemplate.java:484) [spring-jdbc.jar:4.3.9.RELEASE]
at org.ovirt.engine.core.dao.provider.ProviderDaoImpl.getAllWithQuery(ProviderDaoImpl.java:189) [dal.jar:]
at org.ovirt.engine.core.bll.SearchQuery.genericSearch(SearchQuery.java:337) [bll.jar:]
at org.ovirt.engine.core.bll.SearchQuery.searchProviders(SearchQuery.java:408) [bll.jar:]
at org.ovirt.engine.core.bll.SearchQuery.executeQueryCommand(SearchQuery.java:215) [bll.jar:]
at org.ovirt.engine.core.bll.QueriesCommandBase.executeCommand(QueriesCommandBase.java:106) [bll.jar:]
at org.ovirt.engine.core.dal.VdcCommandBase.execute(VdcCommandBase.java:33) [dal.jar:]
at org.ovirt.engine.core.bll.executor.DefaultBackendQueryExecutor.execute(DefaultBackendQueryExecutor.java:14) [bll.jar:]
at org.ovirt.engine.core.bll.Backend.runQueryImpl(Backend.java:579) [bll.jar:]
at org.ovirt.engine.core.bll.Backend.runQuery(Backend.java:548) [bll.jar:]
at sun.reflect.NativeMethodAccessorImpl.invoke0(Native Method) [rt.jar:1.8.0_151]
at sun.reflect.NativeMethodAccessorImpl.invoke(NativeMethodAccessorImpl.java:62) [rt.jar:1.8.0_151]
at sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:43) [rt.jar:1.8.0_151]
at java.lang.reflect.Method.invoke(Method.java:498) [rt.jar:1.8.0_151]
at org.jboss.as.ee.component.ManagedReferenceMethodInterceptor.processInvocation(ManagedReferenceMethodInterceptor.java:52)
at org.jboss.invocation.InterceptorContext.proceed(InterceptorContext.java:422)
at org.jboss.invocation.InterceptorContext$Invocation.proceed(InterceptorContext.java:509)
at org.jboss.as.weld.ejb.DelegatingInterceptorInvocationContext.proceed(DelegatingInterceptorInvocationContext.java:92) [wildfly-weld-ejb-11.0.0.Final.jar:11.0.0.Final]
at org.jboss.weld.interceptor.proxy.WeldInvocationContext.interceptorChainCompleted(WeldInvocationContext.java:98) [weld-core-impl-2.4.3.Final.jar:2.4.3.Final]
at org.jboss.weld.interceptor.proxy.WeldInvocationContext.proceed(WeldInvocationContext.java:117) [weld-core-impl-2.4.3.Final.jar:2.4.3.Final]
at org.ovirt.engine.core.common.di.interceptor.LoggingInterceptor.apply(LoggingInterceptor.java:12) [common.jar:]
at sun.reflect.GeneratedMethodAccessor69.invoke(Unknown Source) [:1.8.0_151]
at sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:43) [rt.jar:1.8.0_151]
at java.lang.reflect.Method.invoke(Method.java:498) [rt.jar:1.8.0_151]
at org.jboss.weld.interceptor.reader.SimpleInterceptorInvocation$SimpleMethodInvocation.invoke(SimpleInterceptorInvocation.java:73) [weld-core-impl-2.4.3.Final.jar:2.4.3.Final]
at org.jboss.weld.interceptor.proxy.WeldInvocationContext.invokeNext(WeldInvocationContext.java:83) [weld-core-impl-2.4.3.Final.jar:2.4.3.Final]
at org.jboss.weld.interceptor.proxy.WeldInvocationContext.proceed(WeldInvocationContext.java:115) [weld-core-impl-2.4.3.Final.jar:2.4.3.Final]
at org.jboss.weld.bean.InterceptorImpl.intercept(InterceptorImpl.java:108) [weld-core-impl-2.4.3.Final.jar:2.4.3.Final]
at org.jboss.as.weld.ejb.DelegatingInterceptorInvocationContext.proceed(DelegatingInterceptorInvocationContext.java:82) [wildfly-weld-ejb-11.0.0.Final.jar:11.0.0.Final]
at org.jboss.as.weld.interceptors.EjbComponentInterceptorSupport.delegateInterception(EjbComponentInterceptorSupport.java:60)
at org.jboss.as.weld.interceptors.Jsr299BindingsInterceptor.delegateInterception(Jsr299BindingsInterceptor.java:76)
at org.jboss.as.weld.interceptors.Jsr299BindingsInterceptor.doMethodInterception(Jsr299BindingsInterceptor.java:88)
at org.jboss.as.weld.interceptors.Jsr299BindingsInterceptor.processInvocation(Jsr299BindingsInterceptor.java:101)
at org.jboss.as.ee.component.interceptors.UserInterceptorFactory$1.processInvocation(UserInterceptorFactory.java:63)
at org.jboss.invocation.InterceptorContext.proceed(InterceptorContext.java:422)
at org.jboss.invocation.InterceptorContext$Invocation.proceed(InterceptorContext.java:509)
at org.ovirt.engine.core.bll.interceptors.CorrelationIdTrackerInterceptor.aroundInvoke(CorrelationIdTrackerInterceptor.java:13) [bll.jar:]
at sun.reflect.GeneratedMethodAccessor66.invoke(Unknown Source) [:1.8.0_151]
at sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:43) [rt.jar:1.8.0_151]
at java.lang.reflect.Method.invoke(Method.java:498) [rt.jar:1.8.0_151]
at org.jboss.as.ee.component.ManagedReferenceLifecycleMethodInterceptor.processInvocation(ManagedReferenceLifecycleMethodInterceptor.java:89)
at org.jboss.invocation.InterceptorContext.proceed(InterceptorContext.java:422)
at org.jboss.as.ejb3.component.invocationmetrics.ExecutionTimeInterceptor.processInvocation(ExecutionTimeInterceptor.java:43) [wildfly-ejb3-11.0.0.Final.jar:11.0.0.Final]
at org.jboss.invocation.InterceptorContext.proceed(InterceptorContext.java:422)
at org.jboss.as.ee.concurrent.ConcurrentContextInterceptor.processInvocation(ConcurrentContextInterceptor.java:45) [wildfly-ee-11.0.0.Final.jar:11.0.0.Final]
at org.jboss.invocation.InterceptorContext.proceed(InterceptorContext.java:422)
at org.jboss.invocation.InitialInterceptor.processInvocation(InitialInterceptor.java:40)
at org.jboss.invocation.InterceptorContext.proceed(InterceptorContext.java:422)
at org.jboss.invocation.ChainedInterceptor.processInvocation(ChainedInterceptor.java:53)
at org.jboss.as.ee.component.interceptors.ComponentDispatcherInterceptor.processInvocation(ComponentDispatcherInterceptor.java:52)
at org.jboss.invocation.InterceptorContext.proceed(InterceptorContext.java:422)
at org.jboss.as.ejb3.component.singleton.SingletonComponentInstanceAssociationInterceptor.processInvocation(SingletonComponentInstanceAssociationInterceptor.java:53) [wildfly-ejb3-11.0.0.Final.jar:11.0.0.Final]
at org.jboss.invocation.InterceptorContext.proceed(InterceptorContext.java:422)
at org.jboss.as.ejb3.tx.CMTTxInterceptor.invokeInNoTx(CMTTxInterceptor.java:264) [wildfly-ejb3-11.0.0.Final.jar:11.0.0.Final]
at org.jboss.as.ejb3.tx.CMTTxInterceptor.supports(CMTTxInterceptor.java:379) [wildfly-ejb3-11.0.0.Final.jar:11.0.0.Final]
at org.jboss.as.ejb3.tx.CMTTxInterceptor.processInvocation(CMTTxInterceptor.java:244) [wildfly-ejb3-11.0.0.Final.jar:11.0.0.Final]
at org.jboss.invocation.InterceptorContext.proceed(InterceptorContext.java:422)
at org.jboss.invocation.InterceptorContext$Invocation.proceed(InterceptorContext.java:509)
at org.jboss.weld.ejb.AbstractEJBRequestScopeActivationInterceptor.aroundInvoke(AbstractEJBRequestScopeActivationInterceptor.java:73) [weld-core-impl-2.4.3.Final.jar:2.4.3.Final]
at org.jboss.as.weld.ejb.EjbRequestScopeActivationInterceptor.processInvocation(EjbRequestScopeActivationInterceptor.java:89)
at org.jboss.invocation.InterceptorContext.proceed(InterceptorContext.java:422)
at org.jboss.as.ejb3.component.interceptors.CurrentInvocationContextInterceptor.processInvocation(CurrentInvocationContextInterceptor.java:41) [wildfly-ejb3-11.0.0.Final.jar:11.0.0.Final]
at org.jboss.invocation.InterceptorContext.proceed(InterceptorContext.java:422)
at org.jboss.as.ejb3.component.invocationmetrics.WaitTimeInterceptor.processInvocation(WaitTimeInterceptor.java:47) [wildfly-ejb3-11.0.0.Final.jar:11.0.0.Final]
at org.jboss.invocation.InterceptorContext.proceed(InterceptorContext.java:422)
at org.jboss.as.ejb3.security.SecurityContextInterceptor.processInvocation(SecurityContextInterceptor.java:100) [wildfly-ejb3-11.0.0.Final.jar:11.0.0.Final]
at org.jboss.invocation.InterceptorContext.proceed(InterceptorContext.java:422)
at org.jboss.as.ejb3.deployment.processors.StartupAwaitInterceptor.processInvocation(StartupAwaitInterceptor.java:22) [wildfly-ejb3-11.0.0.Final.jar:11.0.0.Final]
at org.jboss.invocation.InterceptorContext.proceed(InterceptorContext.java:422)
at org.jboss.as.ejb3.component.interceptors.ShutDownInterceptorFactory$1.processInvocation(ShutDownInterceptorFactory.java:64) [wildfly-ejb3-11.0.0.Final.jar:11.0.0.Final]
at org.jboss.invocation.InterceptorContext.proceed(InterceptorContext.java:422)
at org.jboss.as.ejb3.component.interceptors.LoggingInterceptor.processInvocation(LoggingInterceptor.java:67) [wildfly-ejb3-11.0.0.Final.jar:11.0.0.Final]
at org.jboss.invocation.InterceptorContext.proceed(InterceptorContext.java:422)
at org.jboss.as.ee.component.NamespaceContextInterceptor.processInvocation(NamespaceContextInterceptor.java:50)
at org.jboss.invocation.InterceptorContext.proceed(InterceptorContext.java:422)
at org.jboss.invocation.ContextClassLoaderInterceptor.processInvocation(ContextClassLoaderInterceptor.java:60)
at org.jboss.invocation.InterceptorContext.proceed(InterceptorContext.java:422)
at org.jboss.invocation.InterceptorContext.run(InterceptorContext.java:438)
at org.wildfly.security.manager.WildFlySecurityManager.doChecked(WildFlySecurityManager.java:609)
at org.jboss.invocation.AccessCheckingInterceptor.processInvocation(AccessCheckingInterceptor.java:57)
at org.jboss.invocation.InterceptorContext.proceed(InterceptorContext.java:422)
at org.jboss.invocation.ChainedInterceptor.processInvocation(ChainedInterceptor.java:53)
at org.jboss.as.ee.component.ViewService$View.invoke(ViewService.java:198)
at org.jboss.as.ee.component.ViewDescription$1.processInvocation(ViewDescription.java:185)
at org.jboss.as.ee.component.ProxyInvocationHandler.invoke(ProxyInvocationHandler.java:81)
at org.ovirt.engine.core.common.interfaces.BackendLocal$$$view3.runQuery(Unknown Source) [common.jar:]
at org.ovirt.engine.api.restapi.resource.BackendResource.runQuery(BackendResource.java:82)
at org.ovirt.engine.api.restapi.resource.BackendResource.getBackendCollection(BackendResource.java:152)
at org.ovirt.engine.api.restapi.resource.AbstractBackendCollectionResource.getBackendCollection(AbstractBackendCollectionResource.java:48)
at org.ovirt.engine.api.restapi.resource.openstack.BackendOpenStackNetworkProvidersResource.getBackendCollection(BackendOpenStackNetworkProvidersResource.java:70)
at org.ovirt.engine.api.restapi.resource.openstack.BackendOpenStackNetworkProvidersResource.list(BackendOpenStackNetworkProvidersResource.java:50)
at org.ovirt.engine.api.resource.openstack.OpenstackNetworkProvidersResource.doList(OpenstackNetworkProvidersResource.java:52)
at sun.reflect.NativeMethodAccessorImpl.invoke0(Native Method) [rt.jar:1.8.0_151]
at sun.reflect.NativeMethodAccessorImpl.invoke(NativeMethodAccessorImpl.java:62) [rt.jar:1.8.0_151]
at sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:43) [rt.jar:1.8.0_151]
at java.lang.reflect.Method.invoke(Method.java:498) [rt.jar:1.8.0_151]
at org.jboss.resteasy.core.MethodInjectorImpl.invoke(MethodInjectorImpl.java:140) [resteasy-jaxrs-3.0.24.Final.jar:3.0.24.Final]
at org.jboss.resteasy.core.ResourceMethodInvoker.invokeOnTarget(ResourceMethodInvoker.java:295) [resteasy-jaxrs-3.0.24.Final.jar:3.0.24.Final]
at org.jboss.resteasy.core.ResourceMethodInvoker.invoke(ResourceMethodInvoker.java:249) [resteasy-jaxrs-3.0.24.Final.jar:3.0.24.Final]
at org.jboss.resteasy.core.ResourceLocatorInvoker.invokeOnTargetObject(ResourceLocatorInvoker.java:138) [resteasy-jaxrs-3.0.24.Final.jar:3.0.24.Final]
at org.jboss.resteasy.core.ResourceLocatorInvoker.invoke(ResourceLocatorInvoker.java:101) [resteasy-jaxrs-3.0.24.Final.jar:3.0.24.Final]
at org.jboss.resteasy.core.SynchronousDispatcher.invoke(SynchronousDispatcher.java:406) [resteasy-jaxrs-3.0.24.Final.jar:3.0.24.Final]
at org.jboss.resteasy.core.SynchronousDispatcher.invoke(SynchronousDispatcher.java:213) [resteasy-jaxrs-3.0.24.Final.jar:3.0.24.Final]
at org.jboss.resteasy.plugins.server.servlet.ServletContainerDispatcher.service(ServletContainerDispatcher.java:228) [resteasy-jaxrs-3.0.24.Final.jar:3.0.24.Final]
at org.jboss.resteasy.plugins.server.servlet.HttpServletDispatcher.service(HttpServletDispatcher.java:56) [resteasy-jaxrs-3.0.24.Final.jar:3.0.24.Final]
at org.jboss.resteasy.plugins.server.servlet.HttpServletDispatcher.service(HttpServletDispatcher.java:51) [resteasy-jaxrs-3.0.24.Final.jar:3.0.24.Final]
at javax.servlet.http.HttpServlet.service(HttpServlet.java:790) [jboss-servlet-api_3.1_spec-1.0.0.Final.jar:1.0.0.Final]
at io.undertow.servlet.handlers.ServletHandler.handleRequest(ServletHandler.java:85)
at io.undertow.servlet.handlers.FilterHandler.handleRequest(FilterHandler.java:81)
at io.undertow.servlet.handlers.security.ServletSecurityRoleHandler.handleRequest(ServletSecurityRoleHandler.java:62)
at io.undertow.servlet.handlers.ServletDispatchingHandler.handleRequest(ServletDispatchingHandler.java:36)
at io.undertow.server.handlers.PredicateHandler.handleRequest(PredicateHandler.java:43)
at io.undertow.server.handlers.PredicateHandler.handleRequest(PredicateHandler.java:43)
at io.undertow.server.handlers.PredicateHandler.handleRequest(PredicateHandler.java:43)
at io.undertow.servlet.handlers.ServletInitialHandler.dispatchRequest(ServletInitialHandler.java:274)
at io.undertow.servlet.handlers.ServletInitialHandler.dispatchToPath(ServletInitialHandler.java:209)
at io.undertow.servlet.spec.RequestDispatcherImpl.forwardImpl(RequestDispatcherImpl.java:221)
at io.undertow.servlet.spec.RequestDispatcherImpl.forwardImplSetup(RequestDispatcherImpl.java:147)
at io.undertow.servlet.spec.RequestDispatcherImpl.forward(RequestDispatcherImpl.java:111)
at org.ovirt.engine.api.restapi.invocation.VersionFilter.doFilter(VersionFilter.java:180)
at org.ovirt.engine.api.restapi.invocation.VersionFilter.doFilter(VersionFilter.java:98)
at io.undertow.servlet.core.ManagedFilter.doFilter(ManagedFilter.java:61)
at io.undertow.servlet.handlers.FilterHandler$FilterChainImpl.doFilter(FilterHandler.java:131)
at org.ovirt.engine.api.restapi.invocation.CurrentFilter.doFilter(CurrentFilter.java:117)
at org.ovirt.engine.api.restapi.invocation.CurrentFilter.doFilter(CurrentFilter.java:72)
at io.undertow.servlet.core.ManagedFilter.doFilter(ManagedFilter.java:61)
at io.undertow.servlet.handlers.FilterHandler$FilterChainImpl.doFilter(FilterHandler.java:131)
at org.ovirt.engine.core.aaa.filters.RestApiSessionMgmtFilter.doFilter(RestApiSessionMgmtFilter.java:78) [aaa.jar:]
at io.undertow.servlet.core.ManagedFilter.doFilter(ManagedFilter.java:61)
at io.undertow.servlet.handlers.FilterHandler$FilterChainImpl.doFilter(FilterHandler.java:131)
at org.ovirt.engine.core.aaa.filters.EnforceAuthFilter.doFilter(EnforceAuthFilter.java:42) [aaa.jar:]
at io.undertow.servlet.core.ManagedFilter.doFilter(ManagedFilter.java:61)
at io.undertow.servlet.handlers.FilterHandler$FilterChainImpl.doFilter(FilterHandler.java:131)
at org.ovirt.engine.core.aaa.filters.SsoRestApiNegotiationFilter.doFilter(SsoRestApiNegotiationFilter.java:84) [aaa.jar:]
at io.undertow.servlet.core.ManagedFilter.doFilter(ManagedFilter.java:61)
at io.undertow.servlet.handlers.FilterHandler$FilterChainImpl.doFilter(FilterHandler.java:131)
at org.ovirt.engine.core.aaa.filters.SsoRestApiAuthFilter.doFilter(SsoRestApiAuthFilter.java:47) [aaa.jar:]
at io.undertow.servlet.core.ManagedFilter.doFilter(ManagedFilter.java:61)
at io.undertow.servlet.handlers.FilterHandler$FilterChainImpl.doFilter(FilterHandler.java:131)
at org.ovirt.engine.core.aaa.filters.SessionValidationFilter.doFilter(SessionValidationFilter.java:59) [aaa.jar:]
at io.undertow.servlet.core.ManagedFilter.doFilter(ManagedFilter.java:61)
at io.undertow.servlet.handlers.FilterHandler$FilterChainImpl.doFilter(FilterHandler.java:131)
at org.ovirt.engine.core.aaa.filters.RestApiSessionValidationFilter.doFilter(RestApiSessionValidationFilter.java:35) [aaa.jar:]
at io.undertow.servlet.core.ManagedFilter.doFilter(ManagedFilter.java:61)
at io.undertow.servlet.handlers.FilterHandler$FilterChainImpl.doFilter(FilterHandler.java:131)
at org.ovirt.engine.api.restapi.security.CSRFProtectionFilter.doFilter(CSRFProtectionFilter.java:111)
at org.ovirt.engine.api.restapi.security.CSRFProtectionFilter.doFilter(CSRFProtectionFilter.java:102)
at io.undertow.servlet.core.ManagedFilter.doFilter(ManagedFilter.java:61)
at io.undertow.servlet.handlers.FilterHandler$FilterChainImpl.doFilter(FilterHandler.java:131)
at org.ovirt.engine.core.utils.servlet.CORSSupportFilter.doFilter(CORSSupportFilter.java:284) [utils.jar:]
at io.undertow.servlet.core.ManagedFilter.doFilter(ManagedFilter.java:61)
at io.undertow.servlet.handlers.FilterHandler$FilterChainImpl.doFilter(FilterHandler.java:131)
at io.undertow.servlet.handlers.FilterHandler.handleRequest(FilterHandler.java:84)
at io.undertow.servlet.handlers.security.ServletSecurityRoleHandler.handleRequest(ServletSecurityRoleHandler.java:62)
at io.undertow.servlet.handlers.ServletDispatchingHandler.handleRequest(ServletDispatchingHandler.java:36)
at org.wildfly.extension.undertow.security.SecurityContextAssociationHandler.handleRequest(SecurityContextAssociationHandler.java:78)
at io.undertow.server.handlers.PredicateHandler.handleRequest(PredicateHandler.java:43)
at io.undertow.servlet.handlers.security.SSLInformationAssociationHandler.handleRequest(SSLInformationAssociationHandler.java:131)
at io.undertow.servlet.handlers.security.ServletAuthenticationCallHandler.handleRequest(ServletAuthenticationCallHandler.java:57)
at io.undertow.server.handlers.PredicateHandler.handleRequest(PredicateHandler.java:43)
at io.undertow.security.handlers.AuthenticationConstraintHandler.handleRequest(AuthenticationConstraintHandler.java:53)
at io.undertow.security.handlers.AbstractConfidentialityHandler.handleRequest(AbstractConfidentialityHandler.java:46)
at io.undertow.servlet.handlers.security.ServletConfidentialityConstraintHandler.handleRequest(ServletConfidentialityConstraintHandler.java:64)
at io.undertow.servlet.handlers.security.ServletSecurityConstraintHandler.handleRequest(ServletSecurityConstraintHandler.java:59)
at io.undertow.security.handlers.AuthenticationMechanismsHandler.handleRequest(AuthenticationMechanismsHandler.java:60)
at io.undertow.servlet.handlers.security.CachedAuthenticatedSessionHandler.handleRequest(CachedAuthenticatedSessionHandler.java:77)
at io.undertow.security.handlers.NotificationReceiverHandler.handleRequest(NotificationReceiverHandler.java:50)
at io.undertow.security.handlers.AbstractSecurityContextAssociationHandler.handleRequest(AbstractSecurityContextAssociationHandler.java:43)
at io.undertow.server.handlers.PredicateHandler.handleRequest(PredicateHandler.java:43)
at org.wildfly.extension.undertow.security.jacc.JACCContextIdHandler.handleRequest(JACCContextIdHandler.java:61)
at io.undertow.server.handlers.PredicateHandler.handleRequest(PredicateHandler.java:43)
at io.undertow.server.handlers.PredicateHandler.handleRequest(PredicateHandler.java:43)
at io.undertow.servlet.handlers.ServletInitialHandler.handleFirstRequest(ServletInitialHandler.java:292)
at io.undertow.servlet.handlers.ServletInitialHandler.access$100(ServletInitialHandler.java:81)
at io.undertow.servlet.handlers.ServletInitialHandler$2.call(ServletInitialHandler.java:138)
at io.undertow.servlet.handlers.ServletInitialHandler$2.call(ServletInitialHandler.java:135)
at io.undertow.servlet.core.ServletRequestContextThreadSetupAction$1.call(ServletRequestContextThreadSetupAction.java:48)
at io.undertow.servlet.core.ContextClassLoaderSetupAction$1.call(ContextClassLoaderSetupAction.java:43)
at org.wildfly.extension.undertow.security.SecurityContextThreadSetupAction.lambda$create$0(SecurityContextThreadSetupAction.java:105)
at org.wildfly.extension.undertow.deployment.UndertowDeploymentInfoService$UndertowThreadSetupAction.lambda$create$0(UndertowDeploymentInfoService.java:1508)
at org.wildfly.extension.undertow.deployment.UndertowDeploymentInfoService$UndertowThreadSetupAction.lambda$create$0(UndertowDeploymentInfoService.java:1508)
at org.wildfly.extension.undertow.deployment.UndertowDeploymentInfoService$UndertowThreadSetupAction.lambda$create$0(UndertowDeploymentInfoService.java:1508)
at io.undertow.servlet.handlers.ServletInitialHandler.dispatchRequest(ServletInitialHandler.java:272)
at io.undertow.servlet.handlers.ServletInitialHandler.access$000(ServletInitialHandler.java:81)
at io.undertow.servlet.handlers.ServletInitialHandler$1.handleRequest(ServletInitialHandler.java:104)
at io.undertow.server.Connectors.executeRootHandler(Connectors.java:326)
at io.undertow.server.HttpServerExchange$1.run(HttpServerExchange.java:812)
at java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1149) [rt.jar:1.8.0_151]
at java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:624) [rt.jar:1.8.0_151]
at java.lang.Thread.run(Thread.java:748) [rt.jar:1.8.0_151]
2017-11-06 10:29:55,378-05 DEBUG [org.ovirt.engine.core.common.di.interceptor.DebugLoggingInterceptor] (default task-7) [d5f76e37-ef60-4195-a735-400c1ed293be] method: runQuery, params: [Search, SearchParameters:{refresh='false', filtered='false', searchType='Provider', searchPattern='Providers: type=EXTERNAL_NETWORK', caseSensitive='false', from='0', max='2147483647'}], timeElapsed: 49ms
2017-11-06 10:29:55,384-05 DEBUG [org.ovirt.engine.core.aaa.filters.SsoRestApiAuthFilter] (default task-8) [] Entered SsoRestApiAuthFilter
2017-11-06 10:29:55,384-05 DEBUG [org.ovirt.engine.core.aaa.filters.SsoRestApiAuthFilter] (default task-8) [] SsoRestApiAuthFilter authenticating with sso
2017-11-06 10:29:55,384-05 DEBUG [org.ovirt.engine.core.aaa.filters.SsoRestApiAuthFilter] (default task-8) [] SsoRestApiAuthFilter authenticating using BEARER header
2017-11-06 10:29:55,387-05 DEBUG [org.ovirt.engine.core.aaa.filters.SsoRestApiAuthFilter] (default task-8) [] SsoRestApiAuthFilter successfully authenticated using BEARER header
2017-11-06 10:29:55,387-05 DEBUG [org.ovirt.engine.core.aaa.filters.SsoRestApiNegotiationFilter] (default task-8) [] Entered SsoRestApiNegotiationFilter
2017-11-06 10:29:55,389-05 DEBUG [org.ovirt.engine.core.aaa.filters.SsoRestApiNegotiationFilter] (default task-8) [] SsoRestApiNegotiationFilter Not performing Negotiate Auth
**
*</error>*
*
*
--------------F7FAC86DB10D28BA8B32AB1D
Content-Type: text/html; charset=utf-8
Content-Transfer-Encoding: 7bit
<html>
<head>
<meta http-equiv="content-type" content="text/html; charset=utf-8">
</head>
<body text="#000000" bgcolor="#FFFFFF">
<p>Hi, <br>
</p>
<p>We have a failure in test 002_bootstrap.add_hosts</p>
<p>I can see that the hosts are added but there are errors in
engine.log that suggest there is a problem with crypto. <br>
</p>
<p><b style="font-weight:normal;"
id="docs-internal-guid-5859b7a1-9250-1b11-068e-84f67c0c1437">
<p dir="ltr"
style="line-height:1.38;margin-top:0pt;margin-bottom:0pt;"><span style="font-size:11pt;font-family:Arial;color:#000000;background-color:transparent;font-weight:400;font-style:normal;font-variant:normal;text-decoration:none;vertical-align:baseline;white-space:pre-wrap;">Link to suspected patches: <a class="moz-txt-link-freetext" href="https://gerrit.ovirt.org/#/c/83367/">https://gerrit.ovirt.org/#/c/83367/</a></span></p>
<br>
<p dir="ltr"
style="line-height:1.38;margin-top:0pt;margin-bottom:0pt;"><span style="font-size:11pt;font-family:Arial;color:#000000;background-color:transparent;font-weight:400;font-style:normal;font-variant:normal;text-decoration:none;vertical-align:baseline;white-space:pre-wrap;">Link to Job: <a class="moz-txt-link-freetext" href="http://jenkins.ovirt.org/job/ovirt-master_change-queue-tester/3636/">http://jenkins.ovirt.org/job/ovirt-master_change-queue-tester/3636/</a></span></p>
<br>
<p dir="ltr"
style="line-height:1.38;margin-top:0pt;margin-bottom:0pt;"><span style="font-size:11pt;font-family:Arial;color:#000000;background-color:transparent;font-weight:400;font-style:normal;font-variant:normal;text-decoration:none;vertical-align:baseline;white-space:pre-wrap;">Link to all logs: <a class="moz-txt-link-freetext" href="http://jenkins.ovirt.org/job/ovirt-master_change-queue-tester/3636/artifact/">http://jenkins.ovirt.org/job/ovirt-master_change-queue-tester/3636/artifact/</a></span></p>
<p dir="ltr"
style="line-height:1.38;margin-top:0pt;margin-bottom:0pt;"><span style="font-size:11pt;font-family:Arial;color:#000000;background-color:transparent;font-weight:400;font-style:normal;font-variant:normal;text-decoration:none;vertical-align:baseline;white-space:pre-wrap;">
</span></p>
</b><b style="font-weight:normal;"
id="docs-internal-guid-5859b7a1-9250-1b11-068e-84f67c0c1437">
<p dir="ltr"
style="line-height:1.38;margin-top:0pt;margin-bottom:0pt;"><span style="font-size:11pt;font-family:Arial;color:#000000;background-color:transparent;font-weight:400;font-style:normal;font-variant:normal;text-decoration:none;vertical-align:baseline;white-space:pre-wrap;"><a class="moz-txt-link-freetext" href="http://jenkins.ovirt.org/job/ovirt-master_change-queue-tester/3636/artifact…">http://jenkins.ovirt.org/job/ovirt-master_change-queue-tester/3636/artifact…</a>
</span></p>
<br>
<p dir="ltr"
style="line-height:1.38;margin-top:0pt;margin-bottom:0pt;"><span style="font-size:11pt;font-family:Arial;color:#000000;background-color:transparent;font-weight:400;font-style:normal;font-variant:normal;text-decoration:none;vertical-align:baseline;white-space:pre-wrap;">(Relevant) error snippet from the log: </span></p>
<p dir="ltr"
style="line-height:1.38;margin-top:0pt;margin-bottom:0pt;"><span style="font-size:11pt;font-family:Arial;color:#000000;background-color:transparent;font-weight:400;font-style:normal;font-variant:normal;text-decoration:none;vertical-align:baseline;white-space:pre-wrap;"><error></span></p>
<br>
</b><br>
</p>
<pre style="color: rgb(0, 0, 0); font-style: normal; font-variant-ligatures: normal; font-variant-caps: normal; font-weight: normal; letter-spacing: normal; orphans: 2; text-align: start; text-indent: 0px; text-transform: none; widows: 2; word-spacing: 0px; -webkit-text-stroke-width: 0px; text-decoration-style: initial; text-decoration-color: initial;">2017-11-06 10:29:55,367-05 DEBUG [org.ovirt.engine.core.dal.dbbroker.DbFacadeUtils] (default task-7) [d5f76e37-ef60-4195-a735-400c1ed293be] Failed to decrypt password: javax.crypto.BadPaddingException: Decryption error
at sun.security.rsa.RSAPadding.unpadV15(RSAPadding.java:380) [rt.jar:1.8.0_151]
at sun.security.rsa.RSAPadding.unpad(RSAPadding.java:291) [rt.jar:1.8.0_151]
at com.sun.crypto.provider.RSACipher.doFinal(RSACipher.java:363) [sunjce_provider.jar:1.8.0_151]
at com.sun.crypto.provider.RSACipher.engineDoFinal(RSACipher.java:389) [sunjce_provider.jar:1.8.0_151]
at javax.crypto.Cipher.doFinal(Cipher.java:2165) [jce.jar:1.8.0_151]
at org.ovirt.engine.core.utils.crypt.EngineEncryptionUtils.decrypt(EngineEncryptionUtils.java:179) [utils.jar:]
at org.ovirt.engine.core.dal.dbbroker.DbFacadeUtils.decryptPassword(DbFacadeUtils.java:35) [dal.jar:]
at org.ovirt.engine.core.dao.provider.ProviderDaoImpl$ProviderRowMapper.mapRow(ProviderDaoImpl.java:137) [dal.jar:]
at org.ovirt.engine.core.dao.provider.ProviderDaoImpl$ProviderRowMapper.mapRow(ProviderDaoImpl.java:119) [dal.jar:]
at org.springframework.jdbc.core.RowMapperResultSetExtractor.extractData(RowMapperResultSetExtractor.java:93) [spring-jdbc.jar:4.3.9.RELEASE]
at org.springframework.jdbc.core.RowMapperResultSetExtractor.extractData(RowMapperResultSetExtractor.java:60) [spring-jdbc.jar:4.3.9.RELEASE]
at org.springframework.jdbc.core.JdbcTemplate$1QueryStatementCallback.doInStatement(JdbcTemplate.java:463) [spring-jdbc.jar:4.3.9.RELEASE]
at org.springframework.jdbc.core.JdbcTemplate.execute(JdbcTemplate.java:408) [spring-jdbc.jar:4.3.9.RELEASE]
at org.springframework.jdbc.core.JdbcTemplate.query(JdbcTemplate.java:474) [spring-jdbc.jar:4.3.9.RELEASE]
at org.springframework.jdbc.core.JdbcTemplate.query(JdbcTemplate.java:484) [spring-jdbc.jar:4.3.9.RELEASE]
at org.ovirt.engine.core.dao.provider.ProviderDaoImpl.getAllWithQuery(ProviderDaoImpl.java:189) [dal.jar:]
at org.ovirt.engine.core.bll.SearchQuery.genericSearch(SearchQuery.java:337) [bll.jar:]
at org.ovirt.engine.core.bll.SearchQuery.searchProviders(SearchQuery.java:408) [bll.jar:]
at org.ovirt.engine.core.bll.SearchQuery.executeQueryCommand(SearchQuery.java:215) [bll.jar:]
at org.ovirt.engine.core.bll.QueriesCommandBase.executeCommand(QueriesCommandBase.java:106) [bll.jar:]
at org.ovirt.engine.core.dal.VdcCommandBase.execute(VdcCommandBase.java:33) [dal.jar:]
at org.ovirt.engine.core.bll.executor.DefaultBackendQueryExecutor.execute(DefaultBackendQueryExecutor.java:14) [bll.jar:]
at org.ovirt.engine.core.bll.Backend.runQueryImpl(Backend.java:579) [bll.jar:]
at org.ovirt.engine.core.bll.Backend.runQuery(Backend.java:548) [bll.jar:]
at sun.reflect.NativeMethodAccessorImpl.invoke0(Native Method) [rt.jar:1.8.0_151]
at sun.reflect.NativeMethodAccessorImpl.invoke(NativeMethodAccessorImpl.java:62) [rt.jar:1.8.0_151]
at sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:43) [rt.jar:1.8.0_151]
at java.lang.reflect.Method.invoke(Method.java:498) [rt.jar:1.8.0_151]
at org.jboss.as.ee.component.ManagedReferenceMethodInterceptor.processInvocation(ManagedReferenceMethodInterceptor.java:52)
at org.jboss.invocation.InterceptorContext.proceed(InterceptorContext.java:422)
at org.jboss.invocation.InterceptorContext$Invocation.proceed(InterceptorContext.java:509)
at org.jboss.as.weld.ejb.DelegatingInterceptorInvocationContext.proceed(DelegatingInterceptorInvocationContext.java:92) [wildfly-weld-ejb-11.0.0.Final.jar:11.0.0.Final]
at org.jboss.weld.interceptor.proxy.WeldInvocationContext.interceptorChainCompleted(WeldInvocationContext.java:98) [weld-core-impl-2.4.3.Final.jar:2.4.3.Final]
at org.jboss.weld.interceptor.proxy.WeldInvocationContext.proceed(WeldInvocationContext.java:117) [weld-core-impl-2.4.3.Final.jar:2.4.3.Final]
at org.ovirt.engine.core.common.di.interceptor.LoggingInterceptor.apply(LoggingInterceptor.java:12) [common.jar:]
at sun.reflect.GeneratedMethodAccessor69.invoke(Unknown Source) [:1.8.0_151]
at sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:43) [rt.jar:1.8.0_151]
at java.lang.reflect.Method.invoke(Method.java:498) [rt.jar:1.8.0_151]
at org.jboss.weld.interceptor.reader.SimpleInterceptorInvocation$SimpleMethodInvocation.invoke(SimpleInterceptorInvocation.java:73) [weld-core-impl-2.4.3.Final.jar:2.4.3.Final]
at org.jboss.weld.interceptor.proxy.WeldInvocationContext.invokeNext(WeldInvocationContext.java:83) [weld-core-impl-2.4.3.Final.jar:2.4.3.Final]
at org.jboss.weld.interceptor.proxy.WeldInvocationContext.proceed(WeldInvocationContext.java:115) [weld-core-impl-2.4.3.Final.jar:2.4.3.Final]
at org.jboss.weld.bean.InterceptorImpl.intercept(InterceptorImpl.java:108) [weld-core-impl-2.4.3.Final.jar:2.4.3.Final]
at org.jboss.as.weld.ejb.DelegatingInterceptorInvocationContext.proceed(DelegatingInterceptorInvocationContext.java:82) [wildfly-weld-ejb-11.0.0.Final.jar:11.0.0.Final]
at org.jboss.as.weld.interceptors.EjbComponentInterceptorSupport.delegateInterception(EjbComponentInterceptorSupport.java:60)
at org.jboss.as.weld.interceptors.Jsr299BindingsInterceptor.delegateInterception(Jsr299BindingsInterceptor.java:76)
at org.jboss.as.weld.interceptors.Jsr299BindingsInterceptor.doMethodInterception(Jsr299BindingsInterceptor.java:88)
at org.jboss.as.weld.interceptors.Jsr299BindingsInterceptor.processInvocation(Jsr299BindingsInterceptor.java:101)
at org.jboss.as.ee.component.interceptors.UserInterceptorFactory$1.processInvocation(UserInterceptorFactory.java:63)
at org.jboss.invocation.InterceptorContext.proceed(InterceptorContext.java:422)
at org.jboss.invocation.InterceptorContext$Invocation.proceed(InterceptorContext.java:509)
at org.ovirt.engine.core.bll.interceptors.CorrelationIdTrackerInterceptor.aroundInvoke(CorrelationIdTrackerInterceptor.java:13) [bll.jar:]
at sun.reflect.GeneratedMethodAccessor66.invoke(Unknown Source) [:1.8.0_151]
at sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:43) [rt.jar:1.8.0_151]
at java.lang.reflect.Method.invoke(Method.java:498) [rt.jar:1.8.0_151]
at org.jboss.as.ee.component.ManagedReferenceLifecycleMethodInterceptor.processInvocation(ManagedReferenceLifecycleMethodInterceptor.java:89)
at org.jboss.invocation.InterceptorContext.proceed(InterceptorContext.java:422)
at org.jboss.as.ejb3.component.invocationmetrics.ExecutionTimeInterceptor.processInvocation(ExecutionTimeInterceptor.java:43) [wildfly-ejb3-11.0.0.Final.jar:11.0.0.Final]
at org.jboss.invocation.InterceptorContext.proceed(InterceptorContext.java:422)
at org.jboss.as.ee.concurrent.ConcurrentContextInterceptor.processInvocation(ConcurrentContextInterceptor.java:45) [wildfly-ee-11.0.0.Final.jar:11.0.0.Final]
at org.jboss.invocation.InterceptorContext.proceed(InterceptorContext.java:422)
at org.jboss.invocation.InitialInterceptor.processInvocation(InitialInterceptor.java:40)
at org.jboss.invocation.InterceptorContext.proceed(InterceptorContext.java:422)
at org.jboss.invocation.ChainedInterceptor.processInvocation(ChainedInterceptor.java:53)
at org.jboss.as.ee.component.interceptors.ComponentDispatcherInterceptor.processInvocation(ComponentDispatcherInterceptor.java:52)
at org.jboss.invocation.InterceptorContext.proceed(InterceptorContext.java:422)
at org.jboss.as.ejb3.component.singleton.SingletonComponentInstanceAssociationInterceptor.processInvocation(SingletonComponentInstanceAssociationInterceptor.java:53) [wildfly-ejb3-11.0.0.Final.jar:11.0.0.Final]
at org.jboss.invocation.InterceptorContext.proceed(InterceptorContext.java:422)
at org.jboss.as.ejb3.tx.CMTTxInterceptor.invokeInNoTx(CMTTxInterceptor.java:264) [wildfly-ejb3-11.0.0.Final.jar:11.0.0.Final]
at org.jboss.as.ejb3.tx.CMTTxInterceptor.supports(CMTTxInterceptor.java:379) [wildfly-ejb3-11.0.0.Final.jar:11.0.0.Final]
at org.jboss.as.ejb3.tx.CMTTxInterceptor.processInvocation(CMTTxInterceptor.java:244) [wildfly-ejb3-11.0.0.Final.jar:11.0.0.Final]
at org.jboss.invocation.InterceptorContext.proceed(InterceptorContext.java:422)
at org.jboss.invocation.InterceptorContext$Invocation.proceed(InterceptorContext.java:509)
at org.jboss.weld.ejb.AbstractEJBRequestScopeActivationInterceptor.aroundInvoke(AbstractEJBRequestScopeActivationInterceptor.java:73) [weld-core-impl-2.4.3.Final.jar:2.4.3.Final]
at org.jboss.as.weld.ejb.EjbRequestScopeActivationInterceptor.processInvocation(EjbRequestScopeActivationInterceptor.java:89)
at org.jboss.invocation.InterceptorContext.proceed(InterceptorContext.java:422)
at org.jboss.as.ejb3.component.interceptors.CurrentInvocationContextInterceptor.processInvocation(CurrentInvocationContextInterceptor.java:41) [wildfly-ejb3-11.0.0.Final.jar:11.0.0.Final]
at org.jboss.invocation.InterceptorContext.proceed(InterceptorContext.java:422)
at org.jboss.as.ejb3.component.invocationmetrics.WaitTimeInterceptor.processInvocation(WaitTimeInterceptor.java:47) [wildfly-ejb3-11.0.0.Final.jar:11.0.0.Final]
at org.jboss.invocation.InterceptorContext.proceed(InterceptorContext.java:422)
at org.jboss.as.ejb3.security.SecurityContextInterceptor.processInvocation(SecurityContextInterceptor.java:100) [wildfly-ejb3-11.0.0.Final.jar:11.0.0.Final]
at org.jboss.invocation.InterceptorContext.proceed(InterceptorContext.java:422)
at org.jboss.as.ejb3.deployment.processors.StartupAwaitInterceptor.processInvocation(StartupAwaitInterceptor.java:22) [wildfly-ejb3-11.0.0.Final.jar:11.0.0.Final]
at org.jboss.invocation.InterceptorContext.proceed(InterceptorContext.java:422)
at org.jboss.as.ejb3.component.interceptors.ShutDownInterceptorFactory$1.processInvocation(ShutDownInterceptorFactory.java:64) [wildfly-ejb3-11.0.0.Final.jar:11.0.0.Final]
at org.jboss.invocation.InterceptorContext.proceed(InterceptorContext.java:422)
at org.jboss.as.ejb3.component.interceptors.LoggingInterceptor.processInvocation(LoggingInterceptor.java:67) [wildfly-ejb3-11.0.0.Final.jar:11.0.0.Final]
at org.jboss.invocation.InterceptorContext.proceed(InterceptorContext.java:422)
at org.jboss.as.ee.component.NamespaceContextInterceptor.processInvocation(NamespaceContextInterceptor.java:50)
at org.jboss.invocation.InterceptorContext.proceed(InterceptorContext.java:422)
at org.jboss.invocation.ContextClassLoaderInterceptor.processInvocation(ContextClassLoaderInterceptor.java:60)
at org.jboss.invocation.InterceptorContext.proceed(InterceptorContext.java:422)
at org.jboss.invocation.InterceptorContext.run(InterceptorContext.java:438)
at org.wildfly.security.manager.WildFlySecurityManager.doChecked(WildFlySecurityManager.java:609)
at org.jboss.invocation.AccessCheckingInterceptor.processInvocation(AccessCheckingInterceptor.java:57)
at org.jboss.invocation.InterceptorContext.proceed(InterceptorContext.java:422)
at org.jboss.invocation.ChainedInterceptor.processInvocation(ChainedInterceptor.java:53)
at org.jboss.as.ee.component.ViewService$View.invoke(ViewService.java:198)
at org.jboss.as.ee.component.ViewDescription$1.processInvocation(ViewDescription.java:185)
at org.jboss.as.ee.component.ProxyInvocationHandler.invoke(ProxyInvocationHandler.java:81)
at org.ovirt.engine.core.common.interfaces.BackendLocal$$$view3.runQuery(Unknown Source) [common.jar:]
at org.ovirt.engine.api.restapi.resource.BackendResource.runQuery(BackendResource.java:82)
at org.ovirt.engine.api.restapi.resource.BackendResource.getBackendCollection(BackendResource.java:152)
at org.ovirt.engine.api.restapi.resource.AbstractBackendCollectionResource.getBackendCollection(AbstractBackendCollectionResource.java:48)
at org.ovirt.engine.api.restapi.resource.openstack.BackendOpenStackNetworkProvidersResource.getBackendCollection(BackendOpenStackNetworkProvidersResource.java:70)
at org.ovirt.engine.api.restapi.resource.openstack.BackendOpenStackNetworkProvidersResource.list(BackendOpenStackNetworkProvidersResource.java:50)
at org.ovirt.engine.api.resource.openstack.OpenstackNetworkProvidersResource.doList(OpenstackNetworkProvidersResource.java:52)
at sun.reflect.NativeMethodAccessorImpl.invoke0(Native Method) [rt.jar:1.8.0_151]
at sun.reflect.NativeMethodAccessorImpl.invoke(NativeMethodAccessorImpl.java:62) [rt.jar:1.8.0_151]
at sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:43) [rt.jar:1.8.0_151]
at java.lang.reflect.Method.invoke(Method.java:498) [rt.jar:1.8.0_151]
at org.jboss.resteasy.core.MethodInjectorImpl.invoke(MethodInjectorImpl.java:140) [resteasy-jaxrs-3.0.24.Final.jar:3.0.24.Final]
at org.jboss.resteasy.core.ResourceMethodInvoker.invokeOnTarget(ResourceMethodInvoker.java:295) [resteasy-jaxrs-3.0.24.Final.jar:3.0.24.Final]
at org.jboss.resteasy.core.ResourceMethodInvoker.invoke(ResourceMethodInvoker.java:249) [resteasy-jaxrs-3.0.24.Final.jar:3.0.24.Final]
at org.jboss.resteasy.core.ResourceLocatorInvoker.invokeOnTargetObject(ResourceLocatorInvoker.java:138) [resteasy-jaxrs-3.0.24.Final.jar:3.0.24.Final]
at org.jboss.resteasy.core.ResourceLocatorInvoker.invoke(ResourceLocatorInvoker.java:101) [resteasy-jaxrs-3.0.24.Final.jar:3.0.24.Final]
at org.jboss.resteasy.core.SynchronousDispatcher.invoke(SynchronousDispatcher.java:406) [resteasy-jaxrs-3.0.24.Final.jar:3.0.24.Final]
at org.jboss.resteasy.core.SynchronousDispatcher.invoke(SynchronousDispatcher.java:213) [resteasy-jaxrs-3.0.24.Final.jar:3.0.24.Final]
at org.jboss.resteasy.plugins.server.servlet.ServletContainerDispatcher.service(ServletContainerDispatcher.java:228) [resteasy-jaxrs-3.0.24.Final.jar:3.0.24.Final]
at org.jboss.resteasy.plugins.server.servlet.HttpServletDispatcher.service(HttpServletDispatcher.java:56) [resteasy-jaxrs-3.0.24.Final.jar:3.0.24.Final]
at org.jboss.resteasy.plugins.server.servlet.HttpServletDispatcher.service(HttpServletDispatcher.java:51) [resteasy-jaxrs-3.0.24.Final.jar:3.0.24.Final]
at javax.servlet.http.HttpServlet.service(HttpServlet.java:790) [jboss-servlet-api_3.1_spec-1.0.0.Final.jar:1.0.0.Final]
at io.undertow.servlet.handlers.ServletHandler.handleRequest(ServletHandler.java:85)
at io.undertow.servlet.handlers.FilterHandler.handleRequest(FilterHandler.java:81)
at io.undertow.servlet.handlers.security.ServletSecurityRoleHandler.handleRequest(ServletSecurityRoleHandler.java:62)
at io.undertow.servlet.handlers.ServletDispatchingHandler.handleRequest(ServletDispatchingHandler.java:36)
at io.undertow.server.handlers.PredicateHandler.handleRequest(PredicateHandler.java:43)
at io.undertow.server.handlers.PredicateHandler.handleRequest(PredicateHandler.java:43)
at io.undertow.server.handlers.PredicateHandler.handleRequest(PredicateHandler.java:43)
at io.undertow.servlet.handlers.ServletInitialHandler.dispatchRequest(ServletInitialHandler.java:274)
at io.undertow.servlet.handlers.ServletInitialHandler.dispatchToPath(ServletInitialHandler.java:209)
at io.undertow.servlet.spec.RequestDispatcherImpl.forwardImpl(RequestDispatcherImpl.java:221)
at io.undertow.servlet.spec.RequestDispatcherImpl.forwardImplSetup(RequestDispatcherImpl.java:147)
at io.undertow.servlet.spec.RequestDispatcherImpl.forward(RequestDispatcherImpl.java:111)
at org.ovirt.engine.api.restapi.invocation.VersionFilter.doFilter(VersionFilter.java:180)
at org.ovirt.engine.api.restapi.invocation.VersionFilter.doFilter(VersionFilter.java:98)
at io.undertow.servlet.core.ManagedFilter.doFilter(ManagedFilter.java:61)
at io.undertow.servlet.handlers.FilterHandler$FilterChainImpl.doFilter(FilterHandler.java:131)
at org.ovirt.engine.api.restapi.invocation.CurrentFilter.doFilter(CurrentFilter.java:117)
at org.ovirt.engine.api.restapi.invocation.CurrentFilter.doFilter(CurrentFilter.java:72)
at io.undertow.servlet.core.ManagedFilter.doFilter(ManagedFilter.java:61)
at io.undertow.servlet.handlers.FilterHandler$FilterChainImpl.doFilter(FilterHandler.java:131)
at org.ovirt.engine.core.aaa.filters.RestApiSessionMgmtFilter.doFilter(RestApiSessionMgmtFilter.java:78) [aaa.jar:]
at io.undertow.servlet.core.ManagedFilter.doFilter(ManagedFilter.java:61)
at io.undertow.servlet.handlers.FilterHandler$FilterChainImpl.doFilter(FilterHandler.java:131)
at org.ovirt.engine.core.aaa.filters.EnforceAuthFilter.doFilter(EnforceAuthFilter.java:42) [aaa.jar:]
at io.undertow.servlet.core.ManagedFilter.doFilter(ManagedFilter.java:61)
at io.undertow.servlet.handlers.FilterHandler$FilterChainImpl.doFilter(FilterHandler.java:131)
at org.ovirt.engine.core.aaa.filters.SsoRestApiNegotiationFilter.doFilter(SsoRestApiNegotiationFilter.java:84) [aaa.jar:]
at io.undertow.servlet.core.ManagedFilter.doFilter(ManagedFilter.java:61)
at io.undertow.servlet.handlers.FilterHandler$FilterChainImpl.doFilter(FilterHandler.java:131)
at org.ovirt.engine.core.aaa.filters.SsoRestApiAuthFilter.doFilter(SsoRestApiAuthFilter.java:47) [aaa.jar:]
at io.undertow.servlet.core.ManagedFilter.doFilter(ManagedFilter.java:61)
at io.undertow.servlet.handlers.FilterHandler$FilterChainImpl.doFilter(FilterHandler.java:131)
at org.ovirt.engine.core.aaa.filters.SessionValidationFilter.doFilter(SessionValidationFilter.java:59) [aaa.jar:]
at io.undertow.servlet.core.ManagedFilter.doFilter(ManagedFilter.java:61)
at io.undertow.servlet.handlers.FilterHandler$FilterChainImpl.doFilter(FilterHandler.java:131)
at org.ovirt.engine.core.aaa.filters.RestApiSessionValidationFilter.doFilter(RestApiSessionValidationFilter.java:35) [aaa.jar:]
at io.undertow.servlet.core.ManagedFilter.doFilter(ManagedFilter.java:61)
at io.undertow.servlet.handlers.FilterHandler$FilterChainImpl.doFilter(FilterHandler.java:131)
at org.ovirt.engine.api.restapi.security.CSRFProtectionFilter.doFilter(CSRFProtectionFilter.java:111)
at org.ovirt.engine.api.restapi.security.CSRFProtectionFilter.doFilter(CSRFProtectionFilter.java:102)
at io.undertow.servlet.core.ManagedFilter.doFilter(ManagedFilter.java:61)
at io.undertow.servlet.handlers.FilterHandler$FilterChainImpl.doFilter(FilterHandler.java:131)
at org.ovirt.engine.core.utils.servlet.CORSSupportFilter.doFilter(CORSSupportFilter.java:284) [utils.jar:]
at io.undertow.servlet.core.ManagedFilter.doFilter(ManagedFilter.java:61)
at io.undertow.servlet.handlers.FilterHandler$FilterChainImpl.doFilter(FilterHandler.java:131)
at io.undertow.servlet.handlers.FilterHandler.handleRequest(FilterHandler.java:84)
at io.undertow.servlet.handlers.security.ServletSecurityRoleHandler.handleRequest(ServletSecurityRoleHandler.java:62)
at io.undertow.servlet.handlers.ServletDispatchingHandler.handleRequest(ServletDispatchingHandler.java:36)
at org.wildfly.extension.undertow.security.SecurityContextAssociationHandler.handleRequest(SecurityContextAssociationHandler.java:78)
at io.undertow.server.handlers.PredicateHandler.handleRequest(PredicateHandler.java:43)
at io.undertow.servlet.handlers.security.SSLInformationAssociationHandler.handleRequest(SSLInformationAssociationHandler.java:131)
at io.undertow.servlet.handlers.security.ServletAuthenticationCallHandler.handleRequest(ServletAuthenticationCallHandler.java:57)
at io.undertow.server.handlers.PredicateHandler.handleRequest(PredicateHandler.java:43)
at io.undertow.security.handlers.AuthenticationConstraintHandler.handleRequest(AuthenticationConstraintHandler.java:53)
at io.undertow.security.handlers.AbstractConfidentialityHandler.handleRequest(AbstractConfidentialityHandler.java:46)
at io.undertow.servlet.handlers.security.ServletConfidentialityConstraintHandler.handleRequest(ServletConfidentialityConstraintHandler.java:64)
at io.undertow.servlet.handlers.security.ServletSecurityConstraintHandler.handleRequest(ServletSecurityConstraintHandler.java:59)
at io.undertow.security.handlers.AuthenticationMechanismsHandler.handleRequest(AuthenticationMechanismsHandler.java:60)
at io.undertow.servlet.handlers.security.CachedAuthenticatedSessionHandler.handleRequest(CachedAuthenticatedSessionHandler.java:77)
at io.undertow.security.handlers.NotificationReceiverHandler.handleRequest(NotificationReceiverHandler.java:50)
at io.undertow.security.handlers.AbstractSecurityContextAssociationHandler.handleRequest(AbstractSecurityContextAssociationHandler.java:43)
at io.undertow.server.handlers.PredicateHandler.handleRequest(PredicateHandler.java:43)
at org.wildfly.extension.undertow.security.jacc.JACCContextIdHandler.handleRequest(JACCContextIdHandler.java:61)
at io.undertow.server.handlers.PredicateHandler.handleRequest(PredicateHandler.java:43)
at io.undertow.server.handlers.PredicateHandler.handleRequest(PredicateHandler.java:43)
at io.undertow.servlet.handlers.ServletInitialHandler.handleFirstRequest(ServletInitialHandler.java:292)
at io.undertow.servlet.handlers.ServletInitialHandler.access$100(ServletInitialHandler.java:81)
at io.undertow.servlet.handlers.ServletInitialHandler$2.call(ServletInitialHandler.java:138)
at io.undertow.servlet.handlers.ServletInitialHandler$2.call(ServletInitialHandler.java:135)
at io.undertow.servlet.core.ServletRequestContextThreadSetupAction$1.call(ServletRequestContextThreadSetupAction.java:48)
at io.undertow.servlet.core.ContextClassLoaderSetupAction$1.call(ContextClassLoaderSetupAction.java:43)
at org.wildfly.extension.undertow.security.SecurityContextThreadSetupAction.lambda$create$0(SecurityContextThreadSetupAction.java:105)
at org.wildfly.extension.undertow.deployment.UndertowDeploymentInfoService$UndertowThreadSetupAction.lambda$create$0(UndertowDeploymentInfoService.java:1508)
at org.wildfly.extension.undertow.deployment.UndertowDeploymentInfoService$UndertowThreadSetupAction.lambda$create$0(UndertowDeploymentInfoService.java:1508)
at org.wildfly.extension.undertow.deployment.UndertowDeploymentInfoService$UndertowThreadSetupAction.lambda$create$0(UndertowDeploymentInfoService.java:1508)
at io.undertow.servlet.handlers.ServletInitialHandler.dispatchRequest(ServletInitialHandler.java:272)
at io.undertow.servlet.handlers.ServletInitialHandler.access$000(ServletInitialHandler.java:81)
at io.undertow.servlet.handlers.ServletInitialHandler$1.handleRequest(ServletInitialHandler.java:104)
at io.undertow.server.Connectors.executeRootHandler(Connectors.java:326)
at io.undertow.server.HttpServerExchange$1.run(HttpServerExchange.java:812)
at java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1149) [rt.jar:1.8.0_151]
at java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:624) [rt.jar:1.8.0_151]
at java.lang.Thread.run(Thread.java:748) [rt.jar:1.8.0_151]
2017-11-06 10:29:55,378-05 DEBUG [org.ovirt.engine.core.common.di.interceptor.DebugLoggingInterceptor] (default task-7) [d5f76e37-ef60-4195-a735-400c1ed293be] method: runQuery, params: [Search, SearchParameters:{refresh='false', filtered='false', searchType='Provider', searchPattern='Providers: type=EXTERNAL_NETWORK', caseSensitive='false', from='0', max='2147483647'}], timeElapsed: 49ms
2017-11-06 10:29:55,384-05 DEBUG [org.ovirt.engine.core.aaa.filters.SsoRestApiAuthFilter] (default task-8) [] Entered SsoRestApiAuthFilter
2017-11-06 10:29:55,384-05 DEBUG [org.ovirt.engine.core.aaa.filters.SsoRestApiAuthFilter] (default task-8) [] SsoRestApiAuthFilter authenticating with sso
2017-11-06 10:29:55,384-05 DEBUG [org.ovirt.engine.core.aaa.filters.SsoRestApiAuthFilter] (default task-8) [] SsoRestApiAuthFilter authenticating using BEARER header
2017-11-06 10:29:55,387-05 DEBUG [org.ovirt.engine.core.aaa.filters.SsoRestApiAuthFilter] (default task-8) [] SsoRestApiAuthFilter successfully authenticated using BEARER header
2017-11-06 10:29:55,387-05 DEBUG [org.ovirt.engine.core.aaa.filters.SsoRestApiNegotiationFilter] (default task-8) [] Entered SsoRestApiNegotiationFilter
2017-11-06 10:29:55,389-05 DEBUG [org.ovirt.engine.core.aaa.filters.SsoRestApiNegotiationFilter] (default task-8) [] SsoRestApiNegotiationFilter Not performing Negotiate Auth</pre>
<p><b style="font-weight:normal;"
id="docs-internal-guid-5859b7a1-9250-1b11-068e-84f67c0c1437">
<p dir="ltr"
style="line-height:1.38;margin-top:0pt;margin-bottom:0pt;"><span style="font-size:11pt;font-family:Arial;color:#000000;background-color:transparent;font-weight:400;font-style:normal;font-variant:normal;text-decoration:none;vertical-align:baseline;white-space:pre-wrap;"></error></span></p>
<br>
</b></p>
</body>
</html>
--------------F7FAC86DB10D28BA8B32AB1D--
3
2
Hi,
Alexander fixed a long overlooked bug (
https://bugzilla.redhat.com/show_bug.cgi?id=1508033) where bookmarked
webadmin main pages were not respected when a user was logged out and
clicked the bookmark. Instead, the user was always redirected back to the
dashboard.
Now, navigating to /ovirt-engine/webadmin/#somePlace while logged out will
properly redirect you to the "somePlace" main page after the login form is
submitted.
Thanks Alexander for fixing and Vojtech and Ravi for thoughts and reviews.
And what we'd like to do next is make the UI *fully* bookmarkable by
allowing very expressive links directly to specific entities. I filed an
RFE for this: https://bugzilla.redhat.com/show_bug.cgi?id=1509065 There are
lots of ways this could go, but we imaging some kind of synchronization
with the search bar. /ovirt-engine/webadmin/#vms?name=my-vm or something
like that.
Thoughts are welcome.
Best wishes,
Greg
--
GREG SHEREMETA
SENIOR SOFTWARE ENGINEER - TEAM LEAD - RHV UX
Red Hat NA
<https://www.redhat.com/>
gshereme(a)redhat.com IRC: gshereme
<https://red.ht/sig>
2
1