[ OST Failure Report ] [ oVirt 4.2 ] [ 22-11-2017 ] [ 002_bootstrap.add_dc ]
This is a multi-part message in MIME format. --------------4E0BDA3C10A37D37E4CBA152 Content-Type: text/plain; charset=utf-8 Content-Transfer-Encoding: 7bit Hi, I am not sure we should be alerting on 4.2 versions yet but since we are getting several cq failures for the same reasons and test, I decided this should be communicated. The failed test is 002_bootstrap.add_dc in OST 4.2. and all are pointing to one patch. Remove mandatory option for cluster in register operation - https://gerrit.ovirt.org/#/c/84359/* * can you please have a look? I was unable to determine from what log the error was coming from as well but looking at the failed test and the patch reported there seem to be a connection. ** *Link and headline of suspected patches: * ** Patch that is marked as root cause: Remove mandatory option for cluster in register operation - https://gerrit.ovirt.org/#/c/84359/* * Patches reported as failed: 1. Add discoveredTargets property to HostService - https://gerrit.ovirt.org/#/c/84489/ *2. Document the fact that sparse/raw isn't supported on block - https://gerrit.ovirt.org/#/c/84490/* 3. Update CHANGES.adoc for release 4.2.25 - https://gerrit.ovirt.org/#/c/84499/* * 4. [maven-release-plugin] prepare release 4.2.26 - https://gerrit.ovirt.org/#/c/84500/* * ** * Link to Job: http://jenkins.ovirt.org/job/ovirt-4.2_change-queue-tester/* * * *Link to all logs:* *http://jenkins.ovirt.org/job/ovirt-4.2_change-queue-tester/75/artifact/* * * *(Relevant) error snippet from the log: * * <error> * Traceback (most recent call last): File "/usr/lib64/python2.7/unittest/case.py", line 369, in run testMethod() File "/usr/lib/python2.7/site-packages/nose/case.py", line 197, in runTest self.test(*self.arg) File "/usr/lib/python2.7/site-packages/ovirtlago/testlib.py", line 129, in wrapped_test test() File "/usr/lib/python2.7/site-packages/ovirtlago/testlib.py", line 59, in wrapper return func(get_test_prefix(), *args, **kwargs) File "/home/jenkins/workspace/ovirt-4.2_change-queue-tester/ovirt-system-tests/basic-suite-4.2/test-scenarios/002_bootstrap.py", line 109, in add_dc api = prefix.virt_env.engine_vm().get_api(api_ver=4) File "/usr/lib/python2.7/site-packages/ovirtlago/virt.py", line 329, in get_api return self.get_api_v4() File "/usr/lib/python2.7/site-packages/ovirtlago/virt.py", line 338, in get_api_v4 self._api_v4 = self._get_api(api_ver=4) File "/usr/lib/python2.7/site-packages/ovirtlago/virt.py", line 316, in _get_api while not testapi.test(): File "/usr/lib64/python2.7/site-packages/ovirtsdk4/__init__.py", line 744, in test six.reraise(*sys.exc_info()) File "/usr/lib64/python2.7/site-packages/ovirtsdk4/__init__.py", line 741, in test self.system_service().get() File "/usr/lib64/python2.7/site-packages/ovirtsdk4/services.py", line 21865, in get return self._internal_get(headers, query, wait) File "/usr/lib64/python2.7/site-packages/ovirtsdk4/service.py", line 202, in _internal_get context = self._connection.send(request) File "/usr/lib64/python2.7/site-packages/ovirtsdk4/__init__.py", line 370, in send return self.__send(request) File "/usr/lib64/python2.7/site-packages/ovirtsdk4/__init__.py", line 388, in __send self.authenticate() File "/usr/lib64/python2.7/site-packages/ovirtsdk4/__init__.py", line 381, in authenticate self._sso_token = self._get_access_token() File "/usr/lib64/python2.7/site-packages/ovirtsdk4/__init__.py", line 617, in _get_access_token sso_response = self._get_sso_response(self._sso_url, post_data) File "/usr/lib64/python2.7/site-packages/ovirtsdk4/__init__.py", line 702, in _get_sso_response self._check_content_type(self.__JSON_CONTENT_TYPE_RE, 'JSON', header_lines) File "/usr/lib64/python2.7/site-packages/ovirtsdk4/__init__.py", line 887, in _check_content_type raise Error(msg) Error: The response content type 'text/html; charset=UTF-8' isn't the expected JSON ** *</error>* ** --------------4E0BDA3C10A37D37E4CBA152 Content-Type: text/html; charset=utf-8 Content-Transfer-Encoding: 7bit <html> <head> <meta http-equiv="content-type" content="text/html; charset=utf-8"> </head> <body text="#000000" bgcolor="#FFFFFF"> <p>Hi, <br> </p> <p>I am not sure we should be alerting on 4.2 versions yet but since we are getting several cq failures for the same reasons and test, I decided this should be communicated. <br> </p> <p>The failed test is 002_bootstrap.add_dc in OST 4.2. and all are pointing to one patch. <br> </p> <p><span style="font-weight:normal;">Remove mandatory option for cluster in register operation - <a class="moz-txt-link-freetext" href="https://gerrit.ovirt.org/#/c/84359/">https://gerrit.ovirt.org/#/c/84359/</a></span><b style="font-weight:normal;" id="docs-internal-guid-5859b7a1-e37b-8f20-ebe3-4aa825e533e3"><br> </b></p> <p>can you please have a look? <br> </p> <p>I was unable to determine from what log the error was coming from as well but looking at the failed test and the patch reported there seem to be a connection. <br> </p> <p><b style="font-weight:normal;" id="docs-internal-guid-5859b7a1-e37b-8f20-ebe3-4aa825e533e3"> <p dir="ltr" style="line-height:1.38;margin-top:0pt;margin-bottom:0pt;"><span style="font-size:11pt;font-family:Arial;color:#000000;background-color:transparent;font-weight:400;font-style:normal;font-variant:normal;text-decoration:none;vertical-align:baseline;white-space:pre-wrap;">Link and headline of suspected patches: </span></p> </b></p> <p><span style="font-weight:normal;">Patch that is marked as root cause: <br> </span></p> <p><span style="font-weight:normal;">Remove mandatory option for cluster in register operation - <a class="moz-txt-link-freetext" href="https://gerrit.ovirt.org/#/c/84359/">https://gerrit.ovirt.org/#/c/84359/</a></span><b style="font-weight:normal;" id="docs-internal-guid-5859b7a1-e37b-8f20-ebe3-4aa825e533e3"><br> </b></p> Patches reported as failed: <br> 1. Add discoveredTargets property to HostService - <a class="moz-txt-link-freetext" href="https://gerrit.ovirt.org/#/c/84489/">https://gerrit.ovirt.org/#/c/84489/</a><br> <p><b style="font-weight:normal;" id="docs-internal-guid-5859b7a1-e37b-8f20-ebe3-4aa825e533e3">2. Document the fact that sparse/raw isn't supported on block - <a class="moz-txt-link-freetext" href="https://gerrit.ovirt.org/#/c/84490/">https://gerrit.ovirt.org/#/c/84490/</a></b></p> <p><span style="font-weight:normal;">3. Update CHANGES.adoc for release 4.2.25 - <a class="moz-txt-link-freetext" href="https://gerrit.ovirt.org/#/c/84499/">https://gerrit.ovirt.org/#/c/84499/</a></span><b style="font-weight:normal;" id="docs-internal-guid-5859b7a1-e37b-8f20-ebe3-4aa825e533e3"><br> </b></p> <p><span style="font-weight:normal;">4. [maven-release-plugin] prepare release 4.2.26 - <a class="moz-txt-link-freetext" href="https://gerrit.ovirt.org/#/c/84500/">https://gerrit.ovirt.org/#/c/84500/</a></span><b style="font-weight:normal;" id="docs-internal-guid-5859b7a1-e37b-8f20-ebe3-4aa825e533e3"><br> <p dir="ltr" style="line-height:1.38;margin-top:0pt;margin-bottom:0pt;"><span style="font-size:11pt;font-family:Arial;color:#000000;background-color:transparent;font-weight:400;font-style:normal;font-variant:normal;text-decoration:none;vertical-align:baseline;white-space:pre-wrap;"> </span></p> <p dir="ltr" style="line-height:1.38;margin-top:0pt;margin-bottom:0pt;"><span style="font-size:11pt;font-family:Arial;color:#000000;background-color:transparent;font-weight:400;font-style:normal;font-variant:normal;text-decoration:none;vertical-align:baseline;white-space:pre-wrap;">Link to Job:</span></p> <a class="moz-txt-link-freetext" href="http://jenkins.ovirt.org/job/ovirt-4.2_change-queue-tester/">http://jenkins.ovirt.org/job/ovirt-4.2_change-queue-tester/</a></b></p> <p><b style="font-weight:normal;" id="docs-internal-guid-5859b7a1-e37b-8f20-ebe3-4aa825e533e3"><br> <p dir="ltr" style="line-height:1.38;margin-top:0pt;margin-bottom:0pt;"><span style="font-size:11pt;font-family:Arial;color:#000000;background-color:transparent;font-weight:400;font-style:normal;font-variant:normal;text-decoration:none;vertical-align:baseline;white-space:pre-wrap;">Link to all logs:</span></p> <a class="moz-txt-link-freetext" href="http://jenkins.ovirt.org/job/ovirt-4.2_change-queue-tester/75/artifact/">http://jenkins.ovirt.org/job/ovirt-4.2_change-queue-tester/75/artifact/</a></b></p> <p><b style="font-weight:normal;" id="docs-internal-guid-5859b7a1-e37b-8f20-ebe3-4aa825e533e3"><br> <p dir="ltr" style="line-height:1.38;margin-top:0pt;margin-bottom:0pt;"><span style="font-size:11pt;font-family:Arial;color:#000000;background-color:transparent;font-weight:400;font-style:normal;font-variant:normal;text-decoration:none;vertical-align:baseline;white-space:pre-wrap;">(Relevant) error snippet from the log: </span></p> <p dir="ltr" style="line-height:1.38;margin-top:0pt;margin-bottom:0pt;"><span style="font-size:11pt;font-family:Arial;color:#000000;background-color:transparent;font-weight:400;font-style:normal;font-variant:normal;text-decoration:none;vertical-align:baseline;white-space:pre-wrap;"><error></span></p> <br> <br> </b><br> </p> <pre style="box-sizing: border-box; white-space: pre-wrap; word-wrap: break-word; margin: 0px; color: rgb(51, 51, 51); font-size: 14px; font-style: normal; font-variant-ligatures: normal; font-variant-caps: normal; font-weight: normal; letter-spacing: normal; orphans: 2; text-align: start; text-indent: 0px; text-transform: none; widows: 2; word-spacing: 0px; -webkit-text-stroke-width: 0px; text-decoration-style: initial; text-decoration-color: initial;">Traceback (most recent call last): File "/usr/lib64/python2.7/unittest/case.py", line 369, in run testMethod() File "/usr/lib/python2.7/site-packages/nose/case.py", line 197, in runTest self.test(*self.arg) File "/usr/lib/python2.7/site-packages/ovirtlago/testlib.py", line 129, in wrapped_test test() File "/usr/lib/python2.7/site-packages/ovirtlago/testlib.py", line 59, in wrapper return func(get_test_prefix(), *args, **kwargs) File "/home/jenkins/workspace/ovirt-4.2_change-queue-tester/ovirt-system-tests/basic-suite-4.2/test-scenarios/002_bootstrap.py", line 109, in add_dc api = prefix.virt_env.engine_vm().get_api(api_ver=4) File "/usr/lib/python2.7/site-packages/ovirtlago/virt.py", line 329, in get_api return self.get_api_v4() File "/usr/lib/python2.7/site-packages/ovirtlago/virt.py", line 338, in get_api_v4 self._api_v4 = self._get_api(api_ver=4) File "/usr/lib/python2.7/site-packages/ovirtlago/virt.py", line 316, in _get_api while not testapi.test(): File "/usr/lib64/python2.7/site-packages/ovirtsdk4/__init__.py", line 744, in test six.reraise(*sys.exc_info()) File "/usr/lib64/python2.7/site-packages/ovirtsdk4/__init__.py", line 741, in test self.system_service().get() File "/usr/lib64/python2.7/site-packages/ovirtsdk4/services.py", line 21865, in get return self._internal_get(headers, query, wait) File "/usr/lib64/python2.7/site-packages/ovirtsdk4/service.py", line 202, in _internal_get context = self._connection.send(request) File "/usr/lib64/python2.7/site-packages/ovirtsdk4/__init__.py", line 370, in send return self.__send(request) File "/usr/lib64/python2.7/site-packages/ovirtsdk4/__init__.py", line 388, in __send self.authenticate() File "/usr/lib64/python2.7/site-packages/ovirtsdk4/__init__.py", line 381, in authenticate self._sso_token = self._get_access_token() File "/usr/lib64/python2.7/site-packages/ovirtsdk4/__init__.py", line 617, in _get_access_token sso_response = self._get_sso_response(self._sso_url, post_data) File "/usr/lib64/python2.7/site-packages/ovirtsdk4/__init__.py", line 702, in _get_sso_response self._check_content_type(self.__JSON_CONTENT_TYPE_RE, 'JSON', header_lines) File "/usr/lib64/python2.7/site-packages/ovirtsdk4/__init__.py", line 887, in _check_content_type raise Error(msg) Error: The response content type 'text/html; charset=UTF-8' isn't the expected JSON</pre> <p><b style="font-weight:normal;" id="docs-internal-guid-5859b7a1-e37b-8f20-ebe3-4aa825e533e3"> <p dir="ltr" style="line-height:1.38;margin-top:0pt;margin-bottom:0pt;"><span style="font-size:11pt;font-family:Arial;color:#000000;background-color:transparent;font-weight:400;font-style:normal;font-variant:normal;text-decoration:none;vertical-align:baseline;white-space:pre-wrap;"></error></span></p> </b><br class="Apple-interchange-newline"> </p> <p><br> </p> </body> </html> --------------4E0BDA3C10A37D37E4CBA152--
On 11/22/2017 12:40 PM, Dafna Ron wrote:
Hi,
I am not sure we should be alerting on 4.2 versions yet but since we are getting several cq failures for the same reasons and test, I decided this should be communicated.
The failed test is 002_bootstrap.add_dc in OST 4.2. and all are pointing to one patch.
Remove mandatory option for cluster in register operation - https://gerrit.ovirt.org/#/c/84359/* *
can you please have a look?
I was unable to determine from what log the error was coming from as well but looking at the failed test and the patch reported there seem to be a connection.
It is almost impossible that that patch causes that problem. That patch is included in a release of the specification of the API that the engine and the SDKs aren't using yet. The stack trace below looks similar to a recent problem in version 4.2 (not yet released) of the Python SDK, which was fixed by this patch: No exceptions when 'raise_exception=False' https://gerrit.ovirt.org/83565 Please make sure that you use either a released version of the SDK, or else a pre-release build that contains that fix. Latest build from Jenkins should work fine.
**
*Link and headline of suspected patches: *
**
Patch that is marked as root cause:
Remove mandatory option for cluster in register operation - https://gerrit.ovirt.org/#/c/84359/* *
Patches reported as failed: 1. Add discoveredTargets property to HostService - https://gerrit.ovirt.org/#/c/84489/
*2. Document the fact that sparse/raw isn't supported on block - https://gerrit.ovirt.org/#/c/84490/*
3. Update CHANGES.adoc for release 4.2.25 - https://gerrit.ovirt.org/#/c/84499/* *
4. [maven-release-plugin] prepare release 4.2.26 - https://gerrit.ovirt.org/#/c/84500/* *
**
*
Link to Job:
http://jenkins.ovirt.org/job/ovirt-4.2_change-queue-tester/*
* *
*Link to all logs:*
*http://jenkins.ovirt.org/job/ovirt-4.2_change-queue-tester/75/artifact/*
* *
*(Relevant) error snippet from the log: *
*
<error>
*
Traceback (most recent call last): File "/usr/lib64/python2.7/unittest/case.py", line 369, in run testMethod() File "/usr/lib/python2.7/site-packages/nose/case.py", line 197, in runTest self.test(*self.arg) File "/usr/lib/python2.7/site-packages/ovirtlago/testlib.py", line 129, in wrapped_test test() File "/usr/lib/python2.7/site-packages/ovirtlago/testlib.py", line 59, in wrapper return func(get_test_prefix(), *args, **kwargs) File "/home/jenkins/workspace/ovirt-4.2_change-queue-tester/ovirt-system-tests/basic-suite-4.2/test-scenarios/002_bootstrap.py", line 109, in add_dc api = prefix.virt_env.engine_vm().get_api(api_ver=4) File "/usr/lib/python2.7/site-packages/ovirtlago/virt.py", line 329, in get_api return self.get_api_v4() File "/usr/lib/python2.7/site-packages/ovirtlago/virt.py", line 338, in get_api_v4 self._api_v4 = self._get_api(api_ver=4) File "/usr/lib/python2.7/site-packages/ovirtlago/virt.py", line 316, in _get_api while not testapi.test(): File "/usr/lib64/python2.7/site-packages/ovirtsdk4/__init__.py", line 744, in test six.reraise(*sys.exc_info()) File "/usr/lib64/python2.7/site-packages/ovirtsdk4/__init__.py", line 741, in test self.system_service().get() File "/usr/lib64/python2.7/site-packages/ovirtsdk4/services.py", line 21865, in get return self._internal_get(headers, query, wait) File "/usr/lib64/python2.7/site-packages/ovirtsdk4/service.py", line 202, in _internal_get context = self._connection.send(request) File "/usr/lib64/python2.7/site-packages/ovirtsdk4/__init__.py", line 370, in send return self.__send(request) File "/usr/lib64/python2.7/site-packages/ovirtsdk4/__init__.py", line 388, in __send self.authenticate() File "/usr/lib64/python2.7/site-packages/ovirtsdk4/__init__.py", line 381, in authenticate self._sso_token = self._get_access_token() File "/usr/lib64/python2.7/site-packages/ovirtsdk4/__init__.py", line 617, in _get_access_token sso_response = self._get_sso_response(self._sso_url, post_data) File "/usr/lib64/python2.7/site-packages/ovirtsdk4/__init__.py", line 702, in _get_sso_response self._check_content_type(self.__JSON_CONTENT_TYPE_RE, 'JSON', header_lines) File "/usr/lib64/python2.7/site-packages/ovirtsdk4/__init__.py", line 887, in _check_content_type raise Error(msg) Error: The response content type 'text/html; charset=UTF-8' isn't the expected JSON
**
*</error>*
**
On Wed, Nov 22, 2017 at 2:01 PM, Juan Hernández <jhernand@redhat.com> wrote:
On 11/22/2017 12:40 PM, Dafna Ron wrote:
Hi,
I am not sure we should be alerting on 4.2 versions yet but since we are getting several cq failures for the same reasons and test, I decided this should be communicated.
The failed test is 002_bootstrap.add_dc in OST 4.2. and all are pointing to one patch.
Remove mandatory option for cluster in register operation - https://gerrit.ovirt.org/#/c/84359/* *
can you please have a look?
I was unable to determine from what log the error was coming from as well but looking at the failed test and the patch reported there seem to be a connection.
It is almost impossible that that patch causes that problem. That patch is included in a release of the specification of the API that the engine and the SDKs aren't using yet.
The stack trace below looks similar to a recent problem in version 4.2 (not yet released) of the Python SDK, which was fixed by this patch:
No exceptions when 'raise_exception=False' https://gerrit.ovirt.org/83565
Please make sure that you use either a released version of the SDK, or else a pre-release build that contains that fix. Latest build from Jenkins should work fine.
The 4.2 flow is using 4.1 code mostly now, it might be the reason, once we'll release 4.2 we can update the flow to use the 4.2 release. The reason for that is we can't gurantee that if we'll use the 'master' or 'pre' repos pkg versions won't change and contains higher '4.3' versions, like we've seen in the past, something that takes a very long time to recover from and involves a lot of manual work. If there is no point in running the current flow and we need to wait until we'll switch to 4.2, then it might make sense to ignore it or disable it until then.
**
*Link and headline of suspected patches: *
**
Patch that is marked as root cause:
Remove mandatory option for cluster in register operation - https://gerrit.ovirt.org/#/c/84359/* *
Patches reported as failed: 1. Add discoveredTargets property to HostService - https://gerrit.ovirt.org/#/c/84489/
*2. Document the fact that sparse/raw isn't supported on block - https://gerrit.ovirt.org/#/c/84490/*
3. Update CHANGES.adoc for release 4.2.25 - https://gerrit.ovirt.org/#/c/84499/* *
4. [maven-release-plugin] prepare release 4.2.26 - https://gerrit.ovirt.org/#/c/84500/* *
**
*
Link to Job:
http://jenkins.ovirt.org/job/ovirt-4.2_change-queue-tester/*
* *
*Link to all logs:*
*http://jenkins.ovirt.org/job/ovirt-4.2_change-queue-tester/75/artifact/*
* *
*(Relevant) error snippet from the log: *
*
<error>
*
Traceback (most recent call last): File "/usr/lib64/python2.7/unittest/case.py", line 369, in run testMethod() File "/usr/lib/python2.7/site-packages/nose/case.py", line 197, in runTest self.test(*self.arg) File "/usr/lib/python2.7/site-packages/ovirtlago/testlib.py", line 129, in wrapped_test test() File "/usr/lib/python2.7/site-packages/ovirtlago/testlib.py", line 59, in wrapper return func(get_test_prefix(), *args, **kwargs) File "/home/jenkins/workspace/ovirt-4.2_change-queue-tester/ovirt -system-tests/basic-suite-4.2/test-scenarios/002_bootstrap.py", line 109, in add_dc api = prefix.virt_env.engine_vm().get_api(api_ver=4) File "/usr/lib/python2.7/site-packages/ovirtlago/virt.py", line 329, in get_api return self.get_api_v4() File "/usr/lib/python2.7/site-packages/ovirtlago/virt.py", line 338, in get_api_v4 self._api_v4 = self._get_api(api_ver=4) File "/usr/lib/python2.7/site-packages/ovirtlago/virt.py", line 316, in _get_api while not testapi.test(): File "/usr/lib64/python2.7/site-packages/ovirtsdk4/__init__.py", line 744, in test six.reraise(*sys.exc_info()) File "/usr/lib64/python2.7/site-packages/ovirtsdk4/__init__.py", line 741, in test self.system_service().get() File "/usr/lib64/python2.7/site-packages/ovirtsdk4/services.py", line 21865, in get return self._internal_get(headers, query, wait) File "/usr/lib64/python2.7/site-packages/ovirtsdk4/service.py", line 202, in _internal_get context = self._connection.send(request) File "/usr/lib64/python2.7/site-packages/ovirtsdk4/__init__.py", line 370, in send return self.__send(request) File "/usr/lib64/python2.7/site-packages/ovirtsdk4/__init__.py", line 388, in __send self.authenticate() File "/usr/lib64/python2.7/site-packages/ovirtsdk4/__init__.py", line 381, in authenticate self._sso_token = self._get_access_token() File "/usr/lib64/python2.7/site-packages/ovirtsdk4/__init__.py", line 617, in _get_access_token sso_response = self._get_sso_response(self._sso_url, post_data) File "/usr/lib64/python2.7/site-packages/ovirtsdk4/__init__.py", line 702, in _get_sso_response self._check_content_type(self.__JSON_CONTENT_TYPE_RE, 'JSON', header_lines) File "/usr/lib64/python2.7/site-packages/ovirtsdk4/__init__.py", line 887, in _check_content_type raise Error(msg) Error: The response content type 'text/html; charset=UTF-8' isn't the expected JSON
**
*</error>*
**
_______________________________________________ Devel mailing list Devel@ovirt.org http://lists.ovirt.org/mailman/listinfo/devel
-- Eyal edri MANAGER RHV DevOps EMEA VIRTUALIZATION R&D Red Hat EMEA <https://www.redhat.com/> <https://red.ht/sig> TRIED. TESTED. TRUSTED. <https://redhat.com/trusted> phone: +972-9-7692018 irc: eedri (on #tlv #rhev-dev #rhev-integ)
participants (3)
-
Dafna Ron -
Eyal Edri -
Juan Hernández