- Infra - oVirt List Archives

oVirt infra daily report - unstable production jobs - 614
by jenkins＠jenkins.phx.ovirt.org 15 Oct '18

15 Oct '18

Good morning! Attached is the HTML page with the jenkins status report. You can see it also here: - http://jenkins.ovirt.org/job/system_jenkins-report/614//artifact/exported-a… Cheers, Jenkins

1 0

[JENKINS] Failed to setup proejct kubevirt_kubevirt_standard-check-pr
by jenkins＠jenkins.phx.ovirt.org 15 Oct '18

15 Oct '18

Failed to run project_setup.sh for: #2226 kubevirt [check-patch]. It probably means that docker_cleanup.py failed. This step doesn't fail the job, but we do collect data about such failures to find the root cause. Infra owner, ensure that we're not running out of disk space on ovirt-srv04.phx.ovirt.org-container-6.

1 0

[oVirt Jenkins] ovirt-system-tests_hc-basic-suite-4.2 - Build # 566 - Failure!
by jenkins＠jenkins.phx.ovirt.org 15 Oct '18

15 Oct '18

Project: http://jenkins.ovirt.org/job/ovirt-system-tests_hc-basic-suite-4.2/ Build: http://jenkins.ovirt.org/job/ovirt-system-tests_hc-basic-suite-4.2/566/ Build Number: 566 Build Status: Failure Triggered By: Started by timer ------------------------------------- Changes Since Last Success: ------------------------------------- Changes for Build #566 [Eitan Raviv] networking: connect NFS to engine over IPv6 ----------------- Failed Tests: ----------------- 1 tests failed. FAILED: 002_bootstrap.add_hosts Error Message: Failed to read response: [(<pycurl.Curl object at 0x7fdf38bea050>, 7, 'TCP connection reset by peer')] -------------------- >> begin captured logging << -------------------- ovirtlago.testlib: ERROR: * Unhandled exception in <function _host_is_up_4 at 0x7fdf38b8d578> Traceback (most recent call last): File "/usr/lib/python2.7/site-packages/ovirtlago/testlib.py", line 234, in assert_equals_within res = func() File "/home/jenkins/workspace/ovirt-system-tests_hc-basic-suite-4.2/ovirt-system-tests/hc-basic-suite-4.2/test-scenarios/002_bootstrap.py", line 254, in _host_is_up_4 host_obj = host_service.get() File "/usr/lib64/python2.7/site-packages/ovirtsdk4/services.py", line 37093, in get return self._internal_get(headers, query, wait) File "/usr/lib64/python2.7/site-packages/ovirtsdk4/service.py", line 211, in _internal_get return future.wait() if wait else future File "/usr/lib64/python2.7/site-packages/ovirtsdk4/service.py", line 54, in wait response = self._connection.wait(self._context) File "/usr/lib64/python2.7/site-packages/ovirtsdk4/__init__.py", line 496, in wait return self.__wait(context, failed_auth) File "/usr/lib64/python2.7/site-packages/ovirtsdk4/__init__.py", line 510, in __wait raise Error("Failed to read response: {}".format(err_list)) Error: Failed to read response: [(<pycurl.Curl object at 0x7fdf38bea050>, 7, 'TCP connection reset by peer')] --------------------- >> end captured logging << --------------------- Stack Trace: File "/usr/lib64/python2.7/unittest/case.py", line 369, in run testMethod() File "/usr/lib/python2.7/site-packages/nose/case.py", line 197, in runTest self.test(*self.arg) File "/usr/lib/python2.7/site-packages/ovirtlago/testlib.py", line 142, in wrapped_test test() File "/usr/lib/python2.7/site-packages/ovirtlago/testlib.py", line 60, in wrapper return func(get_test_prefix(), *args, **kwargs) File "/home/jenkins/workspace/ovirt-system-tests_hc-basic-suite-4.2/ovirt-system-tests/hc-basic-suite-4.2/test-scenarios/002_bootstrap.py", line 229, in add_hosts add_hosts_4(prefix) File "/home/jenkins/workspace/ovirt-system-tests_hc-basic-suite-4.2/ovirt-system-tests/hc-basic-suite-4.2/test-scenarios/002_bootstrap.py", line 273, in add_hosts_4 testlib.assert_true_within(_host_is_up_4, timeout=15*60) File "/usr/lib/python2.7/site-packages/ovirtlago/testlib.py", line 278, in assert_true_within assert_equals_within(func, True, timeout, allowed_exceptions) File "/usr/lib/python2.7/site-packages/ovirtlago/testlib.py", line 234, in assert_equals_within res = func() File "/home/jenkins/workspace/ovirt-system-tests_hc-basic-suite-4.2/ovirt-system-tests/hc-basic-suite-4.2/test-scenarios/002_bootstrap.py", line 254, in _host_is_up_4 host_obj = host_service.get() File "/usr/lib64/python2.7/site-packages/ovirtsdk4/services.py", line 37093, in get return self._internal_get(headers, query, wait) File "/usr/lib64/python2.7/site-packages/ovirtsdk4/service.py", line 211, in _internal_get return future.wait() if wait else future File "/usr/lib64/python2.7/site-packages/ovirtsdk4/service.py", line 54, in wait response = self._connection.wait(self._context) File "/usr/lib64/python2.7/site-packages/ovirtsdk4/__init__.py", line 496, in wait return self.__wait(context, failed_auth) File "/usr/lib64/python2.7/site-packages/ovirtsdk4/__init__.py", line 510, in __wait raise Error("Failed to read response: {}".format(err_list)) 'Failed to read response: [(<pycurl.Curl object at 0x7fdf38bea050>, 7, \'TCP connection reset by peer\')]\n-------------------- >> begin captured logging << --------------------\novirtlago.testlib: ERROR: * Unhandled exception in <function _host_is_up_4 at 0x7fdf38b8d578>\nTraceback (most recent call last):\n File "/usr/lib/python2.7/site-packages/ovirtlago/testlib.py", line 234, in assert_equals_within\n res = func()\n File "/home/jenkins/workspace/ovirt-system-tests_hc-basic-suite-4.2/ovirt-system-tests/hc-basic-suite-4.2/test-scenarios/002_bootstrap.py", line 254, in _host_is_up_4\n host_obj = host_service.get()\n File "/usr/lib64/python2.7/site-packages/ovirtsdk4/services.py", line 37093, in get\n return self._internal_get(headers, query, wait)\n File "/usr/lib64/python2.7/site-packages/ovirtsdk4/service.py", line 211, in _internal_get\n return future.wait() if wait else future\n File "/usr/lib64/python2.7/site-packages/ovirtsdk4/service.py", line 54, in wait\n response = self._connection.wait(self._context)\n File "/usr/lib64/python2.7/site-packages/ovirtsdk4/__init__.py", line 496, in wait\n return self.__wait(context, failed_auth)\n File "/usr/lib64/python2.7/site-packages/ovirtsdk4/__init__.py", line 510, in __wait\n raise Error("Failed to read response: {}".format(err_list))\nError: Failed to read response: [(<pycurl.Curl object at 0x7fdf38bea050>, 7, \'TCP connection reset by peer\')]\n--------------------- >> end captured logging << ---------------------'

1 1

[CQ]: 94646,3 (ovirt-engine) failed "ovirt-master" system tests
by oVirt Jenkins 15 Oct '18

15 Oct '18

Change 94646,3 (ovirt-engine) is probably the reason behind recent system test failures in the "ovirt-master" change queue and needs to be fixed. This change had been removed from the testing queue. Artifacts build from this change will not be released until it is fixed. For further details about the change see: https://gerrit.ovirt.org/#/c/94646/3 For failed test results see: http://jenkins.ovirt.org/job/ovirt-master_change-queue-tester/10604/

2 1

[JIRA] (OVIRT-2538) ovirt-engine master failure on test 008_basic_ui_sanity.initialize_chrome
by Greg Sheremeta (oVirt JIRA) 15 Oct '18

15 Oct '18

[ https://ovirt-jira.atlassian.net/browse/OVIRT-2538?page=com.atlassian.jira.… ] Greg Sheremeta commented on OVIRT-2538: --------------------------------------- view-source:https://jenkins.ovirt.org/job/ovirt-master_change-queue-tester/… right click view source on that raise exception_class(message, screen, stacktrace) 'Message: timeout\n (Session info: chrome=64.0.3282.140)\n (Driver info: chromedriver=2.35.528139 (47ead77cb35ad2a9a83248b292151462a66cd881),platform=Linux 3.10.0-862.3.2.el7.x86_64 x86_64)\n\n-------------------- >> begin captured stdout << ---------------------\n _init_browser, connecting to hub at http://172.18.0.2:4444/wd/hub setting window size navigating to engine at https://192.168.201.4/ovirt-engine/webadmin --------------------- and then it times out. Seems like the browser isn't starting. The containers are up and did respond to wget's. Very odd. > ovirt-engine master failure on test 008_basic_ui_sanity.initialize_chrome > --------------------------------------------------------------------------- > > Key: OVIRT-2538 > URL: https://ovirt-jira.atlassian.net/browse/OVIRT-2538 > Project: oVirt - virtualization made easy > Issue Type: Bug > Reporter: Dafna Ron > Assignee: infra > Priority: High > Labels: ost_failures, ost_race > > I have seen random failures of test 008_basic_ui_sanity.initialize_chrome which are not related to the changes. > Here is the latest failure which I confirmed with didi is not a real failure and is not related to the change > https://gerrit.ovirt.org/#/c/89572/ > https://jenkins.ovirt.org/job/ovirt-master_change-queue-tester/10601/ > I cannot see anything in the http logs and in engine logs. > I can see a warn on ssl and some issues which seems to be resolved later on domain names: > '[Mon Oct 15 02:56:26.041204 2018] [ssl:warn] [pid 18661] AH01909: RSA certificate configured for 192.168.201.4:443 does NOT include an ID which matches the server name' > I think the issue is the test since the only errors I can see are in the lago logs. > I saved logs to allow debugging of the test failure. > [~gbenhaim(a)redhat.com] > [~gshereme(a)redhat.com] > Can you please check this issue? -- This message was sent by Atlassian Jira (v1001.0.0-SNAPSHOT#100094)

1 0

[JIRA] (OVIRT-2538) ovirt-engine master failure on test 008_basic_ui_sanity.initialize_chrome
by Dafna Ron (oVirt JIRA) 15 Oct '18

15 Oct '18

[ https://ovirt-jira.atlassian.net/browse/OVIRT-2538?page=com.atlassian.jira.… ] Dafna Ron commented on OVIRT-2538: ---------------------------------- Greg gave example and screen shots should be saved when basic suite is run under: artifact/basic-suite.el7.x86_64/screenshots/ it did not collect screen shots for this run: https://jenkins.ovirt.org/job/ovirt-master_change-queue-tester/10601/artifa… [~bkorren(a)redhat.com] [~gbenhaim(a)redhat.com] any idea why it would not collect the screenshots? any other way for us to debug this in future? > ovirt-engine master failure on test 008_basic_ui_sanity.initialize_chrome > --------------------------------------------------------------------------- > > Key: OVIRT-2538 > URL: https://ovirt-jira.atlassian.net/browse/OVIRT-2538 > Project: oVirt - virtualization made easy > Issue Type: Bug > Reporter: Dafna Ron > Assignee: infra > Priority: High > Labels: ost_failures, ost_race > > I have seen random failures of test 008_basic_ui_sanity.initialize_chrome which are not related to the changes. > Here is the latest failure which I confirmed with didi is not a real failure and is not related to the change > https://gerrit.ovirt.org/#/c/89572/ > https://jenkins.ovirt.org/job/ovirt-master_change-queue-tester/10601/ > I cannot see anything in the http logs and in engine logs. > I can see a warn on ssl and some issues which seems to be resolved later on domain names: > '[Mon Oct 15 02:56:26.041204 2018] [ssl:warn] [pid 18661] AH01909: RSA certificate configured for 192.168.201.4:443 does NOT include an ID which matches the server name' > I think the issue is the test since the only errors I can see are in the lago logs. > I saved logs to allow debugging of the test failure. > [~gbenhaim(a)redhat.com] > [~gshereme(a)redhat.com] > Can you please check this issue? -- This message was sent by Atlassian Jira (v1001.0.0-SNAPSHOT#100094)

1 0

[JIRA] (OVIRT-2538) ovirt-engine master failure on test 008_basic_ui_sanity.initialize_chrome
by Greg Sheremeta (oVirt JIRA) 15 Oct '18

15 Oct '18

[ https://ovirt-jira.atlassian.net/browse/OVIRT-2538?page=com.atlassian.jira.… ] Greg Sheremeta commented on OVIRT-2538: --------------------------------------- There should be a 'screenshots' directory in the OST run artifacts. I'm not sure how to get from a change queue link to the OST link. > ovirt-engine master failure on test 008_basic_ui_sanity.initialize_chrome > --------------------------------------------------------------------------- > > Key: OVIRT-2538 > URL: https://ovirt-jira.atlassian.net/browse/OVIRT-2538 > Project: oVirt - virtualization made easy > Issue Type: Bug > Reporter: Dafna Ron > Assignee: infra > Priority: High > Labels: ost_failures, ost_race > > I have seen random failures of test 008_basic_ui_sanity.initialize_chrome which are not related to the changes. > Here is the latest failure which I confirmed with didi is not a real failure and is not related to the change > https://gerrit.ovirt.org/#/c/89572/ > https://jenkins.ovirt.org/job/ovirt-master_change-queue-tester/10601/ > I cannot see anything in the http logs and in engine logs. > I can see a warn on ssl and some issues which seems to be resolved later on domain names: > '[Mon Oct 15 02:56:26.041204 2018] [ssl:warn] [pid 18661] AH01909: RSA certificate configured for 192.168.201.4:443 does NOT include an ID which matches the server name' > I think the issue is the test since the only errors I can see are in the lago logs. > I saved logs to allow debugging of the test failure. > [~gbenhaim(a)redhat.com] > [~gshereme(a)redhat.com] > Can you please check this issue? -- This message was sent by Atlassian Jira (v1001.0.0-SNAPSHOT#100094)

1 0

[JIRA] (OVIRT-2540) improving the upgrade suites
by Dafna Ron (oVirt JIRA) 15 Oct '18

15 Oct '18

Dafna Ron created OVIRT-2540: -------------------------------- Summary: improving the upgrade suites Key: OVIRT-2540 URL: https://ovirt-jira.atlassian.net/browse/OVIRT-2540 Project: oVirt - virtualization made easy Issue Type: Bug Reporter: Dafna Ron Assignee: infra Priority: High our upgrade suites are testing engine upgrade with no objects created pre upgrade and after upgrade are running a basic sanity test. 1. upgrade should not take more then basic sanity and as we are running both for the upgrade suite we need to change it 2. we are not testing vdsm/host upgrade and we are also not testing basic objects upgrade the upgrade suites should be modified. [~mwperina] said he would write a design and until we have modified the test-cases we should perhaps stop running upgrade on every run as I do not think its contributing to our testing. -- This message was sent by Atlassian Jira (v1001.0.0-SNAPSHOT#100094)

1 0

[JIRA] (OVIRT-2538) ovirt-engine master failure on test 008_basic_ui_sanity.initialize_chrome
by Dafna Ron (oVirt JIRA) 15 Oct '18

15 Oct '18

[ https://ovirt-jira.atlassian.net/browse/OVIRT-2538?page=com.atlassian.jira.… ] Dafna Ron commented on OVIRT-2538: ---------------------------------- Thanks [~gshereme(a)redhat.com] I don't know how often it happens but I remember the test enough to know i have seen it fail a few times for no actual regression :) I opened this Jira so we we can watch it more intently Do we have the screenshots collected in the artifacts (in CQ Jenkins)? where can we find them? > ovirt-engine master failure on test 008_basic_ui_sanity.initialize_chrome > --------------------------------------------------------------------------- > > Key: OVIRT-2538 > URL: https://ovirt-jira.atlassian.net/browse/OVIRT-2538 > Project: oVirt - virtualization made easy > Issue Type: Bug > Reporter: Dafna Ron > Assignee: infra > Priority: High > Labels: ost_failures, ost_race > > I have seen random failures of test 008_basic_ui_sanity.initialize_chrome which are not related to the changes. > Here is the latest failure which I confirmed with didi is not a real failure and is not related to the change > https://gerrit.ovirt.org/#/c/89572/ > https://jenkins.ovirt.org/job/ovirt-master_change-queue-tester/10601/ > I cannot see anything in the http logs and in engine logs. > I can see a warn on ssl and some issues which seems to be resolved later on domain names: > '[Mon Oct 15 02:56:26.041204 2018] [ssl:warn] [pid 18661] AH01909: RSA certificate configured for 192.168.201.4:443 does NOT include an ID which matches the server name' > I think the issue is the test since the only errors I can see are in the lago logs. > I saved logs to allow debugging of the test failure. > [~gbenhaim(a)redhat.com] > [~gshereme(a)redhat.com] > Can you please check this issue? -- This message was sent by Atlassian Jira (v1001.0.0-SNAPSHOT#100094)

1 0

[JIRA] (OVIRT-2538) ovirt-engine master failure on test 008_basic_ui_sanity.initialize_chrome
by Greg Sheremeta (oVirt JIRA) 15 Oct '18

15 Oct '18

[ https://ovirt-jira.atlassian.net/browse/OVIRT-2538?page=com.atlassian.jira.… ] Greg Sheremeta commented on OVIRT-2538: --------------------------------------- How often has this been happening? The most useful thing to look at when this happens are the screenshots. Last time Didi showed this to me, the app didn't load, but it wasn't clear why. And you can only tell that by looking at the screen. > ovirt-engine master failure on test 008_basic_ui_sanity.initialize_chrome > --------------------------------------------------------------------------- > > Key: OVIRT-2538 > URL: https://ovirt-jira.atlassian.net/browse/OVIRT-2538 > Project: oVirt - virtualization made easy > Issue Type: Bug > Reporter: Dafna Ron > Assignee: infra > Priority: High > Labels: ost_failures, ost_race > > I have seen random failures of test 008_basic_ui_sanity.initialize_chrome which are not related to the changes. > Here is the latest failure which I confirmed with didi is not a real failure and is not related to the change > https://gerrit.ovirt.org/#/c/89572/ > https://jenkins.ovirt.org/job/ovirt-master_change-queue-tester/10601/ > I cannot see anything in the http logs and in engine logs. > I can see a warn on ssl and some issues which seems to be resolved later on domain names: > '[Mon Oct 15 02:56:26.041204 2018] [ssl:warn] [pid 18661] AH01909: RSA certificate configured for 192.168.201.4:443 does NOT include an ID which matches the server name' > I think the issue is the test since the only errors I can see are in the lago logs. > I saved logs to allow debugging of the test failure. > [~gbenhaim(a)redhat.com] > [~gshereme(a)redhat.com] > Can you please check this issue? -- This message was sent by Atlassian Jira (v1001.0.0-SNAPSHOT#100094)

1 0