March 2020 - Infra - oVirt List Archives

[CQ]: 107382, 2 (ovirt-engine-nodejs-modules) failed "ovirt-4.3" system tests
by oVirt Jenkins 04 Mar '20

04 Mar '20

Change 107382,2 (ovirt-engine-nodejs-modules) is probably the reason behind recent system test failures in the "ovirt-4.3" change queue and needs to be fixed. This change had been removed from the testing queue. Artifacts build from this change will not be released until it is fixed. For further details about the change see: https://gerrit.ovirt.org/#/c/107382/2 For failed test results see: https://jenkins.ovirt.org/job/ovirt-4.3_change-queue-tester/3050/

1 0

[CQ]: 107382, 2 (ovirt-engine-nodejs-modules) failed "ovirt-master" system tests
by oVirt Jenkins 04 Mar '20

04 Mar '20

Change 107382,2 (ovirt-engine-nodejs-modules) is probably the reason behind recent system test failures in the "ovirt-master" change queue and needs to be fixed. This change had been removed from the testing queue. Artifacts build from this change will not be released until it is fixed. For further details about the change see: https://gerrit.ovirt.org/#/c/107382/2 For failed test results see: https://jenkins.ovirt.org/job/ovirt-master_change-queue-tester/21053/

1 0

[CQ]: 107382, 2 (ovirt-engine-nodejs-modules) failed "ovirt-4.3" system tests
by oVirt Jenkins 04 Mar '20

04 Mar '20

Change 107382,2 (ovirt-engine-nodejs-modules) is probably the reason behind recent system test failures in the "ovirt-4.3" change queue and needs to be fixed. This change had been removed from the testing queue. Artifacts build from this change will not be released until it is fixed. For further details about the change see: https://gerrit.ovirt.org/#/c/107382/2 For failed test results see: https://jenkins.ovirt.org/job/ovirt-4.3_change-queue-tester/3048/

1 0

[CQ]: 107382, 2 (ovirt-engine-nodejs-modules) failed "ovirt-master" system tests
by oVirt Jenkins 04 Mar '20

04 Mar '20

Change 107382,2 (ovirt-engine-nodejs-modules) is probably the reason behind recent system test failures in the "ovirt-master" change queue and needs to be fixed. This change had been removed from the testing queue. Artifacts build from this change will not be released until it is fixed. For further details about the change see: https://gerrit.ovirt.org/#/c/107382/2 For failed test results see: https://jenkins.ovirt.org/job/ovirt-master_change-queue-tester/21051/

1 0

[CQ]: 107376,5 (ovirt-engine) failed "ovirt-master" system tests
by oVirt Jenkins 04 Mar '20

04 Mar '20

Change 107376,5 (ovirt-engine) is probably the reason behind recent system test failures in the "ovirt-master" change queue and needs to be fixed. This change had been removed from the testing queue. Artifacts build from this change will not be released until it is fixed. For further details about the change see: https://gerrit.ovirt.org/#/c/107376/5 For failed test results see: https://jenkins.ovirt.org/job/ovirt-master_change-queue-tester/21049/

1 0

Sanlock lockspace inquire failure (was: [oVirt Jenkins] ovirt-system-tests_he-basic-suite-4.3 - Build # 366 - Failure!)
by Yedidyah Bar David 04 Mar '20

04 Mar '20

Hi all, On Sun, Mar 1, 2020 at 6:06 AM <jenkins(a)jenkins.phx.ovirt.org> wrote: > > Project: https://jenkins.ovirt.org/job/ovirt-system-tests_he-basic-suite-4.3/ > Build: https://jenkins.ovirt.org/job/ovirt-system-tests_he-basic-suite-4.3/366/ I think the root cause is: https://jenkins.ovirt.org/job/ovirt-system-tests_he-basic-suite-4.3/366/art… StatusStorageThread::ERROR::2020-02-29 23:03:04,671::status_broker::90::ovirt_hosted_engine_ha.broker.status_broker.StatusBroker.Update::(run) Failed to update state. Traceback (most recent call last): File "/usr/lib/python2.7/site-packages/ovirt_hosted_engine_ha/broker/status_broker.py", line 82, in run if (self._status_broker._inquire_whiteboard_lock() or File "/usr/lib/python2.7/site-packages/ovirt_hosted_engine_ha/broker/status_broker.py", line 195, in _inquire_whiteboard_lock self.host_id, self._lease_file) SanlockException: (104, 'Sanlock lockspace inquire failure', 'Connection reset by peer') This caused the broker to restart itself, and while it was doing that, OST did 'hosted-engine --vm-status --json', which failed, thus failing the build. This seems to me like another case of a communication problem in CI. Not sure what else could have caused it to fail to inquire the status of the lock. This (communication) issue was mentioned several times in the past already. Are we doing anything re this? Thanks and best regards, > Build Number: 366 > Build Status: Failure > Triggered By: Started by timer > > ------------------------------------- > Changes Since Last Success: > ------------------------------------- > Changes for Build #366 > [Marcin Sobczyk] el8: Don't try to collect whole '/etc/httpd' dir > > > > > ----------------- > Failed Tests: > ----------------- > 1 tests failed. > FAILED: 008_restart_he_vm.clear_global_maintenance > > Error Message: > 1 != 0 > -------------------- >> begin captured logging << -------------------- > root: INFO: Waiting For System Stability... > lago.ssh: DEBUG: start task:29a79ef5-e211-4672-ac5b-12bf0e5f8ee9:Get ssh client for lago-he-basic-suite-4-3-host-0: > lago.ssh: DEBUG: end task:29a79ef5-e211-4672-ac5b-12bf0e5f8ee9:Get ssh client for lago-he-basic-suite-4-3-host-0: > lago.ssh: DEBUG: Running 9a90ca60 on lago-he-basic-suite-4-3-host-0: hosted-engine --set-maintenance --mode=none > lago.ssh: DEBUG: Command 9a90ca60 on lago-he-basic-suite-4-3-host-0 returned with 1 > lago.ssh: DEBUG: Command 9a90ca60 on lago-he-basic-suite-4-3-host-0 errors: > Cannot connect to the HA daemon, please check the logs. > > ovirtlago.testlib: ERROR: * Unhandled exception in <function <lambda> at 0x7f52673872a8> > Traceback (most recent call last): > File "/usr/lib/python2.7/site-packages/ovirtlago/testlib.py", line 234, in assert_equals_within > res = func() > File "/home/jenkins/agent/workspace/ovirt-system-tests_he-basic-suite-4.3/ovirt-system-tests/he-basic-suite-4.3/test-scenarios/008_restart_he_vm.py", line 87, in <lambda> > lambda: _set_and_test_maintenance_mode(host, False) > File "/home/jenkins/agent/workspace/ovirt-system-tests_he-basic-suite-4.3/ovirt-system-tests/he-basic-suite-4.3/test-scenarios/008_restart_he_vm.py", line 108, in _set_and_test_maintenance_mode > nt.assert_equals(ret.code, 0) > File "/usr/lib64/python2.7/unittest/case.py", line 553, in assertEqual > assertion_func(first, second, msg=msg) > File "/usr/lib64/python2.7/unittest/case.py", line 546, in _baseAssertEqual > raise self.failureException(msg) > AssertionError: 1 != 0 > --------------------- >> end captured logging << --------------------- > > Stack Trace: > File "/usr/lib64/python2.7/unittest/case.py", line 369, in run > testMethod() > File "/usr/lib/python2.7/site-packages/nose/case.py", line 197, in runTest > self.test(*self.arg) > File "/usr/lib/python2.7/site-packages/ovirtlago/testlib.py", line 142, in wrapped_test > test() > File "/usr/lib/python2.7/site-packages/ovirtlago/testlib.py", line 60, in wrapper > return func(get_test_prefix(), *args, **kwargs) > File "/home/jenkins/agent/workspace/ovirt-system-tests_he-basic-suite-4.3/ovirt-system-tests/he-basic-suite-4.3/test-scenarios/008_restart_he_vm.py", line 87, in clear_global_maintenance > lambda: _set_and_test_maintenance_mode(host, False) > File "/usr/lib/python2.7/site-packages/ovirtlago/testlib.py", line 282, in assert_true_within_short > assert_equals_within_short(func, True, allowed_exceptions) > File "/usr/lib/python2.7/site-packages/ovirtlago/testlib.py", line 266, in assert_equals_within_short > func, value, SHORT_TIMEOUT, allowed_exceptions=allowed_exceptions > File "/usr/lib/python2.7/site-packages/ovirtlago/testlib.py", line 234, in assert_equals_within > res = func() > File "/home/jenkins/agent/workspace/ovirt-system-tests_he-basic-suite-4.3/ovirt-system-tests/he-basic-suite-4.3/test-scenarios/008_restart_he_vm.py", line 87, in <lambda> > lambda: _set_and_test_maintenance_mode(host, False) > File "/home/jenkins/agent/workspace/ovirt-system-tests_he-basic-suite-4.3/ovirt-system-tests/he-basic-suite-4.3/test-scenarios/008_restart_he_vm.py", line 108, in _set_and_test_maintenance_mode > nt.assert_equals(ret.code, 0) > File "/usr/lib64/python2.7/unittest/case.py", line 553, in assertEqual > assertion_func(first, second, msg=msg) > File "/usr/lib64/python2.7/unittest/case.py", line 546, in _baseAssertEqual > raise self.failureException(msg) > '1 != 0\n-------------------- >> begin captured logging << --------------------\nroot: INFO: Waiting For System Stability...\nlago.ssh: DEBUG: start task:29a79ef5-e211-4672-ac5b-12bf0e5f8ee9:Get ssh client for lago-he-basic-suite-4-3-host-0:\nlago.ssh: DEBUG: end task:29a79ef5-e211-4672-ac5b-12bf0e5f8ee9:Get ssh client for lago-he-basic-suite-4-3-host-0:\nlago.ssh: DEBUG: Running 9a90ca60 on lago-he-basic-suite-4-3-host-0: hosted-engine --set-maintenance --mode=none\nlago.ssh: DEBUG: Command 9a90ca60 on lago-he-basic-suite-4-3-host-0 returned with 1\nlago.ssh: DEBUG: Command 9a90ca60 on lago-he-basic-suite-4-3-host-0 errors:\n Cannot connect to the HA daemon, please check the logs.\n\novirtlago.testlib: ERROR: * Unhandled exception in <function <lambda> at 0x7f52673872a8>\nTraceback (most recent call last):\n File "/usr/lib/python2.7/site-packages/ovirtlago/testlib.py", line 234, in assert_equals_within\n res = func()\n File "/home/jenkins/agent/workspace/ovirt-system-tests_he-basic-suite-4.3/ovirt-system-tests/he-basic-suite-4.3/test-scenarios/008_restart_he_vm.py", line 87, in <lambda>\n lambda: _set_and_test_maintenance_mode(host, False)\n File "/home/jenkins/agent/workspace/ovirt-system-tests_he-basic-suite-4.3/ovirt-system-tests/he-basic-suite-4.3/test-scenarios/008_restart_he_vm.py", line 108, in _set_and_test_maintenance_mode\n nt.assert_equals(ret.code, 0)\n File "/usr/lib64/python2.7/unittest/case.py", line 553, in assertEqual\n assertion_func(first, second, msg=msg)\n File "/usr/lib64/python2.7/unittest/case.py", line 546, in _baseAssertEqual\n raise self.failureException(msg)\nAssertionError: 1 != 0\n--------------------- >> end captured logging << ---------------------' -- Didi

2 3

oVirt infra daily report - unstable production jobs - 1120
by jenkins＠jenkins.phx.ovirt.org 04 Mar '20

04 Mar '20

Good morning! Attached is the HTML page with the jenkins status report. You can see it also here: - https://jenkins.ovirt.org/job/system_jenkins-report/1120//artifact/exported… Cheers, Jenkins

1 0

[CQ]: 50c2cb9 (ovirt-ansible-cluster-upgrade) failed "ovirt-master" system tests, but isn't the failure root cause
by oVirt Jenkins 04 Mar '20

04 Mar '20

A system test invoked by the "ovirt-master" change queue including change 50c2cb9 (ovirt-ansible-cluster-upgrade) failed. However, this change seems not to be the root cause for this failure. Change 78057dd (ovirt-ansible-cluster- upgrade) that this change depends on or is based on, was detected as the cause of the testing failures. This change had been removed from the testing queue. Artifacts built from this change will not be released until either change 78057dd (ovirt-ansible-cluster- upgrade) is fixed and this change is updated to refer to or rebased on the fixed version, or this change is modified to no longer depend on it. For further details about the change see: https://github.com/oVirt/ovirt-ansible-cluster-upgrade/commit/50c2cb94eeccb… For further details about the change that seems to be the root cause behind the testing failures see: https://github.com/oVirt/ovirt-ansible-cluster-upgrade/commit/78057dd4efc79… For failed test results see: https://jenkins.ovirt.org/job/ovirt-master_change-queue-tester/21046/

1 0

[CQ]: 101744,79 (vdsm) failed "ovirt-master" system tests
by oVirt Jenkins 04 Mar '20

04 Mar '20

Change 101744,79 (vdsm) is probably the reason behind recent system test failures in the "ovirt-master" change queue and needs to be fixed. This change had been removed from the testing queue. Artifacts build from this change will not be released until it is fixed. For further details about the change see: https://gerrit.ovirt.org/#/c/101744/79 For failed test results see: https://jenkins.ovirt.org/job/ovirt-master_change-queue-tester/21044/

1 0

[CQ]: 107146, 3 (ovirt-imageio) failed "ovirt-master" system tests, but isn't the failure root cause
by oVirt Jenkins 04 Mar '20

04 Mar '20

A system test invoked by the "ovirt-master" change queue including change 107146,3 (ovirt-imageio) failed. However, this change seems not to be the root cause for this failure. Change 106905,1 (ovirt-imageio) that this change depends on or is based on, was detected as the cause of the testing failures. This change had been removed from the testing queue. Artifacts built from this change will not be released until either change 106905,1 (ovirt-imageio) is fixed and this change is updated to refer to or rebased on the fixed version, or this change is modified to no longer depend on it. For further details about the change see: https://gerrit.ovirt.org/#/c/107146/3 For further details about the change that seems to be the root cause behind the testing failures see: https://gerrit.ovirt.org/#/c/106905/1 For failed test results see: https://jenkins.ovirt.org/job/ovirt-master_change-queue-tester/21036/

1 0