[oVirt Jenkins] ovirt-system-tests_compat-4.3-suite-master - Build # 394 - Failure!
Project: https://jenkins.ovirt.org/job/ovirt-system-tests_compat-4.3-suite-master/ Build: https://jenkins.ovirt.org/job/ovirt-system-tests_compat-4.3-suite-master/394... Build Number: 394 Build Status: Failure Triggered By: Started by timer ------------------------------------- Changes Since Last Success: ------------------------------------- Changes for Build #394 [Sandro Bonazzola] repos: fix Advanced Virtualization repos [Sandro Bonazzola] mirrors: drop gluster 5 mirroring ----------------- Failed Tests: ----------------- No tests ran.
Project: https://jenkins.ovirt.org/job/ovirt-system-tests_compat-4.3-suite-master/ Build: https://jenkins.ovirt.org/job/ovirt-system-tests_compat-4.3-suite-master/395... Build Number: 395 Build Status: Still Failing Triggered By: Started by timer ------------------------------------- Changes Since Last Success: ------------------------------------- Changes for Build #394 [Sandro Bonazzola] repos: fix Advanced Virtualization repos [Sandro Bonazzola] mirrors: drop gluster 5 mirroring Changes for Build #395 [Galit Rosenthal] Fix the repo for suites that weren't moved to no reposync ----------------- Failed Tests: ----------------- 1 tests failed. FAILED: basic-suite-master.test-scenarios.002_bootstrap_pytest.test_verify_add_hosts Error Message: RuntimeError: 1 hosts failed installation: lago-compat-4-3-suite-master-host-0: non_responsive Stack Trace: api_v4 = <ovirtsdk4.Connection object at 0x7fc8bb0e9bd0> @pytest.mark.run(order=30) def test_verify_add_hosts(api_v4): hosts_service = api_v4.system_service().hosts_service() hosts_status = hosts_service.list(search='datacenter={}'.format(DC_NAME)) total_hosts = len(hosts_status) dump_hosts = _host_status_to_print(hosts_service, hosts_status) LOGGER.debug('Host status, verify_add_hosts:\n {}'.format(dump_hosts)) testlib.assert_true_within( lambda: _single_host_up(hosts_service, total_hosts),
timeout=constants.ADD_HOST_TIMEOUT
) ../basic-suite-master/test-scenarios/002_bootstrap_pytest.py:361: _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ /usr/lib/python2.7/site-packages/ovirtlago/testlib.py:278: in assert_true_within assert_equals_within(func, True, timeout, allowed_exceptions) /usr/lib/python2.7/site-packages/ovirtlago/testlib.py:234: in assert_equals_within res = func() ../basic-suite-master/test-scenarios/002_bootstrap_pytest.py:360: in <lambda> lambda: _single_host_up(hosts_service, total_hosts), ../basic-suite-master/test-scenarios/002_bootstrap_pytest.py:154: in _single_host_up _check_problematic_hosts(hosts_service) _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ hosts_service = <ovirtsdk4.services.HostsService object at 0x7fc8b82359d0> def _check_problematic_hosts(hosts_service): problematic_hosts = hosts_service.list(search='datacenter={} AND status != installing and status != initializing and status != up'.format(DC_NAME)) if len(problematic_hosts): dump_hosts = '%s hosts failed installation:\n' % len(problematic_hosts) for host in problematic_hosts: host_service = hosts_service.host_service(host.id) dump_hosts += '%s: %s\n' % (host.name, host_service.get().status)
raise RuntimeError(dump_hosts)
E RuntimeError: 1 hosts failed installation: E lago-compat-4-3-suite-master-host-0: non_responsive ../basic-suite-master/test-scenarios/002_bootstrap_pytest.py:163: RuntimeError
Project: https://jenkins.ovirt.org/job/ovirt-system-tests_compat-4.3-suite-master/ Build: https://jenkins.ovirt.org/job/ovirt-system-tests_compat-4.3-suite-master/396... Build Number: 396 Build Status: Still Failing Triggered By: Started by timer ------------------------------------- Changes Since Last Success: ------------------------------------- Changes for Build #394 [Sandro Bonazzola] repos: fix Advanced Virtualization repos [Sandro Bonazzola] mirrors: drop gluster 5 mirroring Changes for Build #395 [Galit Rosenthal] Fix the repo for suites that weren't moved to no reposync Changes for Build #396 [Galit Rosenthal] Fix the repo for suites that weren't moved to no reposync [Shlomi] Added mirror cleanup functionality [Evgheni Dereveanchin] Modules support for mirror_mgr.sh ----------------- Failed Tests: ----------------- 1 tests failed. FAILED: basic-suite-master.test-scenarios.002_bootstrap_pytest.test_verify_add_hosts Error Message: RuntimeError: 1 hosts failed installation: lago-compat-4-3-suite-master-host-0: non_responsive Stack Trace: api_v4 = <ovirtsdk4.Connection object at 0x7f3d73466bd0> @pytest.mark.run(order=30) def test_verify_add_hosts(api_v4): hosts_service = api_v4.system_service().hosts_service() hosts_status = hosts_service.list(search='datacenter={}'.format(DC_NAME)) total_hosts = len(hosts_status) dump_hosts = _host_status_to_print(hosts_service, hosts_status) LOGGER.debug('Host status, verify_add_hosts:\n {}'.format(dump_hosts)) testlib.assert_true_within( lambda: _single_host_up(hosts_service, total_hosts),
timeout=constants.ADD_HOST_TIMEOUT
) ../basic-suite-master/test-scenarios/002_bootstrap_pytest.py:361: _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ /usr/lib/python2.7/site-packages/ovirtlago/testlib.py:278: in assert_true_within assert_equals_within(func, True, timeout, allowed_exceptions) /usr/lib/python2.7/site-packages/ovirtlago/testlib.py:234: in assert_equals_within res = func() ../basic-suite-master/test-scenarios/002_bootstrap_pytest.py:360: in <lambda> lambda: _single_host_up(hosts_service, total_hosts), ../basic-suite-master/test-scenarios/002_bootstrap_pytest.py:154: in _single_host_up _check_problematic_hosts(hosts_service) _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ hosts_service = <ovirtsdk4.services.HostsService object at 0x7f3d705c1bd0> def _check_problematic_hosts(hosts_service): problematic_hosts = hosts_service.list(search='datacenter={} AND status != installing and status != initializing and status != up'.format(DC_NAME)) if len(problematic_hosts): dump_hosts = '%s hosts failed installation:\n' % len(problematic_hosts) for host in problematic_hosts: host_service = hosts_service.host_service(host.id) dump_hosts += '%s: %s\n' % (host.name, host_service.get().status)
raise RuntimeError(dump_hosts)
E RuntimeError: 1 hosts failed installation: E lago-compat-4-3-suite-master-host-0: non_responsive ../basic-suite-master/test-scenarios/002_bootstrap_pytest.py:163: RuntimeError
Project: https://jenkins.ovirt.org/job/ovirt-system-tests_compat-4.3-suite-master/ Build: https://jenkins.ovirt.org/job/ovirt-system-tests_compat-4.3-suite-master/397... Build Number: 397 Build Status: Still Failing Triggered By: Started by timer ------------------------------------- Changes Since Last Success: ------------------------------------- Changes for Build #394 [Sandro Bonazzola] repos: fix Advanced Virtualization repos [Sandro Bonazzola] mirrors: drop gluster 5 mirroring Changes for Build #395 [Galit Rosenthal] Fix the repo for suites that weren't moved to no reposync Changes for Build #396 [Galit Rosenthal] Fix the repo for suites that weren't moved to no reposync [Shlomi] Added mirror cleanup functionality [Evgheni Dereveanchin] Modules support for mirror_mgr.sh Changes for Build #397 [Galit Rosenthal] Fix alosync not recognize returning old 002_bootstrap [Evgheni Dereveanchin] Define mirrors for CentOS 8 [Evgheni Dereveanchin] fix gpg keys for COPR repos [Shlomi] Fixed an issue with cleanup of newly created mirrors. [Ehud Yonasi] ost: add custom repos to test nightly suites. ----------------- Failed Tests: ----------------- 1 tests failed. FAILED: 002_bootstrap.verify_add_hosts_43 Error Message: 1 hosts failed installation: lago-compat-4-3-suite-master-host-0: non_responsive -------------------- >> begin captured logging << -------------------- 002_bootstrap: DEBUG: Host status, verify_add_hosts: lago-compat-4-3-suite-master-host-0: installing lago-compat-4-3-suite-master-host-1: installing ovirtlago.testlib: ERROR: * Unhandled exception in <function <lambda> at 0x7fae087f85f0> Traceback (most recent call last): File "/usr/lib/python2.7/site-packages/ovirtlago/testlib.py", line 234, in assert_equals_within res = func() File "/home/jenkins/agent/workspace/ovirt-system-tests_compat-4.3-suite-master/ovirt-system-tests/compat-4.3-suite-master/test-scenarios/002_bootstrap.py", line 368, in <lambda> lambda: _single_host_up(hosts_service, total_hosts), File "/home/jenkins/agent/workspace/ovirt-system-tests_compat-4.3-suite-master/ovirt-system-tests/compat-4.3-suite-master/test-scenarios/002_bootstrap.py", line 154, in _single_host_up _check_problematic_hosts(hosts_service) File "/home/jenkins/agent/workspace/ovirt-system-tests_compat-4.3-suite-master/ovirt-system-tests/compat-4.3-suite-master/test-scenarios/002_bootstrap.py", line 163, in _check_problematic_hosts raise RuntimeError(dump_hosts) RuntimeError: 1 hosts failed installation: lago-compat-4-3-suite-master-host-0: non_responsive --------------------- >> end captured logging << --------------------- Stack Trace: File "/usr/lib64/python2.7/unittest/case.py", line 369, in run testMethod() File "/usr/lib/python2.7/site-packages/nose/case.py", line 197, in runTest self.test(*self.arg) File "/usr/lib/python2.7/site-packages/ovirtlago/testlib.py", line 142, in wrapped_test test() File "/usr/lib/python2.7/site-packages/ovirtlago/testlib.py", line 60, in wrapper return func(get_test_prefix(), *args, **kwargs) File "/usr/lib/python2.7/site-packages/ovirtlago/testlib.py", line 79, in wrapper prefix.virt_env.engine_vm().get_api(api_ver=4), *args, **kwargs File "/home/jenkins/agent/workspace/ovirt-system-tests_compat-4.3-suite-master/ovirt-system-tests/compat-4.3-suite-master/test-scenarios/002_bootstrap.py", line 369, in verify_add_hosts timeout=constants.ADD_HOST_TIMEOUT File "/usr/lib/python2.7/site-packages/ovirtlago/testlib.py", line 278, in assert_true_within assert_equals_within(func, True, timeout, allowed_exceptions) File "/usr/lib/python2.7/site-packages/ovirtlago/testlib.py", line 234, in assert_equals_within res = func() File "/home/jenkins/agent/workspace/ovirt-system-tests_compat-4.3-suite-master/ovirt-system-tests/compat-4.3-suite-master/test-scenarios/002_bootstrap.py", line 368, in <lambda> lambda: _single_host_up(hosts_service, total_hosts), File "/home/jenkins/agent/workspace/ovirt-system-tests_compat-4.3-suite-master/ovirt-system-tests/compat-4.3-suite-master/test-scenarios/002_bootstrap.py", line 154, in _single_host_up _check_problematic_hosts(hosts_service) File "/home/jenkins/agent/workspace/ovirt-system-tests_compat-4.3-suite-master/ovirt-system-tests/compat-4.3-suite-master/test-scenarios/002_bootstrap.py", line 163, in _check_problematic_hosts raise RuntimeError(dump_hosts) '1 hosts failed installation:\nlago-compat-4-3-suite-master-host-0: non_responsive\n\n-------------------- >> begin captured logging << --------------------\n002_bootstrap: DEBUG: Host status, verify_add_hosts:\n lago-compat-4-3-suite-master-host-0: installing\nlago-compat-4-3-suite-master-host-1: installing\n\novirtlago.testlib: ERROR: * Unhandled exception in <function <lambda> at 0x7fae087f85f0>\nTraceback (most recent call last):\n File "/usr/lib/python2.7/site-packages/ovirtlago/testlib.py", line 234, in assert_equals_within\n res = func()\n File "/home/jenkins/agent/workspace/ovirt-system-tests_compat-4.3-suite-master/ovirt-system-tests/compat-4.3-suite-master/test-scenarios/002_bootstrap.py", line 368, in <lambda>\n lambda: _single_host_up(hosts_service, total_hosts),\n File "/home/jenkins/agent/workspace/ovirt-system-tests_compat-4.3-suite-master/ovirt-system-tests/compat-4.3-suite-master/test-scenarios/002_bootstrap.py", line 154, in _single_host_up\n _check_problematic_hosts(hosts_service)\n File "/home/jenkins/agent/workspace/ovirt-system-tests_compat-4.3-suite-master/ovirt-system-tests/compat-4.3-suite-master/test-scenarios/002_bootstrap.py", line 163, in _check_problematic_hosts\n raise RuntimeError(dump_hosts)\nRuntimeError: 1 hosts failed installation:\nlago-compat-4-3-suite-master-host-0: non_responsive\n\n--------------------- >> end captured logging << ---------------------'
Project: https://jenkins.ovirt.org/job/ovirt-system-tests_compat-4.3-suite-master/ Build: https://jenkins.ovirt.org/job/ovirt-system-tests_compat-4.3-suite-master/398... Build Number: 398 Build Status: Still Failing Triggered By: Started by timer ------------------------------------- Changes Since Last Success: ------------------------------------- Changes for Build #394 [Sandro Bonazzola] repos: fix Advanced Virtualization repos [Sandro Bonazzola] mirrors: drop gluster 5 mirroring Changes for Build #395 [Galit Rosenthal] Fix the repo for suites that weren't moved to no reposync Changes for Build #396 [Galit Rosenthal] Fix the repo for suites that weren't moved to no reposync [Shlomi] Added mirror cleanup functionality [Evgheni Dereveanchin] Modules support for mirror_mgr.sh Changes for Build #397 [Galit Rosenthal] Fix alosync not recognize returning old 002_bootstrap [Evgheni Dereveanchin] Define mirrors for CentOS 8 [Evgheni Dereveanchin] fix gpg keys for COPR repos [Shlomi] Fixed an issue with cleanup of newly created mirrors. [Ehud Yonasi] ost: add custom repos to test nightly suites. Changes for Build #398 [Galit Rosenthal] Fix alosync not recognize returning old 002_bootstrap ----------------- Failed Tests: ----------------- 1 tests failed. FAILED: 002_bootstrap.verify_add_hosts_43 Error Message: 1 hosts failed installation: lago-compat-4-3-suite-master-host-0: non_responsive -------------------- >> begin captured logging << -------------------- 002_bootstrap: DEBUG: Host status, verify_add_hosts: lago-compat-4-3-suite-master-host-0: installing lago-compat-4-3-suite-master-host-1: installing ovirtlago.testlib: ERROR: * Unhandled exception in <function <lambda> at 0x7fa11d71f050> Traceback (most recent call last): File "/usr/lib/python2.7/site-packages/ovirtlago/testlib.py", line 234, in assert_equals_within res = func() File "/home/jenkins/agent/workspace/ovirt-system-tests_compat-4.3-suite-master/ovirt-system-tests/compat-4.3-suite-master/test-scenarios/002_bootstrap.py", line 368, in <lambda> lambda: _single_host_up(hosts_service, total_hosts), File "/home/jenkins/agent/workspace/ovirt-system-tests_compat-4.3-suite-master/ovirt-system-tests/compat-4.3-suite-master/test-scenarios/002_bootstrap.py", line 154, in _single_host_up _check_problematic_hosts(hosts_service) File "/home/jenkins/agent/workspace/ovirt-system-tests_compat-4.3-suite-master/ovirt-system-tests/compat-4.3-suite-master/test-scenarios/002_bootstrap.py", line 163, in _check_problematic_hosts raise RuntimeError(dump_hosts) RuntimeError: 1 hosts failed installation: lago-compat-4-3-suite-master-host-0: non_responsive --------------------- >> end captured logging << --------------------- Stack Trace: File "/usr/lib64/python2.7/unittest/case.py", line 369, in run testMethod() File "/usr/lib/python2.7/site-packages/nose/case.py", line 197, in runTest self.test(*self.arg) File "/usr/lib/python2.7/site-packages/ovirtlago/testlib.py", line 142, in wrapped_test test() File "/usr/lib/python2.7/site-packages/ovirtlago/testlib.py", line 60, in wrapper return func(get_test_prefix(), *args, **kwargs) File "/usr/lib/python2.7/site-packages/ovirtlago/testlib.py", line 79, in wrapper prefix.virt_env.engine_vm().get_api(api_ver=4), *args, **kwargs File "/home/jenkins/agent/workspace/ovirt-system-tests_compat-4.3-suite-master/ovirt-system-tests/compat-4.3-suite-master/test-scenarios/002_bootstrap.py", line 369, in verify_add_hosts timeout=constants.ADD_HOST_TIMEOUT File "/usr/lib/python2.7/site-packages/ovirtlago/testlib.py", line 278, in assert_true_within assert_equals_within(func, True, timeout, allowed_exceptions) File "/usr/lib/python2.7/site-packages/ovirtlago/testlib.py", line 234, in assert_equals_within res = func() File "/home/jenkins/agent/workspace/ovirt-system-tests_compat-4.3-suite-master/ovirt-system-tests/compat-4.3-suite-master/test-scenarios/002_bootstrap.py", line 368, in <lambda> lambda: _single_host_up(hosts_service, total_hosts), File "/home/jenkins/agent/workspace/ovirt-system-tests_compat-4.3-suite-master/ovirt-system-tests/compat-4.3-suite-master/test-scenarios/002_bootstrap.py", line 154, in _single_host_up _check_problematic_hosts(hosts_service) File "/home/jenkins/agent/workspace/ovirt-system-tests_compat-4.3-suite-master/ovirt-system-tests/compat-4.3-suite-master/test-scenarios/002_bootstrap.py", line 163, in _check_problematic_hosts raise RuntimeError(dump_hosts) '1 hosts failed installation:\nlago-compat-4-3-suite-master-host-0: non_responsive\n\n-------------------- >> begin captured logging << --------------------\n002_bootstrap: DEBUG: Host status, verify_add_hosts:\n lago-compat-4-3-suite-master-host-0: installing\nlago-compat-4-3-suite-master-host-1: installing\n\novirtlago.testlib: ERROR: * Unhandled exception in <function <lambda> at 0x7fa11d71f050>\nTraceback (most recent call last):\n File "/usr/lib/python2.7/site-packages/ovirtlago/testlib.py", line 234, in assert_equals_within\n res = func()\n File "/home/jenkins/agent/workspace/ovirt-system-tests_compat-4.3-suite-master/ovirt-system-tests/compat-4.3-suite-master/test-scenarios/002_bootstrap.py", line 368, in <lambda>\n lambda: _single_host_up(hosts_service, total_hosts),\n File "/home/jenkins/agent/workspace/ovirt-system-tests_compat-4.3-suite-master/ovirt-system-tests/compat-4.3-suite-master/test-scenarios/002_bootstrap.py", line 154, in _single_host_up\n _check_problematic_hosts(hosts_service)\n File "/home/jenkins/agent/workspace/ovirt-system-tests_compat-4.3-suite-master/ovirt-system-tests/compat-4.3-suite-master/test-scenarios/002_bootstrap.py", line 163, in _check_problematic_hosts\n raise RuntimeError(dump_hosts)\nRuntimeError: 1 hosts failed installation:\nlago-compat-4-3-suite-master-host-0: non_responsive\n\n--------------------- >> end captured logging << ---------------------'
Project: https://jenkins.ovirt.org/job/ovirt-system-tests_compat-4.3-suite-master/ Build: https://jenkins.ovirt.org/job/ovirt-system-tests_compat-4.3-suite-master/399... Build Number: 399 Build Status: Still Failing Triggered By: Started by timer ------------------------------------- Changes Since Last Success: ------------------------------------- Changes for Build #394 [Sandro Bonazzola] repos: fix Advanced Virtualization repos [Sandro Bonazzola] mirrors: drop gluster 5 mirroring Changes for Build #395 [Galit Rosenthal] Fix the repo for suites that weren't moved to no reposync Changes for Build #396 [Galit Rosenthal] Fix the repo for suites that weren't moved to no reposync [Shlomi] Added mirror cleanup functionality [Evgheni Dereveanchin] Modules support for mirror_mgr.sh Changes for Build #397 [Galit Rosenthal] Fix alosync not recognize returning old 002_bootstrap [Evgheni Dereveanchin] Define mirrors for CentOS 8 [Evgheni Dereveanchin] fix gpg keys for COPR repos [Shlomi] Fixed an issue with cleanup of newly created mirrors. [Ehud Yonasi] ost: add custom repos to test nightly suites. Changes for Build #398 [Galit Rosenthal] Fix alosync not recognize returning old 002_bootstrap Changes for Build #399 [Galit Rosenthal] Fix alosync not recognize returning old 002_bootstrap ----------------- Failed Tests: ----------------- 1 tests failed. FAILED: 002_bootstrap.verify_add_hosts_43 Error Message: 1 hosts failed installation: lago-compat-4-3-suite-master-host-0: non_responsive -------------------- >> begin captured logging << -------------------- 002_bootstrap: DEBUG: Host status, verify_add_hosts: lago-compat-4-3-suite-master-host-0: installing lago-compat-4-3-suite-master-host-1: installing ovirtlago.testlib: ERROR: * Unhandled exception in <function <lambda> at 0x7fad93cd7050> Traceback (most recent call last): File "/usr/lib/python2.7/site-packages/ovirtlago/testlib.py", line 234, in assert_equals_within res = func() File "/home/jenkins/agent/workspace/ovirt-system-tests_compat-4.3-suite-master/ovirt-system-tests/compat-4.3-suite-master/test-scenarios/002_bootstrap.py", line 368, in <lambda> lambda: _single_host_up(hosts_service, total_hosts), File "/home/jenkins/agent/workspace/ovirt-system-tests_compat-4.3-suite-master/ovirt-system-tests/compat-4.3-suite-master/test-scenarios/002_bootstrap.py", line 154, in _single_host_up _check_problematic_hosts(hosts_service) File "/home/jenkins/agent/workspace/ovirt-system-tests_compat-4.3-suite-master/ovirt-system-tests/compat-4.3-suite-master/test-scenarios/002_bootstrap.py", line 163, in _check_problematic_hosts raise RuntimeError(dump_hosts) RuntimeError: 1 hosts failed installation: lago-compat-4-3-suite-master-host-0: non_responsive --------------------- >> end captured logging << --------------------- Stack Trace: File "/usr/lib64/python2.7/unittest/case.py", line 369, in run testMethod() File "/usr/lib/python2.7/site-packages/nose/case.py", line 197, in runTest self.test(*self.arg) File "/usr/lib/python2.7/site-packages/ovirtlago/testlib.py", line 142, in wrapped_test test() File "/usr/lib/python2.7/site-packages/ovirtlago/testlib.py", line 60, in wrapper return func(get_test_prefix(), *args, **kwargs) File "/usr/lib/python2.7/site-packages/ovirtlago/testlib.py", line 79, in wrapper prefix.virt_env.engine_vm().get_api(api_ver=4), *args, **kwargs File "/home/jenkins/agent/workspace/ovirt-system-tests_compat-4.3-suite-master/ovirt-system-tests/compat-4.3-suite-master/test-scenarios/002_bootstrap.py", line 369, in verify_add_hosts timeout=constants.ADD_HOST_TIMEOUT File "/usr/lib/python2.7/site-packages/ovirtlago/testlib.py", line 278, in assert_true_within assert_equals_within(func, True, timeout, allowed_exceptions) File "/usr/lib/python2.7/site-packages/ovirtlago/testlib.py", line 234, in assert_equals_within res = func() File "/home/jenkins/agent/workspace/ovirt-system-tests_compat-4.3-suite-master/ovirt-system-tests/compat-4.3-suite-master/test-scenarios/002_bootstrap.py", line 368, in <lambda> lambda: _single_host_up(hosts_service, total_hosts), File "/home/jenkins/agent/workspace/ovirt-system-tests_compat-4.3-suite-master/ovirt-system-tests/compat-4.3-suite-master/test-scenarios/002_bootstrap.py", line 154, in _single_host_up _check_problematic_hosts(hosts_service) File "/home/jenkins/agent/workspace/ovirt-system-tests_compat-4.3-suite-master/ovirt-system-tests/compat-4.3-suite-master/test-scenarios/002_bootstrap.py", line 163, in _check_problematic_hosts raise RuntimeError(dump_hosts) '1 hosts failed installation:\nlago-compat-4-3-suite-master-host-0: non_responsive\n\n-------------------- >> begin captured logging << --------------------\n002_bootstrap: DEBUG: Host status, verify_add_hosts:\n lago-compat-4-3-suite-master-host-0: installing\nlago-compat-4-3-suite-master-host-1: installing\n\novirtlago.testlib: ERROR: * Unhandled exception in <function <lambda> at 0x7fad93cd7050>\nTraceback (most recent call last):\n File "/usr/lib/python2.7/site-packages/ovirtlago/testlib.py", line 234, in assert_equals_within\n res = func()\n File "/home/jenkins/agent/workspace/ovirt-system-tests_compat-4.3-suite-master/ovirt-system-tests/compat-4.3-suite-master/test-scenarios/002_bootstrap.py", line 368, in <lambda>\n lambda: _single_host_up(hosts_service, total_hosts),\n File "/home/jenkins/agent/workspace/ovirt-system-tests_compat-4.3-suite-master/ovirt-system-tests/compat-4.3-suite-master/test-scenarios/002_bootstrap.py", line 154, in _single_host_up\n _check_problematic_hosts(hosts_service)\n File "/home/jenkins/agent/workspace/ovirt-system-tests_compat-4.3-suite-master/ovirt-system-tests/compat-4.3-suite-master/test-scenarios/002_bootstrap.py", line 163, in _check_problematic_hosts\n raise RuntimeError(dump_hosts)\nRuntimeError: 1 hosts failed installation:\nlago-compat-4-3-suite-master-host-0: non_responsive\n\n--------------------- >> end captured logging << ---------------------'
Project: https://jenkins.ovirt.org/job/ovirt-system-tests_compat-4.3-suite-master/ Build: https://jenkins.ovirt.org/job/ovirt-system-tests_compat-4.3-suite-master/400... Build Number: 400 Build Status: Still Failing Triggered By: Started by timer ------------------------------------- Changes Since Last Success: ------------------------------------- Changes for Build #394 [Sandro Bonazzola] repos: fix Advanced Virtualization repos [Sandro Bonazzola] mirrors: drop gluster 5 mirroring Changes for Build #395 [Galit Rosenthal] Fix the repo for suites that weren't moved to no reposync Changes for Build #396 [Galit Rosenthal] Fix the repo for suites that weren't moved to no reposync [Shlomi] Added mirror cleanup functionality [Evgheni Dereveanchin] Modules support for mirror_mgr.sh Changes for Build #397 [Galit Rosenthal] Fix alosync not recognize returning old 002_bootstrap [Evgheni Dereveanchin] Define mirrors for CentOS 8 [Evgheni Dereveanchin] fix gpg keys for COPR repos [Shlomi] Fixed an issue with cleanup of newly created mirrors. [Ehud Yonasi] ost: add custom repos to test nightly suites. Changes for Build #398 [Galit Rosenthal] Fix alosync not recognize returning old 002_bootstrap Changes for Build #399 [Galit Rosenthal] Fix alosync not recognize returning old 002_bootstrap Changes for Build #400 [Dominik Holler] network: xfailing ovs tests ----------------- Failed Tests: ----------------- 1 tests failed. FAILED: 002_bootstrap.verify_add_hosts_43 Error Message: 1 hosts failed installation: lago-compat-4-3-suite-master-host-0: non_responsive -------------------- >> begin captured logging << -------------------- 002_bootstrap: DEBUG: Host status, verify_add_hosts: lago-compat-4-3-suite-master-host-0: installing lago-compat-4-3-suite-master-host-1: installing ovirtlago.testlib: ERROR: * Unhandled exception in <function <lambda> at 0x7f051c13d050> Traceback (most recent call last): File "/usr/lib/python2.7/site-packages/ovirtlago/testlib.py", line 234, in assert_equals_within res = func() File "/home/jenkins/agent/workspace/ovirt-system-tests_compat-4.3-suite-master/ovirt-system-tests/compat-4.3-suite-master/test-scenarios/002_bootstrap.py", line 368, in <lambda> lambda: _single_host_up(hosts_service, total_hosts), File "/home/jenkins/agent/workspace/ovirt-system-tests_compat-4.3-suite-master/ovirt-system-tests/compat-4.3-suite-master/test-scenarios/002_bootstrap.py", line 154, in _single_host_up _check_problematic_hosts(hosts_service) File "/home/jenkins/agent/workspace/ovirt-system-tests_compat-4.3-suite-master/ovirt-system-tests/compat-4.3-suite-master/test-scenarios/002_bootstrap.py", line 163, in _check_problematic_hosts raise RuntimeError(dump_hosts) RuntimeError: 1 hosts failed installation: lago-compat-4-3-suite-master-host-0: non_responsive --------------------- >> end captured logging << --------------------- Stack Trace: File "/usr/lib64/python2.7/unittest/case.py", line 369, in run testMethod() File "/usr/lib/python2.7/site-packages/nose/case.py", line 197, in runTest self.test(*self.arg) File "/usr/lib/python2.7/site-packages/ovirtlago/testlib.py", line 142, in wrapped_test test() File "/usr/lib/python2.7/site-packages/ovirtlago/testlib.py", line 60, in wrapper return func(get_test_prefix(), *args, **kwargs) File "/usr/lib/python2.7/site-packages/ovirtlago/testlib.py", line 79, in wrapper prefix.virt_env.engine_vm().get_api(api_ver=4), *args, **kwargs File "/home/jenkins/agent/workspace/ovirt-system-tests_compat-4.3-suite-master/ovirt-system-tests/compat-4.3-suite-master/test-scenarios/002_bootstrap.py", line 369, in verify_add_hosts timeout=constants.ADD_HOST_TIMEOUT File "/usr/lib/python2.7/site-packages/ovirtlago/testlib.py", line 278, in assert_true_within assert_equals_within(func, True, timeout, allowed_exceptions) File "/usr/lib/python2.7/site-packages/ovirtlago/testlib.py", line 234, in assert_equals_within res = func() File "/home/jenkins/agent/workspace/ovirt-system-tests_compat-4.3-suite-master/ovirt-system-tests/compat-4.3-suite-master/test-scenarios/002_bootstrap.py", line 368, in <lambda> lambda: _single_host_up(hosts_service, total_hosts), File "/home/jenkins/agent/workspace/ovirt-system-tests_compat-4.3-suite-master/ovirt-system-tests/compat-4.3-suite-master/test-scenarios/002_bootstrap.py", line 154, in _single_host_up _check_problematic_hosts(hosts_service) File "/home/jenkins/agent/workspace/ovirt-system-tests_compat-4.3-suite-master/ovirt-system-tests/compat-4.3-suite-master/test-scenarios/002_bootstrap.py", line 163, in _check_problematic_hosts raise RuntimeError(dump_hosts) '1 hosts failed installation:\nlago-compat-4-3-suite-master-host-0: non_responsive\n\n-------------------- >> begin captured logging << --------------------\n002_bootstrap: DEBUG: Host status, verify_add_hosts:\n lago-compat-4-3-suite-master-host-0: installing\nlago-compat-4-3-suite-master-host-1: installing\n\novirtlago.testlib: ERROR: * Unhandled exception in <function <lambda> at 0x7f051c13d050>\nTraceback (most recent call last):\n File "/usr/lib/python2.7/site-packages/ovirtlago/testlib.py", line 234, in assert_equals_within\n res = func()\n File "/home/jenkins/agent/workspace/ovirt-system-tests_compat-4.3-suite-master/ovirt-system-tests/compat-4.3-suite-master/test-scenarios/002_bootstrap.py", line 368, in <lambda>\n lambda: _single_host_up(hosts_service, total_hosts),\n File "/home/jenkins/agent/workspace/ovirt-system-tests_compat-4.3-suite-master/ovirt-system-tests/compat-4.3-suite-master/test-scenarios/002_bootstrap.py", line 154, in _single_host_up\n _check_problematic_hosts(hosts_service)\n File "/home/jenkins/agent/workspace/ovirt-system-tests_compat-4.3-suite-master/ovirt-system-tests/compat-4.3-suite-master/test-scenarios/002_bootstrap.py", line 163, in _check_problematic_hosts\n raise RuntimeError(dump_hosts)\nRuntimeError: 1 hosts failed installation:\nlago-compat-4-3-suite-master-host-0: non_responsive\n\n--------------------- >> end captured logging << ---------------------'
Project: https://jenkins.ovirt.org/job/ovirt-system-tests_compat-4.3-suite-master/ Build: https://jenkins.ovirt.org/job/ovirt-system-tests_compat-4.3-suite-master/401... Build Number: 401 Build Status: Still Failing Triggered By: Started by timer ------------------------------------- Changes Since Last Success: ------------------------------------- Changes for Build #394 [Sandro Bonazzola] repos: fix Advanced Virtualization repos [Sandro Bonazzola] mirrors: drop gluster 5 mirroring Changes for Build #395 [Galit Rosenthal] Fix the repo for suites that weren't moved to no reposync Changes for Build #396 [Galit Rosenthal] Fix the repo for suites that weren't moved to no reposync [Shlomi] Added mirror cleanup functionality [Evgheni Dereveanchin] Modules support for mirror_mgr.sh Changes for Build #397 [Galit Rosenthal] Fix alosync not recognize returning old 002_bootstrap [Evgheni Dereveanchin] Define mirrors for CentOS 8 [Evgheni Dereveanchin] fix gpg keys for COPR repos [Shlomi] Fixed an issue with cleanup of newly created mirrors. [Ehud Yonasi] ost: add custom repos to test nightly suites. Changes for Build #398 [Galit Rosenthal] Fix alosync not recognize returning old 002_bootstrap Changes for Build #399 [Galit Rosenthal] Fix alosync not recognize returning old 002_bootstrap Changes for Build #400 [Dominik Holler] network: xfailing ovs tests Changes for Build #401 [Evgheni Dereveanchin] Move from git proto to https [Evgheni Dereveanchin] add suffix to el8 ppc64le repo names in mock configs ----------------- Failed Tests: ----------------- 1 tests failed. FAILED: 002_bootstrap.verify_add_hosts_43 Error Message: 1 hosts failed installation: lago-compat-4-3-suite-master-host-0: non_responsive -------------------- >> begin captured logging << -------------------- 002_bootstrap: DEBUG: Host status, verify_add_hosts: lago-compat-4-3-suite-master-host-0: installing lago-compat-4-3-suite-master-host-1: installing ovirtlago.testlib: ERROR: * Unhandled exception in <function <lambda> at 0x7f9aef167050> Traceback (most recent call last): File "/usr/lib/python2.7/site-packages/ovirtlago/testlib.py", line 234, in assert_equals_within res = func() File "/home/jenkins/agent/workspace/ovirt-system-tests_compat-4.3-suite-master/ovirt-system-tests/compat-4.3-suite-master/test-scenarios/002_bootstrap.py", line 368, in <lambda> lambda: _single_host_up(hosts_service, total_hosts), File "/home/jenkins/agent/workspace/ovirt-system-tests_compat-4.3-suite-master/ovirt-system-tests/compat-4.3-suite-master/test-scenarios/002_bootstrap.py", line 154, in _single_host_up _check_problematic_hosts(hosts_service) File "/home/jenkins/agent/workspace/ovirt-system-tests_compat-4.3-suite-master/ovirt-system-tests/compat-4.3-suite-master/test-scenarios/002_bootstrap.py", line 163, in _check_problematic_hosts raise RuntimeError(dump_hosts) RuntimeError: 1 hosts failed installation: lago-compat-4-3-suite-master-host-0: non_responsive --------------------- >> end captured logging << --------------------- Stack Trace: File "/usr/lib64/python2.7/unittest/case.py", line 369, in run testMethod() File "/usr/lib/python2.7/site-packages/nose/case.py", line 197, in runTest self.test(*self.arg) File "/usr/lib/python2.7/site-packages/ovirtlago/testlib.py", line 142, in wrapped_test test() File "/usr/lib/python2.7/site-packages/ovirtlago/testlib.py", line 60, in wrapper return func(get_test_prefix(), *args, **kwargs) File "/usr/lib/python2.7/site-packages/ovirtlago/testlib.py", line 79, in wrapper prefix.virt_env.engine_vm().get_api(api_ver=4), *args, **kwargs File "/home/jenkins/agent/workspace/ovirt-system-tests_compat-4.3-suite-master/ovirt-system-tests/compat-4.3-suite-master/test-scenarios/002_bootstrap.py", line 369, in verify_add_hosts timeout=constants.ADD_HOST_TIMEOUT File "/usr/lib/python2.7/site-packages/ovirtlago/testlib.py", line 278, in assert_true_within assert_equals_within(func, True, timeout, allowed_exceptions) File "/usr/lib/python2.7/site-packages/ovirtlago/testlib.py", line 234, in assert_equals_within res = func() File "/home/jenkins/agent/workspace/ovirt-system-tests_compat-4.3-suite-master/ovirt-system-tests/compat-4.3-suite-master/test-scenarios/002_bootstrap.py", line 368, in <lambda> lambda: _single_host_up(hosts_service, total_hosts), File "/home/jenkins/agent/workspace/ovirt-system-tests_compat-4.3-suite-master/ovirt-system-tests/compat-4.3-suite-master/test-scenarios/002_bootstrap.py", line 154, in _single_host_up _check_problematic_hosts(hosts_service) File "/home/jenkins/agent/workspace/ovirt-system-tests_compat-4.3-suite-master/ovirt-system-tests/compat-4.3-suite-master/test-scenarios/002_bootstrap.py", line 163, in _check_problematic_hosts raise RuntimeError(dump_hosts) '1 hosts failed installation:\nlago-compat-4-3-suite-master-host-0: non_responsive\n\n-------------------- >> begin captured logging << --------------------\n002_bootstrap: DEBUG: Host status, verify_add_hosts:\n lago-compat-4-3-suite-master-host-0: installing\nlago-compat-4-3-suite-master-host-1: installing\n\novirtlago.testlib: ERROR: * Unhandled exception in <function <lambda> at 0x7f9aef167050>\nTraceback (most recent call last):\n File "/usr/lib/python2.7/site-packages/ovirtlago/testlib.py", line 234, in assert_equals_within\n res = func()\n File "/home/jenkins/agent/workspace/ovirt-system-tests_compat-4.3-suite-master/ovirt-system-tests/compat-4.3-suite-master/test-scenarios/002_bootstrap.py", line 368, in <lambda>\n lambda: _single_host_up(hosts_service, total_hosts),\n File "/home/jenkins/agent/workspace/ovirt-system-tests_compat-4.3-suite-master/ovirt-system-tests/compat-4.3-suite-master/test-scenarios/002_bootstrap.py", line 154, in _single_host_up\n _check_problematic_hosts(hosts_service)\n File "/home/jenkins/agent/workspace/ovirt-system-tests_compat-4.3-suite-master/ovirt-system-tests/compat-4.3-suite-master/test-scenarios/002_bootstrap.py", line 163, in _check_problematic_hosts\n raise RuntimeError(dump_hosts)\nRuntimeError: 1 hosts failed installation:\nlago-compat-4-3-suite-master-host-0: non_responsive\n\n--------------------- >> end captured logging << ---------------------'
Project: https://jenkins.ovirt.org/job/ovirt-system-tests_compat-4.3-suite-master/ Build: https://jenkins.ovirt.org/job/ovirt-system-tests_compat-4.3-suite-master/402... Build Number: 402 Build Status: Still Failing Triggered By: Started by timer ------------------------------------- Changes Since Last Success: ------------------------------------- Changes for Build #394 [Sandro Bonazzola] repos: fix Advanced Virtualization repos [Sandro Bonazzola] mirrors: drop gluster 5 mirroring Changes for Build #395 [Galit Rosenthal] Fix the repo for suites that weren't moved to no reposync Changes for Build #396 [Galit Rosenthal] Fix the repo for suites that weren't moved to no reposync [Shlomi] Added mirror cleanup functionality [Evgheni Dereveanchin] Modules support for mirror_mgr.sh Changes for Build #397 [Galit Rosenthal] Fix alosync not recognize returning old 002_bootstrap [Evgheni Dereveanchin] Define mirrors for CentOS 8 [Evgheni Dereveanchin] fix gpg keys for COPR repos [Shlomi] Fixed an issue with cleanup of newly created mirrors. [Ehud Yonasi] ost: add custom repos to test nightly suites. Changes for Build #398 [Galit Rosenthal] Fix alosync not recognize returning old 002_bootstrap Changes for Build #399 [Galit Rosenthal] Fix alosync not recognize returning old 002_bootstrap Changes for Build #400 [Dominik Holler] network: xfailing ovs tests Changes for Build #401 [Evgheni Dereveanchin] Move from git proto to https [Evgheni Dereveanchin] add suffix to el8 ppc64le repo names in mock configs Changes for Build #402 [Michal Skrivanek] minor compatibility and resiliency fixes [Martin Perina] mock_configs: Enable PG 12 module in EL8 mocks ----------------- Failed Tests: ----------------- 1 tests failed. FAILED: basic-suite-master.test-scenarios.004_basic_sanity_pytest.test_make_snapshot_with_memory Error Message: AssertionError: False != True after 600 seconds Stack Trace: api_v4 = <ovirtsdk4.Connection object at 0x7f4113589a50> @pytest.mark.run(order=34) def test_make_snapshot_with_memory(api_v4): engine = api_v4.system_service() vm_service = test_utils.get_vm_service(engine, VM0_NAME) disks_service = engine.disks_service() vm_disks_service = \ test_utils.get_disk_attachments_service(engine, VM0_NAME) vm_disks = [disks_service.disk_service(attachment.disk.id).get() for attachment in vm_disks_service.list()] disk_attachments = [types.DiskAttachment(disk=types.Disk(id=disk.id)) for disk in vm_disks if disk.storage_type != types.DiskStorageType.LUN] snapshots_service = vm_service.snapshots_service() snapshot_params = types.Snapshot( description=SNAPSHOT_DESC_MEM, persist_memorystate=True, disk_attachments=disk_attachments ) with test_utils.TestEvent(engine, 45): # USER_CREATE_SNAPSHOT event
snapshots_service.add(snapshot_params)
../basic-suite-master/test-scenarios/004_basic_sanity_pytest.py:447: _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ /usr/lib64/python2.7/contextlib.py:35: in __exit__ self.gen.throw(type, value, traceback) ../compat-4.3-suite-master/test_utils/__init__.py:290: in TestEvent lambda: /usr/lib/python2.7/site-packages/ovirtlago/testlib.py:286: in assert_true_within_long assert_equals_within_long(func, True, allowed_exceptions) /usr/lib/python2.7/site-packages/ovirtlago/testlib.py:273: in assert_equals_within_long func, value, LONG_TIMEOUT, allowed_exceptions=allowed_exceptions _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ func = <function <lambda> at 0x7f41105725f0>, value = True, timeout = 600 allowed_exceptions = [], initial_wait = 0 def assert_equals_within( func, value, timeout, allowed_exceptions=None, initial_wait=10 ): allowed_exceptions = allowed_exceptions or [] with utils.EggTimer(timeout) as timer: while not timer.elapsed(): try: res = func() if res == value: return except Exception as exc: if _instance_of_any(exc, allowed_exceptions): time.sleep(3) continue LOGGER.exception("Unhandled exception in %s", func) raise if initial_wait == 0: time.sleep(3) else: time.sleep(initial_wait) initial_wait = 0 try: raise AssertionError(
'%s != %s after %s seconds' % (res, value, timeout)
) E AssertionError: False != True after 600 seconds /usr/lib/python2.7/site-packages/ovirtlago/testlib.py:252: AssertionError
Project: https://jenkins.ovirt.org/job/ovirt-system-tests_compat-4.3-suite-master/ Build: https://jenkins.ovirt.org/job/ovirt-system-tests_compat-4.3-suite-master/403... Build Number: 403 Build Status: Still Failing Triggered By: Started by timer ------------------------------------- Changes Since Last Success: ------------------------------------- Changes for Build #394 [Sandro Bonazzola] repos: fix Advanced Virtualization repos [Sandro Bonazzola] mirrors: drop gluster 5 mirroring Changes for Build #395 [Galit Rosenthal] Fix the repo for suites that weren't moved to no reposync Changes for Build #396 [Galit Rosenthal] Fix the repo for suites that weren't moved to no reposync [Shlomi] Added mirror cleanup functionality [Evgheni Dereveanchin] Modules support for mirror_mgr.sh Changes for Build #397 [Galit Rosenthal] Fix alosync not recognize returning old 002_bootstrap [Evgheni Dereveanchin] Define mirrors for CentOS 8 [Evgheni Dereveanchin] fix gpg keys for COPR repos [Shlomi] Fixed an issue with cleanup of newly created mirrors. [Ehud Yonasi] ost: add custom repos to test nightly suites. Changes for Build #398 [Galit Rosenthal] Fix alosync not recognize returning old 002_bootstrap Changes for Build #399 [Galit Rosenthal] Fix alosync not recognize returning old 002_bootstrap Changes for Build #400 [Dominik Holler] network: xfailing ovs tests Changes for Build #401 [Evgheni Dereveanchin] Move from git proto to https [Evgheni Dereveanchin] add suffix to el8 ppc64le repo names in mock configs Changes for Build #402 [Michal Skrivanek] minor compatibility and resiliency fixes [Martin Perina] mock_configs: Enable PG 12 module in EL8 mocks Changes for Build #403 [Michal Skrivanek] use a q35 custom emulated machine now ----------------- Failed Tests: ----------------- 1 tests failed. FAILED: basic-suite-master.test-scenarios.004_basic_sanity_pytest.test_ha_recovery Error Message: AssertionError: False != True after 600 seconds Stack Trace: prefix = <ovirtlago.prefix.OvirtPrefix object at 0x7f843cbd4f50> @pytest.mark.run(order=18) def test_ha_recovery(prefix): engine = prefix.virt_env.engine_vm().get_api_v4().system_service() with test_utils.TestEvent(engine, [119, 9602, 506]): # VM_DOWN_ERROR event(119) # HA_VM_FAILED event event(9602) # VDS_INITIATED_RUN_VM event(506) vm_host = _vm_host(prefix, VM2_NAME) pid = vm_host.ssh(['pgrep', '-f', 'qemu.*guest=vm2']) vm_host.ssh(['kill', '-KILL', pid.out.decode('utf-8')]) vm_service = test_utils.get_vm_service(engine, VM2_NAME) testlib.assert_true_within_long(
lambda:
vm_service.get().status == types.VmStatus.UP ) ../basic-suite-master/test-scenarios/004_basic_sanity_pytest.py:739: _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ /usr/lib/python2.7/site-packages/ovirtlago/testlib.py:286: in assert_true_within_long assert_equals_within_long(func, True, allowed_exceptions) /usr/lib/python2.7/site-packages/ovirtlago/testlib.py:273: in assert_equals_within_long func, value, LONG_TIMEOUT, allowed_exceptions=allowed_exceptions _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ func = <function <lambda> at 0x7f8439be7b90>, value = True, timeout = 600 allowed_exceptions = [], initial_wait = 0 def assert_equals_within( func, value, timeout, allowed_exceptions=None, initial_wait=10 ): allowed_exceptions = allowed_exceptions or [] with utils.EggTimer(timeout) as timer: while not timer.elapsed(): try: res = func() if res == value: return except Exception as exc: if _instance_of_any(exc, allowed_exceptions): time.sleep(3) continue LOGGER.exception("Unhandled exception in %s", func) raise if initial_wait == 0: time.sleep(3) else: time.sleep(initial_wait) initial_wait = 0 try: raise AssertionError(
'%s != %s after %s seconds' % (res, value, timeout)
) E AssertionError: False != True after 600 seconds /usr/lib/python2.7/site-packages/ovirtlago/testlib.py:252: AssertionError
Project: https://jenkins.ovirt.org/job/ovirt-system-tests_compat-4.3-suite-master/ Build: https://jenkins.ovirt.org/job/ovirt-system-tests_compat-4.3-suite-master/404... Build Number: 404 Build Status: Still Failing Triggered By: Started by timer ------------------------------------- Changes Since Last Success: ------------------------------------- Changes for Build #394 [Sandro Bonazzola] repos: fix Advanced Virtualization repos [Sandro Bonazzola] mirrors: drop gluster 5 mirroring Changes for Build #395 [Galit Rosenthal] Fix the repo for suites that weren't moved to no reposync Changes for Build #396 [Galit Rosenthal] Fix the repo for suites that weren't moved to no reposync [Shlomi] Added mirror cleanup functionality [Evgheni Dereveanchin] Modules support for mirror_mgr.sh Changes for Build #397 [Galit Rosenthal] Fix alosync not recognize returning old 002_bootstrap [Evgheni Dereveanchin] Define mirrors for CentOS 8 [Evgheni Dereveanchin] fix gpg keys for COPR repos [Shlomi] Fixed an issue with cleanup of newly created mirrors. [Ehud Yonasi] ost: add custom repos to test nightly suites. Changes for Build #398 [Galit Rosenthal] Fix alosync not recognize returning old 002_bootstrap Changes for Build #399 [Galit Rosenthal] Fix alosync not recognize returning old 002_bootstrap Changes for Build #400 [Dominik Holler] network: xfailing ovs tests Changes for Build #401 [Evgheni Dereveanchin] Move from git proto to https [Evgheni Dereveanchin] add suffix to el8 ppc64le repo names in mock configs Changes for Build #402 [Michal Skrivanek] minor compatibility and resiliency fixes [Martin Perina] mock_configs: Enable PG 12 module in EL8 mocks Changes for Build #403 [Michal Skrivanek] use a q35 custom emulated machine now Changes for Build #404 [Marcin Sobczyk] pytest: Add 'repo_server' fixture ----------------- Failed Tests: ----------------- 1 tests failed. FAILED: basic-suite-master.test-scenarios.004_basic_sanity_pytest.test_ha_recovery Error Message: AssertionError: False != True after 600 seconds Stack Trace: prefix = <ovirtlago.prefix.OvirtPrefix object at 0x7fc9e4786150> @pytest.mark.run(order=18) def test_ha_recovery(prefix): engine = prefix.virt_env.engine_vm().get_api_v4().system_service() with test_utils.TestEvent(engine, [119, 9602, 506]): # VM_DOWN_ERROR event(119) # HA_VM_FAILED event event(9602) # VDS_INITIATED_RUN_VM event(506) vm_host = _vm_host(prefix, VM2_NAME) pid = vm_host.ssh(['pgrep', '-f', 'qemu.*guest=vm2']) vm_host.ssh(['kill', '-KILL', pid.out.decode('utf-8')]) vm_service = test_utils.get_vm_service(engine, VM2_NAME) testlib.assert_true_within_long(
lambda:
vm_service.get().status == types.VmStatus.UP ) ../basic-suite-master/test-scenarios/004_basic_sanity_pytest.py:739: _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ /usr/lib/python2.7/site-packages/ovirtlago/testlib.py:286: in assert_true_within_long assert_equals_within_long(func, True, allowed_exceptions) /usr/lib/python2.7/site-packages/ovirtlago/testlib.py:273: in assert_equals_within_long func, value, LONG_TIMEOUT, allowed_exceptions=allowed_exceptions _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ func = <function <lambda> at 0x7fc9e0c79a28>, value = True, timeout = 600 allowed_exceptions = [], initial_wait = 0 def assert_equals_within( func, value, timeout, allowed_exceptions=None, initial_wait=10 ): allowed_exceptions = allowed_exceptions or [] with utils.EggTimer(timeout) as timer: while not timer.elapsed(): try: res = func() if res == value: return except Exception as exc: if _instance_of_any(exc, allowed_exceptions): time.sleep(3) continue LOGGER.exception("Unhandled exception in %s", func) raise if initial_wait == 0: time.sleep(3) else: time.sleep(initial_wait) initial_wait = 0 try: raise AssertionError(
'%s != %s after %s seconds' % (res, value, timeout)
) E AssertionError: False != True after 600 seconds /usr/lib/python2.7/site-packages/ovirtlago/testlib.py:252: AssertionError
Project: https://jenkins.ovirt.org/job/ovirt-system-tests_compat-4.3-suite-master/ Build: https://jenkins.ovirt.org/job/ovirt-system-tests_compat-4.3-suite-master/405... Build Number: 405 Build Status: Still Failing Triggered By: Started by user Galit Rosenthal ------------------------------------- Changes Since Last Success: ------------------------------------- Changes for Build #395 [Galit Rosenthal] Fix the repo for suites that weren't moved to no reposync Changes for Build #396 [Galit Rosenthal] Fix the repo for suites that weren't moved to no reposync [Shlomi] Added mirror cleanup functionality [Evgheni Dereveanchin] Modules support for mirror_mgr.sh Changes for Build #397 [Galit Rosenthal] Fix alosync not recognize returning old 002_bootstrap [Evgheni Dereveanchin] Define mirrors for CentOS 8 [Evgheni Dereveanchin] fix gpg keys for COPR repos [Shlomi] Fixed an issue with cleanup of newly created mirrors. [Ehud Yonasi] ost: add custom repos to test nightly suites. Changes for Build #398 [Galit Rosenthal] Fix alosync not recognize returning old 002_bootstrap Changes for Build #399 [Galit Rosenthal] Fix alosync not recognize returning old 002_bootstrap Changes for Build #400 [Dominik Holler] network: xfailing ovs tests Changes for Build #401 [Evgheni Dereveanchin] Move from git proto to https [Evgheni Dereveanchin] add suffix to el8 ppc64le repo names in mock configs Changes for Build #402 [Michal Skrivanek] minor compatibility and resiliency fixes [Martin Perina] mock_configs: Enable PG 12 module in EL8 mocks Changes for Build #403 [Michal Skrivanek] use a q35 custom emulated machine now Changes for Build #404 [Marcin Sobczyk] pytest: Add 'repo_server' fixture Changes for Build #405 [Galit] Remove wrong link ----------------- Failed Tests: ----------------- 1 tests failed. FAILED: basic-suite-master.test-scenarios.004_basic_sanity_pytest.test_ha_recovery Error Message: AssertionError: False != True after 600 seconds Stack Trace: prefix = <ovirtlago.prefix.OvirtPrefix object at 0x7f24ab4ce150> @pytest.mark.run(order=18) def test_ha_recovery(prefix): engine = prefix.virt_env.engine_vm().get_api_v4().system_service() with test_utils.TestEvent(engine, [119, 9602, 506]): # VM_DOWN_ERROR event(119) # HA_VM_FAILED event event(9602) # VDS_INITIATED_RUN_VM event(506) vm_host = _vm_host(prefix, VM2_NAME) pid = vm_host.ssh(['pgrep', '-f', 'qemu.*guest=vm2']) vm_host.ssh(['kill', '-KILL', pid.out.decode('utf-8')]) vm_service = test_utils.get_vm_service(engine, VM2_NAME) testlib.assert_true_within_long(
lambda:
vm_service.get().status == types.VmStatus.UP ) ../basic-suite-master/test-scenarios/004_basic_sanity_pytest.py:739: _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ /usr/lib/python2.7/site-packages/ovirtlago/testlib.py:286: in assert_true_within_long assert_equals_within_long(func, True, allowed_exceptions) /usr/lib/python2.7/site-packages/ovirtlago/testlib.py:273: in assert_equals_within_long func, value, LONG_TIMEOUT, allowed_exceptions=allowed_exceptions _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ func = <function <lambda> at 0x7f24a809dcf8>, value = True, timeout = 600 allowed_exceptions = [], initial_wait = 0 def assert_equals_within( func, value, timeout, allowed_exceptions=None, initial_wait=10 ): allowed_exceptions = allowed_exceptions or [] with utils.EggTimer(timeout) as timer: while not timer.elapsed(): try: res = func() if res == value: return except Exception as exc: if _instance_of_any(exc, allowed_exceptions): time.sleep(3) continue LOGGER.exception("Unhandled exception in %s", func) raise if initial_wait == 0: time.sleep(3) else: time.sleep(initial_wait) initial_wait = 0 try: raise AssertionError(
'%s != %s after %s seconds' % (res, value, timeout)
) E AssertionError: False != True after 600 seconds /usr/lib/python2.7/site-packages/ovirtlago/testlib.py:252: AssertionError
Project: https://jenkins.ovirt.org/job/ovirt-system-tests_compat-4.3-suite-master/ Build: https://jenkins.ovirt.org/job/ovirt-system-tests_compat-4.3-suite-master/406... Build Number: 406 Build Status: Still Failing Triggered By: Started by timer ------------------------------------- Changes Since Last Success: ------------------------------------- Changes for Build #395 [Galit Rosenthal] Fix the repo for suites that weren't moved to no reposync Changes for Build #396 [Galit Rosenthal] Fix the repo for suites that weren't moved to no reposync [Shlomi] Added mirror cleanup functionality [Evgheni Dereveanchin] Modules support for mirror_mgr.sh Changes for Build #397 [Galit Rosenthal] Fix alosync not recognize returning old 002_bootstrap [Evgheni Dereveanchin] Define mirrors for CentOS 8 [Evgheni Dereveanchin] fix gpg keys for COPR repos [Shlomi] Fixed an issue with cleanup of newly created mirrors. [Ehud Yonasi] ost: add custom repos to test nightly suites. Changes for Build #398 [Galit Rosenthal] Fix alosync not recognize returning old 002_bootstrap Changes for Build #399 [Galit Rosenthal] Fix alosync not recognize returning old 002_bootstrap Changes for Build #400 [Dominik Holler] network: xfailing ovs tests Changes for Build #401 [Evgheni Dereveanchin] Move from git proto to https [Evgheni Dereveanchin] add suffix to el8 ppc64le repo names in mock configs Changes for Build #402 [Michal Skrivanek] minor compatibility and resiliency fixes [Martin Perina] mock_configs: Enable PG 12 module in EL8 mocks Changes for Build #403 [Michal Skrivanek] use a q35 custom emulated machine now Changes for Build #404 [Marcin Sobczyk] pytest: Add 'repo_server' fixture Changes for Build #405 [Galit] Remove wrong link Changes for Build #406 [Michal Skrivanek] javapackages-tools module only required on CentOS ----------------- Failed Tests: ----------------- 1 tests failed. FAILED: basic-suite-master.test-scenarios.004_basic_sanity_pytest.test_ha_recovery Error Message: AssertionError: False != True after 600 seconds Stack Trace: prefix = <ovirtlago.prefix.OvirtPrefix object at 0x7f417f7bd150> @pytest.mark.run(order=18) def test_ha_recovery(prefix): engine = prefix.virt_env.engine_vm().get_api_v4().system_service() with test_utils.TestEvent(engine, [119, 9602, 506]): # VM_DOWN_ERROR event(119) # HA_VM_FAILED event event(9602) # VDS_INITIATED_RUN_VM event(506) vm_host = _vm_host(prefix, VM2_NAME) pid = vm_host.ssh(['pgrep', '-f', 'qemu.*guest=vm2']) vm_host.ssh(['kill', '-KILL', pid.out.decode('utf-8')]) vm_service = test_utils.get_vm_service(engine, VM2_NAME) testlib.assert_true_within_long(
lambda:
vm_service.get().status == types.VmStatus.UP ) ../basic-suite-master/test-scenarios/004_basic_sanity_pytest.py:739: _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ /usr/lib/python2.7/site-packages/ovirtlago/testlib.py:286: in assert_true_within_long assert_equals_within_long(func, True, allowed_exceptions) /usr/lib/python2.7/site-packages/ovirtlago/testlib.py:273: in assert_equals_within_long func, value, LONG_TIMEOUT, allowed_exceptions=allowed_exceptions _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ func = <function <lambda> at 0x7f416fc66e60>, value = True, timeout = 600 allowed_exceptions = [], initial_wait = 0 def assert_equals_within( func, value, timeout, allowed_exceptions=None, initial_wait=10 ): allowed_exceptions = allowed_exceptions or [] with utils.EggTimer(timeout) as timer: while not timer.elapsed(): try: res = func() if res == value: return except Exception as exc: if _instance_of_any(exc, allowed_exceptions): time.sleep(3) continue LOGGER.exception("Unhandled exception in %s", func) raise if initial_wait == 0: time.sleep(3) else: time.sleep(initial_wait) initial_wait = 0 try: raise AssertionError(
'%s != %s after %s seconds' % (res, value, timeout)
) E AssertionError: False != True after 600 seconds /usr/lib/python2.7/site-packages/ovirtlago/testlib.py:252: AssertionError
Project: https://jenkins.ovirt.org/job/ovirt-system-tests_compat-4.3-suite-master/ Build: https://jenkins.ovirt.org/job/ovirt-system-tests_compat-4.3-suite-master/407... Build Number: 407 Build Status: Still Failing Triggered By: Started by timer ------------------------------------- Changes Since Last Success: ------------------------------------- Changes for Build #396 [Galit Rosenthal] Fix the repo for suites that weren't moved to no reposync [Shlomi] Added mirror cleanup functionality [Evgheni Dereveanchin] Modules support for mirror_mgr.sh Changes for Build #397 [Galit Rosenthal] Fix alosync not recognize returning old 002_bootstrap [Evgheni Dereveanchin] Define mirrors for CentOS 8 [Evgheni Dereveanchin] fix gpg keys for COPR repos [Shlomi] Fixed an issue with cleanup of newly created mirrors. [Ehud Yonasi] ost: add custom repos to test nightly suites. Changes for Build #398 [Galit Rosenthal] Fix alosync not recognize returning old 002_bootstrap Changes for Build #399 [Galit Rosenthal] Fix alosync not recognize returning old 002_bootstrap Changes for Build #400 [Dominik Holler] network: xfailing ovs tests Changes for Build #401 [Evgheni Dereveanchin] Move from git proto to https [Evgheni Dereveanchin] add suffix to el8 ppc64le repo names in mock configs Changes for Build #402 [Michal Skrivanek] minor compatibility and resiliency fixes [Martin Perina] mock_configs: Enable PG 12 module in EL8 mocks Changes for Build #403 [Michal Skrivanek] use a q35 custom emulated machine now Changes for Build #404 [Marcin Sobczyk] pytest: Add 'repo_server' fixture Changes for Build #405 [Galit] Remove wrong link Changes for Build #406 [Michal Skrivanek] javapackages-tools module only required on CentOS Changes for Build #407 [Michal Skrivanek] javapackages-tools module only required on CentOS ----------------- Failed Tests: ----------------- 1 tests failed. FAILED: basic-suite-master.test-scenarios.004_basic_sanity_pytest.test_ha_recovery Error Message: AssertionError: False != True after 600 seconds Stack Trace: prefix = <ovirtlago.prefix.OvirtPrefix object at 0x7f9f8c160150> @pytest.mark.run(order=18) def test_ha_recovery(prefix): engine = prefix.virt_env.engine_vm().get_api_v4().system_service() with test_utils.TestEvent(engine, [119, 9602, 506]): # VM_DOWN_ERROR event(119) # HA_VM_FAILED event event(9602) # VDS_INITIATED_RUN_VM event(506) vm_host = _vm_host(prefix, VM2_NAME) pid = vm_host.ssh(['pgrep', '-f', 'qemu.*guest=vm2']) vm_host.ssh(['kill', '-KILL', pid.out.decode('utf-8')]) vm_service = test_utils.get_vm_service(engine, VM2_NAME) testlib.assert_true_within_long(
lambda:
vm_service.get().status == types.VmStatus.UP ) ../basic-suite-master/test-scenarios/004_basic_sanity_pytest.py:739: _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ /usr/lib/python2.7/site-packages/ovirtlago/testlib.py:286: in assert_true_within_long assert_equals_within_long(func, True, allowed_exceptions) /usr/lib/python2.7/site-packages/ovirtlago/testlib.py:273: in assert_equals_within_long func, value, LONG_TIMEOUT, allowed_exceptions=allowed_exceptions _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ func = <function <lambda> at 0x7f9f88652938>, value = True, timeout = 600 allowed_exceptions = [], initial_wait = 0 def assert_equals_within( func, value, timeout, allowed_exceptions=None, initial_wait=10 ): allowed_exceptions = allowed_exceptions or [] with utils.EggTimer(timeout) as timer: while not timer.elapsed(): try: res = func() if res == value: return except Exception as exc: if _instance_of_any(exc, allowed_exceptions): time.sleep(3) continue LOGGER.exception("Unhandled exception in %s", func) raise if initial_wait == 0: time.sleep(3) else: time.sleep(initial_wait) initial_wait = 0 try: raise AssertionError(
'%s != %s after %s seconds' % (res, value, timeout)
) E AssertionError: False != True after 600 seconds /usr/lib/python2.7/site-packages/ovirtlago/testlib.py:252: AssertionError
Project: https://jenkins.ovirt.org/job/ovirt-system-tests_compat-4.3-suite-master/ Build: https://jenkins.ovirt.org/job/ovirt-system-tests_compat-4.3-suite-master/408... Build Number: 408 Build Status: Still Failing Triggered By: Started by timer ------------------------------------- Changes Since Last Success: ------------------------------------- Changes for Build #397 [Galit Rosenthal] Fix alosync not recognize returning old 002_bootstrap [Evgheni Dereveanchin] Define mirrors for CentOS 8 [Evgheni Dereveanchin] fix gpg keys for COPR repos [Shlomi] Fixed an issue with cleanup of newly created mirrors. [Ehud Yonasi] ost: add custom repos to test nightly suites. Changes for Build #398 [Galit Rosenthal] Fix alosync not recognize returning old 002_bootstrap Changes for Build #399 [Galit Rosenthal] Fix alosync not recognize returning old 002_bootstrap Changes for Build #400 [Dominik Holler] network: xfailing ovs tests Changes for Build #401 [Evgheni Dereveanchin] Move from git proto to https [Evgheni Dereveanchin] add suffix to el8 ppc64le repo names in mock configs Changes for Build #402 [Michal Skrivanek] minor compatibility and resiliency fixes [Martin Perina] mock_configs: Enable PG 12 module in EL8 mocks Changes for Build #403 [Michal Skrivanek] use a q35 custom emulated machine now Changes for Build #404 [Marcin Sobczyk] pytest: Add 'repo_server' fixture Changes for Build #405 [Galit] Remove wrong link Changes for Build #406 [Michal Skrivanek] javapackages-tools module only required on CentOS Changes for Build #407 [Michal Skrivanek] javapackages-tools module only required on CentOS Changes for Build #408 [Michal Skrivanek] javapackages-tools module only required on CentOS ----------------- Failed Tests: ----------------- 1 tests failed. FAILED: basic-suite-master.test-scenarios.004_basic_sanity_pytest.test_ha_recovery Error Message: AssertionError: False != True after 600 seconds Stack Trace: prefix = <ovirtlago.prefix.OvirtPrefix object at 0x7fd84b24b150> @pytest.mark.run(order=18) def test_ha_recovery(prefix): engine = prefix.virt_env.engine_vm().get_api_v4().system_service() with test_utils.TestEvent(engine, [119, 9602, 506]): # VM_DOWN_ERROR event(119) # HA_VM_FAILED event event(9602) # VDS_INITIATED_RUN_VM event(506) vm_host = _vm_host(prefix, VM2_NAME) pid = vm_host.ssh(['pgrep', '-f', 'qemu.*guest=vm2']) vm_host.ssh(['kill', '-KILL', pid.out.decode('utf-8')]) vm_service = test_utils.get_vm_service(engine, VM2_NAME) testlib.assert_true_within_long(
lambda:
vm_service.get().status == types.VmStatus.UP ) ../basic-suite-master/test-scenarios/004_basic_sanity_pytest.py:739: _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ /usr/lib/python2.7/site-packages/ovirtlago/testlib.py:286: in assert_true_within_long assert_equals_within_long(func, True, allowed_exceptions) /usr/lib/python2.7/site-packages/ovirtlago/testlib.py:273: in assert_equals_within_long func, value, LONG_TIMEOUT, allowed_exceptions=allowed_exceptions _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ func = <function <lambda> at 0x7fd8481f0cf8>, value = True, timeout = 600 allowed_exceptions = [], initial_wait = 0 def assert_equals_within( func, value, timeout, allowed_exceptions=None, initial_wait=10 ): allowed_exceptions = allowed_exceptions or [] with utils.EggTimer(timeout) as timer: while not timer.elapsed(): try: res = func() if res == value: return except Exception as exc: if _instance_of_any(exc, allowed_exceptions): time.sleep(3) continue LOGGER.exception("Unhandled exception in %s", func) raise if initial_wait == 0: time.sleep(3) else: time.sleep(initial_wait) initial_wait = 0 try: raise AssertionError(
'%s != %s after %s seconds' % (res, value, timeout)
) E AssertionError: False != True after 600 seconds /usr/lib/python2.7/site-packages/ovirtlago/testlib.py:252: AssertionError
Project: https://jenkins.ovirt.org/job/ovirt-system-tests_compat-4.3-suite-master/ Build: https://jenkins.ovirt.org/job/ovirt-system-tests_compat-4.3-suite-master/409... Build Number: 409 Build Status: Still Failing Triggered By: Started by timer ------------------------------------- Changes Since Last Success: ------------------------------------- Changes for Build #398 [Galit Rosenthal] Fix alosync not recognize returning old 002_bootstrap Changes for Build #399 [Galit Rosenthal] Fix alosync not recognize returning old 002_bootstrap Changes for Build #400 [Dominik Holler] network: xfailing ovs tests Changes for Build #401 [Evgheni Dereveanchin] Move from git proto to https [Evgheni Dereveanchin] add suffix to el8 ppc64le repo names in mock configs Changes for Build #402 [Michal Skrivanek] minor compatibility and resiliency fixes [Martin Perina] mock_configs: Enable PG 12 module in EL8 mocks Changes for Build #403 [Michal Skrivanek] use a q35 custom emulated machine now Changes for Build #404 [Marcin Sobczyk] pytest: Add 'repo_server' fixture Changes for Build #405 [Galit] Remove wrong link Changes for Build #406 [Michal Skrivanek] javapackages-tools module only required on CentOS Changes for Build #407 [Michal Skrivanek] javapackages-tools module only required on CentOS Changes for Build #408 [Michal Skrivanek] javapackages-tools module only required on CentOS Changes for Build #409 [Michal Skrivanek] javapackages-tools module only required on CentOS ----------------- Failed Tests: ----------------- 1 tests failed. FAILED: basic-suite-master.test-scenarios.004_basic_sanity_pytest.test_ha_recovery Error Message: AssertionError: False != True after 600 seconds Stack Trace: prefix = <ovirtlago.prefix.OvirtPrefix object at 0x7f624f603150> @pytest.mark.run(order=18) def test_ha_recovery(prefix): engine = prefix.virt_env.engine_vm().get_api_v4().system_service() with test_utils.TestEvent(engine, [119, 9602, 506]): # VM_DOWN_ERROR event(119) # HA_VM_FAILED event event(9602) # VDS_INITIATED_RUN_VM event(506) vm_host = _vm_host(prefix, VM2_NAME) pid = vm_host.ssh(['pgrep', '-f', 'qemu.*guest=vm2']) vm_host.ssh(['kill', '-KILL', pid.out.decode('utf-8')]) vm_service = test_utils.get_vm_service(engine, VM2_NAME) testlib.assert_true_within_long(
lambda:
vm_service.get().status == types.VmStatus.UP ) ../basic-suite-master/test-scenarios/004_basic_sanity_pytest.py:739: _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ /usr/lib/python2.7/site-packages/ovirtlago/testlib.py:286: in assert_true_within_long assert_equals_within_long(func, True, allowed_exceptions) /usr/lib/python2.7/site-packages/ovirtlago/testlib.py:273: in assert_equals_within_long func, value, LONG_TIMEOUT, allowed_exceptions=allowed_exceptions _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ func = <function <lambda> at 0x7f623fa89cf8>, value = True, timeout = 600 allowed_exceptions = [], initial_wait = 0 def assert_equals_within( func, value, timeout, allowed_exceptions=None, initial_wait=10 ): allowed_exceptions = allowed_exceptions or [] with utils.EggTimer(timeout) as timer: while not timer.elapsed(): try: res = func() if res == value: return except Exception as exc: if _instance_of_any(exc, allowed_exceptions): time.sleep(3) continue LOGGER.exception("Unhandled exception in %s", func) raise if initial_wait == 0: time.sleep(3) else: time.sleep(initial_wait) initial_wait = 0 try: raise AssertionError(
'%s != %s after %s seconds' % (res, value, timeout)
) E AssertionError: False != True after 600 seconds /usr/lib/python2.7/site-packages/ovirtlago/testlib.py:252: AssertionError
Project: https://jenkins.ovirt.org/job/ovirt-system-tests_compat-4.3-suite-master/ Build: https://jenkins.ovirt.org/job/ovirt-system-tests_compat-4.3-suite-master/410... Build Number: 410 Build Status: Still Failing Triggered By: Started by timer ------------------------------------- Changes Since Last Success: ------------------------------------- Changes for Build #399 [Galit Rosenthal] Fix alosync not recognize returning old 002_bootstrap Changes for Build #400 [Dominik Holler] network: xfailing ovs tests Changes for Build #401 [Evgheni Dereveanchin] Move from git proto to https [Evgheni Dereveanchin] add suffix to el8 ppc64le repo names in mock configs Changes for Build #402 [Michal Skrivanek] minor compatibility and resiliency fixes [Martin Perina] mock_configs: Enable PG 12 module in EL8 mocks Changes for Build #403 [Michal Skrivanek] use a q35 custom emulated machine now Changes for Build #404 [Marcin Sobczyk] pytest: Add 'repo_server' fixture Changes for Build #405 [Galit] Remove wrong link Changes for Build #406 [Michal Skrivanek] javapackages-tools module only required on CentOS Changes for Build #407 [Michal Skrivanek] javapackages-tools module only required on CentOS Changes for Build #408 [Michal Skrivanek] javapackages-tools module only required on CentOS Changes for Build #409 [Michal Skrivanek] javapackages-tools module only required on CentOS Changes for Build #410 [Marcin Sobczyk] basic: Use 'order_by' in 004_basic_sanity tests [Evgheni Dereveanchin] Only run mirror cleanup if a sync is needed ----------------- Failed Tests: ----------------- 1 tests failed. FAILED: basic-suite-master.test-scenarios.004_basic_sanity_pytest.test_ha_recovery Error Message: AssertionError: False != True after 600 seconds Stack Trace: prefix = <ovirtlago.prefix.OvirtPrefix object at 0x7fca8ec9c610> @order_by(_TEST_LIST) def test_ha_recovery(prefix): engine = prefix.virt_env.engine_vm().get_api_v4().system_service() with test_utils.TestEvent(engine, [119, 9602, 506]): # VM_DOWN_ERROR event(119) # HA_VM_FAILED event event(9602) # VDS_INITIATED_RUN_VM event(506) vm_host = _vm_host(prefix, VM2_NAME) pid = vm_host.ssh(['pgrep', '-f', 'qemu.*guest=vm2']) vm_host.ssh(['kill', '-KILL', pid.out.decode('utf-8')]) vm_service = test_utils.get_vm_service(engine, VM2_NAME) testlib.assert_true_within_long(
lambda:
vm_service.get().status == types.VmStatus.UP ) ../basic-suite-master/test-scenarios/004_basic_sanity_pytest.py:788: _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ /usr/lib/python2.7/site-packages/ovirtlago/testlib.py:286: in assert_true_within_long assert_equals_within_long(func, True, allowed_exceptions) /usr/lib/python2.7/site-packages/ovirtlago/testlib.py:273: in assert_equals_within_long func, value, LONG_TIMEOUT, allowed_exceptions=allowed_exceptions _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ func = <function <lambda> at 0x7fca8c0958c0>, value = True, timeout = 600 allowed_exceptions = [], initial_wait = 0 def assert_equals_within( func, value, timeout, allowed_exceptions=None, initial_wait=10 ): allowed_exceptions = allowed_exceptions or [] with utils.EggTimer(timeout) as timer: while not timer.elapsed(): try: res = func() if res == value: return except Exception as exc: if _instance_of_any(exc, allowed_exceptions): time.sleep(3) continue LOGGER.exception("Unhandled exception in %s", func) raise if initial_wait == 0: time.sleep(3) else: time.sleep(initial_wait) initial_wait = 0 try: raise AssertionError(
'%s != %s after %s seconds' % (res, value, timeout)
) E AssertionError: False != True after 600 seconds /usr/lib/python2.7/site-packages/ovirtlago/testlib.py:252: AssertionError
Project: https://jenkins.ovirt.org/job/ovirt-system-tests_compat-4.3-suite-master/ Build: https://jenkins.ovirt.org/job/ovirt-system-tests_compat-4.3-suite-master/411... Build Number: 411 Build Status: Still Failing Triggered By: Started by timer ------------------------------------- Changes Since Last Success: ------------------------------------- Changes for Build #400 [Dominik Holler] network: xfailing ovs tests Changes for Build #401 [Evgheni Dereveanchin] Move from git proto to https [Evgheni Dereveanchin] add suffix to el8 ppc64le repo names in mock configs Changes for Build #402 [Michal Skrivanek] minor compatibility and resiliency fixes [Martin Perina] mock_configs: Enable PG 12 module in EL8 mocks Changes for Build #403 [Michal Skrivanek] use a q35 custom emulated machine now Changes for Build #404 [Marcin Sobczyk] pytest: Add 'repo_server' fixture Changes for Build #405 [Galit] Remove wrong link Changes for Build #406 [Michal Skrivanek] javapackages-tools module only required on CentOS Changes for Build #407 [Michal Skrivanek] javapackages-tools module only required on CentOS Changes for Build #408 [Michal Skrivanek] javapackages-tools module only required on CentOS Changes for Build #409 [Michal Skrivanek] javapackages-tools module only required on CentOS Changes for Build #410 [Marcin Sobczyk] basic: Use 'order_by' in 004_basic_sanity tests [Evgheni Dereveanchin] Only run mirror cleanup if a sync is needed Changes for Build #411 [Martin Necas] ansible: update ANSIBLE_URL_PREFIX for ansible collection ----------------- Failed Tests: ----------------- 1 tests failed. FAILED: basic-suite-master.test-scenarios.004_basic_sanity_pytest.test_ha_recovery Error Message: AssertionError: False != True after 600 seconds Stack Trace: prefix = <ovirtlago.prefix.OvirtPrefix object at 0x7f75483f9610> @order_by(_TEST_LIST) def test_ha_recovery(prefix): engine = prefix.virt_env.engine_vm().get_api_v4().system_service() with test_utils.TestEvent(engine, [119, 9602, 506]): # VM_DOWN_ERROR event(119) # HA_VM_FAILED event event(9602) # VDS_INITIATED_RUN_VM event(506) vm_host = _vm_host(prefix, VM2_NAME) pid = vm_host.ssh(['pgrep', '-f', 'qemu.*guest=vm2']) vm_host.ssh(['kill', '-KILL', pid.out.decode('utf-8')]) vm_service = test_utils.get_vm_service(engine, VM2_NAME) testlib.assert_true_within_long(
lambda:
vm_service.get().status == types.VmStatus.UP ) ../basic-suite-master/test-scenarios/004_basic_sanity_pytest.py:788: _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ /usr/lib/python2.7/site-packages/ovirtlago/testlib.py:286: in assert_true_within_long assert_equals_within_long(func, True, allowed_exceptions) /usr/lib/python2.7/site-packages/ovirtlago/testlib.py:273: in assert_equals_within_long func, value, LONG_TIMEOUT, allowed_exceptions=allowed_exceptions _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ func = <function <lambda> at 0x7f75448a89b0>, value = True, timeout = 600 allowed_exceptions = [], initial_wait = 0 def assert_equals_within( func, value, timeout, allowed_exceptions=None, initial_wait=10 ): allowed_exceptions = allowed_exceptions or [] with utils.EggTimer(timeout) as timer: while not timer.elapsed(): try: res = func() if res == value: return except Exception as exc: if _instance_of_any(exc, allowed_exceptions): time.sleep(3) continue LOGGER.exception("Unhandled exception in %s", func) raise if initial_wait == 0: time.sleep(3) else: time.sleep(initial_wait) initial_wait = 0 try: raise AssertionError(
'%s != %s after %s seconds' % (res, value, timeout)
) E AssertionError: False != True after 600 seconds /usr/lib/python2.7/site-packages/ovirtlago/testlib.py:252: AssertionError
Project: https://jenkins.ovirt.org/job/ovirt-system-tests_compat-4.3-suite-master/ Build: https://jenkins.ovirt.org/job/ovirt-system-tests_compat-4.3-suite-master/412... Build Number: 412 Build Status: Still Failing Triggered By: Started by timer ------------------------------------- Changes Since Last Success: ------------------------------------- Changes for Build #401 [Evgheni Dereveanchin] Move from git proto to https [Evgheni Dereveanchin] add suffix to el8 ppc64le repo names in mock configs Changes for Build #402 [Michal Skrivanek] minor compatibility and resiliency fixes [Martin Perina] mock_configs: Enable PG 12 module in EL8 mocks Changes for Build #403 [Michal Skrivanek] use a q35 custom emulated machine now Changes for Build #404 [Marcin Sobczyk] pytest: Add 'repo_server' fixture Changes for Build #405 [Galit] Remove wrong link Changes for Build #406 [Michal Skrivanek] javapackages-tools module only required on CentOS Changes for Build #407 [Michal Skrivanek] javapackages-tools module only required on CentOS Changes for Build #408 [Michal Skrivanek] javapackages-tools module only required on CentOS Changes for Build #409 [Michal Skrivanek] javapackages-tools module only required on CentOS Changes for Build #410 [Marcin Sobczyk] basic: Use 'order_by' in 004_basic_sanity tests [Evgheni Dereveanchin] Only run mirror cleanup if a sync is needed Changes for Build #411 [Martin Necas] ansible: update ANSIBLE_URL_PREFIX for ansible collection Changes for Build #412 [Marcin Sobczyk] selenium: Mount '/dev/shm' when using podman [Evgheni Dereveanchin] Define more CentOS 8 repo mirrors [Evgheni Dereveanchin] Fix typo in centos-extras-el8 mirror definition [Ehud Yonasi] OST: add metrics nightly suite for 4.3 [Evgheni Dereveanchin] Use repodata directory as modfyrepo path ----------------- Failed Tests: ----------------- No tests ran.
Project: https://jenkins.ovirt.org/job/ovirt-system-tests_compat-4.3-suite-master/ Build: https://jenkins.ovirt.org/job/ovirt-system-tests_compat-4.3-suite-master/413... Build Number: 413 Build Status: Still Failing Triggered By: Started by timer ------------------------------------- Changes Since Last Success: ------------------------------------- Changes for Build #402 [Michal Skrivanek] minor compatibility and resiliency fixes [Martin Perina] mock_configs: Enable PG 12 module in EL8 mocks Changes for Build #403 [Michal Skrivanek] use a q35 custom emulated machine now Changes for Build #404 [Marcin Sobczyk] pytest: Add 'repo_server' fixture Changes for Build #405 [Galit] Remove wrong link Changes for Build #406 [Michal Skrivanek] javapackages-tools module only required on CentOS Changes for Build #407 [Michal Skrivanek] javapackages-tools module only required on CentOS Changes for Build #408 [Michal Skrivanek] javapackages-tools module only required on CentOS Changes for Build #409 [Michal Skrivanek] javapackages-tools module only required on CentOS Changes for Build #410 [Marcin Sobczyk] basic: Use 'order_by' in 004_basic_sanity tests [Evgheni Dereveanchin] Only run mirror cleanup if a sync is needed Changes for Build #411 [Martin Necas] ansible: update ANSIBLE_URL_PREFIX for ansible collection Changes for Build #412 [Marcin Sobczyk] selenium: Mount '/dev/shm' when using podman [Evgheni Dereveanchin] Define more CentOS 8 repo mirrors [Evgheni Dereveanchin] Fix typo in centos-extras-el8 mirror definition [Ehud Yonasi] OST: add metrics nightly suite for 4.3 [Evgheni Dereveanchin] Use repodata directory as modfyrepo path Changes for Build #413 [Marcin Sobczyk] selenium: Mount '/dev/shm' when using podman ----------------- Failed Tests: ----------------- 1 tests failed. FAILED: basic-suite-master.test-scenarios.004_basic_sanity_pytest.test_ha_recovery Error Message: AssertionError: False != True after 600 seconds Stack Trace: prefix = <ovirtlago.prefix.OvirtPrefix object at 0x7f19438df610> @order_by(_TEST_LIST) def test_ha_recovery(prefix): engine = prefix.virt_env.engine_vm().get_api_v4().system_service() with test_utils.TestEvent(engine, [119, 9602, 506]): # VM_DOWN_ERROR event(119) # HA_VM_FAILED event event(9602) # VDS_INITIATED_RUN_VM event(506) vm_host = _vm_host(prefix, VM2_NAME) pid = vm_host.ssh(['pgrep', '-f', 'qemu.*guest=vm2']) vm_host.ssh(['kill', '-KILL', pid.out.decode('utf-8')]) vm_service = test_utils.get_vm_service(engine, VM2_NAME) testlib.assert_true_within_long(
lambda:
vm_service.get().status == types.VmStatus.UP ) ../basic-suite-master/test-scenarios/004_basic_sanity_pytest.py:788: _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ /usr/lib/python2.7/site-packages/ovirtlago/testlib.py:286: in assert_true_within_long assert_equals_within_long(func, True, allowed_exceptions) /usr/lib/python2.7/site-packages/ovirtlago/testlib.py:273: in assert_equals_within_long func, value, LONG_TIMEOUT, allowed_exceptions=allowed_exceptions _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ func = <function <lambda> at 0x7f19400bbed8>, value = True, timeout = 600 allowed_exceptions = [], initial_wait = 0 def assert_equals_within( func, value, timeout, allowed_exceptions=None, initial_wait=10 ): allowed_exceptions = allowed_exceptions or [] with utils.EggTimer(timeout) as timer: while not timer.elapsed(): try: res = func() if res == value: return except Exception as exc: if _instance_of_any(exc, allowed_exceptions): time.sleep(3) continue LOGGER.exception("Unhandled exception in %s", func) raise if initial_wait == 0: time.sleep(3) else: time.sleep(initial_wait) initial_wait = 0 try: raise AssertionError(
'%s != %s after %s seconds' % (res, value, timeout)
) E AssertionError: False != True after 600 seconds /usr/lib/python2.7/site-packages/ovirtlago/testlib.py:252: AssertionError
Project: https://jenkins.ovirt.org/job/ovirt-system-tests_compat-4.3-suite-master/ Build: https://jenkins.ovirt.org/job/ovirt-system-tests_compat-4.3-suite-master/414... Build Number: 414 Build Status: Still Failing Triggered By: Started by timer ------------------------------------- Changes Since Last Success: ------------------------------------- Changes for Build #403 [Michal Skrivanek] use a q35 custom emulated machine now Changes for Build #404 [Marcin Sobczyk] pytest: Add 'repo_server' fixture Changes for Build #405 [Galit] Remove wrong link Changes for Build #406 [Michal Skrivanek] javapackages-tools module only required on CentOS Changes for Build #407 [Michal Skrivanek] javapackages-tools module only required on CentOS Changes for Build #408 [Michal Skrivanek] javapackages-tools module only required on CentOS Changes for Build #409 [Michal Skrivanek] javapackages-tools module only required on CentOS Changes for Build #410 [Marcin Sobczyk] basic: Use 'order_by' in 004_basic_sanity tests [Evgheni Dereveanchin] Only run mirror cleanup if a sync is needed Changes for Build #411 [Martin Necas] ansible: update ANSIBLE_URL_PREFIX for ansible collection Changes for Build #412 [Marcin Sobczyk] selenium: Mount '/dev/shm' when using podman [Evgheni Dereveanchin] Define more CentOS 8 repo mirrors [Evgheni Dereveanchin] Fix typo in centos-extras-el8 mirror definition [Ehud Yonasi] OST: add metrics nightly suite for 4.3 [Evgheni Dereveanchin] Use repodata directory as modfyrepo path Changes for Build #413 [Marcin Sobczyk] selenium: Mount '/dev/shm' when using podman Changes for Build #414 [Marcin Sobczyk] selenium: Mount '/dev/shm' when using podman ----------------- Failed Tests: ----------------- 1 tests failed. FAILED: basic-suite-master.test-scenarios.004_basic_sanity_pytest.test_ha_recovery Error Message: AssertionError: False != True after 600 seconds Stack Trace: prefix = <ovirtlago.prefix.OvirtPrefix object at 0x7f6460a4f610> @order_by(_TEST_LIST) def test_ha_recovery(prefix): engine = prefix.virt_env.engine_vm().get_api_v4().system_service() with test_utils.TestEvent(engine, [119, 9602, 506]): # VM_DOWN_ERROR event(119) # HA_VM_FAILED event event(9602) # VDS_INITIATED_RUN_VM event(506) vm_host = _vm_host(prefix, VM2_NAME) pid = vm_host.ssh(['pgrep', '-f', 'qemu.*guest=vm2']) vm_host.ssh(['kill', '-KILL', pid.out.decode('utf-8')]) vm_service = test_utils.get_vm_service(engine, VM2_NAME) testlib.assert_true_within_long(
lambda:
vm_service.get().status == types.VmStatus.UP ) ../basic-suite-master/test-scenarios/004_basic_sanity_pytest.py:788: _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ /usr/lib/python2.7/site-packages/ovirtlago/testlib.py:286: in assert_true_within_long assert_equals_within_long(func, True, allowed_exceptions) /usr/lib/python2.7/site-packages/ovirtlago/testlib.py:273: in assert_equals_within_long func, value, LONG_TIMEOUT, allowed_exceptions=allowed_exceptions _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ func = <function <lambda> at 0x7f645cf449b0>, value = True, timeout = 600 allowed_exceptions = [], initial_wait = 0 def assert_equals_within( func, value, timeout, allowed_exceptions=None, initial_wait=10 ): allowed_exceptions = allowed_exceptions or [] with utils.EggTimer(timeout) as timer: while not timer.elapsed(): try: res = func() if res == value: return except Exception as exc: if _instance_of_any(exc, allowed_exceptions): time.sleep(3) continue LOGGER.exception("Unhandled exception in %s", func) raise if initial_wait == 0: time.sleep(3) else: time.sleep(initial_wait) initial_wait = 0 try: raise AssertionError(
'%s != %s after %s seconds' % (res, value, timeout)
) E AssertionError: False != True after 600 seconds /usr/lib/python2.7/site-packages/ovirtlago/testlib.py:252: AssertionError
Project: https://jenkins.ovirt.org/job/ovirt-system-tests_compat-4.3-suite-master/ Build: https://jenkins.ovirt.org/job/ovirt-system-tests_compat-4.3-suite-master/415... Build Number: 415 Build Status: Still Failing Triggered By: Started by timer ------------------------------------- Changes Since Last Success: ------------------------------------- Changes for Build #404 [Marcin Sobczyk] pytest: Add 'repo_server' fixture Changes for Build #405 [Galit] Remove wrong link Changes for Build #406 [Michal Skrivanek] javapackages-tools module only required on CentOS Changes for Build #407 [Michal Skrivanek] javapackages-tools module only required on CentOS Changes for Build #408 [Michal Skrivanek] javapackages-tools module only required on CentOS Changes for Build #409 [Michal Skrivanek] javapackages-tools module only required on CentOS Changes for Build #410 [Marcin Sobczyk] basic: Use 'order_by' in 004_basic_sanity tests [Evgheni Dereveanchin] Only run mirror cleanup if a sync is needed Changes for Build #411 [Martin Necas] ansible: update ANSIBLE_URL_PREFIX for ansible collection Changes for Build #412 [Marcin Sobczyk] selenium: Mount '/dev/shm' when using podman [Evgheni Dereveanchin] Define more CentOS 8 repo mirrors [Evgheni Dereveanchin] Fix typo in centos-extras-el8 mirror definition [Ehud Yonasi] OST: add metrics nightly suite for 4.3 [Evgheni Dereveanchin] Use repodata directory as modfyrepo path Changes for Build #413 [Marcin Sobczyk] selenium: Mount '/dev/shm' when using podman Changes for Build #414 [Marcin Sobczyk] selenium: Mount '/dev/shm' when using podman Changes for Build #415 [Marcin Sobczyk] selenium: Mount '/dev/shm' when using podman ----------------- Failed Tests: ----------------- 1 tests failed. FAILED: basic-suite-master.test-scenarios.004_basic_sanity_pytest.test_ha_recovery Error Message: AssertionError: False != True after 600 seconds Stack Trace: prefix = <ovirtlago.prefix.OvirtPrefix object at 0x7f044148f610> @order_by(_TEST_LIST) def test_ha_recovery(prefix): engine = prefix.virt_env.engine_vm().get_api_v4().system_service() with test_utils.TestEvent(engine, [119, 9602, 506]): # VM_DOWN_ERROR event(119) # HA_VM_FAILED event event(9602) # VDS_INITIATED_RUN_VM event(506) vm_host = _vm_host(prefix, VM2_NAME) pid = vm_host.ssh(['pgrep', '-f', 'qemu.*guest=vm2']) vm_host.ssh(['kill', '-KILL', pid.out.decode('utf-8')]) vm_service = test_utils.get_vm_service(engine, VM2_NAME) testlib.assert_true_within_long(
lambda:
vm_service.get().status == types.VmStatus.UP ) ../basic-suite-master/test-scenarios/004_basic_sanity_pytest.py:788: _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ /usr/lib/python2.7/site-packages/ovirtlago/testlib.py:286: in assert_true_within_long assert_equals_within_long(func, True, allowed_exceptions) /usr/lib/python2.7/site-packages/ovirtlago/testlib.py:273: in assert_equals_within_long func, value, LONG_TIMEOUT, allowed_exceptions=allowed_exceptions _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ func = <function <lambda> at 0x7f043d93ded8>, value = True, timeout = 600 allowed_exceptions = [], initial_wait = 0 def assert_equals_within( func, value, timeout, allowed_exceptions=None, initial_wait=10 ): allowed_exceptions = allowed_exceptions or [] with utils.EggTimer(timeout) as timer: while not timer.elapsed(): try: res = func() if res == value: return except Exception as exc: if _instance_of_any(exc, allowed_exceptions): time.sleep(3) continue LOGGER.exception("Unhandled exception in %s", func) raise if initial_wait == 0: time.sleep(3) else: time.sleep(initial_wait) initial_wait = 0 try: raise AssertionError(
'%s != %s after %s seconds' % (res, value, timeout)
) E AssertionError: False != True after 600 seconds /usr/lib/python2.7/site-packages/ovirtlago/testlib.py:252: AssertionError
Project: https://jenkins.ovirt.org/job/ovirt-system-tests_compat-4.3-suite-master/ Build: https://jenkins.ovirt.org/job/ovirt-system-tests_compat-4.3-suite-master/416... Build Number: 416 Build Status: Still Failing Triggered By: Started by timer ------------------------------------- Changes Since Last Success: ------------------------------------- Changes for Build #405 [Galit] Remove wrong link Changes for Build #406 [Michal Skrivanek] javapackages-tools module only required on CentOS Changes for Build #407 [Michal Skrivanek] javapackages-tools module only required on CentOS Changes for Build #408 [Michal Skrivanek] javapackages-tools module only required on CentOS Changes for Build #409 [Michal Skrivanek] javapackages-tools module only required on CentOS Changes for Build #410 [Marcin Sobczyk] basic: Use 'order_by' in 004_basic_sanity tests [Evgheni Dereveanchin] Only run mirror cleanup if a sync is needed Changes for Build #411 [Martin Necas] ansible: update ANSIBLE_URL_PREFIX for ansible collection Changes for Build #412 [Marcin Sobczyk] selenium: Mount '/dev/shm' when using podman [Evgheni Dereveanchin] Define more CentOS 8 repo mirrors [Evgheni Dereveanchin] Fix typo in centos-extras-el8 mirror definition [Ehud Yonasi] OST: add metrics nightly suite for 4.3 [Evgheni Dereveanchin] Use repodata directory as modfyrepo path Changes for Build #413 [Marcin Sobczyk] selenium: Mount '/dev/shm' when using podman Changes for Build #414 [Marcin Sobczyk] selenium: Mount '/dev/shm' when using podman Changes for Build #415 [Marcin Sobczyk] selenium: Mount '/dev/shm' when using podman Changes for Build #416 [Galit Rosenthal] Use specific LagoinitFile for hc suite 4.3 el7 [Evgeny Slutsky] Add eslutsky@redhat.com to jenkins Recipient lists. ----------------- Failed Tests: ----------------- No tests ran.
Project: https://jenkins.ovirt.org/job/ovirt-system-tests_compat-4.3-suite-master/ Build: https://jenkins.ovirt.org/job/ovirt-system-tests_compat-4.3-suite-master/417... Build Number: 417 Build Status: Still Failing Triggered By: Started by timer ------------------------------------- Changes Since Last Success: ------------------------------------- Changes for Build #407 [Michal Skrivanek] javapackages-tools module only required on CentOS Changes for Build #408 [Michal Skrivanek] javapackages-tools module only required on CentOS Changes for Build #409 [Michal Skrivanek] javapackages-tools module only required on CentOS Changes for Build #410 [Marcin Sobczyk] basic: Use 'order_by' in 004_basic_sanity tests [Evgheni Dereveanchin] Only run mirror cleanup if a sync is needed Changes for Build #411 [Martin Necas] ansible: update ANSIBLE_URL_PREFIX for ansible collection Changes for Build #412 [Marcin Sobczyk] selenium: Mount '/dev/shm' when using podman [Evgheni Dereveanchin] Define more CentOS 8 repo mirrors [Evgheni Dereveanchin] Fix typo in centos-extras-el8 mirror definition [Ehud Yonasi] OST: add metrics nightly suite for 4.3 [Evgheni Dereveanchin] Use repodata directory as modfyrepo path Changes for Build #413 [Marcin Sobczyk] selenium: Mount '/dev/shm' when using podman Changes for Build #414 [Marcin Sobczyk] selenium: Mount '/dev/shm' when using podman Changes for Build #415 [Marcin Sobczyk] selenium: Mount '/dev/shm' when using podman Changes for Build #416 [Galit Rosenthal] Use specific LagoinitFile for hc suite 4.3 el7 [Evgeny Slutsky] Add eslutsky@redhat.com to jenkins Recipient lists. Changes for Build #417 [Dominik Holler] Revert "network, ovirtlib: Set custom_emulated_machine" ----------------- Failed Tests: ----------------- 1 tests failed. FAILED: basic-suite-master.test-scenarios.004_basic_sanity_pytest.test_ha_recovery Error Message: AssertionError: False != True after 600 seconds Stack Trace: prefix = <ovirtlago.prefix.OvirtPrefix object at 0x7f81f6703610> @order_by(_TEST_LIST) def test_ha_recovery(prefix): engine = prefix.virt_env.engine_vm().get_api_v4().system_service() with test_utils.TestEvent(engine, [119, 9602, 506]): # VM_DOWN_ERROR event(119) # HA_VM_FAILED event event(9602) # VDS_INITIATED_RUN_VM event(506) vm_host = _vm_host(prefix, VM2_NAME) pid = vm_host.ssh(['pgrep', '-f', 'qemu.*guest=vm2']) vm_host.ssh(['kill', '-KILL', pid.out.decode('utf-8')]) vm_service = test_utils.get_vm_service(engine, VM2_NAME) testlib.assert_true_within_long(
lambda:
vm_service.get().status == types.VmStatus.UP ) ../basic-suite-master/test-scenarios/004_basic_sanity_pytest.py:788: _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ /usr/lib/python2.7/site-packages/ovirtlago/testlib.py:286: in assert_true_within_long assert_equals_within_long(func, True, allowed_exceptions) /usr/lib/python2.7/site-packages/ovirtlago/testlib.py:273: in assert_equals_within_long func, value, LONG_TIMEOUT, allowed_exceptions=allowed_exceptions _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ func = <function <lambda> at 0x7f81f410a9b0>, value = True, timeout = 600 allowed_exceptions = [], initial_wait = 0 def assert_equals_within( func, value, timeout, allowed_exceptions=None, initial_wait=10 ): allowed_exceptions = allowed_exceptions or [] with utils.EggTimer(timeout) as timer: while not timer.elapsed(): try: res = func() if res == value: return except Exception as exc: if _instance_of_any(exc, allowed_exceptions): time.sleep(3) continue LOGGER.exception("Unhandled exception in %s", func) raise if initial_wait == 0: time.sleep(3) else: time.sleep(initial_wait) initial_wait = 0 try: raise AssertionError(
'%s != %s after %s seconds' % (res, value, timeout)
) E AssertionError: False != True after 600 seconds /usr/lib/python2.7/site-packages/ovirtlago/testlib.py:252: AssertionError
Project: https://jenkins.ovirt.org/job/ovirt-system-tests_compat-4.3-suite-master/ Build: https://jenkins.ovirt.org/job/ovirt-system-tests_compat-4.3-suite-master/418... Build Number: 418 Build Status: Still Failing Triggered By: Started by timer ------------------------------------- Changes Since Last Success: ------------------------------------- Changes for Build #408 [Michal Skrivanek] javapackages-tools module only required on CentOS Changes for Build #409 [Michal Skrivanek] javapackages-tools module only required on CentOS Changes for Build #410 [Marcin Sobczyk] basic: Use 'order_by' in 004_basic_sanity tests [Evgheni Dereveanchin] Only run mirror cleanup if a sync is needed Changes for Build #411 [Martin Necas] ansible: update ANSIBLE_URL_PREFIX for ansible collection Changes for Build #412 [Marcin Sobczyk] selenium: Mount '/dev/shm' when using podman [Evgheni Dereveanchin] Define more CentOS 8 repo mirrors [Evgheni Dereveanchin] Fix typo in centos-extras-el8 mirror definition [Ehud Yonasi] OST: add metrics nightly suite for 4.3 [Evgheni Dereveanchin] Use repodata directory as modfyrepo path Changes for Build #413 [Marcin Sobczyk] selenium: Mount '/dev/shm' when using podman Changes for Build #414 [Marcin Sobczyk] selenium: Mount '/dev/shm' when using podman Changes for Build #415 [Marcin Sobczyk] selenium: Mount '/dev/shm' when using podman Changes for Build #416 [Galit Rosenthal] Use specific LagoinitFile for hc suite 4.3 el7 [Evgeny Slutsky] Add eslutsky@redhat.com to jenkins Recipient lists. Changes for Build #417 [Dominik Holler] Revert "network, ovirtlib: Set custom_emulated_machine" Changes for Build #418 [Yedidyah Bar David] HE suites: verify_engine_backup [Sandro Bonazzola] ovirt-engine: drop jobs for legacy versions ----------------- Failed Tests: ----------------- 1 tests failed. FAILED: basic-suite-master.test-scenarios.004_basic_sanity_pytest.test_ha_recovery Error Message: AssertionError: False != True after 600 seconds Stack Trace: prefix = <ovirtlago.prefix.OvirtPrefix object at 0x7f438e95b610> @order_by(_TEST_LIST) def test_ha_recovery(prefix): engine = prefix.virt_env.engine_vm().get_api_v4().system_service() with test_utils.TestEvent(engine, [119, 9602, 506]): # VM_DOWN_ERROR event(119) # HA_VM_FAILED event event(9602) # VDS_INITIATED_RUN_VM event(506) vm_host = _vm_host(prefix, VM2_NAME) pid = vm_host.ssh(['pgrep', '-f', 'qemu.*guest=vm2']) vm_host.ssh(['kill', '-KILL', pid.out.decode('utf-8')]) vm_service = test_utils.get_vm_service(engine, VM2_NAME) testlib.assert_true_within_long(
lambda:
vm_service.get().status == types.VmStatus.UP ) ../basic-suite-master/test-scenarios/004_basic_sanity_pytest.py:788: _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ /usr/lib/python2.7/site-packages/ovirtlago/testlib.py:286: in assert_true_within_long assert_equals_within_long(func, True, allowed_exceptions) /usr/lib/python2.7/site-packages/ovirtlago/testlib.py:273: in assert_equals_within_long func, value, LONG_TIMEOUT, allowed_exceptions=allowed_exceptions _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ func = <function <lambda> at 0x7f438c1296e0>, value = True, timeout = 600 allowed_exceptions = [], initial_wait = 0 def assert_equals_within( func, value, timeout, allowed_exceptions=None, initial_wait=10 ): allowed_exceptions = allowed_exceptions or [] with utils.EggTimer(timeout) as timer: while not timer.elapsed(): try: res = func() if res == value: return except Exception as exc: if _instance_of_any(exc, allowed_exceptions): time.sleep(3) continue LOGGER.exception("Unhandled exception in %s", func) raise if initial_wait == 0: time.sleep(3) else: time.sleep(initial_wait) initial_wait = 0 try: raise AssertionError(
'%s != %s after %s seconds' % (res, value, timeout)
) E AssertionError: False != True after 600 seconds /usr/lib/python2.7/site-packages/ovirtlago/testlib.py:252: AssertionError
Project: https://jenkins.ovirt.org/job/ovirt-system-tests_compat-4.3-suite-master/ Build: https://jenkins.ovirt.org/job/ovirt-system-tests_compat-4.3-suite-master/419... Build Number: 419 Build Status: Still Failing Triggered By: Started by timer ------------------------------------- Changes Since Last Success: ------------------------------------- Changes for Build #409 [Michal Skrivanek] javapackages-tools module only required on CentOS Changes for Build #410 [Marcin Sobczyk] basic: Use 'order_by' in 004_basic_sanity tests [Evgheni Dereveanchin] Only run mirror cleanup if a sync is needed Changes for Build #411 [Martin Necas] ansible: update ANSIBLE_URL_PREFIX for ansible collection Changes for Build #412 [Marcin Sobczyk] selenium: Mount '/dev/shm' when using podman [Evgheni Dereveanchin] Define more CentOS 8 repo mirrors [Evgheni Dereveanchin] Fix typo in centos-extras-el8 mirror definition [Ehud Yonasi] OST: add metrics nightly suite for 4.3 [Evgheni Dereveanchin] Use repodata directory as modfyrepo path Changes for Build #413 [Marcin Sobczyk] selenium: Mount '/dev/shm' when using podman Changes for Build #414 [Marcin Sobczyk] selenium: Mount '/dev/shm' when using podman Changes for Build #415 [Marcin Sobczyk] selenium: Mount '/dev/shm' when using podman Changes for Build #416 [Galit Rosenthal] Use specific LagoinitFile for hc suite 4.3 el7 [Evgeny Slutsky] Add eslutsky@redhat.com to jenkins Recipient lists. Changes for Build #417 [Dominik Holler] Revert "network, ovirtlib: Set custom_emulated_machine" Changes for Build #418 [Yedidyah Bar David] HE suites: verify_engine_backup [Sandro Bonazzola] ovirt-engine: drop jobs for legacy versions Changes for Build #419 [Benny Zlotnik] poll update disk [Shlomi] Added ability to ignore other repos when sync_mirror runs ----------------- Failed Tests: ----------------- 1 tests failed. FAILED: basic-suite-master.test-scenarios.004_basic_sanity_pytest.test_ha_recovery Error Message: AssertionError: False != True after 600 seconds Stack Trace: prefix = <ovirtlago.prefix.OvirtPrefix object at 0x7f0f84714590> @order_by(_TEST_LIST) def test_ha_recovery(prefix): engine = prefix.virt_env.engine_vm().get_api_v4().system_service() with test_utils.TestEvent(engine, [119, 9602, 506]): # VM_DOWN_ERROR event(119) # HA_VM_FAILED event event(9602) # VDS_INITIATED_RUN_VM event(506) vm_host = _vm_host(prefix, VM2_NAME) pid = vm_host.ssh(['pgrep', '-f', 'qemu.*guest=vm2']) vm_host.ssh(['kill', '-KILL', pid.out.decode('utf-8')]) vm_service = test_utils.get_vm_service(engine, VM2_NAME) testlib.assert_true_within_long(
lambda:
vm_service.get().status == types.VmStatus.UP ) ../basic-suite-master/test-scenarios/004_basic_sanity_pytest.py:788: _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ /usr/lib/python2.7/site-packages/ovirtlago/testlib.py:286: in assert_true_within_long assert_equals_within_long(func, True, allowed_exceptions) /usr/lib/python2.7/site-packages/ovirtlago/testlib.py:273: in assert_equals_within_long func, value, LONG_TIMEOUT, allowed_exceptions=allowed_exceptions _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ func = <function <lambda> at 0x7f0f80fc69b0>, value = True, timeout = 600 allowed_exceptions = [], initial_wait = 0 def assert_equals_within( func, value, timeout, allowed_exceptions=None, initial_wait=10 ): allowed_exceptions = allowed_exceptions or [] with utils.EggTimer(timeout) as timer: while not timer.elapsed(): try: res = func() if res == value: return except Exception as exc: if _instance_of_any(exc, allowed_exceptions): time.sleep(3) continue LOGGER.exception("Unhandled exception in %s", func) raise if initial_wait == 0: time.sleep(3) else: time.sleep(initial_wait) initial_wait = 0 try: raise AssertionError(
'%s != %s after %s seconds' % (res, value, timeout)
) E AssertionError: False != True after 600 seconds /usr/lib/python2.7/site-packages/ovirtlago/testlib.py:252: AssertionError
Project: https://jenkins.ovirt.org/job/ovirt-system-tests_compat-4.3-suite-master/ Build: https://jenkins.ovirt.org/job/ovirt-system-tests_compat-4.3-suite-master/420... Build Number: 420 Build Status: Still Failing Triggered By: Started by timer ------------------------------------- Changes Since Last Success: ------------------------------------- Changes for Build #410 [Marcin Sobczyk] basic: Use 'order_by' in 004_basic_sanity tests [Evgheni Dereveanchin] Only run mirror cleanup if a sync is needed Changes for Build #411 [Martin Necas] ansible: update ANSIBLE_URL_PREFIX for ansible collection Changes for Build #412 [Marcin Sobczyk] selenium: Mount '/dev/shm' when using podman [Evgheni Dereveanchin] Define more CentOS 8 repo mirrors [Evgheni Dereveanchin] Fix typo in centos-extras-el8 mirror definition [Ehud Yonasi] OST: add metrics nightly suite for 4.3 [Evgheni Dereveanchin] Use repodata directory as modfyrepo path Changes for Build #413 [Marcin Sobczyk] selenium: Mount '/dev/shm' when using podman Changes for Build #414 [Marcin Sobczyk] selenium: Mount '/dev/shm' when using podman Changes for Build #415 [Marcin Sobczyk] selenium: Mount '/dev/shm' when using podman Changes for Build #416 [Galit Rosenthal] Use specific LagoinitFile for hc suite 4.3 el7 [Evgeny Slutsky] Add eslutsky@redhat.com to jenkins Recipient lists. Changes for Build #417 [Dominik Holler] Revert "network, ovirtlib: Set custom_emulated_machine" Changes for Build #418 [Yedidyah Bar David] HE suites: verify_engine_backup [Sandro Bonazzola] ovirt-engine: drop jobs for legacy versions Changes for Build #419 [Benny Zlotnik] poll update disk [Shlomi] Added ability to ignore other repos when sync_mirror runs Changes for Build #420 [Michal Skrivanek] we really should be installing vmconsole ----------------- Failed Tests: ----------------- 1 tests failed. FAILED: basic-suite-master.test-scenarios.004_basic_sanity_pytest.test_ha_recovery Error Message: AssertionError: False != True after 600 seconds Stack Trace: prefix = <ovirtlago.prefix.OvirtPrefix object at 0x7f80c186d590> @order_by(_TEST_LIST) def test_ha_recovery(prefix): engine = prefix.virt_env.engine_vm().get_api_v4().system_service() with test_utils.TestEvent(engine, [119, 9602, 506]): # VM_DOWN_ERROR event(119) # HA_VM_FAILED event event(9602) # VDS_INITIATED_RUN_VM event(506) vm_host = _vm_host(prefix, VM2_NAME) pid = vm_host.ssh(['pgrep', '-f', 'qemu.*guest=vm2']) vm_host.ssh(['kill', '-KILL', pid.out.decode('utf-8')]) vm_service = test_utils.get_vm_service(engine, VM2_NAME) testlib.assert_true_within_long(
lambda:
vm_service.get().status == types.VmStatus.UP ) ../basic-suite-master/test-scenarios/004_basic_sanity_pytest.py:788: _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ /usr/lib/python2.7/site-packages/ovirtlago/testlib.py:286: in assert_true_within_long assert_equals_within_long(func, True, allowed_exceptions) /usr/lib/python2.7/site-packages/ovirtlago/testlib.py:273: in assert_equals_within_long func, value, LONG_TIMEOUT, allowed_exceptions=allowed_exceptions _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ func = <function <lambda> at 0x7f80ba117c80>, value = True, timeout = 600 allowed_exceptions = [], initial_wait = 0 def assert_equals_within( func, value, timeout, allowed_exceptions=None, initial_wait=10 ): allowed_exceptions = allowed_exceptions or [] with utils.EggTimer(timeout) as timer: while not timer.elapsed(): try: res = func() if res == value: return except Exception as exc: if _instance_of_any(exc, allowed_exceptions): time.sleep(3) continue LOGGER.exception("Unhandled exception in %s", func) raise if initial_wait == 0: time.sleep(3) else: time.sleep(initial_wait) initial_wait = 0 try: raise AssertionError(
'%s != %s after %s seconds' % (res, value, timeout)
) E AssertionError: False != True after 600 seconds /usr/lib/python2.7/site-packages/ovirtlago/testlib.py:252: AssertionError
Project: https://jenkins.ovirt.org/job/ovirt-system-tests_compat-4.3-suite-master/ Build: https://jenkins.ovirt.org/job/ovirt-system-tests_compat-4.3-suite-master/421... Build Number: 421 Build Status: Still Failing Triggered By: Started by timer ------------------------------------- Changes Since Last Success: ------------------------------------- Changes for Build #411 [Martin Necas] ansible: update ANSIBLE_URL_PREFIX for ansible collection Changes for Build #412 [Marcin Sobczyk] selenium: Mount '/dev/shm' when using podman [Evgheni Dereveanchin] Define more CentOS 8 repo mirrors [Evgheni Dereveanchin] Fix typo in centos-extras-el8 mirror definition [Ehud Yonasi] OST: add metrics nightly suite for 4.3 [Evgheni Dereveanchin] Use repodata directory as modfyrepo path Changes for Build #413 [Marcin Sobczyk] selenium: Mount '/dev/shm' when using podman Changes for Build #414 [Marcin Sobczyk] selenium: Mount '/dev/shm' when using podman Changes for Build #415 [Marcin Sobczyk] selenium: Mount '/dev/shm' when using podman Changes for Build #416 [Galit Rosenthal] Use specific LagoinitFile for hc suite 4.3 el7 [Evgeny Slutsky] Add eslutsky@redhat.com to jenkins Recipient lists. Changes for Build #417 [Dominik Holler] Revert "network, ovirtlib: Set custom_emulated_machine" Changes for Build #418 [Yedidyah Bar David] HE suites: verify_engine_backup [Sandro Bonazzola] ovirt-engine: drop jobs for legacy versions Changes for Build #419 [Benny Zlotnik] poll update disk [Shlomi] Added ability to ignore other repos when sync_mirror runs Changes for Build #420 [Michal Skrivanek] we really should be installing vmconsole Changes for Build #421 [Michal Skrivanek] we really should be installing vmconsole ----------------- Failed Tests: ----------------- 1 tests failed. FAILED: basic-suite-master.test-scenarios.004_basic_sanity_pytest.test_ha_recovery Error Message: AssertionError: False != True after 600 seconds Stack Trace: prefix = <ovirtlago.prefix.OvirtPrefix object at 0x7f2b38c04590> @order_by(_TEST_LIST) def test_ha_recovery(prefix): engine = prefix.virt_env.engine_vm().get_api_v4().system_service() with test_utils.TestEvent(engine, [119, 9602, 506]): # VM_DOWN_ERROR event(119) # HA_VM_FAILED event event(9602) # VDS_INITIATED_RUN_VM event(506) vm_host = _vm_host(prefix, VM2_NAME) pid = vm_host.ssh(['pgrep', '-f', 'qemu.*guest=vm2']) vm_host.ssh(['kill', '-KILL', pid.out.decode('utf-8')]) vm_service = test_utils.get_vm_service(engine, VM2_NAME) testlib.assert_true_within_long(
lambda:
vm_service.get().status == types.VmStatus.UP ) ../basic-suite-master/test-scenarios/004_basic_sanity_pytest.py:788: _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ /usr/lib/python2.7/site-packages/ovirtlago/testlib.py:286: in assert_true_within_long assert_equals_within_long(func, True, allowed_exceptions) /usr/lib/python2.7/site-packages/ovirtlago/testlib.py:273: in assert_equals_within_long func, value, LONG_TIMEOUT, allowed_exceptions=allowed_exceptions _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ func = <function <lambda> at 0x7f2b354a7758>, value = True, timeout = 600 allowed_exceptions = [], initial_wait = 0 def assert_equals_within( func, value, timeout, allowed_exceptions=None, initial_wait=10 ): allowed_exceptions = allowed_exceptions or [] with utils.EggTimer(timeout) as timer: while not timer.elapsed(): try: res = func() if res == value: return except Exception as exc: if _instance_of_any(exc, allowed_exceptions): time.sleep(3) continue LOGGER.exception("Unhandled exception in %s", func) raise if initial_wait == 0: time.sleep(3) else: time.sleep(initial_wait) initial_wait = 0 try: raise AssertionError(
'%s != %s after %s seconds' % (res, value, timeout)
) E AssertionError: False != True after 600 seconds /usr/lib/python2.7/site-packages/ovirtlago/testlib.py:252: AssertionError
Project: https://jenkins.ovirt.org/job/ovirt-system-tests_compat-4.3-suite-master/ Build: https://jenkins.ovirt.org/job/ovirt-system-tests_compat-4.3-suite-master/422... Build Number: 422 Build Status: Still Failing Triggered By: Started by timer ------------------------------------- Changes Since Last Success: ------------------------------------- Changes for Build #412 [Marcin Sobczyk] selenium: Mount '/dev/shm' when using podman [Evgheni Dereveanchin] Define more CentOS 8 repo mirrors [Evgheni Dereveanchin] Fix typo in centos-extras-el8 mirror definition [Ehud Yonasi] OST: add metrics nightly suite for 4.3 [Evgheni Dereveanchin] Use repodata directory as modfyrepo path Changes for Build #413 [Marcin Sobczyk] selenium: Mount '/dev/shm' when using podman Changes for Build #414 [Marcin Sobczyk] selenium: Mount '/dev/shm' when using podman Changes for Build #415 [Marcin Sobczyk] selenium: Mount '/dev/shm' when using podman Changes for Build #416 [Galit Rosenthal] Use specific LagoinitFile for hc suite 4.3 el7 [Evgeny Slutsky] Add eslutsky@redhat.com to jenkins Recipient lists. Changes for Build #417 [Dominik Holler] Revert "network, ovirtlib: Set custom_emulated_machine" Changes for Build #418 [Yedidyah Bar David] HE suites: verify_engine_backup [Sandro Bonazzola] ovirt-engine: drop jobs for legacy versions Changes for Build #419 [Benny Zlotnik] poll update disk [Shlomi] Added ability to ignore other repos when sync_mirror runs Changes for Build #420 [Michal Skrivanek] we really should be installing vmconsole Changes for Build #421 [Michal Skrivanek] we really should be installing vmconsole Changes for Build #422 [Michal Skrivanek] we really should be installing vmconsole ----------------- Failed Tests: ----------------- 1 tests failed. FAILED: basic-suite-master.test-scenarios.002_bootstrap_pytest.test_verify_add_all_hosts Error Message: RuntimeError: 1 hosts failed installation: lago-compat-4-3-suite-master-host-1: install_failed Stack Trace: prefix = <ovirtlago.prefix.OvirtPrefix object at 0x7f1706193e10> @order_by(_TEST_LIST) def test_verify_add_all_hosts(prefix): api = prefix.virt_env.engine_vm().get_api_v4() hosts_service = api.system_service().hosts_service() total_hosts = len(hosts_service.list(search='datacenter={}'.format(DC_NAME))) testlib.assert_true_within( lambda: _all_hosts_up(hosts_service, total_hosts),
timeout=constants.ADD_HOST_TIMEOUT
) ../basic-suite-master/test-scenarios/002_bootstrap_pytest.py:443: _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ /usr/lib/python2.7/site-packages/ovirtlago/testlib.py:278: in assert_true_within assert_equals_within(func, True, timeout, allowed_exceptions) /usr/lib/python2.7/site-packages/ovirtlago/testlib.py:234: in assert_equals_within res = func() ../basic-suite-master/test-scenarios/002_bootstrap_pytest.py:442: in <lambda> lambda: _all_hosts_up(hosts_service, total_hosts), ../basic-suite-master/test-scenarios/002_bootstrap_pytest.py:217: in _all_hosts_up _check_problematic_hosts(hosts_service) _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ hosts_service = <ovirtsdk4.services.HostsService object at 0x7f170654ce10> def _check_problematic_hosts(hosts_service): problematic_hosts = hosts_service.list(search='datacenter={} AND status != installing and status != initializing and status != up'.format(DC_NAME)) if len(problematic_hosts): dump_hosts = '%s hosts failed installation:\n' % len(problematic_hosts) for host in problematic_hosts: host_service = hosts_service.host_service(host.id) dump_hosts += '%s: %s\n' % (host.name, host_service.get().status)
raise RuntimeError(dump_hosts)
E RuntimeError: 1 hosts failed installation: E lago-compat-4-3-suite-master-host-1: install_failed ../basic-suite-master/test-scenarios/002_bootstrap_pytest.py:237: RuntimeError
Project: https://jenkins.ovirt.org/job/ovirt-system-tests_compat-4.3-suite-master/ Build: https://jenkins.ovirt.org/job/ovirt-system-tests_compat-4.3-suite-master/423... Build Number: 423 Build Status: Still Failing Triggered By: Started by timer ------------------------------------- Changes Since Last Success: ------------------------------------- Changes for Build #413 [Marcin Sobczyk] selenium: Mount '/dev/shm' when using podman Changes for Build #414 [Marcin Sobczyk] selenium: Mount '/dev/shm' when using podman Changes for Build #415 [Marcin Sobczyk] selenium: Mount '/dev/shm' when using podman Changes for Build #416 [Galit Rosenthal] Use specific LagoinitFile for hc suite 4.3 el7 [Evgeny Slutsky] Add eslutsky@redhat.com to jenkins Recipient lists. Changes for Build #417 [Dominik Holler] Revert "network, ovirtlib: Set custom_emulated_machine" Changes for Build #418 [Yedidyah Bar David] HE suites: verify_engine_backup [Sandro Bonazzola] ovirt-engine: drop jobs for legacy versions Changes for Build #419 [Benny Zlotnik] poll update disk [Shlomi] Added ability to ignore other repos when sync_mirror runs Changes for Build #420 [Michal Skrivanek] we really should be installing vmconsole Changes for Build #421 [Michal Skrivanek] we really should be installing vmconsole Changes for Build #422 [Michal Skrivanek] we really should be installing vmconsole Changes for Build #423 [Galit] move hc suite to sdk4 [Evgheni Dereveanchin] Remove ACME annotations from TLS routes ----------------- Failed Tests: ----------------- 1 tests failed. FAILED: basic-suite-master.test-scenarios.004_basic_sanity_pytest.test_ha_recovery Error Message: AssertionError: False != True after 600 seconds Stack Trace: prefix = <ovirtlago.prefix.OvirtPrefix object at 0x7fad0614c590> @order_by(_TEST_LIST) def test_ha_recovery(prefix): engine = prefix.virt_env.engine_vm().get_api_v4().system_service() with test_utils.TestEvent(engine, [119, 9602, 506]): # VM_DOWN_ERROR event(119) # HA_VM_FAILED event event(9602) # VDS_INITIATED_RUN_VM event(506) vm_host = _vm_host(prefix, VM2_NAME) pid = vm_host.ssh(['pgrep', '-f', 'qemu.*guest=vm2']) vm_host.ssh(['kill', '-KILL', pid.out.decode('utf-8')]) vm_service = test_utils.get_vm_service(engine, VM2_NAME) testlib.assert_true_within_long(
lambda:
vm_service.get().status == types.VmStatus.UP ) ../basic-suite-master/test-scenarios/004_basic_sanity_pytest.py:788: _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ /usr/lib/python2.7/site-packages/ovirtlago/testlib.py:286: in assert_true_within_long assert_equals_within_long(func, True, allowed_exceptions) /usr/lib/python2.7/site-packages/ovirtlago/testlib.py:273: in assert_equals_within_long func, value, LONG_TIMEOUT, allowed_exceptions=allowed_exceptions _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ func = <function <lambda> at 0x7fad041149b0>, value = True, timeout = 600 allowed_exceptions = [], initial_wait = 0 def assert_equals_within( func, value, timeout, allowed_exceptions=None, initial_wait=10 ): allowed_exceptions = allowed_exceptions or [] with utils.EggTimer(timeout) as timer: while not timer.elapsed(): try: res = func() if res == value: return except Exception as exc: if _instance_of_any(exc, allowed_exceptions): time.sleep(3) continue LOGGER.exception("Unhandled exception in %s", func) raise if initial_wait == 0: time.sleep(3) else: time.sleep(initial_wait) initial_wait = 0 try: raise AssertionError(
'%s != %s after %s seconds' % (res, value, timeout)
) E AssertionError: False != True after 600 seconds /usr/lib/python2.7/site-packages/ovirtlago/testlib.py:252: AssertionError
Project: https://jenkins.ovirt.org/job/ovirt-system-tests_compat-4.3-suite-master/ Build: https://jenkins.ovirt.org/job/ovirt-system-tests_compat-4.3-suite-master/424... Build Number: 424 Build Status: Still Failing Triggered By: Started by timer ------------------------------------- Changes Since Last Success: ------------------------------------- Changes for Build #414 [Marcin Sobczyk] selenium: Mount '/dev/shm' when using podman Changes for Build #415 [Marcin Sobczyk] selenium: Mount '/dev/shm' when using podman Changes for Build #416 [Galit Rosenthal] Use specific LagoinitFile for hc suite 4.3 el7 [Evgeny Slutsky] Add eslutsky@redhat.com to jenkins Recipient lists. Changes for Build #417 [Dominik Holler] Revert "network, ovirtlib: Set custom_emulated_machine" Changes for Build #418 [Yedidyah Bar David] HE suites: verify_engine_backup [Sandro Bonazzola] ovirt-engine: drop jobs for legacy versions Changes for Build #419 [Benny Zlotnik] poll update disk [Shlomi] Added ability to ignore other repos when sync_mirror runs Changes for Build #420 [Michal Skrivanek] we really should be installing vmconsole Changes for Build #421 [Michal Skrivanek] we really should be installing vmconsole Changes for Build #422 [Michal Skrivanek] we really should be installing vmconsole Changes for Build #423 [Galit] move hc suite to sdk4 [Evgheni Dereveanchin] Remove ACME annotations from TLS routes Changes for Build #424 [Lucia Jelinkova] UI tests: Added wait for left menu ----------------- Failed Tests: ----------------- 1 tests failed. FAILED: basic-suite-master.test-scenarios.004_basic_sanity_pytest.test_ha_recovery Error Message: AssertionError: False != True after 600 seconds Stack Trace: prefix = <ovirtlago.prefix.OvirtPrefix object at 0x7f357e52a590> @order_by(_TEST_LIST) def test_ha_recovery(prefix): engine = prefix.virt_env.engine_vm().get_api_v4().system_service() with test_utils.TestEvent(engine, [119, 9602, 506]): # VM_DOWN_ERROR event(119) # HA_VM_FAILED event event(9602) # VDS_INITIATED_RUN_VM event(506) vm_host = _vm_host(prefix, VM2_NAME) pid = vm_host.ssh(['pgrep', '-f', 'qemu.*guest=vm2']) vm_host.ssh(['kill', '-KILL', pid.out.decode('utf-8')]) vm_service = test_utils.get_vm_service(engine, VM2_NAME) testlib.assert_true_within_long(
lambda:
vm_service.get().status == types.VmStatus.UP ) ../basic-suite-master/test-scenarios/004_basic_sanity_pytest.py:788: _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ /usr/lib/python2.7/site-packages/ovirtlago/testlib.py:286: in assert_true_within_long assert_equals_within_long(func, True, allowed_exceptions) /usr/lib/python2.7/site-packages/ovirtlago/testlib.py:273: in assert_equals_within_long func, value, LONG_TIMEOUT, allowed_exceptions=allowed_exceptions _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ func = <function <lambda> at 0x7f357c080e60>, value = True, timeout = 600 allowed_exceptions = [], initial_wait = 0 def assert_equals_within( func, value, timeout, allowed_exceptions=None, initial_wait=10 ): allowed_exceptions = allowed_exceptions or [] with utils.EggTimer(timeout) as timer: while not timer.elapsed(): try: res = func() if res == value: return except Exception as exc: if _instance_of_any(exc, allowed_exceptions): time.sleep(3) continue LOGGER.exception("Unhandled exception in %s", func) raise if initial_wait == 0: time.sleep(3) else: time.sleep(initial_wait) initial_wait = 0 try: raise AssertionError(
'%s != %s after %s seconds' % (res, value, timeout)
) E AssertionError: False != True after 600 seconds /usr/lib/python2.7/site-packages/ovirtlago/testlib.py:252: AssertionError
Project: https://jenkins.ovirt.org/job/ovirt-system-tests_compat-4.3-suite-master/ Build: https://jenkins.ovirt.org/job/ovirt-system-tests_compat-4.3-suite-master/425... Build Number: 425 Build Status: Still Failing Triggered By: Started by timer ------------------------------------- Changes Since Last Success: ------------------------------------- Changes for Build #415 [Marcin Sobczyk] selenium: Mount '/dev/shm' when using podman Changes for Build #416 [Galit Rosenthal] Use specific LagoinitFile for hc suite 4.3 el7 [Evgeny Slutsky] Add eslutsky@redhat.com to jenkins Recipient lists. Changes for Build #417 [Dominik Holler] Revert "network, ovirtlib: Set custom_emulated_machine" Changes for Build #418 [Yedidyah Bar David] HE suites: verify_engine_backup [Sandro Bonazzola] ovirt-engine: drop jobs for legacy versions Changes for Build #419 [Benny Zlotnik] poll update disk [Shlomi] Added ability to ignore other repos when sync_mirror runs Changes for Build #420 [Michal Skrivanek] we really should be installing vmconsole Changes for Build #421 [Michal Skrivanek] we really should be installing vmconsole Changes for Build #422 [Michal Skrivanek] we really should be installing vmconsole Changes for Build #423 [Galit] move hc suite to sdk4 [Evgheni Dereveanchin] Remove ACME annotations from TLS routes Changes for Build #424 [Lucia Jelinkova] UI tests: Added wait for left menu Changes for Build #425 [Sandro Bonazzola] python3: use print function [Sandro Bonazzola] mirrors: drop gluster 3 mirrors [Sandro Bonazzola] cleanup: drop gluster7 dismissed repos mirroring ----------------- Failed Tests: ----------------- 1 tests failed. FAILED: basic-suite-master.test-scenarios.002_bootstrap_pytest.test_verify_add_hosts Error Message: RuntimeError: 2 hosts failed installation: lago-compat-4-3-suite-master-host-0: install_failed lago-compat-4-3-suite-master-host-1: install_failed Stack Trace: api_v4 = <ovirtsdk4.Connection object at 0x7f1a64dcda50> @order_by(_TEST_LIST) def test_verify_add_hosts(api_v4): hosts_service = api_v4.system_service().hosts_service() hosts_status = hosts_service.list(search='datacenter={}'.format(DC_NAME)) total_hosts = len(hosts_status) dump_hosts = _host_status_to_print(hosts_service, hosts_status) LOGGER.debug('Host status, verify_add_hosts:\n {}'.format(dump_hosts)) testlib.assert_true_within( lambda: _single_host_up(hosts_service, total_hosts),
timeout=constants.ADD_HOST_TIMEOUT
) ../basic-suite-master/test-scenarios/002_bootstrap_pytest.py:432: _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ /usr/lib/python2.7/site-packages/ovirtlago/testlib.py:278: in assert_true_within assert_equals_within(func, True, timeout, allowed_exceptions) /usr/lib/python2.7/site-packages/ovirtlago/testlib.py:234: in assert_equals_within res = func() ../basic-suite-master/test-scenarios/002_bootstrap_pytest.py:431: in <lambda> lambda: _single_host_up(hosts_service, total_hosts), ../basic-suite-master/test-scenarios/002_bootstrap_pytest.py:228: in _single_host_up _check_problematic_hosts(hosts_service) _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ hosts_service = <ovirtsdk4.services.HostsService object at 0x7f1a64f9f550> def _check_problematic_hosts(hosts_service): problematic_hosts = hosts_service.list(search='datacenter={} AND status != installing and status != initializing and status != up'.format(DC_NAME)) if len(problematic_hosts): dump_hosts = '%s hosts failed installation:\n' % len(problematic_hosts) for host in problematic_hosts: host_service = hosts_service.host_service(host.id) dump_hosts += '%s: %s\n' % (host.name, host_service.get().status)
raise RuntimeError(dump_hosts)
E RuntimeError: 2 hosts failed installation: E lago-compat-4-3-suite-master-host-0: install_failed E lago-compat-4-3-suite-master-host-1: install_failed ../basic-suite-master/test-scenarios/002_bootstrap_pytest.py:237: RuntimeError
Project: https://jenkins.ovirt.org/job/ovirt-system-tests_compat-4.3-suite-master/ Build: https://jenkins.ovirt.org/job/ovirt-system-tests_compat-4.3-suite-master/426... Build Number: 426 Build Status: Still Failing Triggered By: Started by timer ------------------------------------- Changes Since Last Success: ------------------------------------- Changes for Build #416 [Galit Rosenthal] Use specific LagoinitFile for hc suite 4.3 el7 [Evgeny Slutsky] Add eslutsky@redhat.com to jenkins Recipient lists. Changes for Build #417 [Dominik Holler] Revert "network, ovirtlib: Set custom_emulated_machine" Changes for Build #418 [Yedidyah Bar David] HE suites: verify_engine_backup [Sandro Bonazzola] ovirt-engine: drop jobs for legacy versions Changes for Build #419 [Benny Zlotnik] poll update disk [Shlomi] Added ability to ignore other repos when sync_mirror runs Changes for Build #420 [Michal Skrivanek] we really should be installing vmconsole Changes for Build #421 [Michal Skrivanek] we really should be installing vmconsole Changes for Build #422 [Michal Skrivanek] we really should be installing vmconsole Changes for Build #423 [Galit] move hc suite to sdk4 [Evgheni Dereveanchin] Remove ACME annotations from TLS routes Changes for Build #424 [Lucia Jelinkova] UI tests: Added wait for left menu Changes for Build #425 [Sandro Bonazzola] python3: use print function [Sandro Bonazzola] mirrors: drop gluster 3 mirrors [Sandro Bonazzola] cleanup: drop gluster7 dismissed repos mirroring Changes for Build #426 [Sandro Bonazzola] python3: use print function ----------------- Failed Tests: ----------------- 1 tests failed. FAILED: basic-suite-master.test-scenarios.002_bootstrap_pytest.test_verify_add_hosts Error Message: RuntimeError: 2 hosts failed installation: lago-compat-4-3-suite-master-host-0: install_failed lago-compat-4-3-suite-master-host-1: install_failed Stack Trace: api_v4 = <ovirtsdk4.Connection object at 0x7f3796d6ca50> @order_by(_TEST_LIST) def test_verify_add_hosts(api_v4): hosts_service = api_v4.system_service().hosts_service() hosts_status = hosts_service.list(search='datacenter={}'.format(DC_NAME)) total_hosts = len(hosts_status) dump_hosts = _host_status_to_print(hosts_service, hosts_status) LOGGER.debug('Host status, verify_add_hosts:\n {}'.format(dump_hosts)) testlib.assert_true_within( lambda: _single_host_up(hosts_service, total_hosts),
timeout=constants.ADD_HOST_TIMEOUT
) ../basic-suite-master/test-scenarios/002_bootstrap_pytest.py:432: _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ /usr/lib/python2.7/site-packages/ovirtlago/testlib.py:278: in assert_true_within assert_equals_within(func, True, timeout, allowed_exceptions) /usr/lib/python2.7/site-packages/ovirtlago/testlib.py:234: in assert_equals_within res = func() ../basic-suite-master/test-scenarios/002_bootstrap_pytest.py:431: in <lambda> lambda: _single_host_up(hosts_service, total_hosts), ../basic-suite-master/test-scenarios/002_bootstrap_pytest.py:228: in _single_host_up _check_problematic_hosts(hosts_service) _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ hosts_service = <ovirtsdk4.services.HostsService object at 0x7f3796f55550> def _check_problematic_hosts(hosts_service): problematic_hosts = hosts_service.list(search='datacenter={} AND status != installing and status != initializing and status != up'.format(DC_NAME)) if len(problematic_hosts): dump_hosts = '%s hosts failed installation:\n' % len(problematic_hosts) for host in problematic_hosts: host_service = hosts_service.host_service(host.id) dump_hosts += '%s: %s\n' % (host.name, host_service.get().status)
raise RuntimeError(dump_hosts)
E RuntimeError: 2 hosts failed installation: E lago-compat-4-3-suite-master-host-0: install_failed E lago-compat-4-3-suite-master-host-1: install_failed ../basic-suite-master/test-scenarios/002_bootstrap_pytest.py:237: RuntimeError
Project: https://jenkins.ovirt.org/job/ovirt-system-tests_compat-4.3-suite-master/ Build: https://jenkins.ovirt.org/job/ovirt-system-tests_compat-4.3-suite-master/427... Build Number: 427 Build Status: Still Failing Triggered By: Started by timer ------------------------------------- Changes Since Last Success: ------------------------------------- Changes for Build #417 [Dominik Holler] Revert "network, ovirtlib: Set custom_emulated_machine" Changes for Build #418 [Yedidyah Bar David] HE suites: verify_engine_backup [Sandro Bonazzola] ovirt-engine: drop jobs for legacy versions Changes for Build #419 [Benny Zlotnik] poll update disk [Shlomi] Added ability to ignore other repos when sync_mirror runs Changes for Build #420 [Michal Skrivanek] we really should be installing vmconsole Changes for Build #421 [Michal Skrivanek] we really should be installing vmconsole Changes for Build #422 [Michal Skrivanek] we really should be installing vmconsole Changes for Build #423 [Galit] move hc suite to sdk4 [Evgheni Dereveanchin] Remove ACME annotations from TLS routes Changes for Build #424 [Lucia Jelinkova] UI tests: Added wait for left menu Changes for Build #425 [Sandro Bonazzola] python3: use print function [Sandro Bonazzola] mirrors: drop gluster 3 mirrors [Sandro Bonazzola] cleanup: drop gluster7 dismissed repos mirroring Changes for Build #426 [Sandro Bonazzola] python3: use print function Changes for Build #427 [Sandro Bonazzola] python3: use print function ----------------- Failed Tests: ----------------- 1 tests failed. FAILED: basic-suite-master.test-scenarios.002_bootstrap_pytest.test_verify_add_hosts Error Message: RuntimeError: 2 hosts failed installation: lago-compat-4-3-suite-master-host-0: install_failed lago-compat-4-3-suite-master-host-1: install_failed Stack Trace: api_v4 = <ovirtsdk4.Connection object at 0x7fd184e35a50> @order_by(_TEST_LIST) def test_verify_add_hosts(api_v4): hosts_service = api_v4.system_service().hosts_service() hosts_status = hosts_service.list(search='datacenter={}'.format(DC_NAME)) total_hosts = len(hosts_status) dump_hosts = _host_status_to_print(hosts_service, hosts_status) LOGGER.debug('Host status, verify_add_hosts:\n {}'.format(dump_hosts)) testlib.assert_true_within( lambda: _single_host_up(hosts_service, total_hosts),
timeout=constants.ADD_HOST_TIMEOUT
) ../basic-suite-master/test-scenarios/002_bootstrap_pytest.py:432: _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ /usr/lib/python2.7/site-packages/ovirtlago/testlib.py:278: in assert_true_within assert_equals_within(func, True, timeout, allowed_exceptions) /usr/lib/python2.7/site-packages/ovirtlago/testlib.py:234: in assert_equals_within res = func() ../basic-suite-master/test-scenarios/002_bootstrap_pytest.py:431: in <lambda> lambda: _single_host_up(hosts_service, total_hosts), ../basic-suite-master/test-scenarios/002_bootstrap_pytest.py:228: in _single_host_up _check_problematic_hosts(hosts_service) _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ hosts_service = <ovirtsdk4.services.HostsService object at 0x7fd185007550> def _check_problematic_hosts(hosts_service): problematic_hosts = hosts_service.list(search='datacenter={} AND status != installing and status != initializing and status != up'.format(DC_NAME)) if len(problematic_hosts): dump_hosts = '%s hosts failed installation:\n' % len(problematic_hosts) for host in problematic_hosts: host_service = hosts_service.host_service(host.id) dump_hosts += '%s: %s\n' % (host.name, host_service.get().status)
raise RuntimeError(dump_hosts)
E RuntimeError: 2 hosts failed installation: E lago-compat-4-3-suite-master-host-0: install_failed E lago-compat-4-3-suite-master-host-1: install_failed ../basic-suite-master/test-scenarios/002_bootstrap_pytest.py:237: RuntimeError
Project: https://jenkins.ovirt.org/job/ovirt-system-tests_compat-4.3-suite-master/ Build: https://jenkins.ovirt.org/job/ovirt-system-tests_compat-4.3-suite-master/428... Build Number: 428 Build Status: Still Failing Triggered By: Started by timer ------------------------------------- Changes Since Last Success: ------------------------------------- Changes for Build #418 [Yedidyah Bar David] HE suites: verify_engine_backup [Sandro Bonazzola] ovirt-engine: drop jobs for legacy versions Changes for Build #419 [Benny Zlotnik] poll update disk [Shlomi] Added ability to ignore other repos when sync_mirror runs Changes for Build #420 [Michal Skrivanek] we really should be installing vmconsole Changes for Build #421 [Michal Skrivanek] we really should be installing vmconsole Changes for Build #422 [Michal Skrivanek] we really should be installing vmconsole Changes for Build #423 [Galit] move hc suite to sdk4 [Evgheni Dereveanchin] Remove ACME annotations from TLS routes Changes for Build #424 [Lucia Jelinkova] UI tests: Added wait for left menu Changes for Build #425 [Sandro Bonazzola] python3: use print function [Sandro Bonazzola] mirrors: drop gluster 3 mirrors [Sandro Bonazzola] cleanup: drop gluster7 dismissed repos mirroring Changes for Build #426 [Sandro Bonazzola] python3: use print function Changes for Build #427 [Sandro Bonazzola] python3: use print function Changes for Build #428 [Sandro Bonazzola] python3: use print function ----------------- Failed Tests: ----------------- 1 tests failed. FAILED: basic-suite-master.test-scenarios.002_bootstrap_pytest.test_verify_add_hosts Error Message: RuntimeError: 2 hosts failed installation: lago-compat-4-3-suite-master-host-0: install_failed lago-compat-4-3-suite-master-host-1: install_failed Stack Trace: api_v4 = <ovirtsdk4.Connection object at 0x7fcaa865aa50> @order_by(_TEST_LIST) def test_verify_add_hosts(api_v4): hosts_service = api_v4.system_service().hosts_service() hosts_status = hosts_service.list(search='datacenter={}'.format(DC_NAME)) total_hosts = len(hosts_status) dump_hosts = _host_status_to_print(hosts_service, hosts_status) LOGGER.debug('Host status, verify_add_hosts:\n {}'.format(dump_hosts)) testlib.assert_true_within( lambda: _single_host_up(hosts_service, total_hosts),
timeout=constants.ADD_HOST_TIMEOUT
) ../basic-suite-master/test-scenarios/002_bootstrap_pytest.py:432: _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ /usr/lib/python2.7/site-packages/ovirtlago/testlib.py:278: in assert_true_within assert_equals_within(func, True, timeout, allowed_exceptions) /usr/lib/python2.7/site-packages/ovirtlago/testlib.py:234: in assert_equals_within res = func() ../basic-suite-master/test-scenarios/002_bootstrap_pytest.py:431: in <lambda> lambda: _single_host_up(hosts_service, total_hosts), ../basic-suite-master/test-scenarios/002_bootstrap_pytest.py:228: in _single_host_up _check_problematic_hosts(hosts_service) _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ hosts_service = <ovirtsdk4.services.HostsService object at 0x7fcaa8843550> def _check_problematic_hosts(hosts_service): problematic_hosts = hosts_service.list(search='datacenter={} AND status != installing and status != initializing and status != up'.format(DC_NAME)) if len(problematic_hosts): dump_hosts = '%s hosts failed installation:\n' % len(problematic_hosts) for host in problematic_hosts: host_service = hosts_service.host_service(host.id) dump_hosts += '%s: %s\n' % (host.name, host_service.get().status)
raise RuntimeError(dump_hosts)
E RuntimeError: 2 hosts failed installation: E lago-compat-4-3-suite-master-host-0: install_failed E lago-compat-4-3-suite-master-host-1: install_failed ../basic-suite-master/test-scenarios/002_bootstrap_pytest.py:237: RuntimeError
Project: https://jenkins.ovirt.org/job/ovirt-system-tests_compat-4.3-suite-master/ Build: https://jenkins.ovirt.org/job/ovirt-system-tests_compat-4.3-suite-master/429... Build Number: 429 Build Status: Still Failing Triggered By: Started by timer ------------------------------------- Changes Since Last Success: ------------------------------------- Changes for Build #419 [Benny Zlotnik] poll update disk [Shlomi] Added ability to ignore other repos when sync_mirror runs Changes for Build #420 [Michal Skrivanek] we really should be installing vmconsole Changes for Build #421 [Michal Skrivanek] we really should be installing vmconsole Changes for Build #422 [Michal Skrivanek] we really should be installing vmconsole Changes for Build #423 [Galit] move hc suite to sdk4 [Evgheni Dereveanchin] Remove ACME annotations from TLS routes Changes for Build #424 [Lucia Jelinkova] UI tests: Added wait for left menu Changes for Build #425 [Sandro Bonazzola] python3: use print function [Sandro Bonazzola] mirrors: drop gluster 3 mirrors [Sandro Bonazzola] cleanup: drop gluster7 dismissed repos mirroring Changes for Build #426 [Sandro Bonazzola] python3: use print function Changes for Build #427 [Sandro Bonazzola] python3: use print function Changes for Build #428 [Sandro Bonazzola] python3: use print function Changes for Build #429 [Sandro Bonazzola] python3: use print function ----------------- Failed Tests: ----------------- 1 tests failed. FAILED: basic-suite-master.test-scenarios.002_bootstrap_pytest.test_verify_add_hosts Error Message: RuntimeError: 2 hosts failed installation: lago-compat-4-3-suite-master-host-0: install_failed lago-compat-4-3-suite-master-host-1: install_failed Stack Trace: api_v4 = <ovirtsdk4.Connection object at 0x7ffad959ca50> @order_by(_TEST_LIST) def test_verify_add_hosts(api_v4): hosts_service = api_v4.system_service().hosts_service() hosts_status = hosts_service.list(search='datacenter={}'.format(DC_NAME)) total_hosts = len(hosts_status) dump_hosts = _host_status_to_print(hosts_service, hosts_status) LOGGER.debug('Host status, verify_add_hosts:\n {}'.format(dump_hosts)) testlib.assert_true_within( lambda: _single_host_up(hosts_service, total_hosts),
timeout=constants.ADD_HOST_TIMEOUT
) ../basic-suite-master/test-scenarios/002_bootstrap_pytest.py:432: _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ /usr/lib/python2.7/site-packages/ovirtlago/testlib.py:278: in assert_true_within assert_equals_within(func, True, timeout, allowed_exceptions) /usr/lib/python2.7/site-packages/ovirtlago/testlib.py:234: in assert_equals_within res = func() ../basic-suite-master/test-scenarios/002_bootstrap_pytest.py:431: in <lambda> lambda: _single_host_up(hosts_service, total_hosts), ../basic-suite-master/test-scenarios/002_bootstrap_pytest.py:228: in _single_host_up _check_problematic_hosts(hosts_service) _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ hosts_service = <ovirtsdk4.services.HostsService object at 0x7ffad976e550> def _check_problematic_hosts(hosts_service): problematic_hosts = hosts_service.list(search='datacenter={} AND status != installing and status != initializing and status != up'.format(DC_NAME)) if len(problematic_hosts): dump_hosts = '%s hosts failed installation:\n' % len(problematic_hosts) for host in problematic_hosts: host_service = hosts_service.host_service(host.id) dump_hosts += '%s: %s\n' % (host.name, host_service.get().status)
raise RuntimeError(dump_hosts)
E RuntimeError: 2 hosts failed installation: E lago-compat-4-3-suite-master-host-0: install_failed E lago-compat-4-3-suite-master-host-1: install_failed ../basic-suite-master/test-scenarios/002_bootstrap_pytest.py:237: RuntimeError
Project: https://jenkins.ovirt.org/job/ovirt-system-tests_compat-4.3-suite-master/ Build: https://jenkins.ovirt.org/job/ovirt-system-tests_compat-4.3-suite-master/430... Build Number: 430 Build Status: Still Failing Triggered By: Started by timer ------------------------------------- Changes Since Last Success: ------------------------------------- Changes for Build #421 [Michal Skrivanek] we really should be installing vmconsole Changes for Build #422 [Michal Skrivanek] we really should be installing vmconsole Changes for Build #423 [Galit] move hc suite to sdk4 [Evgheni Dereveanchin] Remove ACME annotations from TLS routes Changes for Build #424 [Lucia Jelinkova] UI tests: Added wait for left menu Changes for Build #425 [Sandro Bonazzola] python3: use print function [Sandro Bonazzola] mirrors: drop gluster 3 mirrors [Sandro Bonazzola] cleanup: drop gluster7 dismissed repos mirroring Changes for Build #426 [Sandro Bonazzola] python3: use print function Changes for Build #427 [Sandro Bonazzola] python3: use print function Changes for Build #428 [Sandro Bonazzola] python3: use print function Changes for Build #429 [Sandro Bonazzola] python3: use print function Changes for Build #430 [Michal Skrivanek] fix list of packages for storage setup ----------------- Failed Tests: ----------------- 1 tests failed. FAILED: basic-suite-master.test-scenarios.002_bootstrap_pytest.test_verify_engine_backup Error Message: AssertionError: False != True after 600 seconds Stack Trace: prefix = <ovirtlago.prefix.OvirtPrefix object at 0x7f91faf84ed0> @order_by(_TEST_LIST) def test_verify_engine_backup(prefix): engine_vm = prefix.virt_env.engine_vm() engine_vm.ssh( [ 'mkdir', '/var/log/ost-engine-backup', ], ) api = prefix.virt_env.engine_vm().get_api_v4() engine = api.system_service() with test_utils.TestEvent(engine, [9024, 9025]): #backup started event, completed result = engine_vm.ssh( [ 'engine-backup', '--mode=backup', '--file=/var/log/ost-engine-backup/backup.tgz', '--log=/var/log/ost-engine-backup/log.txt', ], )
assert result.code == 0, \
'Failed to run engine-backup with code {0}. Output: {1}'.format(result.code, result.out) ../basic-suite-master/test-scenarios/002_bootstrap_pytest.py:1303: _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ /usr/lib64/python2.7/contextlib.py:35: in __exit__ self.gen.throw(type, value, traceback) ../compat-4.3-suite-master/test_utils/__init__.py:290: in TestEvent lambda: /usr/lib/python2.7/site-packages/ovirtlago/testlib.py:286: in assert_true_within_long assert_equals_within_long(func, True, allowed_exceptions) /usr/lib/python2.7/site-packages/ovirtlago/testlib.py:273: in assert_equals_within_long func, value, LONG_TIMEOUT, allowed_exceptions=allowed_exceptions _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ func = <function <lambda> at 0x7f91fada7ed8>, value = True, timeout = 600 allowed_exceptions = [], initial_wait = 0 def assert_equals_within( func, value, timeout, allowed_exceptions=None, initial_wait=10 ): allowed_exceptions = allowed_exceptions or [] with utils.EggTimer(timeout) as timer: while not timer.elapsed(): try: res = func() if res == value: return except Exception as exc: if _instance_of_any(exc, allowed_exceptions): time.sleep(3) continue LOGGER.exception("Unhandled exception in %s", func) raise if initial_wait == 0: time.sleep(3) else: time.sleep(initial_wait) initial_wait = 0 try: raise AssertionError(
'%s != %s after %s seconds' % (res, value, timeout)
) E AssertionError: False != True after 600 seconds /usr/lib/python2.7/site-packages/ovirtlago/testlib.py:252: AssertionError
Project: https://jenkins.ovirt.org/job/ovirt-system-tests_compat-4.3-suite-master/ Build: https://jenkins.ovirt.org/job/ovirt-system-tests_compat-4.3-suite-master/431... Build Number: 431 Build Status: Still Failing Triggered By: Started by timer ------------------------------------- Changes Since Last Success: ------------------------------------- Changes for Build #422 [Michal Skrivanek] we really should be installing vmconsole Changes for Build #423 [Galit] move hc suite to sdk4 [Evgheni Dereveanchin] Remove ACME annotations from TLS routes Changes for Build #424 [Lucia Jelinkova] UI tests: Added wait for left menu Changes for Build #425 [Sandro Bonazzola] python3: use print function [Sandro Bonazzola] mirrors: drop gluster 3 mirrors [Sandro Bonazzola] cleanup: drop gluster7 dismissed repos mirroring Changes for Build #426 [Sandro Bonazzola] python3: use print function Changes for Build #427 [Sandro Bonazzola] python3: use print function Changes for Build #428 [Sandro Bonazzola] python3: use print function Changes for Build #429 [Sandro Bonazzola] python3: use print function Changes for Build #430 [Michal Skrivanek] fix list of packages for storage setup Changes for Build #431 [Lucia Jelinkova] UI test refactoring ----------------- Failed Tests: ----------------- 1 tests failed. FAILED: basic-suite-master.test-scenarios.002_bootstrap_pytest.test_verify_engine_backup Error Message: AssertionError: False != True after 600 seconds Stack Trace: prefix = <ovirtlago.prefix.OvirtPrefix object at 0x7f0a8937ded0> @order_by(_TEST_LIST) def test_verify_engine_backup(prefix): engine_vm = prefix.virt_env.engine_vm() engine_vm.ssh( [ 'mkdir', '/var/log/ost-engine-backup', ], ) api = prefix.virt_env.engine_vm().get_api_v4() engine = api.system_service() with test_utils.TestEvent(engine, [9024, 9025]): #backup started event, completed result = engine_vm.ssh( [ 'engine-backup', '--mode=backup', '--file=/var/log/ost-engine-backup/backup.tgz', '--log=/var/log/ost-engine-backup/log.txt', ], )
assert result.code == 0, \
'Failed to run engine-backup with code {0}. Output: {1}'.format(result.code, result.out) ../basic-suite-master/test-scenarios/002_bootstrap_pytest.py:1303: _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ /usr/lib64/python2.7/contextlib.py:35: in __exit__ self.gen.throw(type, value, traceback) ../compat-4.3-suite-master/test_utils/__init__.py:290: in TestEvent lambda: /usr/lib/python2.7/site-packages/ovirtlago/testlib.py:286: in assert_true_within_long assert_equals_within_long(func, True, allowed_exceptions) /usr/lib/python2.7/site-packages/ovirtlago/testlib.py:273: in assert_equals_within_long func, value, LONG_TIMEOUT, allowed_exceptions=allowed_exceptions _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ func = <function <lambda> at 0x7f0a894b3410>, value = True, timeout = 600 allowed_exceptions = [], initial_wait = 0 def assert_equals_within( func, value, timeout, allowed_exceptions=None, initial_wait=10 ): allowed_exceptions = allowed_exceptions or [] with utils.EggTimer(timeout) as timer: while not timer.elapsed(): try: res = func() if res == value: return except Exception as exc: if _instance_of_any(exc, allowed_exceptions): time.sleep(3) continue LOGGER.exception("Unhandled exception in %s", func) raise if initial_wait == 0: time.sleep(3) else: time.sleep(initial_wait) initial_wait = 0 try: raise AssertionError(
'%s != %s after %s seconds' % (res, value, timeout)
) E AssertionError: False != True after 600 seconds /usr/lib/python2.7/site-packages/ovirtlago/testlib.py:252: AssertionError
Project: https://jenkins.ovirt.org/job/ovirt-system-tests_compat-4.3-suite-master/ Build: https://jenkins.ovirt.org/job/ovirt-system-tests_compat-4.3-suite-master/432... Build Number: 432 Build Status: Still Failing Triggered By: Started by timer ------------------------------------- Changes Since Last Success: ------------------------------------- Changes for Build #423 [Galit] move hc suite to sdk4 [Evgheni Dereveanchin] Remove ACME annotations from TLS routes Changes for Build #424 [Lucia Jelinkova] UI tests: Added wait for left menu Changes for Build #425 [Sandro Bonazzola] python3: use print function [Sandro Bonazzola] mirrors: drop gluster 3 mirrors [Sandro Bonazzola] cleanup: drop gluster7 dismissed repos mirroring Changes for Build #426 [Sandro Bonazzola] python3: use print function Changes for Build #427 [Sandro Bonazzola] python3: use print function Changes for Build #428 [Sandro Bonazzola] python3: use print function Changes for Build #429 [Sandro Bonazzola] python3: use print function Changes for Build #430 [Michal Skrivanek] fix list of packages for storage setup Changes for Build #431 [Lucia Jelinkova] UI test refactoring Changes for Build #432 [Galit Rosenthal] Remove reposync from upgrade-from-prevrelease-suite-4.3 ----------------- Failed Tests: ----------------- 1 tests failed. FAILED: basic-suite-master.test-scenarios.002_bootstrap_pytest.test_verify_engine_backup Error Message: AssertionError: False != True after 600 seconds Stack Trace: prefix = <ovirtlago.prefix.OvirtPrefix object at 0x7f41953d8ed0> @order_by(_TEST_LIST) def test_verify_engine_backup(prefix): engine_vm = prefix.virt_env.engine_vm() engine_vm.ssh( [ 'mkdir', '/var/log/ost-engine-backup', ], ) api = prefix.virt_env.engine_vm().get_api_v4() engine = api.system_service() with test_utils.TestEvent(engine, [9024, 9025]): #backup started event, completed result = engine_vm.ssh( [ 'engine-backup', '--mode=backup', '--file=/var/log/ost-engine-backup/backup.tgz', '--log=/var/log/ost-engine-backup/log.txt', ], )
assert result.code == 0, \
'Failed to run engine-backup with code {0}. Output: {1}'.format(result.code, result.out) ../basic-suite-master/test-scenarios/002_bootstrap_pytest.py:1303: _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ /usr/lib64/python2.7/contextlib.py:35: in __exit__ self.gen.throw(type, value, traceback) ../compat-4.3-suite-master/test_utils/__init__.py:290: in TestEvent lambda: /usr/lib/python2.7/site-packages/ovirtlago/testlib.py:286: in assert_true_within_long assert_equals_within_long(func, True, allowed_exceptions) /usr/lib/python2.7/site-packages/ovirtlago/testlib.py:273: in assert_equals_within_long func, value, LONG_TIMEOUT, allowed_exceptions=allowed_exceptions _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ func = <function <lambda> at 0x7f41951fbde8>, value = True, timeout = 600 allowed_exceptions = [], initial_wait = 0 def assert_equals_within( func, value, timeout, allowed_exceptions=None, initial_wait=10 ): allowed_exceptions = allowed_exceptions or [] with utils.EggTimer(timeout) as timer: while not timer.elapsed(): try: res = func() if res == value: return except Exception as exc: if _instance_of_any(exc, allowed_exceptions): time.sleep(3) continue LOGGER.exception("Unhandled exception in %s", func) raise if initial_wait == 0: time.sleep(3) else: time.sleep(initial_wait) initial_wait = 0 try: raise AssertionError(
'%s != %s after %s seconds' % (res, value, timeout)
) E AssertionError: False != True after 600 seconds /usr/lib/python2.7/site-packages/ovirtlago/testlib.py:252: AssertionError
Project: https://jenkins.ovirt.org/job/ovirt-system-tests_compat-4.3-suite-master/ Build: https://jenkins.ovirt.org/job/ovirt-system-tests_compat-4.3-suite-master/433... Build Number: 433 Build Status: Still Failing Triggered By: Started by timer ------------------------------------- Changes Since Last Success: ------------------------------------- Changes for Build #424 [Lucia Jelinkova] UI tests: Added wait for left menu Changes for Build #425 [Sandro Bonazzola] python3: use print function [Sandro Bonazzola] mirrors: drop gluster 3 mirrors [Sandro Bonazzola] cleanup: drop gluster7 dismissed repos mirroring Changes for Build #426 [Sandro Bonazzola] python3: use print function Changes for Build #427 [Sandro Bonazzola] python3: use print function Changes for Build #428 [Sandro Bonazzola] python3: use print function Changes for Build #429 [Sandro Bonazzola] python3: use print function Changes for Build #430 [Michal Skrivanek] fix list of packages for storage setup Changes for Build #431 [Lucia Jelinkova] UI test refactoring Changes for Build #432 [Galit Rosenthal] Remove reposync from upgrade-from-prevrelease-suite-4.3 Changes for Build #433 [Marcin Sobczyk] selenium: podman: Add grid setup retries ----------------- Failed Tests: ----------------- 1 tests failed. FAILED: basic-suite-master.test-scenarios.004_basic_sanity_pytest.test_ha_recovery Error Message: AssertionError: False != True after 600 seconds Stack Trace: prefix = <ovirtlago.prefix.OvirtPrefix object at 0x7f77a42be190> @order_by(_TEST_LIST) def test_ha_recovery(prefix): engine = prefix.virt_env.engine_vm().get_api_v4().system_service() with test_utils.TestEvent(engine, [119, 9602, 506]): # VM_DOWN_ERROR event(119) # HA_VM_FAILED event event(9602) # VDS_INITIATED_RUN_VM event(506) vm_host = _vm_host(prefix, VM2_NAME) pid = vm_host.ssh(['pgrep', '-f', 'qemu.*guest=vm2']) vm_host.ssh(['kill', '-KILL', pid.out.decode('utf-8')]) vm_service = test_utils.get_vm_service(engine, VM2_NAME) testlib.assert_true_within_long(
lambda:
vm_service.get().status == types.VmStatus.UP ) ../basic-suite-master/test-scenarios/004_basic_sanity_pytest.py:788: _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ /usr/lib/python2.7/site-packages/ovirtlago/testlib.py:286: in assert_true_within_long assert_equals_within_long(func, True, allowed_exceptions) /usr/lib/python2.7/site-packages/ovirtlago/testlib.py:273: in assert_equals_within_long func, value, LONG_TIMEOUT, allowed_exceptions=allowed_exceptions _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ func = <function <lambda> at 0x7f77a0b84a28>, value = True, timeout = 600 allowed_exceptions = [], initial_wait = 0 def assert_equals_within( func, value, timeout, allowed_exceptions=None, initial_wait=10 ): allowed_exceptions = allowed_exceptions or [] with utils.EggTimer(timeout) as timer: while not timer.elapsed(): try: res = func() if res == value: return except Exception as exc: if _instance_of_any(exc, allowed_exceptions): time.sleep(3) continue LOGGER.exception("Unhandled exception in %s", func) raise if initial_wait == 0: time.sleep(3) else: time.sleep(initial_wait) initial_wait = 0 try: raise AssertionError(
'%s != %s after %s seconds' % (res, value, timeout)
) E AssertionError: False != True after 600 seconds /usr/lib/python2.7/site-packages/ovirtlago/testlib.py:252: AssertionError
Project: https://jenkins.ovirt.org/job/ovirt-system-tests_compat-4.3-suite-master/ Build: https://jenkins.ovirt.org/job/ovirt-system-tests_compat-4.3-suite-master/434... Build Number: 434 Build Status: Still Failing Triggered By: Started by timer ------------------------------------- Changes Since Last Success: ------------------------------------- Changes for Build #425 [Sandro Bonazzola] python3: use print function [Sandro Bonazzola] mirrors: drop gluster 3 mirrors [Sandro Bonazzola] cleanup: drop gluster7 dismissed repos mirroring Changes for Build #426 [Sandro Bonazzola] python3: use print function Changes for Build #427 [Sandro Bonazzola] python3: use print function Changes for Build #428 [Sandro Bonazzola] python3: use print function Changes for Build #429 [Sandro Bonazzola] python3: use print function Changes for Build #430 [Michal Skrivanek] fix list of packages for storage setup Changes for Build #431 [Lucia Jelinkova] UI test refactoring Changes for Build #432 [Galit Rosenthal] Remove reposync from upgrade-from-prevrelease-suite-4.3 Changes for Build #433 [Marcin Sobczyk] selenium: podman: Add grid setup retries Changes for Build #434 [Marcin Sobczyk] selenium: podman: Add grid setup retries ----------------- Failed Tests: ----------------- 1 tests failed. FAILED: basic-suite-master.test-scenarios.004_basic_sanity_pytest.test_ha_recovery Error Message: AssertionError: False != True after 600 seconds Stack Trace: prefix = <ovirtlago.prefix.OvirtPrefix object at 0x7fe0c8e2c190> @order_by(_TEST_LIST) def test_ha_recovery(prefix): engine = prefix.virt_env.engine_vm().get_api_v4().system_service() with test_utils.TestEvent(engine, [119, 9602, 506]): # VM_DOWN_ERROR event(119) # HA_VM_FAILED event event(9602) # VDS_INITIATED_RUN_VM event(506) vm_host = _vm_host(prefix, VM2_NAME) pid = vm_host.ssh(['pgrep', '-f', 'qemu.*guest=vm2']) vm_host.ssh(['kill', '-KILL', pid.out.decode('utf-8')]) vm_service = test_utils.get_vm_service(engine, VM2_NAME) testlib.assert_true_within_long(
lambda:
vm_service.get().status == types.VmStatus.UP ) ../basic-suite-master/test-scenarios/004_basic_sanity_pytest.py:788: _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ /usr/lib/python2.7/site-packages/ovirtlago/testlib.py:286: in assert_true_within_long assert_equals_within_long(func, True, allowed_exceptions) /usr/lib/python2.7/site-packages/ovirtlago/testlib.py:273: in assert_equals_within_long func, value, LONG_TIMEOUT, allowed_exceptions=allowed_exceptions _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ func = <function <lambda> at 0x7fe0c569dc08>, value = True, timeout = 600 allowed_exceptions = [], initial_wait = 0 def assert_equals_within( func, value, timeout, allowed_exceptions=None, initial_wait=10 ): allowed_exceptions = allowed_exceptions or [] with utils.EggTimer(timeout) as timer: while not timer.elapsed(): try: res = func() if res == value: return except Exception as exc: if _instance_of_any(exc, allowed_exceptions): time.sleep(3) continue LOGGER.exception("Unhandled exception in %s", func) raise if initial_wait == 0: time.sleep(3) else: time.sleep(initial_wait) initial_wait = 0 try: raise AssertionError(
'%s != %s after %s seconds' % (res, value, timeout)
) E AssertionError: False != True after 600 seconds /usr/lib/python2.7/site-packages/ovirtlago/testlib.py:252: AssertionError
Project: https://jenkins.ovirt.org/job/ovirt-system-tests_compat-4.3-suite-master/ Build: https://jenkins.ovirt.org/job/ovirt-system-tests_compat-4.3-suite-master/435... Build Number: 435 Build Status: Still Failing Triggered By: Started by timer ------------------------------------- Changes Since Last Success: ------------------------------------- Changes for Build #426 [Sandro Bonazzola] python3: use print function Changes for Build #427 [Sandro Bonazzola] python3: use print function Changes for Build #428 [Sandro Bonazzola] python3: use print function Changes for Build #429 [Sandro Bonazzola] python3: use print function Changes for Build #430 [Michal Skrivanek] fix list of packages for storage setup Changes for Build #431 [Lucia Jelinkova] UI test refactoring Changes for Build #432 [Galit Rosenthal] Remove reposync from upgrade-from-prevrelease-suite-4.3 Changes for Build #433 [Marcin Sobczyk] selenium: podman: Add grid setup retries Changes for Build #434 [Marcin Sobczyk] selenium: podman: Add grid setup retries Changes for Build #435 [Marcin Sobczyk] selenium: podman: Add grid setup retries ----------------- Failed Tests: ----------------- 1 tests failed. FAILED: basic-suite-master.test-scenarios.004_basic_sanity_pytest.test_ha_recovery Error Message: AssertionError: False != True after 600 seconds Stack Trace: prefix = <ovirtlago.prefix.OvirtPrefix object at 0x7f5eac7c2190> @order_by(_TEST_LIST) def test_ha_recovery(prefix): engine = prefix.virt_env.engine_vm().get_api_v4().system_service() with test_utils.TestEvent(engine, [119, 9602, 506]): # VM_DOWN_ERROR event(119) # HA_VM_FAILED event event(9602) # VDS_INITIATED_RUN_VM event(506) vm_host = _vm_host(prefix, VM2_NAME) pid = vm_host.ssh(['pgrep', '-f', 'qemu.*guest=vm2']) vm_host.ssh(['kill', '-KILL', pid.out.decode('utf-8')]) vm_service = test_utils.get_vm_service(engine, VM2_NAME) testlib.assert_true_within_long(
lambda:
vm_service.get().status == types.VmStatus.UP ) ../basic-suite-master/test-scenarios/004_basic_sanity_pytest.py:788: _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ /usr/lib/python2.7/site-packages/ovirtlago/testlib.py:286: in assert_true_within_long assert_equals_within_long(func, True, allowed_exceptions) /usr/lib/python2.7/site-packages/ovirtlago/testlib.py:273: in assert_equals_within_long func, value, LONG_TIMEOUT, allowed_exceptions=allowed_exceptions _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ func = <function <lambda> at 0x7f5ea9087aa0>, value = True, timeout = 600 allowed_exceptions = [], initial_wait = 0 def assert_equals_within( func, value, timeout, allowed_exceptions=None, initial_wait=10 ): allowed_exceptions = allowed_exceptions or [] with utils.EggTimer(timeout) as timer: while not timer.elapsed(): try: res = func() if res == value: return except Exception as exc: if _instance_of_any(exc, allowed_exceptions): time.sleep(3) continue LOGGER.exception("Unhandled exception in %s", func) raise if initial_wait == 0: time.sleep(3) else: time.sleep(initial_wait) initial_wait = 0 try: raise AssertionError(
'%s != %s after %s seconds' % (res, value, timeout)
) E AssertionError: False != True after 600 seconds /usr/lib/python2.7/site-packages/ovirtlago/testlib.py:252: AssertionError
Project: https://jenkins.ovirt.org/job/ovirt-system-tests_compat-4.3-suite-master/ Build: https://jenkins.ovirt.org/job/ovirt-system-tests_compat-4.3-suite-master/436... Build Number: 436 Build Status: Still Failing Triggered By: Started by timer ------------------------------------- Changes Since Last Success: ------------------------------------- Changes for Build #427 [Sandro Bonazzola] python3: use print function Changes for Build #428 [Sandro Bonazzola] python3: use print function Changes for Build #429 [Sandro Bonazzola] python3: use print function Changes for Build #430 [Michal Skrivanek] fix list of packages for storage setup Changes for Build #431 [Lucia Jelinkova] UI test refactoring Changes for Build #432 [Galit Rosenthal] Remove reposync from upgrade-from-prevrelease-suite-4.3 Changes for Build #433 [Marcin Sobczyk] selenium: podman: Add grid setup retries Changes for Build #434 [Marcin Sobczyk] selenium: podman: Add grid setup retries Changes for Build #435 [Marcin Sobczyk] selenium: podman: Add grid setup retries Changes for Build #436 [Marcin Sobczyk] selenium: podman: Add grid setup retries ----------------- Failed Tests: ----------------- 1 tests failed. FAILED: basic-suite-master.test-scenarios.002_bootstrap_pytest.test_verify_engine_backup Error Message: AssertionError: False != True after 600 seconds Stack Trace: prefix = <ovirtlago.prefix.OvirtPrefix object at 0x7f830dab1ed0> @order_by(_TEST_LIST) def test_verify_engine_backup(prefix): engine_vm = prefix.virt_env.engine_vm() engine_vm.ssh( [ 'mkdir', '/var/log/ost-engine-backup', ], ) api = prefix.virt_env.engine_vm().get_api_v4() engine = api.system_service() with test_utils.TestEvent(engine, [9024, 9025]): #backup started event, completed result = engine_vm.ssh( [ 'engine-backup', '--mode=backup', '--file=/var/log/ost-engine-backup/backup.tgz', '--log=/var/log/ost-engine-backup/log.txt', ], )
assert result.code == 0, \
'Failed to run engine-backup with code {0}. Output: {1}'.format(result.code, result.out) ../basic-suite-master/test-scenarios/002_bootstrap_pytest.py:1303: _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ /usr/lib64/python2.7/contextlib.py:35: in __exit__ self.gen.throw(type, value, traceback) ../compat-4.3-suite-master/test_utils/__init__.py:290: in TestEvent lambda: /usr/lib/python2.7/site-packages/ovirtlago/testlib.py:286: in assert_true_within_long assert_equals_within_long(func, True, allowed_exceptions) /usr/lib/python2.7/site-packages/ovirtlago/testlib.py:273: in assert_equals_within_long func, value, LONG_TIMEOUT, allowed_exceptions=allowed_exceptions _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ func = <function <lambda> at 0x7f830d8d4de8>, value = True, timeout = 600 allowed_exceptions = [], initial_wait = 0 def assert_equals_within( func, value, timeout, allowed_exceptions=None, initial_wait=10 ): allowed_exceptions = allowed_exceptions or [] with utils.EggTimer(timeout) as timer: while not timer.elapsed(): try: res = func() if res == value: return except Exception as exc: if _instance_of_any(exc, allowed_exceptions): time.sleep(3) continue LOGGER.exception("Unhandled exception in %s", func) raise if initial_wait == 0: time.sleep(3) else: time.sleep(initial_wait) initial_wait = 0 try: raise AssertionError(
'%s != %s after %s seconds' % (res, value, timeout)
) E AssertionError: False != True after 600 seconds /usr/lib/python2.7/site-packages/ovirtlago/testlib.py:252: AssertionError
Project: https://jenkins.ovirt.org/job/ovirt-system-tests_compat-4.3-suite-master/ Build: https://jenkins.ovirt.org/job/ovirt-system-tests_compat-4.3-suite-master/437... Build Number: 437 Build Status: Still Failing Triggered By: Started by timer ------------------------------------- Changes Since Last Success: ------------------------------------- Changes for Build #428 [Sandro Bonazzola] python3: use print function Changes for Build #429 [Sandro Bonazzola] python3: use print function Changes for Build #430 [Michal Skrivanek] fix list of packages for storage setup Changes for Build #431 [Lucia Jelinkova] UI test refactoring Changes for Build #432 [Galit Rosenthal] Remove reposync from upgrade-from-prevrelease-suite-4.3 Changes for Build #433 [Marcin Sobczyk] selenium: podman: Add grid setup retries Changes for Build #434 [Marcin Sobczyk] selenium: podman: Add grid setup retries Changes for Build #435 [Marcin Sobczyk] selenium: podman: Add grid setup retries Changes for Build #436 [Marcin Sobczyk] selenium: podman: Add grid setup retries Changes for Build #437 [Evgeny Slutsky] fixes for he-basic-role-remote-suite [Shlomi] Added ability to generate an HTML report, containing all repos ----------------- Failed Tests: ----------------- 1 tests failed. FAILED: basic-suite-master.test-scenarios.004_basic_sanity_pytest.test_ha_recovery Error Message: AssertionError: False != True after 600 seconds Stack Trace: prefix = <ovirtlago.prefix.OvirtPrefix object at 0x7f50dcc5f190> @order_by(_TEST_LIST) def test_ha_recovery(prefix): engine = prefix.virt_env.engine_vm().get_api_v4().system_service() with test_utils.TestEvent(engine, [119, 9602, 506]): # VM_DOWN_ERROR event(119) # HA_VM_FAILED event event(9602) # VDS_INITIATED_RUN_VM event(506) vm_host = _vm_host(prefix, VM2_NAME) pid = vm_host.ssh(['pgrep', '-f', 'qemu.*guest=vm2']) vm_host.ssh(['kill', '-KILL', pid.out.decode('utf-8')]) vm_service = test_utils.get_vm_service(engine, VM2_NAME) testlib.assert_true_within_long(
lambda:
vm_service.get().status == types.VmStatus.UP ) ../basic-suite-master/test-scenarios/004_basic_sanity_pytest.py:788: _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ /usr/lib/python2.7/site-packages/ovirtlago/testlib.py:286: in assert_true_within_long assert_equals_within_long(func, True, allowed_exceptions) /usr/lib/python2.7/site-packages/ovirtlago/testlib.py:273: in assert_equals_within_long func, value, LONG_TIMEOUT, allowed_exceptions=allowed_exceptions _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ func = <function <lambda> at 0x7f50d94e0848>, value = True, timeout = 600 allowed_exceptions = [], initial_wait = 0 def assert_equals_within( func, value, timeout, allowed_exceptions=None, initial_wait=10 ): allowed_exceptions = allowed_exceptions or [] with utils.EggTimer(timeout) as timer: while not timer.elapsed(): try: res = func() if res == value: return except Exception as exc: if _instance_of_any(exc, allowed_exceptions): time.sleep(3) continue LOGGER.exception("Unhandled exception in %s", func) raise if initial_wait == 0: time.sleep(3) else: time.sleep(initial_wait) initial_wait = 0 try: raise AssertionError(
'%s != %s after %s seconds' % (res, value, timeout)
) E AssertionError: False != True after 600 seconds /usr/lib/python2.7/site-packages/ovirtlago/testlib.py:252: AssertionError
Project: https://jenkins.ovirt.org/job/ovirt-system-tests_compat-4.3-suite-master/ Build: https://jenkins.ovirt.org/job/ovirt-system-tests_compat-4.3-suite-master/438... Build Number: 438 Build Status: Still Failing Triggered By: Started by timer ------------------------------------- Changes Since Last Success: ------------------------------------- Changes for Build #429 [Sandro Bonazzola] python3: use print function Changes for Build #430 [Michal Skrivanek] fix list of packages for storage setup Changes for Build #431 [Lucia Jelinkova] UI test refactoring Changes for Build #432 [Galit Rosenthal] Remove reposync from upgrade-from-prevrelease-suite-4.3 Changes for Build #433 [Marcin Sobczyk] selenium: podman: Add grid setup retries Changes for Build #434 [Marcin Sobczyk] selenium: podman: Add grid setup retries Changes for Build #435 [Marcin Sobczyk] selenium: podman: Add grid setup retries Changes for Build #436 [Marcin Sobczyk] selenium: podman: Add grid setup retries Changes for Build #437 [Evgeny Slutsky] fixes for he-basic-role-remote-suite [Shlomi] Added ability to generate an HTML report, containing all repos Changes for Build #438 [Galit Rosenthal] Move upgrade from release suite to no reposync ----------------- Failed Tests: ----------------- 1 tests failed. FAILED: basic-suite-master.test-scenarios.002_bootstrap_pytest.test_verify_engine_backup Error Message: AssertionError: False != True after 600 seconds Stack Trace: prefix = <ovirtlago.prefix.OvirtPrefix object at 0x7ffbbe09aed0> @order_by(_TEST_LIST) def test_verify_engine_backup(prefix): engine_vm = prefix.virt_env.engine_vm() engine_vm.ssh( [ 'mkdir', '/var/log/ost-engine-backup', ], ) api = prefix.virt_env.engine_vm().get_api_v4() engine = api.system_service() with test_utils.TestEvent(engine, [9024, 9025]): #backup started event, completed result = engine_vm.ssh( [ 'engine-backup', '--mode=backup', '--file=/var/log/ost-engine-backup/backup.tgz', '--log=/var/log/ost-engine-backup/log.txt', ], )
assert result.code == 0, \
'Failed to run engine-backup with code {0}. Output: {1}'.format(result.code, result.out) ../basic-suite-master/test-scenarios/002_bootstrap_pytest.py:1303: _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ /usr/lib64/python2.7/contextlib.py:35: in __exit__ self.gen.throw(type, value, traceback) ../compat-4.3-suite-master/test_utils/__init__.py:290: in TestEvent lambda: /usr/lib/python2.7/site-packages/ovirtlago/testlib.py:286: in assert_true_within_long assert_equals_within_long(func, True, allowed_exceptions) /usr/lib/python2.7/site-packages/ovirtlago/testlib.py:273: in assert_equals_within_long func, value, LONG_TIMEOUT, allowed_exceptions=allowed_exceptions _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ func = <function <lambda> at 0x7ffbbdebdc80>, value = True, timeout = 600 allowed_exceptions = [], initial_wait = 0 def assert_equals_within( func, value, timeout, allowed_exceptions=None, initial_wait=10 ): allowed_exceptions = allowed_exceptions or [] with utils.EggTimer(timeout) as timer: while not timer.elapsed(): try: res = func() if res == value: return except Exception as exc: if _instance_of_any(exc, allowed_exceptions): time.sleep(3) continue LOGGER.exception("Unhandled exception in %s", func) raise if initial_wait == 0: time.sleep(3) else: time.sleep(initial_wait) initial_wait = 0 try: raise AssertionError(
'%s != %s after %s seconds' % (res, value, timeout)
) E AssertionError: False != True after 600 seconds /usr/lib/python2.7/site-packages/ovirtlago/testlib.py:252: AssertionError
Project: https://jenkins.ovirt.org/job/ovirt-system-tests_compat-4.3-suite-master/ Build: https://jenkins.ovirt.org/job/ovirt-system-tests_compat-4.3-suite-master/439... Build Number: 439 Build Status: Still Failing Triggered By: Started by timer ------------------------------------- Changes Since Last Success: ------------------------------------- Changes for Build #430 [Michal Skrivanek] fix list of packages for storage setup Changes for Build #431 [Lucia Jelinkova] UI test refactoring Changes for Build #432 [Galit Rosenthal] Remove reposync from upgrade-from-prevrelease-suite-4.3 Changes for Build #433 [Marcin Sobczyk] selenium: podman: Add grid setup retries Changes for Build #434 [Marcin Sobczyk] selenium: podman: Add grid setup retries Changes for Build #435 [Marcin Sobczyk] selenium: podman: Add grid setup retries Changes for Build #436 [Marcin Sobczyk] selenium: podman: Add grid setup retries Changes for Build #437 [Evgeny Slutsky] fixes for he-basic-role-remote-suite [Shlomi] Added ability to generate an HTML report, containing all repos Changes for Build #438 [Galit Rosenthal] Move upgrade from release suite to no reposync Changes for Build #439 [jenkins CI] Auto create ovirt-4.3 [Lev Veyde] Fixed the deprecated --old-chroot option issue [Evgheni Dereveanchin] remove fc29 mirror sync jobs ----------------- Failed Tests: ----------------- 1 tests failed. FAILED: basic-suite-master.test-scenarios.002_bootstrap_pytest.test_verify_engine_backup Error Message: AssertionError: False != True after 600 seconds Stack Trace: prefix = <ovirtlago.prefix.OvirtPrefix object at 0x7f7d05066ed0> @order_by(_TEST_LIST) def test_verify_engine_backup(prefix): engine_vm = prefix.virt_env.engine_vm() engine_vm.ssh( [ 'mkdir', '/var/log/ost-engine-backup', ], ) api = prefix.virt_env.engine_vm().get_api_v4() engine = api.system_service() with test_utils.TestEvent(engine, [9024, 9025]): #backup started event, completed result = engine_vm.ssh( [ 'engine-backup', '--mode=backup', '--file=/var/log/ost-engine-backup/backup.tgz', '--log=/var/log/ost-engine-backup/log.txt', ], )
assert result.code == 0, \
'Failed to run engine-backup with code {0}. Output: {1}'.format(result.code, result.out) ../basic-suite-master/test-scenarios/002_bootstrap_pytest.py:1303: _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ /usr/lib64/python2.7/contextlib.py:35: in __exit__ self.gen.throw(type, value, traceback) ../compat-4.3-suite-master/test_utils/__init__.py:290: in TestEvent lambda: /usr/lib/python2.7/site-packages/ovirtlago/testlib.py:286: in assert_true_within_long assert_equals_within_long(func, True, allowed_exceptions) /usr/lib/python2.7/site-packages/ovirtlago/testlib.py:273: in assert_equals_within_long func, value, LONG_TIMEOUT, allowed_exceptions=allowed_exceptions _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ func = <function <lambda> at 0x7f7d04e89c08>, value = True, timeout = 600 allowed_exceptions = [], initial_wait = 0 def assert_equals_within( func, value, timeout, allowed_exceptions=None, initial_wait=10 ): allowed_exceptions = allowed_exceptions or [] with utils.EggTimer(timeout) as timer: while not timer.elapsed(): try: res = func() if res == value: return except Exception as exc: if _instance_of_any(exc, allowed_exceptions): time.sleep(3) continue LOGGER.exception("Unhandled exception in %s", func) raise if initial_wait == 0: time.sleep(3) else: time.sleep(initial_wait) initial_wait = 0 try: raise AssertionError(
'%s != %s after %s seconds' % (res, value, timeout)
) E AssertionError: False != True after 600 seconds /usr/lib/python2.7/site-packages/ovirtlago/testlib.py:252: AssertionError
Project: https://jenkins.ovirt.org/job/ovirt-system-tests_compat-4.3-suite-master/ Build: https://jenkins.ovirt.org/job/ovirt-system-tests_compat-4.3-suite-master/440... Build Number: 440 Build Status: Still Failing Triggered By: Started by timer ------------------------------------- Changes Since Last Success: ------------------------------------- Changes for Build #431 [Lucia Jelinkova] UI test refactoring Changes for Build #432 [Galit Rosenthal] Remove reposync from upgrade-from-prevrelease-suite-4.3 Changes for Build #433 [Marcin Sobczyk] selenium: podman: Add grid setup retries Changes for Build #434 [Marcin Sobczyk] selenium: podman: Add grid setup retries Changes for Build #435 [Marcin Sobczyk] selenium: podman: Add grid setup retries Changes for Build #436 [Marcin Sobczyk] selenium: podman: Add grid setup retries Changes for Build #437 [Evgeny Slutsky] fixes for he-basic-role-remote-suite [Shlomi] Added ability to generate an HTML report, containing all repos Changes for Build #438 [Galit Rosenthal] Move upgrade from release suite to no reposync Changes for Build #439 [jenkins CI] Auto create ovirt-4.3 [Lev Veyde] Fixed the deprecated --old-chroot option issue [Evgheni Dereveanchin] remove fc29 mirror sync jobs Changes for Build #440 [jenkins CI] Auto create ovirt-4.3 [Ehud Yonasi] change_queue: disable tests for CQ. [Evgheni Dereveanchin] Download all package versions for modular repos ----------------- Failed Tests: ----------------- 1 tests failed. FAILED: basic-suite-master.test-scenarios.004_basic_sanity_pytest.test_ha_recovery Error Message: AssertionError: False != True after 600 seconds Stack Trace: prefix = <ovirtlago.prefix.OvirtPrefix object at 0x7f1628b2c190> @order_by(_TEST_LIST) def test_ha_recovery(prefix): engine = prefix.virt_env.engine_vm().get_api_v4().system_service() with test_utils.TestEvent(engine, [119, 9602, 506]): # VM_DOWN_ERROR event(119) # HA_VM_FAILED event event(9602) # VDS_INITIATED_RUN_VM event(506) vm_host = _vm_host(prefix, VM2_NAME) pid = vm_host.ssh(['pgrep', '-f', 'qemu.*guest=vm2']) vm_host.ssh(['kill', '-KILL', pid.out.decode('utf-8')]) vm_service = test_utils.get_vm_service(engine, VM2_NAME) testlib.assert_true_within_long(
lambda:
vm_service.get().status == types.VmStatus.UP ) ../basic-suite-master/test-scenarios/004_basic_sanity_pytest.py:788: _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ /usr/lib/python2.7/site-packages/ovirtlago/testlib.py:286: in assert_true_within_long assert_equals_within_long(func, True, allowed_exceptions) /usr/lib/python2.7/site-packages/ovirtlago/testlib.py:273: in assert_equals_within_long func, value, LONG_TIMEOUT, allowed_exceptions=allowed_exceptions _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ func = <function <lambda> at 0x7f16253a91b8>, value = True, timeout = 600 allowed_exceptions = [], initial_wait = 0 def assert_equals_within( func, value, timeout, allowed_exceptions=None, initial_wait=10 ): allowed_exceptions = allowed_exceptions or [] with utils.EggTimer(timeout) as timer: while not timer.elapsed(): try: res = func() if res == value: return except Exception as exc: if _instance_of_any(exc, allowed_exceptions): time.sleep(3) continue LOGGER.exception("Unhandled exception in %s", func) raise if initial_wait == 0: time.sleep(3) else: time.sleep(initial_wait) initial_wait = 0 try: raise AssertionError(
'%s != %s after %s seconds' % (res, value, timeout)
) E AssertionError: False != True after 600 seconds /usr/lib/python2.7/site-packages/ovirtlago/testlib.py:252: AssertionError
Project: https://jenkins.ovirt.org/job/ovirt-system-tests_compat-4.3-suite-master/ Build: https://jenkins.ovirt.org/job/ovirt-system-tests_compat-4.3-suite-master/441... Build Number: 441 Build Status: Still Failing Triggered By: Started by timer ------------------------------------- Changes Since Last Success: ------------------------------------- Changes for Build #432 [Galit Rosenthal] Remove reposync from upgrade-from-prevrelease-suite-4.3 Changes for Build #433 [Marcin Sobczyk] selenium: podman: Add grid setup retries Changes for Build #434 [Marcin Sobczyk] selenium: podman: Add grid setup retries Changes for Build #435 [Marcin Sobczyk] selenium: podman: Add grid setup retries Changes for Build #436 [Marcin Sobczyk] selenium: podman: Add grid setup retries Changes for Build #437 [Evgeny Slutsky] fixes for he-basic-role-remote-suite [Shlomi] Added ability to generate an HTML report, containing all repos Changes for Build #438 [Galit Rosenthal] Move upgrade from release suite to no reposync Changes for Build #439 [jenkins CI] Auto create ovirt-4.3 [Lev Veyde] Fixed the deprecated --old-chroot option issue [Evgheni Dereveanchin] remove fc29 mirror sync jobs Changes for Build #440 [jenkins CI] Auto create ovirt-4.3 [Ehud Yonasi] change_queue: disable tests for CQ. [Evgheni Dereveanchin] Download all package versions for modular repos Changes for Build #441 [jenkins CI] Auto create ovirt-4.3 ----------------- Failed Tests: ----------------- 1 tests failed. FAILED: basic-suite-master.test-scenarios.004_basic_sanity_pytest.test_ha_recovery Error Message: AssertionError: False != True after 600 seconds Stack Trace: prefix = <ovirtlago.prefix.OvirtPrefix object at 0x7fa9d3378190> @order_by(_TEST_LIST) def test_ha_recovery(prefix): engine = prefix.virt_env.engine_vm().get_api_v4().system_service() with test_utils.TestEvent(engine, [119, 9602, 506]): # VM_DOWN_ERROR event(119) # HA_VM_FAILED event event(9602) # VDS_INITIATED_RUN_VM event(506) vm_host = _vm_host(prefix, VM2_NAME) pid = vm_host.ssh(['pgrep', '-f', 'qemu.*guest=vm2']) vm_host.ssh(['kill', '-KILL', pid.out.decode('utf-8')]) vm_service = test_utils.get_vm_service(engine, VM2_NAME) testlib.assert_true_within_long(
lambda:
vm_service.get().status == types.VmStatus.UP ) ../basic-suite-master/test-scenarios/004_basic_sanity_pytest.py:788: _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ /usr/lib/python2.7/site-packages/ovirtlago/testlib.py:286: in assert_true_within_long assert_equals_within_long(func, True, allowed_exceptions) /usr/lib/python2.7/site-packages/ovirtlago/testlib.py:273: in assert_equals_within_long func, value, LONG_TIMEOUT, allowed_exceptions=allowed_exceptions _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ func = <function <lambda> at 0x7fa9d010d5f0>, value = True, timeout = 600 allowed_exceptions = [], initial_wait = 0 def assert_equals_within( func, value, timeout, allowed_exceptions=None, initial_wait=10 ): allowed_exceptions = allowed_exceptions or [] with utils.EggTimer(timeout) as timer: while not timer.elapsed(): try: res = func() if res == value: return except Exception as exc: if _instance_of_any(exc, allowed_exceptions): time.sleep(3) continue LOGGER.exception("Unhandled exception in %s", func) raise if initial_wait == 0: time.sleep(3) else: time.sleep(initial_wait) initial_wait = 0 try: raise AssertionError(
'%s != %s after %s seconds' % (res, value, timeout)
) E AssertionError: False != True after 600 seconds /usr/lib/python2.7/site-packages/ovirtlago/testlib.py:252: AssertionError
Project: https://jenkins.ovirt.org/job/ovirt-system-tests_compat-4.3-suite-master/ Build: https://jenkins.ovirt.org/job/ovirt-system-tests_compat-4.3-suite-master/442... Build Number: 442 Build Status: Still Failing Triggered By: Started by timer ------------------------------------- Changes Since Last Success: ------------------------------------- Changes for Build #433 [Marcin Sobczyk] selenium: podman: Add grid setup retries Changes for Build #434 [Marcin Sobczyk] selenium: podman: Add grid setup retries Changes for Build #435 [Marcin Sobczyk] selenium: podman: Add grid setup retries Changes for Build #436 [Marcin Sobczyk] selenium: podman: Add grid setup retries Changes for Build #437 [Evgeny Slutsky] fixes for he-basic-role-remote-suite [Shlomi] Added ability to generate an HTML report, containing all repos Changes for Build #438 [Galit Rosenthal] Move upgrade from release suite to no reposync Changes for Build #439 [jenkins CI] Auto create ovirt-4.3 [Lev Veyde] Fixed the deprecated --old-chroot option issue [Evgheni Dereveanchin] remove fc29 mirror sync jobs Changes for Build #440 [jenkins CI] Auto create ovirt-4.3 [Ehud Yonasi] change_queue: disable tests for CQ. [Evgheni Dereveanchin] Download all package versions for modular repos Changes for Build #441 [jenkins CI] Auto create ovirt-4.3 Changes for Build #442 [jenkins CI] Auto create ovirt-4.3 ----------------- Failed Tests: ----------------- 1 tests failed. FAILED: basic-suite-master.test-scenarios.004_basic_sanity_pytest.test_ha_recovery Error Message: AssertionError: False != True after 600 seconds Stack Trace: prefix = <ovirtlago.prefix.OvirtPrefix object at 0x7f37e082f190> @order_by(_TEST_LIST) def test_ha_recovery(prefix): engine = prefix.virt_env.engine_vm().get_api_v4().system_service() with test_utils.TestEvent(engine, [119, 9602, 506]): # VM_DOWN_ERROR event(119) # HA_VM_FAILED event event(9602) # VDS_INITIATED_RUN_VM event(506) vm_host = _vm_host(prefix, VM2_NAME) pid = vm_host.ssh(['pgrep', '-f', 'qemu.*guest=vm2']) vm_host.ssh(['kill', '-KILL', pid.out.decode('utf-8')]) vm_service = test_utils.get_vm_service(engine, VM2_NAME) testlib.assert_true_within_long(
lambda:
vm_service.get().status == types.VmStatus.UP ) ../basic-suite-master/test-scenarios/004_basic_sanity_pytest.py:788: _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ /usr/lib/python2.7/site-packages/ovirtlago/testlib.py:286: in assert_true_within_long assert_equals_within_long(func, True, allowed_exceptions) /usr/lib/python2.7/site-packages/ovirtlago/testlib.py:273: in assert_equals_within_long func, value, LONG_TIMEOUT, allowed_exceptions=allowed_exceptions _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ func = <function <lambda> at 0x7f37dd119848>, value = True, timeout = 600 allowed_exceptions = [], initial_wait = 0 def assert_equals_within( func, value, timeout, allowed_exceptions=None, initial_wait=10 ): allowed_exceptions = allowed_exceptions or [] with utils.EggTimer(timeout) as timer: while not timer.elapsed(): try: res = func() if res == value: return except Exception as exc: if _instance_of_any(exc, allowed_exceptions): time.sleep(3) continue LOGGER.exception("Unhandled exception in %s", func) raise if initial_wait == 0: time.sleep(3) else: time.sleep(initial_wait) initial_wait = 0 try: raise AssertionError(
'%s != %s after %s seconds' % (res, value, timeout)
) E AssertionError: False != True after 600 seconds /usr/lib/python2.7/site-packages/ovirtlago/testlib.py:252: AssertionError
Project: https://jenkins.ovirt.org/job/ovirt-system-tests_compat-4.3-suite-master/ Build: https://jenkins.ovirt.org/job/ovirt-system-tests_compat-4.3-suite-master/443... Build Number: 443 Build Status: Still Failing Triggered By: Started by timer ------------------------------------- Changes Since Last Success: ------------------------------------- Changes for Build #434 [Marcin Sobczyk] selenium: podman: Add grid setup retries Changes for Build #435 [Marcin Sobczyk] selenium: podman: Add grid setup retries Changes for Build #436 [Marcin Sobczyk] selenium: podman: Add grid setup retries Changes for Build #437 [Evgeny Slutsky] fixes for he-basic-role-remote-suite [Shlomi] Added ability to generate an HTML report, containing all repos Changes for Build #438 [Galit Rosenthal] Move upgrade from release suite to no reposync Changes for Build #439 [jenkins CI] Auto create ovirt-4.3 [Lev Veyde] Fixed the deprecated --old-chroot option issue [Evgheni Dereveanchin] remove fc29 mirror sync jobs Changes for Build #440 [jenkins CI] Auto create ovirt-4.3 [Ehud Yonasi] change_queue: disable tests for CQ. [Evgheni Dereveanchin] Download all package versions for modular repos Changes for Build #441 [jenkins CI] Auto create ovirt-4.3 Changes for Build #442 [jenkins CI] Auto create ovirt-4.3 Changes for Build #443 [jenkins CI] Auto create ovirt-4.3 ----------------- Failed Tests: ----------------- 1 tests failed. FAILED: basic-suite-master.test-scenarios.004_basic_sanity_pytest.test_ha_recovery Error Message: AssertionError: False != True after 600 seconds Stack Trace: prefix = <ovirtlago.prefix.OvirtPrefix object at 0x7f9478845190> @order_by(_TEST_LIST) def test_ha_recovery(prefix): engine = prefix.virt_env.engine_vm().get_api_v4().system_service() with test_utils.TestEvent(engine, [119, 9602, 506]): # VM_DOWN_ERROR event(119) # HA_VM_FAILED event event(9602) # VDS_INITIATED_RUN_VM event(506) vm_host = _vm_host(prefix, VM2_NAME) pid = vm_host.ssh(['pgrep', '-f', 'qemu.*guest=vm2']) vm_host.ssh(['kill', '-KILL', pid.out.decode('utf-8')]) vm_service = test_utils.get_vm_service(engine, VM2_NAME) testlib.assert_true_within_long(
lambda:
vm_service.get().status == types.VmStatus.UP ) ../basic-suite-master/test-scenarios/004_basic_sanity_pytest.py:788: _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ /usr/lib/python2.7/site-packages/ovirtlago/testlib.py:286: in assert_true_within_long assert_equals_within_long(func, True, allowed_exceptions) /usr/lib/python2.7/site-packages/ovirtlago/testlib.py:273: in assert_equals_within_long func, value, LONG_TIMEOUT, allowed_exceptions=allowed_exceptions _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ func = <function <lambda> at 0x7f94750b6578>, value = True, timeout = 600 allowed_exceptions = [], initial_wait = 0 def assert_equals_within( func, value, timeout, allowed_exceptions=None, initial_wait=10 ): allowed_exceptions = allowed_exceptions or [] with utils.EggTimer(timeout) as timer: while not timer.elapsed(): try: res = func() if res == value: return except Exception as exc: if _instance_of_any(exc, allowed_exceptions): time.sleep(3) continue LOGGER.exception("Unhandled exception in %s", func) raise if initial_wait == 0: time.sleep(3) else: time.sleep(initial_wait) initial_wait = 0 try: raise AssertionError(
'%s != %s after %s seconds' % (res, value, timeout)
) E AssertionError: False != True after 600 seconds /usr/lib/python2.7/site-packages/ovirtlago/testlib.py:252: AssertionError
Project: https://jenkins.ovirt.org/job/ovirt-system-tests_compat-4.3-suite-master/ Build: https://jenkins.ovirt.org/job/ovirt-system-tests_compat-4.3-suite-master/444... Build Number: 444 Build Status: Still Failing Triggered By: Started by timer ------------------------------------- Changes Since Last Success: ------------------------------------- Changes for Build #435 [Marcin Sobczyk] selenium: podman: Add grid setup retries Changes for Build #436 [Marcin Sobczyk] selenium: podman: Add grid setup retries Changes for Build #437 [Evgeny Slutsky] fixes for he-basic-role-remote-suite [Shlomi] Added ability to generate an HTML report, containing all repos Changes for Build #438 [Galit Rosenthal] Move upgrade from release suite to no reposync Changes for Build #439 [jenkins CI] Auto create ovirt-4.3 [Lev Veyde] Fixed the deprecated --old-chroot option issue [Evgheni Dereveanchin] remove fc29 mirror sync jobs Changes for Build #440 [jenkins CI] Auto create ovirt-4.3 [Ehud Yonasi] change_queue: disable tests for CQ. [Evgheni Dereveanchin] Download all package versions for modular repos Changes for Build #441 [jenkins CI] Auto create ovirt-4.3 Changes for Build #442 [jenkins CI] Auto create ovirt-4.3 Changes for Build #443 [jenkins CI] Auto create ovirt-4.3 Changes for Build #444 [hbraha] network: explicitly set the filter to None ----------------- Failed Tests: ----------------- 1 tests failed. FAILED: basic-suite-master.test-scenarios.004_basic_sanity_pytest.test_ha_recovery Error Message: AssertionError: False != True after 600 seconds Stack Trace: prefix = <ovirtlago.prefix.OvirtPrefix object at 0x7f7d23b21210> @order_by(_TEST_LIST) def test_ha_recovery(prefix): engine = prefix.virt_env.engine_vm().get_api_v4().system_service() with test_utils.TestEvent(engine, [119, 9602, 506]): # VM_DOWN_ERROR event(119) # HA_VM_FAILED event event(9602) # VDS_INITIATED_RUN_VM event(506) vm_host = _vm_host(prefix, VM2_NAME) pid = vm_host.ssh(['pgrep', '-f', 'qemu.*guest=vm2']) vm_host.ssh(['kill', '-KILL', pid.out.decode('utf-8')]) vm_service = test_utils.get_vm_service(engine, VM2_NAME) testlib.assert_true_within_long(
lambda:
vm_service.get().status == types.VmStatus.UP ) ../basic-suite-master/test-scenarios/004_basic_sanity_pytest.py:788: _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ /usr/lib/python2.7/site-packages/ovirtlago/testlib.py:286: in assert_true_within_long assert_equals_within_long(func, True, allowed_exceptions) /usr/lib/python2.7/site-packages/ovirtlago/testlib.py:273: in assert_equals_within_long func, value, LONG_TIMEOUT, allowed_exceptions=allowed_exceptions _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ func = <function <lambda> at 0x7f7d204422a8>, value = True, timeout = 600 allowed_exceptions = [], initial_wait = 0 def assert_equals_within( func, value, timeout, allowed_exceptions=None, initial_wait=10 ): allowed_exceptions = allowed_exceptions or [] with utils.EggTimer(timeout) as timer: while not timer.elapsed(): try: res = func() if res == value: return except Exception as exc: if _instance_of_any(exc, allowed_exceptions): time.sleep(3) continue LOGGER.exception("Unhandled exception in %s", func) raise if initial_wait == 0: time.sleep(3) else: time.sleep(initial_wait) initial_wait = 0 try: raise AssertionError(
'%s != %s after %s seconds' % (res, value, timeout)
) E AssertionError: False != True after 600 seconds /usr/lib/python2.7/site-packages/ovirtlago/testlib.py:252: AssertionError
Project: https://jenkins.ovirt.org/job/ovirt-system-tests_compat-4.3-suite-master/ Build: https://jenkins.ovirt.org/job/ovirt-system-tests_compat-4.3-suite-master/445... Build Number: 445 Build Status: Still Failing Triggered By: Started by timer ------------------------------------- Changes Since Last Success: ------------------------------------- Changes for Build #436 [Marcin Sobczyk] selenium: podman: Add grid setup retries Changes for Build #437 [Evgeny Slutsky] fixes for he-basic-role-remote-suite [Shlomi] Added ability to generate an HTML report, containing all repos Changes for Build #438 [Galit Rosenthal] Move upgrade from release suite to no reposync Changes for Build #439 [jenkins CI] Auto create ovirt-4.3 [Lev Veyde] Fixed the deprecated --old-chroot option issue [Evgheni Dereveanchin] remove fc29 mirror sync jobs Changes for Build #440 [jenkins CI] Auto create ovirt-4.3 [Ehud Yonasi] change_queue: disable tests for CQ. [Evgheni Dereveanchin] Download all package versions for modular repos Changes for Build #441 [jenkins CI] Auto create ovirt-4.3 Changes for Build #442 [jenkins CI] Auto create ovirt-4.3 Changes for Build #443 [jenkins CI] Auto create ovirt-4.3 Changes for Build #444 [hbraha] network: explicitly set the filter to None Changes for Build #445 [hbraha] network: explicitly set the filter to None ----------------- Failed Tests: ----------------- 1 tests failed. FAILED: basic-suite-master.test-scenarios.004_basic_sanity_pytest.test_ha_recovery Error Message: AssertionError: False != True after 600 seconds Stack Trace: prefix = <ovirtlago.prefix.OvirtPrefix object at 0x7efbe88e32d0> @order_by(_TEST_LIST) def test_ha_recovery(prefix): engine = prefix.virt_env.engine_vm().get_api_v4().system_service() with test_utils.TestEvent(engine, [119, 9602, 506]): # VM_DOWN_ERROR event(119) # HA_VM_FAILED event event(9602) # VDS_INITIATED_RUN_VM event(506) vm_host = _vm_host(prefix, VM2_NAME) pid = vm_host.ssh(['pgrep', '-f', 'qemu.*guest=vm2']) vm_host.ssh(['kill', '-KILL', pid.out.decode('utf-8')]) vm_service = test_utils.get_vm_service(engine, VM2_NAME) testlib.assert_true_within_long(
lambda:
vm_service.get().status == types.VmStatus.UP ) ../basic-suite-master/test-scenarios/004_basic_sanity_pytest.py:788: _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ /usr/lib/python2.7/site-packages/ovirtlago/testlib.py:286: in assert_true_within_long assert_equals_within_long(func, True, allowed_exceptions) /usr/lib/python2.7/site-packages/ovirtlago/testlib.py:273: in assert_equals_within_long func, value, LONG_TIMEOUT, allowed_exceptions=allowed_exceptions _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ func = <function <lambda> at 0x7efbe5187578>, value = True, timeout = 600 allowed_exceptions = [], initial_wait = 0 def assert_equals_within( func, value, timeout, allowed_exceptions=None, initial_wait=10 ): allowed_exceptions = allowed_exceptions or [] with utils.EggTimer(timeout) as timer: while not timer.elapsed(): try: res = func() if res == value: return except Exception as exc: if _instance_of_any(exc, allowed_exceptions): time.sleep(3) continue LOGGER.exception("Unhandled exception in %s", func) raise if initial_wait == 0: time.sleep(3) else: time.sleep(initial_wait) initial_wait = 0 try: raise AssertionError(
'%s != %s after %s seconds' % (res, value, timeout)
) E AssertionError: False != True after 600 seconds /usr/lib/python2.7/site-packages/ovirtlago/testlib.py:252: AssertionError
Project: https://jenkins.ovirt.org/job/ovirt-system-tests_compat-4.3-suite-master/ Build: https://jenkins.ovirt.org/job/ovirt-system-tests_compat-4.3-suite-master/446... Build Number: 446 Build Status: Still Failing Triggered By: Started by timer ------------------------------------- Changes Since Last Success: ------------------------------------- Changes for Build #437 [Evgeny Slutsky] fixes for he-basic-role-remote-suite [Shlomi] Added ability to generate an HTML report, containing all repos Changes for Build #438 [Galit Rosenthal] Move upgrade from release suite to no reposync Changes for Build #439 [jenkins CI] Auto create ovirt-4.3 [Lev Veyde] Fixed the deprecated --old-chroot option issue [Evgheni Dereveanchin] remove fc29 mirror sync jobs Changes for Build #440 [jenkins CI] Auto create ovirt-4.3 [Ehud Yonasi] change_queue: disable tests for CQ. [Evgheni Dereveanchin] Download all package versions for modular repos Changes for Build #441 [jenkins CI] Auto create ovirt-4.3 Changes for Build #442 [jenkins CI] Auto create ovirt-4.3 Changes for Build #443 [jenkins CI] Auto create ovirt-4.3 Changes for Build #444 [hbraha] network: explicitly set the filter to None Changes for Build #445 [hbraha] network: explicitly set the filter to None Changes for Build #446 [hbraha] network: explicitly set the filter to None ----------------- Failed Tests: ----------------- 1 tests failed. FAILED: basic-suite-master.test-scenarios.004_basic_sanity_pytest.test_ha_recovery Error Message: AssertionError: False != True after 600 seconds Stack Trace: prefix = <ovirtlago.prefix.OvirtPrefix object at 0x7f7b4334b2d0> @order_by(_TEST_LIST) def test_ha_recovery(prefix): engine = prefix.virt_env.engine_vm().get_api_v4().system_service() with test_utils.TestEvent(engine, [119, 9602, 506]): # VM_DOWN_ERROR event(119) # HA_VM_FAILED event event(9602) # VDS_INITIATED_RUN_VM event(506) vm_host = _vm_host(prefix, VM2_NAME) pid = vm_host.ssh(['pgrep', '-f', 'qemu.*guest=vm2']) vm_host.ssh(['kill', '-KILL', pid.out.decode('utf-8')]) vm_service = test_utils.get_vm_service(engine, VM2_NAME) testlib.assert_true_within_long(
lambda:
vm_service.get().status == types.VmStatus.UP ) ../basic-suite-master/test-scenarios/004_basic_sanity_pytest.py:788: _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ /usr/lib/python2.7/site-packages/ovirtlago/testlib.py:286: in assert_true_within_long assert_equals_within_long(func, True, allowed_exceptions) /usr/lib/python2.7/site-packages/ovirtlago/testlib.py:273: in assert_equals_within_long func, value, LONG_TIMEOUT, allowed_exceptions=allowed_exceptions _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ func = <function <lambda> at 0x7f7b401a2758>, value = True, timeout = 600 allowed_exceptions = [], initial_wait = 0 def assert_equals_within( func, value, timeout, allowed_exceptions=None, initial_wait=10 ): allowed_exceptions = allowed_exceptions or [] with utils.EggTimer(timeout) as timer: while not timer.elapsed(): try: res = func() if res == value: return except Exception as exc: if _instance_of_any(exc, allowed_exceptions): time.sleep(3) continue LOGGER.exception("Unhandled exception in %s", func) raise if initial_wait == 0: time.sleep(3) else: time.sleep(initial_wait) initial_wait = 0 try: raise AssertionError(
'%s != %s after %s seconds' % (res, value, timeout)
) E AssertionError: False != True after 600 seconds /usr/lib/python2.7/site-packages/ovirtlago/testlib.py:252: AssertionError
Project: https://jenkins.ovirt.org/job/ovirt-system-tests_compat-4.3-suite-master/ Build: https://jenkins.ovirt.org/job/ovirt-system-tests_compat-4.3-suite-master/447... Build Number: 447 Build Status: Still Failing Triggered By: Started by timer ------------------------------------- Changes Since Last Success: ------------------------------------- Changes for Build #438 [Galit Rosenthal] Move upgrade from release suite to no reposync Changes for Build #439 [jenkins CI] Auto create ovirt-4.3 [Lev Veyde] Fixed the deprecated --old-chroot option issue [Evgheni Dereveanchin] remove fc29 mirror sync jobs Changes for Build #440 [jenkins CI] Auto create ovirt-4.3 [Ehud Yonasi] change_queue: disable tests for CQ. [Evgheni Dereveanchin] Download all package versions for modular repos Changes for Build #441 [jenkins CI] Auto create ovirt-4.3 Changes for Build #442 [jenkins CI] Auto create ovirt-4.3 Changes for Build #443 [jenkins CI] Auto create ovirt-4.3 Changes for Build #444 [hbraha] network: explicitly set the filter to None Changes for Build #445 [hbraha] network: explicitly set the filter to None Changes for Build #446 [hbraha] network: explicitly set the filter to None Changes for Build #447 [hbraha] network: explicitly set the filter to None ----------------- Failed Tests: ----------------- 1 tests failed. FAILED: basic-suite-master.test-scenarios.004_basic_sanity_pytest.test_ha_recovery Error Message: AssertionError: False != True after 600 seconds Stack Trace: prefix = <ovirtlago.prefix.OvirtPrefix object at 0x7f6d4d6dc2d0> @order_by(_TEST_LIST) def test_ha_recovery(prefix): engine = prefix.virt_env.engine_vm().get_api_v4().system_service() with test_utils.TestEvent(engine, [119, 9602, 506]): # VM_DOWN_ERROR event(119) # HA_VM_FAILED event event(9602) # VDS_INITIATED_RUN_VM event(506) vm_host = _vm_host(prefix, VM2_NAME) pid = vm_host.ssh(['pgrep', '-f', 'qemu.*guest=vm2']) vm_host.ssh(['kill', '-KILL', pid.out.decode('utf-8')]) vm_service = test_utils.get_vm_service(engine, VM2_NAME) testlib.assert_true_within_long(
lambda:
vm_service.get().status == types.VmStatus.UP ) ../basic-suite-master/test-scenarios/004_basic_sanity_pytest.py:788: _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ /usr/lib/python2.7/site-packages/ovirtlago/testlib.py:286: in assert_true_within_long assert_equals_within_long(func, True, allowed_exceptions) /usr/lib/python2.7/site-packages/ovirtlago/testlib.py:273: in assert_equals_within_long func, value, LONG_TIMEOUT, allowed_exceptions=allowed_exceptions _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ func = <function <lambda> at 0x7f6d45f59b90>, value = True, timeout = 600 allowed_exceptions = [], initial_wait = 0 def assert_equals_within( func, value, timeout, allowed_exceptions=None, initial_wait=10 ): allowed_exceptions = allowed_exceptions or [] with utils.EggTimer(timeout) as timer: while not timer.elapsed(): try: res = func() if res == value: return except Exception as exc: if _instance_of_any(exc, allowed_exceptions): time.sleep(3) continue LOGGER.exception("Unhandled exception in %s", func) raise if initial_wait == 0: time.sleep(3) else: time.sleep(initial_wait) initial_wait = 0 try: raise AssertionError(
'%s != %s after %s seconds' % (res, value, timeout)
) E AssertionError: False != True after 600 seconds /usr/lib/python2.7/site-packages/ovirtlago/testlib.py:252: AssertionError
Project: https://jenkins.ovirt.org/job/ovirt-system-tests_compat-4.3-suite-master/ Build: https://jenkins.ovirt.org/job/ovirt-system-tests_compat-4.3-suite-master/448... Build Number: 448 Build Status: Still Failing Triggered By: Started by timer ------------------------------------- Changes Since Last Success: ------------------------------------- Changes for Build #439 [jenkins CI] Auto create ovirt-4.3 [Lev Veyde] Fixed the deprecated --old-chroot option issue [Evgheni Dereveanchin] remove fc29 mirror sync jobs Changes for Build #440 [jenkins CI] Auto create ovirt-4.3 [Ehud Yonasi] change_queue: disable tests for CQ. [Evgheni Dereveanchin] Download all package versions for modular repos Changes for Build #441 [jenkins CI] Auto create ovirt-4.3 Changes for Build #442 [jenkins CI] Auto create ovirt-4.3 Changes for Build #443 [jenkins CI] Auto create ovirt-4.3 Changes for Build #444 [hbraha] network: explicitly set the filter to None Changes for Build #445 [hbraha] network: explicitly set the filter to None Changes for Build #446 [hbraha] network: explicitly set the filter to None Changes for Build #447 [hbraha] network: explicitly set the filter to None Changes for Build #448 [Eitan Raviv] Install specific version of decorator from PyPi ----------------- Failed Tests: ----------------- 1 tests failed. FAILED: basic-suite-master.test-scenarios.004_basic_sanity_pytest.test_ha_recovery Error Message: AssertionError: False != True after 600 seconds Stack Trace: prefix = <ovirtlago.prefix.OvirtPrefix object at 0x7fd238e1c2d0> @order_by(_TEST_LIST) def test_ha_recovery(prefix): engine = prefix.virt_env.engine_vm().get_api_v4().system_service() with test_utils.TestEvent(engine, [119, 9602, 506]): # VM_DOWN_ERROR event(119) # HA_VM_FAILED event event(9602) # VDS_INITIATED_RUN_VM event(506) vm_host = _vm_host(prefix, VM2_NAME) pid = vm_host.ssh(['pgrep', '-f', 'qemu.*guest=vm2']) vm_host.ssh(['kill', '-KILL', pid.out.decode('utf-8')]) vm_service = test_utils.get_vm_service(engine, VM2_NAME) testlib.assert_true_within_long(
lambda:
vm_service.get().status == types.VmStatus.UP ) ../basic-suite-master/test-scenarios/004_basic_sanity_pytest.py:788: _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ /usr/lib/python2.7/site-packages/ovirtlago/testlib.py:286: in assert_true_within_long assert_equals_within_long(func, True, allowed_exceptions) /usr/lib/python2.7/site-packages/ovirtlago/testlib.py:273: in assert_equals_within_long func, value, LONG_TIMEOUT, allowed_exceptions=allowed_exceptions _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ func = <function <lambda> at 0x7fd235738d70>, value = True, timeout = 600 allowed_exceptions = [], initial_wait = 0 def assert_equals_within( func, value, timeout, allowed_exceptions=None, initial_wait=10 ): allowed_exceptions = allowed_exceptions or [] with utils.EggTimer(timeout) as timer: while not timer.elapsed(): try: res = func() if res == value: return except Exception as exc: if _instance_of_any(exc, allowed_exceptions): time.sleep(3) continue LOGGER.exception("Unhandled exception in %s", func) raise if initial_wait == 0: time.sleep(3) else: time.sleep(initial_wait) initial_wait = 0 try: raise AssertionError(
'%s != %s after %s seconds' % (res, value, timeout)
) E AssertionError: False != True after 600 seconds /usr/lib/python2.7/site-packages/ovirtlago/testlib.py:252: AssertionError
Project: https://jenkins.ovirt.org/job/ovirt-system-tests_compat-4.3-suite-master/ Build: https://jenkins.ovirt.org/job/ovirt-system-tests_compat-4.3-suite-master/449... Build Number: 449 Build Status: Still Failing Triggered By: Started by timer ------------------------------------- Changes Since Last Success: ------------------------------------- Changes for Build #440 [jenkins CI] Auto create ovirt-4.3 [Ehud Yonasi] change_queue: disable tests for CQ. [Evgheni Dereveanchin] Download all package versions for modular repos Changes for Build #441 [jenkins CI] Auto create ovirt-4.3 Changes for Build #442 [jenkins CI] Auto create ovirt-4.3 Changes for Build #443 [jenkins CI] Auto create ovirt-4.3 Changes for Build #444 [hbraha] network: explicitly set the filter to None Changes for Build #445 [hbraha] network: explicitly set the filter to None Changes for Build #446 [hbraha] network: explicitly set the filter to None Changes for Build #447 [hbraha] network: explicitly set the filter to None Changes for Build #448 [Eitan Raviv] Install specific version of decorator from PyPi Changes for Build #449 [Eitan Raviv] Install specific version of decorator from PyPi ----------------- Failed Tests: ----------------- 1 tests failed. FAILED: basic-suite-master.test-scenarios.004_basic_sanity_pytest.test_ha_recovery Error Message: AssertionError: False != True after 600 seconds Stack Trace: prefix = <ovirtlago.prefix.OvirtPrefix object at 0x7f8d14b1f2d0> @order_by(_TEST_LIST) def test_ha_recovery(prefix): engine = prefix.virt_env.engine_vm().get_api_v4().system_service() with test_utils.TestEvent(engine, [119, 9602, 506]): # VM_DOWN_ERROR event(119) # HA_VM_FAILED event event(9602) # VDS_INITIATED_RUN_VM event(506) vm_host = _vm_host(prefix, VM2_NAME) pid = vm_host.ssh(['pgrep', '-f', 'qemu.*guest=vm2']) vm_host.ssh(['kill', '-KILL', pid.out.decode('utf-8')]) vm_service = test_utils.get_vm_service(engine, VM2_NAME) testlib.assert_true_within_long(
lambda:
vm_service.get().status == types.VmStatus.UP ) ../basic-suite-master/test-scenarios/004_basic_sanity_pytest.py:788: _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ /usr/lib/python2.7/site-packages/ovirtlago/testlib.py:286: in assert_true_within_long assert_equals_within_long(func, True, allowed_exceptions) /usr/lib/python2.7/site-packages/ovirtlago/testlib.py:273: in assert_equals_within_long func, value, LONG_TIMEOUT, allowed_exceptions=allowed_exceptions _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ func = <function <lambda> at 0x7f8d113d2668>, value = True, timeout = 600 allowed_exceptions = [], initial_wait = 0 def assert_equals_within( func, value, timeout, allowed_exceptions=None, initial_wait=10 ): allowed_exceptions = allowed_exceptions or [] with utils.EggTimer(timeout) as timer: while not timer.elapsed(): try: res = func() if res == value: return except Exception as exc: if _instance_of_any(exc, allowed_exceptions): time.sleep(3) continue LOGGER.exception("Unhandled exception in %s", func) raise if initial_wait == 0: time.sleep(3) else: time.sleep(initial_wait) initial_wait = 0 try: raise AssertionError(
'%s != %s after %s seconds' % (res, value, timeout)
) E AssertionError: False != True after 600 seconds /usr/lib/python2.7/site-packages/ovirtlago/testlib.py:252: AssertionError
Project: https://jenkins.ovirt.org/job/ovirt-system-tests_compat-4.3-suite-master/ Build: https://jenkins.ovirt.org/job/ovirt-system-tests_compat-4.3-suite-master/450... Build Number: 450 Build Status: Still Failing Triggered By: Started by timer ------------------------------------- Changes Since Last Success: ------------------------------------- Changes for Build #441 [jenkins CI] Auto create ovirt-4.3 Changes for Build #442 [jenkins CI] Auto create ovirt-4.3 Changes for Build #443 [jenkins CI] Auto create ovirt-4.3 Changes for Build #444 [hbraha] network: explicitly set the filter to None Changes for Build #445 [hbraha] network: explicitly set the filter to None Changes for Build #446 [hbraha] network: explicitly set the filter to None Changes for Build #447 [hbraha] network: explicitly set the filter to None Changes for Build #448 [Eitan Raviv] Install specific version of decorator from PyPi Changes for Build #449 [Eitan Raviv] Install specific version of decorator from PyPi Changes for Build #450 [jenkins CI] Auto create ovirt-4.2-cq ----------------- Failed Tests: ----------------- 1 tests failed. FAILED: basic-suite-master.test-scenarios.004_basic_sanity_pytest.test_ha_recovery Error Message: AssertionError: False != True after 600 seconds Stack Trace: prefix = <ovirtlago.prefix.OvirtPrefix object at 0x7ff20b2142d0> @order_by(_TEST_LIST) def test_ha_recovery(prefix): engine = prefix.virt_env.engine_vm().get_api_v4().system_service() with test_utils.TestEvent(engine, [119, 9602, 506]): # VM_DOWN_ERROR event(119) # HA_VM_FAILED event event(9602) # VDS_INITIATED_RUN_VM event(506) vm_host = _vm_host(prefix, VM2_NAME) pid = vm_host.ssh(['pgrep', '-f', 'qemu.*guest=vm2']) vm_host.ssh(['kill', '-KILL', pid.out.decode('utf-8')]) vm_service = test_utils.get_vm_service(engine, VM2_NAME) testlib.assert_true_within_long(
lambda:
vm_service.get().status == types.VmStatus.UP ) ../basic-suite-master/test-scenarios/004_basic_sanity_pytest.py:788: _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ /usr/lib/python2.7/site-packages/ovirtlago/testlib.py:286: in assert_true_within_long assert_equals_within_long(func, True, allowed_exceptions) /usr/lib/python2.7/site-packages/ovirtlago/testlib.py:273: in assert_equals_within_long func, value, LONG_TIMEOUT, allowed_exceptions=allowed_exceptions _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ func = <function <lambda> at 0x7ff1fbac2b90>, value = True, timeout = 600 allowed_exceptions = [], initial_wait = 0 def assert_equals_within( func, value, timeout, allowed_exceptions=None, initial_wait=10 ): allowed_exceptions = allowed_exceptions or [] with utils.EggTimer(timeout) as timer: while not timer.elapsed(): try: res = func() if res == value: return except Exception as exc: if _instance_of_any(exc, allowed_exceptions): time.sleep(3) continue LOGGER.exception("Unhandled exception in %s", func) raise if initial_wait == 0: time.sleep(3) else: time.sleep(initial_wait) initial_wait = 0 try: raise AssertionError(
'%s != %s after %s seconds' % (res, value, timeout)
) E AssertionError: False != True after 600 seconds /usr/lib/python2.7/site-packages/ovirtlago/testlib.py:252: AssertionError
Project: https://jenkins.ovirt.org/job/ovirt-system-tests_compat-4.3-suite-master/ Build: https://jenkins.ovirt.org/job/ovirt-system-tests_compat-4.3-suite-master/451... Build Number: 451 Build Status: Still Failing Triggered By: Started by timer ------------------------------------- Changes Since Last Success: ------------------------------------- Changes for Build #442 [jenkins CI] Auto create ovirt-4.3 Changes for Build #443 [jenkins CI] Auto create ovirt-4.3 Changes for Build #444 [hbraha] network: explicitly set the filter to None Changes for Build #445 [hbraha] network: explicitly set the filter to None Changes for Build #446 [hbraha] network: explicitly set the filter to None Changes for Build #447 [hbraha] network: explicitly set the filter to None Changes for Build #448 [Eitan Raviv] Install specific version of decorator from PyPi Changes for Build #449 [Eitan Raviv] Install specific version of decorator from PyPi Changes for Build #450 [jenkins CI] Auto create ovirt-4.2-cq Changes for Build #451 [Yedidyah Bar David] HE: try to get status multiple times [Shlomi] Modified alert_old_patches to post additional info ----------------- Failed Tests: ----------------- 1 tests failed. FAILED: basic-suite-master.test-scenarios.004_basic_sanity_pytest.test_ha_recovery Error Message: AssertionError: False != True after 600 seconds Stack Trace: prefix = <ovirtlago.prefix.OvirtPrefix object at 0x7f0ea14502d0> @order_by(_TEST_LIST) def test_ha_recovery(prefix): engine = prefix.virt_env.engine_vm().get_api_v4().system_service() with test_utils.TestEvent(engine, [119, 9602, 506]): # VM_DOWN_ERROR event(119) # HA_VM_FAILED event event(9602) # VDS_INITIATED_RUN_VM event(506) vm_host = _vm_host(prefix, VM2_NAME) pid = vm_host.ssh(['pgrep', '-f', 'qemu.*guest=vm2']) vm_host.ssh(['kill', '-KILL', pid.out.decode('utf-8')]) vm_service = test_utils.get_vm_service(engine, VM2_NAME) testlib.assert_true_within_long(
lambda:
vm_service.get().status == types.VmStatus.UP ) ../basic-suite-master/test-scenarios/004_basic_sanity_pytest.py:788: _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ /usr/lib/python2.7/site-packages/ovirtlago/testlib.py:286: in assert_true_within_long assert_equals_within_long(func, True, allowed_exceptions) /usr/lib/python2.7/site-packages/ovirtlago/testlib.py:273: in assert_equals_within_long func, value, LONG_TIMEOUT, allowed_exceptions=allowed_exceptions _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ func = <function <lambda> at 0x7f0e99cbf488>, value = True, timeout = 600 allowed_exceptions = [], initial_wait = 0 def assert_equals_within( func, value, timeout, allowed_exceptions=None, initial_wait=10 ): allowed_exceptions = allowed_exceptions or [] with utils.EggTimer(timeout) as timer: while not timer.elapsed(): try: res = func() if res == value: return except Exception as exc: if _instance_of_any(exc, allowed_exceptions): time.sleep(3) continue LOGGER.exception("Unhandled exception in %s", func) raise if initial_wait == 0: time.sleep(3) else: time.sleep(initial_wait) initial_wait = 0 try: raise AssertionError(
'%s != %s after %s seconds' % (res, value, timeout)
) E AssertionError: False != True after 600 seconds /usr/lib/python2.7/site-packages/ovirtlago/testlib.py:252: AssertionError
Project: https://jenkins.ovirt.org/job/ovirt-system-tests_compat-4.3-suite-master/ Build: https://jenkins.ovirt.org/job/ovirt-system-tests_compat-4.3-suite-master/452... Build Number: 452 Build Status: Still Failing Triggered By: Started by timer ------------------------------------- Changes Since Last Success: ------------------------------------- Changes for Build #443 [jenkins CI] Auto create ovirt-4.3 Changes for Build #444 [hbraha] network: explicitly set the filter to None Changes for Build #445 [hbraha] network: explicitly set the filter to None Changes for Build #446 [hbraha] network: explicitly set the filter to None Changes for Build #447 [hbraha] network: explicitly set the filter to None Changes for Build #448 [Eitan Raviv] Install specific version of decorator from PyPi Changes for Build #449 [Eitan Raviv] Install specific version of decorator from PyPi Changes for Build #450 [jenkins CI] Auto create ovirt-4.2-cq Changes for Build #451 [Yedidyah Bar David] HE: try to get status multiple times [Shlomi] Modified alert_old_patches to post additional info Changes for Build #452 [Yedidyah Bar David] HE: try to get status multiple times ----------------- Failed Tests: ----------------- 1 tests failed. FAILED: basic-suite-master.test-scenarios.004_basic_sanity_pytest.test_ha_recovery Error Message: AssertionError: False != True after 600 seconds Stack Trace: prefix = <ovirtlago.prefix.OvirtPrefix object at 0x7fe08581f2d0> @order_by(_TEST_LIST) def test_ha_recovery(prefix): engine = prefix.virt_env.engine_vm().get_api_v4().system_service() with test_utils.TestEvent(engine, [119, 9602, 506]): # VM_DOWN_ERROR event(119) # HA_VM_FAILED event event(9602) # VDS_INITIATED_RUN_VM event(506) vm_host = _vm_host(prefix, VM2_NAME) pid = vm_host.ssh(['pgrep', '-f', 'qemu.*guest=vm2']) vm_host.ssh(['kill', '-KILL', pid.out.decode('utf-8')]) vm_service = test_utils.get_vm_service(engine, VM2_NAME) testlib.assert_true_within_long(
lambda:
vm_service.get().status == types.VmStatus.UP ) ../basic-suite-master/test-scenarios/004_basic_sanity_pytest.py:788: _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ /usr/lib/python2.7/site-packages/ovirtlago/testlib.py:286: in assert_true_within_long assert_equals_within_long(func, True, allowed_exceptions) /usr/lib/python2.7/site-packages/ovirtlago/testlib.py:273: in assert_equals_within_long func, value, LONG_TIMEOUT, allowed_exceptions=allowed_exceptions _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ func = <function <lambda> at 0x7fe07e0957d0>, value = True, timeout = 600 allowed_exceptions = [], initial_wait = 0 def assert_equals_within( func, value, timeout, allowed_exceptions=None, initial_wait=10 ): allowed_exceptions = allowed_exceptions or [] with utils.EggTimer(timeout) as timer: while not timer.elapsed(): try: res = func() if res == value: return except Exception as exc: if _instance_of_any(exc, allowed_exceptions): time.sleep(3) continue LOGGER.exception("Unhandled exception in %s", func) raise if initial_wait == 0: time.sleep(3) else: time.sleep(initial_wait) initial_wait = 0 try: raise AssertionError(
'%s != %s after %s seconds' % (res, value, timeout)
) E AssertionError: False != True after 600 seconds /usr/lib/python2.7/site-packages/ovirtlago/testlib.py:252: AssertionError
Project: https://jenkins.ovirt.org/job/ovirt-system-tests_compat-4.3-suite-master/ Build: https://jenkins.ovirt.org/job/ovirt-system-tests_compat-4.3-suite-master/453... Build Number: 453 Build Status: Still Failing Triggered By: Started by timer ------------------------------------- Changes Since Last Success: ------------------------------------- Changes for Build #443 [jenkins CI] Auto create ovirt-4.3 Changes for Build #444 [hbraha] network: explicitly set the filter to None Changes for Build #445 [hbraha] network: explicitly set the filter to None Changes for Build #446 [hbraha] network: explicitly set the filter to None Changes for Build #447 [hbraha] network: explicitly set the filter to None Changes for Build #448 [Eitan Raviv] Install specific version of decorator from PyPi Changes for Build #449 [Eitan Raviv] Install specific version of decorator from PyPi Changes for Build #450 [jenkins CI] Auto create ovirt-4.2-cq Changes for Build #451 [Yedidyah Bar David] HE: try to get status multiple times [Shlomi] Modified alert_old_patches to post additional info Changes for Build #452 [Yedidyah Bar David] HE: try to get status multiple times Changes for Build #453 [Yedidyah Bar David] HE: try to get status multiple times [Ehud Yonasi] ost_nightly: added jobs for nightly tests. ----------------- Failed Tests: ----------------- No tests ran.
Project: https://jenkins.ovirt.org/job/ovirt-system-tests_compat-4.3-suite-master/ Build: https://jenkins.ovirt.org/job/ovirt-system-tests_compat-4.3-suite-master/454... Build Number: 454 Build Status: Still Failing Triggered By: Started by timer ------------------------------------- Changes Since Last Success: ------------------------------------- Changes for Build #445 [hbraha] network: explicitly set the filter to None Changes for Build #446 [hbraha] network: explicitly set the filter to None Changes for Build #447 [hbraha] network: explicitly set the filter to None Changes for Build #448 [Eitan Raviv] Install specific version of decorator from PyPi Changes for Build #449 [Eitan Raviv] Install specific version of decorator from PyPi Changes for Build #450 [jenkins CI] Auto create ovirt-4.2-cq Changes for Build #451 [Yedidyah Bar David] HE: try to get status multiple times [Shlomi] Modified alert_old_patches to post additional info Changes for Build #452 [Yedidyah Bar David] HE: try to get status multiple times Changes for Build #453 [Yedidyah Bar David] HE: try to get status multiple times [Ehud Yonasi] ost_nightly: added jobs for nightly tests. Changes for Build #454 [Yedidyah Bar David] HE: try to get status multiple times [Evgheni Dereveanchin] Bump el7 mirror snapshots ----------------- Failed Tests: ----------------- 1 tests failed. FAILED: basic-suite-master.test-scenarios.004_basic_sanity_pytest.test_ha_recovery Error Message: AssertionError: False != True after 600 seconds Stack Trace: prefix = <ovirtlago.prefix.OvirtPrefix object at 0x7f2b5e5682d0> @order_by(_TEST_LIST) def test_ha_recovery(prefix): engine = prefix.virt_env.engine_vm().get_api_v4().system_service() with test_utils.TestEvent(engine, [119, 9602, 506]): # VM_DOWN_ERROR event(119) # HA_VM_FAILED event event(9602) # VDS_INITIATED_RUN_VM event(506) vm_host = _vm_host(prefix, VM2_NAME) pid = vm_host.ssh(['pgrep', '-f', 'qemu.*guest=vm2']) vm_host.ssh(['kill', '-KILL', pid.out.decode('utf-8')]) vm_service = test_utils.get_vm_service(engine, VM2_NAME) testlib.assert_true_within_long(
lambda:
vm_service.get().status == types.VmStatus.UP ) ../basic-suite-master/test-scenarios/004_basic_sanity_pytest.py:788: _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ /usr/lib/python2.7/site-packages/ovirtlago/testlib.py:286: in assert_true_within_long assert_equals_within_long(func, True, allowed_exceptions) /usr/lib/python2.7/site-packages/ovirtlago/testlib.py:273: in assert_equals_within_long func, value, LONG_TIMEOUT, allowed_exceptions=allowed_exceptions _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ func = <function <lambda> at 0x7f2b5c0f37d0>, value = True, timeout = 600 allowed_exceptions = [], initial_wait = 0 def assert_equals_within( func, value, timeout, allowed_exceptions=None, initial_wait=10 ): allowed_exceptions = allowed_exceptions or [] with utils.EggTimer(timeout) as timer: while not timer.elapsed(): try: res = func() if res == value: return except Exception as exc: if _instance_of_any(exc, allowed_exceptions): time.sleep(3) continue LOGGER.exception("Unhandled exception in %s", func) raise if initial_wait == 0: time.sleep(3) else: time.sleep(initial_wait) initial_wait = 0 try: raise AssertionError(
'%s != %s after %s seconds' % (res, value, timeout)
) E AssertionError: False != True after 600 seconds /usr/lib/python2.7/site-packages/ovirtlago/testlib.py:252: AssertionError
Project: https://jenkins.ovirt.org/job/ovirt-system-tests_compat-4.3-suite-master/ Build: https://jenkins.ovirt.org/job/ovirt-system-tests_compat-4.3-suite-master/455... Build Number: 455 Build Status: Still Failing Triggered By: Started by timer ------------------------------------- Changes Since Last Success: ------------------------------------- Changes for Build #446 [hbraha] network: explicitly set the filter to None Changes for Build #447 [hbraha] network: explicitly set the filter to None Changes for Build #448 [Eitan Raviv] Install specific version of decorator from PyPi Changes for Build #449 [Eitan Raviv] Install specific version of decorator from PyPi Changes for Build #450 [jenkins CI] Auto create ovirt-4.2-cq Changes for Build #451 [Yedidyah Bar David] HE: try to get status multiple times [Shlomi] Modified alert_old_patches to post additional info Changes for Build #452 [Yedidyah Bar David] HE: try to get status multiple times Changes for Build #453 [Yedidyah Bar David] HE: try to get status multiple times [Ehud Yonasi] ost_nightly: added jobs for nightly tests. Changes for Build #454 [Yedidyah Bar David] HE: try to get status multiple times [Evgheni Dereveanchin] Bump el7 mirror snapshots Changes for Build #455 [Marcin Sobczyk] ost-images: Use -f and -F to define image format ----------------- Failed Tests: ----------------- 1 tests failed. FAILED: basic-suite-master.test-scenarios.004_basic_sanity_pytest.test_ha_recovery Error Message: AssertionError: False != True after 600 seconds Stack Trace: prefix = <ovirtlago.prefix.OvirtPrefix object at 0x7f9da96ef2d0> @order_by(_TEST_LIST) def test_ha_recovery(prefix): engine = prefix.virt_env.engine_vm().get_api_v4().system_service() with test_utils.TestEvent(engine, [119, 9602, 506]): # VM_DOWN_ERROR event(119) # HA_VM_FAILED event event(9602) # VDS_INITIATED_RUN_VM event(506) vm_host = _vm_host(prefix, VM2_NAME) pid = vm_host.ssh(['pgrep', '-f', 'qemu.*guest=vm2']) vm_host.ssh(['kill', '-KILL', pid.out.decode('utf-8')]) vm_service = test_utils.get_vm_service(engine, VM2_NAME) testlib.assert_true_within_long(
lambda:
vm_service.get().status == types.VmStatus.UP ) ../basic-suite-master/test-scenarios/004_basic_sanity_pytest.py:788: _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ /usr/lib/python2.7/site-packages/ovirtlago/testlib.py:286: in assert_true_within_long assert_equals_within_long(func, True, allowed_exceptions) /usr/lib/python2.7/site-packages/ovirtlago/testlib.py:273: in assert_equals_within_long func, value, LONG_TIMEOUT, allowed_exceptions=allowed_exceptions _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ func = <function <lambda> at 0x7f9da802d2a8>, value = True, timeout = 600 allowed_exceptions = [], initial_wait = 0 def assert_equals_within( func, value, timeout, allowed_exceptions=None, initial_wait=10 ): allowed_exceptions = allowed_exceptions or [] with utils.EggTimer(timeout) as timer: while not timer.elapsed(): try: res = func() if res == value: return except Exception as exc: if _instance_of_any(exc, allowed_exceptions): time.sleep(3) continue LOGGER.exception("Unhandled exception in %s", func) raise if initial_wait == 0: time.sleep(3) else: time.sleep(initial_wait) initial_wait = 0 try: raise AssertionError(
'%s != %s after %s seconds' % (res, value, timeout)
) E AssertionError: False != True after 600 seconds /usr/lib/python2.7/site-packages/ovirtlago/testlib.py:252: AssertionError
Project: https://jenkins.ovirt.org/job/ovirt-system-tests_compat-4.3-suite-master/ Build: https://jenkins.ovirt.org/job/ovirt-system-tests_compat-4.3-suite-master/456... Build Number: 456 Build Status: Still Failing Triggered By: Started by timer ------------------------------------- Changes Since Last Success: ------------------------------------- Changes for Build #447 [hbraha] network: explicitly set the filter to None Changes for Build #448 [Eitan Raviv] Install specific version of decorator from PyPi Changes for Build #449 [Eitan Raviv] Install specific version of decorator from PyPi Changes for Build #450 [jenkins CI] Auto create ovirt-4.2-cq Changes for Build #451 [Yedidyah Bar David] HE: try to get status multiple times [Shlomi] Modified alert_old_patches to post additional info Changes for Build #452 [Yedidyah Bar David] HE: try to get status multiple times Changes for Build #453 [Yedidyah Bar David] HE: try to get status multiple times [Ehud Yonasi] ost_nightly: added jobs for nightly tests. Changes for Build #454 [Yedidyah Bar David] HE: try to get status multiple times [Evgheni Dereveanchin] Bump el7 mirror snapshots Changes for Build #455 [Marcin Sobczyk] ost-images: Use -f and -F to define image format Changes for Build #456 [Marcin Sobczyk] ost-images: Use -f and -F to define image format ----------------- Failed Tests: ----------------- 1 tests failed. FAILED: basic-suite-master.test-scenarios.004_basic_sanity_pytest.test_ha_recovery Error Message: AssertionError: False != True after 600 seconds Stack Trace: prefix = <ovirtlago.prefix.OvirtPrefix object at 0x7ff2a50bf2d0> @order_by(_TEST_LIST) def test_ha_recovery(prefix): engine = prefix.virt_env.engine_vm().get_api_v4().system_service() with test_utils.TestEvent(engine, [119, 9602, 506]): # VM_DOWN_ERROR event(119) # HA_VM_FAILED event event(9602) # VDS_INITIATED_RUN_VM event(506) vm_host = _vm_host(prefix, VM2_NAME) pid = vm_host.ssh(['pgrep', '-f', 'qemu.*guest=vm2']) vm_host.ssh(['kill', '-KILL', pid.out.decode('utf-8')]) vm_service = test_utils.get_vm_service(engine, VM2_NAME) testlib.assert_true_within_long(
lambda:
vm_service.get().status == types.VmStatus.UP ) ../basic-suite-master/test-scenarios/004_basic_sanity_pytest.py:788: _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ /usr/lib/python2.7/site-packages/ovirtlago/testlib.py:286: in assert_true_within_long assert_equals_within_long(func, True, allowed_exceptions) /usr/lib/python2.7/site-packages/ovirtlago/testlib.py:273: in assert_equals_within_long func, value, LONG_TIMEOUT, allowed_exceptions=allowed_exceptions _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ func = <function <lambda> at 0x7ff2a19dc5f0>, value = True, timeout = 600 allowed_exceptions = [], initial_wait = 0 def assert_equals_within( func, value, timeout, allowed_exceptions=None, initial_wait=10 ): allowed_exceptions = allowed_exceptions or [] with utils.EggTimer(timeout) as timer: while not timer.elapsed(): try: res = func() if res == value: return except Exception as exc: if _instance_of_any(exc, allowed_exceptions): time.sleep(3) continue LOGGER.exception("Unhandled exception in %s", func) raise if initial_wait == 0: time.sleep(3) else: time.sleep(initial_wait) initial_wait = 0 try: raise AssertionError(
'%s != %s after %s seconds' % (res, value, timeout)
) E AssertionError: False != True after 600 seconds /usr/lib/python2.7/site-packages/ovirtlago/testlib.py:252: AssertionError
Project: https://jenkins.ovirt.org/job/ovirt-system-tests_compat-4.3-suite-master/ Build: https://jenkins.ovirt.org/job/ovirt-system-tests_compat-4.3-suite-master/457... Build Number: 457 Build Status: Still Failing Triggered By: Started by timer ------------------------------------- Changes Since Last Success: ------------------------------------- Changes for Build #448 [Eitan Raviv] Install specific version of decorator from PyPi Changes for Build #449 [Eitan Raviv] Install specific version of decorator from PyPi Changes for Build #450 [jenkins CI] Auto create ovirt-4.2-cq Changes for Build #451 [Yedidyah Bar David] HE: try to get status multiple times [Shlomi] Modified alert_old_patches to post additional info Changes for Build #452 [Yedidyah Bar David] HE: try to get status multiple times Changes for Build #453 [Yedidyah Bar David] HE: try to get status multiple times [Ehud Yonasi] ost_nightly: added jobs for nightly tests. Changes for Build #454 [Yedidyah Bar David] HE: try to get status multiple times [Evgheni Dereveanchin] Bump el7 mirror snapshots Changes for Build #455 [Marcin Sobczyk] ost-images: Use -f and -F to define image format Changes for Build #456 [Marcin Sobczyk] ost-images: Use -f and -F to define image format Changes for Build #457 [Marcin Sobczyk] ost-images: Use -f and -F to define image format ----------------- Failed Tests: ----------------- 1 tests failed. FAILED: basic-suite-master.test-scenarios.004_basic_sanity_pytest.test_ha_recovery Error Message: AssertionError: False != True after 600 seconds Stack Trace: prefix = <ovirtlago.prefix.OvirtPrefix object at 0x7f8890f3a2d0> @order_by(_TEST_LIST) def test_ha_recovery(prefix): engine = prefix.virt_env.engine_vm().get_api_v4().system_service() with test_utils.TestEvent(engine, [119, 9602, 506]): # VM_DOWN_ERROR event(119) # HA_VM_FAILED event event(9602) # VDS_INITIATED_RUN_VM event(506) vm_host = _vm_host(prefix, VM2_NAME) pid = vm_host.ssh(['pgrep', '-f', 'qemu.*guest=vm2']) vm_host.ssh(['kill', '-KILL', pid.out.decode('utf-8')]) vm_service = test_utils.get_vm_service(engine, VM2_NAME) testlib.assert_true_within_long(
lambda:
vm_service.get().status == types.VmStatus.UP ) ../basic-suite-master/test-scenarios/004_basic_sanity_pytest.py:788: _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ /usr/lib/python2.7/site-packages/ovirtlago/testlib.py:286: in assert_true_within_long assert_equals_within_long(func, True, allowed_exceptions) /usr/lib/python2.7/site-packages/ovirtlago/testlib.py:273: in assert_equals_within_long func, value, LONG_TIMEOUT, allowed_exceptions=allowed_exceptions _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ func = <function <lambda> at 0x7f888d856de8>, value = True, timeout = 600 allowed_exceptions = [], initial_wait = 0 def assert_equals_within( func, value, timeout, allowed_exceptions=None, initial_wait=10 ): allowed_exceptions = allowed_exceptions or [] with utils.EggTimer(timeout) as timer: while not timer.elapsed(): try: res = func() if res == value: return except Exception as exc: if _instance_of_any(exc, allowed_exceptions): time.sleep(3) continue LOGGER.exception("Unhandled exception in %s", func) raise if initial_wait == 0: time.sleep(3) else: time.sleep(initial_wait) initial_wait = 0 try: raise AssertionError(
'%s != %s after %s seconds' % (res, value, timeout)
) E AssertionError: False != True after 600 seconds /usr/lib/python2.7/site-packages/ovirtlago/testlib.py:252: AssertionError
Project: https://jenkins.ovirt.org/job/ovirt-system-tests_compat-4.3-suite-master/ Build: https://jenkins.ovirt.org/job/ovirt-system-tests_compat-4.3-suite-master/458... Build Number: 458 Build Status: Still Failing Triggered By: Started by timer ------------------------------------- Changes Since Last Success: ------------------------------------- Changes for Build #449 [Eitan Raviv] Install specific version of decorator from PyPi Changes for Build #450 [jenkins CI] Auto create ovirt-4.2-cq Changes for Build #451 [Yedidyah Bar David] HE: try to get status multiple times [Shlomi] Modified alert_old_patches to post additional info Changes for Build #452 [Yedidyah Bar David] HE: try to get status multiple times Changes for Build #453 [Yedidyah Bar David] HE: try to get status multiple times [Ehud Yonasi] ost_nightly: added jobs for nightly tests. Changes for Build #454 [Yedidyah Bar David] HE: try to get status multiple times [Evgheni Dereveanchin] Bump el7 mirror snapshots Changes for Build #455 [Marcin Sobczyk] ost-images: Use -f and -F to define image format Changes for Build #456 [Marcin Sobczyk] ost-images: Use -f and -F to define image format Changes for Build #457 [Marcin Sobczyk] ost-images: Use -f and -F to define image format Changes for Build #458 [Evgeny Slutsky] Missing script package in el7 repo ----------------- Failed Tests: ----------------- 1 tests failed. FAILED: basic-suite-master.test-scenarios.004_basic_sanity_pytest.test_ha_recovery Error Message: AssertionError: False != True after 600 seconds Stack Trace: prefix = <ovirtlago.prefix.OvirtPrefix object at 0x7fd466ec2290> @order_by(_TEST_LIST) def test_ha_recovery(prefix): engine = prefix.virt_env.engine_vm().get_api_v4().system_service() with test_utils.TestEvent(engine, [119, 9602, 506]): # VM_DOWN_ERROR event(119) # HA_VM_FAILED event event(9602) # VDS_INITIATED_RUN_VM event(506) vm_host = _vm_host(prefix, VM2_NAME) pid = vm_host.ssh(['pgrep', '-f', 'qemu.*guest=vm2']) vm_host.ssh(['kill', '-KILL', pid.out.decode('utf-8')]) vm_service = test_utils.get_vm_service(engine, VM2_NAME) testlib.assert_true_within_long(
lambda:
vm_service.get().status == types.VmStatus.UP ) ../basic-suite-master/test-scenarios/004_basic_sanity_pytest.py:788: _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ /usr/lib/python2.7/site-packages/ovirtlago/testlib.py:286: in assert_true_within_long assert_equals_within_long(func, True, allowed_exceptions) /usr/lib/python2.7/site-packages/ovirtlago/testlib.py:273: in assert_equals_within_long func, value, LONG_TIMEOUT, allowed_exceptions=allowed_exceptions _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ func = <function <lambda> at 0x7fd464265398>, value = True, timeout = 600 allowed_exceptions = [], initial_wait = 0 def assert_equals_within( func, value, timeout, allowed_exceptions=None, initial_wait=10 ): allowed_exceptions = allowed_exceptions or [] with utils.EggTimer(timeout) as timer: while not timer.elapsed(): try: res = func() if res == value: return except Exception as exc: if _instance_of_any(exc, allowed_exceptions): time.sleep(3) continue LOGGER.exception("Unhandled exception in %s", func) raise if initial_wait == 0: time.sleep(3) else: time.sleep(initial_wait) initial_wait = 0 try: raise AssertionError(
'%s != %s after %s seconds' % (res, value, timeout)
) E AssertionError: False != True after 600 seconds /usr/lib/python2.7/site-packages/ovirtlago/testlib.py:252: AssertionError
Project: https://jenkins.ovirt.org/job/ovirt-system-tests_compat-4.3-suite-master/ Build: https://jenkins.ovirt.org/job/ovirt-system-tests_compat-4.3-suite-master/459... Build Number: 459 Build Status: Still Failing Triggered By: Started by timer ------------------------------------- Changes Since Last Success: ------------------------------------- Changes for Build #450 [jenkins CI] Auto create ovirt-4.2-cq Changes for Build #451 [Yedidyah Bar David] HE: try to get status multiple times [Shlomi] Modified alert_old_patches to post additional info Changes for Build #452 [Yedidyah Bar David] HE: try to get status multiple times Changes for Build #453 [Yedidyah Bar David] HE: try to get status multiple times [Ehud Yonasi] ost_nightly: added jobs for nightly tests. Changes for Build #454 [Yedidyah Bar David] HE: try to get status multiple times [Evgheni Dereveanchin] Bump el7 mirror snapshots Changes for Build #455 [Marcin Sobczyk] ost-images: Use -f and -F to define image format Changes for Build #456 [Marcin Sobczyk] ost-images: Use -f and -F to define image format Changes for Build #457 [Marcin Sobczyk] ost-images: Use -f and -F to define image format Changes for Build #458 [Evgeny Slutsky] Missing script package in el7 repo Changes for Build #459 [Evgeny Slutsky] Missing script package in el7 repo ----------------- Failed Tests: ----------------- 1 tests failed. FAILED: basic-suite-master.test-scenarios.004_basic_sanity_pytest.test_ha_recovery Error Message: AssertionError: False != True after 600 seconds Stack Trace: prefix = <ovirtlago.prefix.OvirtPrefix object at 0x7f17b5fe2290> @order_by(_TEST_LIST) def test_ha_recovery(prefix): engine = prefix.virt_env.engine_vm().get_api_v4().system_service() with test_utils.TestEvent(engine, [119, 9602, 506]): # VM_DOWN_ERROR event(119) # HA_VM_FAILED event event(9602) # VDS_INITIATED_RUN_VM event(506) vm_host = _vm_host(prefix, VM2_NAME) pid = vm_host.ssh(['pgrep', '-f', 'qemu.*guest=vm2']) vm_host.ssh(['kill', '-KILL', pid.out.decode('utf-8')]) vm_service = test_utils.get_vm_service(engine, VM2_NAME) testlib.assert_true_within_long(
lambda:
vm_service.get().status == types.VmStatus.UP ) ../basic-suite-master/test-scenarios/004_basic_sanity_pytest.py:788: _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ /usr/lib/python2.7/site-packages/ovirtlago/testlib.py:286: in assert_true_within_long assert_equals_within_long(func, True, allowed_exceptions) /usr/lib/python2.7/site-packages/ovirtlago/testlib.py:273: in assert_equals_within_long func, value, LONG_TIMEOUT, allowed_exceptions=allowed_exceptions _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ func = <function <lambda> at 0x7f17ae8e87d0>, value = True, timeout = 600 allowed_exceptions = [], initial_wait = 0 def assert_equals_within( func, value, timeout, allowed_exceptions=None, initial_wait=10 ): allowed_exceptions = allowed_exceptions or [] with utils.EggTimer(timeout) as timer: while not timer.elapsed(): try: res = func() if res == value: return except Exception as exc: if _instance_of_any(exc, allowed_exceptions): time.sleep(3) continue LOGGER.exception("Unhandled exception in %s", func) raise if initial_wait == 0: time.sleep(3) else: time.sleep(initial_wait) initial_wait = 0 try: raise AssertionError(
'%s != %s after %s seconds' % (res, value, timeout)
) E AssertionError: False != True after 600 seconds /usr/lib/python2.7/site-packages/ovirtlago/testlib.py:252: AssertionError
Project: https://jenkins.ovirt.org/job/ovirt-system-tests_compat-4.3-suite-master/ Build: https://jenkins.ovirt.org/job/ovirt-system-tests_compat-4.3-suite-master/460... Build Number: 460 Build Status: Still Failing Triggered By: Started by timer ------------------------------------- Changes Since Last Success: ------------------------------------- Changes for Build #451 [Yedidyah Bar David] HE: try to get status multiple times [Shlomi] Modified alert_old_patches to post additional info Changes for Build #452 [Yedidyah Bar David] HE: try to get status multiple times Changes for Build #453 [Yedidyah Bar David] HE: try to get status multiple times [Ehud Yonasi] ost_nightly: added jobs for nightly tests. Changes for Build #454 [Yedidyah Bar David] HE: try to get status multiple times [Evgheni Dereveanchin] Bump el7 mirror snapshots Changes for Build #455 [Marcin Sobczyk] ost-images: Use -f and -F to define image format Changes for Build #456 [Marcin Sobczyk] ost-images: Use -f and -F to define image format Changes for Build #457 [Marcin Sobczyk] ost-images: Use -f and -F to define image format Changes for Build #458 [Evgeny Slutsky] Missing script package in el7 repo Changes for Build #459 [Evgeny Slutsky] Missing script package in el7 repo Changes for Build #460 [Evgeny Slutsky] Missing script package in el7 repo ----------------- Failed Tests: ----------------- 1 tests failed. FAILED: basic-suite-master.test-scenarios.004_basic_sanity_pytest.test_ha_recovery Error Message: AssertionError: False != True after 600 seconds Stack Trace: prefix = <ovirtlago.prefix.OvirtPrefix object at 0x7ff39a253290> @order_by(_TEST_LIST) def test_ha_recovery(prefix): engine = prefix.virt_env.engine_vm().get_api_v4().system_service() with test_utils.TestEvent(engine, [119, 9602, 506]): # VM_DOWN_ERROR event(119) # HA_VM_FAILED event event(9602) # VDS_INITIATED_RUN_VM event(506) vm_host = _vm_host(prefix, VM2_NAME) pid = vm_host.ssh(['pgrep', '-f', 'qemu.*guest=vm2']) vm_host.ssh(['kill', '-KILL', pid.out.decode('utf-8')]) vm_service = test_utils.get_vm_service(engine, VM2_NAME) testlib.assert_true_within_long(
lambda:
vm_service.get().status == types.VmStatus.UP ) ../basic-suite-master/test-scenarios/004_basic_sanity_pytest.py:788: _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ /usr/lib/python2.7/site-packages/ovirtlago/testlib.py:286: in assert_true_within_long assert_equals_within_long(func, True, allowed_exceptions) /usr/lib/python2.7/site-packages/ovirtlago/testlib.py:273: in assert_equals_within_long func, value, LONG_TIMEOUT, allowed_exceptions=allowed_exceptions _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ func = <function <lambda> at 0x7ff39806bd70>, value = True, timeout = 600 allowed_exceptions = [], initial_wait = 0 def assert_equals_within( func, value, timeout, allowed_exceptions=None, initial_wait=10 ): allowed_exceptions = allowed_exceptions or [] with utils.EggTimer(timeout) as timer: while not timer.elapsed(): try: res = func() if res == value: return except Exception as exc: if _instance_of_any(exc, allowed_exceptions): time.sleep(3) continue LOGGER.exception("Unhandled exception in %s", func) raise if initial_wait == 0: time.sleep(3) else: time.sleep(initial_wait) initial_wait = 0 try: raise AssertionError(
'%s != %s after %s seconds' % (res, value, timeout)
) E AssertionError: False != True after 600 seconds /usr/lib/python2.7/site-packages/ovirtlago/testlib.py:252: AssertionError
Project: https://jenkins.ovirt.org/job/ovirt-system-tests_compat-4.3-suite-master/ Build: https://jenkins.ovirt.org/job/ovirt-system-tests_compat-4.3-suite-master/461... Build Number: 461 Build Status: Still Failing Triggered By: Started by timer ------------------------------------- Changes Since Last Success: ------------------------------------- Changes for Build #452 [Yedidyah Bar David] HE: try to get status multiple times Changes for Build #453 [Yedidyah Bar David] HE: try to get status multiple times [Ehud Yonasi] ost_nightly: added jobs for nightly tests. Changes for Build #454 [Yedidyah Bar David] HE: try to get status multiple times [Evgheni Dereveanchin] Bump el7 mirror snapshots Changes for Build #455 [Marcin Sobczyk] ost-images: Use -f and -F to define image format Changes for Build #456 [Marcin Sobczyk] ost-images: Use -f and -F to define image format Changes for Build #457 [Marcin Sobczyk] ost-images: Use -f and -F to define image format Changes for Build #458 [Evgeny Slutsky] Missing script package in el7 repo Changes for Build #459 [Evgeny Slutsky] Missing script package in el7 repo Changes for Build #460 [Evgeny Slutsky] Missing script package in el7 repo Changes for Build #461 [Dominik Holler] basic-suite-master: remove skipping test_hotplug_nic [Ehud Yonasi] stdci_runner: add node label for templates. ----------------- Failed Tests: ----------------- 1 tests failed. FAILED: basic-suite-master.test-scenarios.004_basic_sanity_pytest.test_ha_recovery Error Message: AssertionError: False != True after 600 seconds Stack Trace: prefix = <ovirtlago.prefix.OvirtPrefix object at 0x7f0cf917f9d0> @order_by(_TEST_LIST) def test_ha_recovery(prefix): engine = prefix.virt_env.engine_vm().get_api_v4().system_service() with test_utils.TestEvent(engine, [119, 9602, 506]): # VM_DOWN_ERROR event(119) # HA_VM_FAILED event event(9602) # VDS_INITIATED_RUN_VM event(506) vm_host = _vm_host(prefix, VM2_NAME) pid = vm_host.ssh(['pgrep', '-f', 'qemu.*guest=vm2']) vm_host.ssh(['kill', '-KILL', pid.out.decode('utf-8')]) vm_service = test_utils.get_vm_service(engine, VM2_NAME) testlib.assert_true_within_long(
lambda:
vm_service.get().status == types.VmStatus.UP ) ../basic-suite-master/test-scenarios/004_basic_sanity_pytest.py:788: _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ /usr/lib/python2.7/site-packages/ovirtlago/testlib.py:286: in assert_true_within_long assert_equals_within_long(func, True, allowed_exceptions) /usr/lib/python2.7/site-packages/ovirtlago/testlib.py:273: in assert_equals_within_long func, value, LONG_TIMEOUT, allowed_exceptions=allowed_exceptions _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ func = <function <lambda> at 0x7f0cf59e9758>, value = True, timeout = 600 allowed_exceptions = [], initial_wait = 0 def assert_equals_within( func, value, timeout, allowed_exceptions=None, initial_wait=10 ): allowed_exceptions = allowed_exceptions or [] with utils.EggTimer(timeout) as timer: while not timer.elapsed(): try: res = func() if res == value: return except Exception as exc: if _instance_of_any(exc, allowed_exceptions): time.sleep(3) continue LOGGER.exception("Unhandled exception in %s", func) raise if initial_wait == 0: time.sleep(3) else: time.sleep(initial_wait) initial_wait = 0 try: raise AssertionError(
'%s != %s after %s seconds' % (res, value, timeout)
) E AssertionError: False != True after 600 seconds /usr/lib/python2.7/site-packages/ovirtlago/testlib.py:252: AssertionError
Project: https://jenkins.ovirt.org/job/ovirt-system-tests_compat-4.3-suite-master/ Build: https://jenkins.ovirt.org/job/ovirt-system-tests_compat-4.3-suite-master/462... Build Number: 462 Build Status: Still Failing Triggered By: Started by timer ------------------------------------- Changes Since Last Success: ------------------------------------- Changes for Build #453 [Yedidyah Bar David] HE: try to get status multiple times [Ehud Yonasi] ost_nightly: added jobs for nightly tests. Changes for Build #454 [Yedidyah Bar David] HE: try to get status multiple times [Evgheni Dereveanchin] Bump el7 mirror snapshots Changes for Build #455 [Marcin Sobczyk] ost-images: Use -f and -F to define image format Changes for Build #456 [Marcin Sobczyk] ost-images: Use -f and -F to define image format Changes for Build #457 [Marcin Sobczyk] ost-images: Use -f and -F to define image format Changes for Build #458 [Evgeny Slutsky] Missing script package in el7 repo Changes for Build #459 [Evgeny Slutsky] Missing script package in el7 repo Changes for Build #460 [Evgeny Slutsky] Missing script package in el7 repo Changes for Build #461 [Dominik Holler] basic-suite-master: remove skipping test_hotplug_nic [Ehud Yonasi] stdci_runner: add node label for templates. Changes for Build #462 [Dominik Holler] basic-suite-master: remove skipping test_hotplug_nic ----------------- Failed Tests: ----------------- 1 tests failed. FAILED: basic-suite-master.test-scenarios.004_basic_sanity_pytest.test_ha_recovery Error Message: AssertionError: False != True after 600 seconds Stack Trace: prefix = <ovirtlago.prefix.OvirtPrefix object at 0x7f11c9242a10> @order_by(_TEST_LIST) def test_ha_recovery(prefix): engine = prefix.virt_env.engine_vm().get_api_v4().system_service() with test_utils.TestEvent(engine, [119, 9602, 506]): # VM_DOWN_ERROR event(119) # HA_VM_FAILED event event(9602) # VDS_INITIATED_RUN_VM event(506) vm_host = _vm_host(prefix, VM2_NAME) pid = vm_host.ssh(['pgrep', '-f', 'qemu.*guest=vm2']) vm_host.ssh(['kill', '-KILL', pid.out.decode('utf-8')]) vm_service = test_utils.get_vm_service(engine, VM2_NAME) testlib.assert_true_within_long(
lambda:
vm_service.get().status == types.VmStatus.UP ) ../basic-suite-master/test-scenarios/004_basic_sanity_pytest.py:788: _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ /usr/lib/python2.7/site-packages/ovirtlago/testlib.py:286: in assert_true_within_long assert_equals_within_long(func, True, allowed_exceptions) /usr/lib/python2.7/site-packages/ovirtlago/testlib.py:273: in assert_equals_within_long func, value, LONG_TIMEOUT, allowed_exceptions=allowed_exceptions _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ func = <function <lambda> at 0x7f11c5a9d5f0>, value = True, timeout = 600 allowed_exceptions = [], initial_wait = 0 def assert_equals_within( func, value, timeout, allowed_exceptions=None, initial_wait=10 ): allowed_exceptions = allowed_exceptions or [] with utils.EggTimer(timeout) as timer: while not timer.elapsed(): try: res = func() if res == value: return except Exception as exc: if _instance_of_any(exc, allowed_exceptions): time.sleep(3) continue LOGGER.exception("Unhandled exception in %s", func) raise if initial_wait == 0: time.sleep(3) else: time.sleep(initial_wait) initial_wait = 0 try: raise AssertionError(
'%s != %s after %s seconds' % (res, value, timeout)
) E AssertionError: False != True after 600 seconds /usr/lib/python2.7/site-packages/ovirtlago/testlib.py:252: AssertionError
Project: https://jenkins.ovirt.org/job/ovirt-system-tests_compat-4.3-suite-master/ Build: https://jenkins.ovirt.org/job/ovirt-system-tests_compat-4.3-suite-master/463... Build Number: 463 Build Status: Still Failing Triggered By: Started by timer ------------------------------------- Changes Since Last Success: ------------------------------------- Changes for Build #454 [Yedidyah Bar David] HE: try to get status multiple times [Evgheni Dereveanchin] Bump el7 mirror snapshots Changes for Build #455 [Marcin Sobczyk] ost-images: Use -f and -F to define image format Changes for Build #456 [Marcin Sobczyk] ost-images: Use -f and -F to define image format Changes for Build #457 [Marcin Sobczyk] ost-images: Use -f and -F to define image format Changes for Build #458 [Evgeny Slutsky] Missing script package in el7 repo Changes for Build #459 [Evgeny Slutsky] Missing script package in el7 repo Changes for Build #460 [Evgeny Slutsky] Missing script package in el7 repo Changes for Build #461 [Dominik Holler] basic-suite-master: remove skipping test_hotplug_nic [Ehud Yonasi] stdci_runner: add node label for templates. Changes for Build #462 [Dominik Holler] basic-suite-master: remove skipping test_hotplug_nic Changes for Build #463 [Dominik Holler] basic-suite-master: remove skipping test_hotplug_nic ----------------- Failed Tests: ----------------- 1 tests failed. FAILED: basic-suite-master.test-scenarios.004_basic_sanity_pytest.test_ha_recovery Error Message: AssertionError: False != True after 600 seconds Stack Trace: prefix = <ovirtlago.prefix.OvirtPrefix object at 0x7f37fdf0aa10> @order_by(_TEST_LIST) def test_ha_recovery(prefix): engine = prefix.virt_env.engine_vm().get_api_v4().system_service() with test_utils.TestEvent(engine, [119, 9602, 506]): # VM_DOWN_ERROR event(119) # HA_VM_FAILED event event(9602) # VDS_INITIATED_RUN_VM event(506) vm_host = _vm_host(prefix, VM2_NAME) pid = vm_host.ssh(['pgrep', '-f', 'qemu.*guest=vm2']) vm_host.ssh(['kill', '-KILL', pid.out.decode('utf-8')]) vm_service = test_utils.get_vm_service(engine, VM2_NAME) testlib.assert_true_within_long(
lambda:
vm_service.get().status == types.VmStatus.UP ) ../basic-suite-master/test-scenarios/004_basic_sanity_pytest.py:788: _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ /usr/lib/python2.7/site-packages/ovirtlago/testlib.py:286: in assert_true_within_long assert_equals_within_long(func, True, allowed_exceptions) /usr/lib/python2.7/site-packages/ovirtlago/testlib.py:273: in assert_equals_within_long func, value, LONG_TIMEOUT, allowed_exceptions=allowed_exceptions _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ func = <function <lambda> at 0x7f37fc0a2578>, value = True, timeout = 600 allowed_exceptions = [], initial_wait = 0 def assert_equals_within( func, value, timeout, allowed_exceptions=None, initial_wait=10 ): allowed_exceptions = allowed_exceptions or [] with utils.EggTimer(timeout) as timer: while not timer.elapsed(): try: res = func() if res == value: return except Exception as exc: if _instance_of_any(exc, allowed_exceptions): time.sleep(3) continue LOGGER.exception("Unhandled exception in %s", func) raise if initial_wait == 0: time.sleep(3) else: time.sleep(initial_wait) initial_wait = 0 try: raise AssertionError(
'%s != %s after %s seconds' % (res, value, timeout)
) E AssertionError: False != True after 600 seconds /usr/lib/python2.7/site-packages/ovirtlago/testlib.py:252: AssertionError
Project: https://jenkins.ovirt.org/job/ovirt-system-tests_compat-4.3-suite-master/ Build: https://jenkins.ovirt.org/job/ovirt-system-tests_compat-4.3-suite-master/464... Build Number: 464 Build Status: Still Failing Triggered By: Started by timer ------------------------------------- Changes Since Last Success: ------------------------------------- Changes for Build #455 [Marcin Sobczyk] ost-images: Use -f and -F to define image format Changes for Build #456 [Marcin Sobczyk] ost-images: Use -f and -F to define image format Changes for Build #457 [Marcin Sobczyk] ost-images: Use -f and -F to define image format Changes for Build #458 [Evgeny Slutsky] Missing script package in el7 repo Changes for Build #459 [Evgeny Slutsky] Missing script package in el7 repo Changes for Build #460 [Evgeny Slutsky] Missing script package in el7 repo Changes for Build #461 [Dominik Holler] basic-suite-master: remove skipping test_hotplug_nic [Ehud Yonasi] stdci_runner: add node label for templates. Changes for Build #462 [Dominik Holler] basic-suite-master: remove skipping test_hotplug_nic Changes for Build #463 [Dominik Holler] basic-suite-master: remove skipping test_hotplug_nic Changes for Build #464 [Dominik Holler] basic-suite-master: remove skipping test_hotplug_nic ----------------- Failed Tests: ----------------- 1 tests failed. FAILED: basic-suite-master.test-scenarios.004_basic_sanity_pytest.test_ha_recovery Error Message: AssertionError: False != True after 600 seconds Stack Trace: prefix = <ovirtlago.prefix.OvirtPrefix object at 0x7f9f82156a10> @order_by(_TEST_LIST) def test_ha_recovery(prefix): engine = prefix.virt_env.engine_vm().get_api_v4().system_service() with test_utils.TestEvent(engine, [119, 9602, 506]): # VM_DOWN_ERROR event(119) # HA_VM_FAILED event event(9602) # VDS_INITIATED_RUN_VM event(506) vm_host = _vm_host(prefix, VM2_NAME) pid = vm_host.ssh(['pgrep', '-f', 'qemu.*guest=vm2']) vm_host.ssh(['kill', '-KILL', pid.out.decode('utf-8')]) vm_service = test_utils.get_vm_service(engine, VM2_NAME) testlib.assert_true_within_long(
lambda:
vm_service.get().status == types.VmStatus.UP ) ../basic-suite-master/test-scenarios/004_basic_sanity_pytest.py:788: _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ /usr/lib/python2.7/site-packages/ovirtlago/testlib.py:286: in assert_true_within_long assert_equals_within_long(func, True, allowed_exceptions) /usr/lib/python2.7/site-packages/ovirtlago/testlib.py:273: in assert_equals_within_long func, value, LONG_TIMEOUT, allowed_exceptions=allowed_exceptions _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ func = <function <lambda> at 0x7f9f8011bd70>, value = True, timeout = 600 allowed_exceptions = [], initial_wait = 0 def assert_equals_within( func, value, timeout, allowed_exceptions=None, initial_wait=10 ): allowed_exceptions = allowed_exceptions or [] with utils.EggTimer(timeout) as timer: while not timer.elapsed(): try: res = func() if res == value: return except Exception as exc: if _instance_of_any(exc, allowed_exceptions): time.sleep(3) continue LOGGER.exception("Unhandled exception in %s", func) raise if initial_wait == 0: time.sleep(3) else: time.sleep(initial_wait) initial_wait = 0 try: raise AssertionError(
'%s != %s after %s seconds' % (res, value, timeout)
) E AssertionError: False != True after 600 seconds /usr/lib/python2.7/site-packages/ovirtlago/testlib.py:252: AssertionError
Project: https://jenkins.ovirt.org/job/ovirt-system-tests_compat-4.3-suite-master/ Build: https://jenkins.ovirt.org/job/ovirt-system-tests_compat-4.3-suite-master/465... Build Number: 465 Build Status: Still Failing Triggered By: Started by timer ------------------------------------- Changes Since Last Success: ------------------------------------- Changes for Build #456 [Marcin Sobczyk] ost-images: Use -f and -F to define image format Changes for Build #457 [Marcin Sobczyk] ost-images: Use -f and -F to define image format Changes for Build #458 [Evgeny Slutsky] Missing script package in el7 repo Changes for Build #459 [Evgeny Slutsky] Missing script package in el7 repo Changes for Build #460 [Evgeny Slutsky] Missing script package in el7 repo Changes for Build #461 [Dominik Holler] basic-suite-master: remove skipping test_hotplug_nic [Ehud Yonasi] stdci_runner: add node label for templates. Changes for Build #462 [Dominik Holler] basic-suite-master: remove skipping test_hotplug_nic Changes for Build #463 [Dominik Holler] basic-suite-master: remove skipping test_hotplug_nic Changes for Build #464 [Dominik Holler] basic-suite-master: remove skipping test_hotplug_nic Changes for Build #465 [Yedidyah Bar David] he-node master: Search node images in el8 repo ----------------- Failed Tests: ----------------- 1 tests failed. FAILED: basic-suite-master.test-scenarios.004_basic_sanity_pytest.test_ha_recovery Error Message: AssertionError: False != True after 600 seconds Stack Trace: prefix = <ovirtlago.prefix.OvirtPrefix object at 0x7fd2e25b0a10> @order_by(_TEST_LIST) def test_ha_recovery(prefix): engine = prefix.virt_env.engine_vm().get_api_v4().system_service() with test_utils.TestEvent(engine, [119, 9602, 506]): # VM_DOWN_ERROR event(119) # HA_VM_FAILED event event(9602) # VDS_INITIATED_RUN_VM event(506) vm_host = _vm_host(prefix, VM2_NAME) pid = vm_host.ssh(['pgrep', '-f', 'qemu.*guest=vm2']) vm_host.ssh(['kill', '-KILL', pid.out.decode('utf-8')]) vm_service = test_utils.get_vm_service(engine, VM2_NAME) testlib.assert_true_within_long(
lambda:
vm_service.get().status == types.VmStatus.UP ) ../basic-suite-master/test-scenarios/004_basic_sanity_pytest.py:788: _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ /usr/lib/python2.7/site-packages/ovirtlago/testlib.py:286: in assert_true_within_long assert_equals_within_long(func, True, allowed_exceptions) /usr/lib/python2.7/site-packages/ovirtlago/testlib.py:273: in assert_equals_within_long func, value, LONG_TIMEOUT, allowed_exceptions=allowed_exceptions _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ func = <function <lambda> at 0x7fd2e01968c0>, value = True, timeout = 600 allowed_exceptions = [], initial_wait = 0 def assert_equals_within( func, value, timeout, allowed_exceptions=None, initial_wait=10 ): allowed_exceptions = allowed_exceptions or [] with utils.EggTimer(timeout) as timer: while not timer.elapsed(): try: res = func() if res == value: return except Exception as exc: if _instance_of_any(exc, allowed_exceptions): time.sleep(3) continue LOGGER.exception("Unhandled exception in %s", func) raise if initial_wait == 0: time.sleep(3) else: time.sleep(initial_wait) initial_wait = 0 try: raise AssertionError(
'%s != %s after %s seconds' % (res, value, timeout)
) E AssertionError: False != True after 600 seconds /usr/lib/python2.7/site-packages/ovirtlago/testlib.py:252: AssertionError
Project: https://jenkins.ovirt.org/job/ovirt-system-tests_compat-4.3-suite-master/ Build: https://jenkins.ovirt.org/job/ovirt-system-tests_compat-4.3-suite-master/466... Build Number: 466 Build Status: Still Failing Triggered By: Started by timer ------------------------------------- Changes Since Last Success: ------------------------------------- Changes for Build #457 [Marcin Sobczyk] ost-images: Use -f and -F to define image format Changes for Build #458 [Evgeny Slutsky] Missing script package in el7 repo Changes for Build #459 [Evgeny Slutsky] Missing script package in el7 repo Changes for Build #460 [Evgeny Slutsky] Missing script package in el7 repo Changes for Build #461 [Dominik Holler] basic-suite-master: remove skipping test_hotplug_nic [Ehud Yonasi] stdci_runner: add node label for templates. Changes for Build #462 [Dominik Holler] basic-suite-master: remove skipping test_hotplug_nic Changes for Build #463 [Dominik Holler] basic-suite-master: remove skipping test_hotplug_nic Changes for Build #464 [Dominik Holler] basic-suite-master: remove skipping test_hotplug_nic Changes for Build #465 [Yedidyah Bar David] he-node master: Search node images in el8 repo Changes for Build #466 [Yedidyah Bar David] he-node master: Search node images in el8 repo ----------------- Failed Tests: ----------------- 1 tests failed. FAILED: basic-suite-master.test-scenarios.004_basic_sanity_pytest.test_ha_recovery Error Message: AssertionError: False != True after 600 seconds Stack Trace: prefix = <ovirtlago.prefix.OvirtPrefix object at 0x7f344d58ea10> @order_by(_TEST_LIST) def test_ha_recovery(prefix): engine = prefix.virt_env.engine_vm().get_api_v4().system_service() with test_utils.TestEvent(engine, [119, 9602, 506]): # VM_DOWN_ERROR event(119) # HA_VM_FAILED event event(9602) # VDS_INITIATED_RUN_VM event(506) vm_host = _vm_host(prefix, VM2_NAME) pid = vm_host.ssh(['pgrep', '-f', 'qemu.*guest=vm2']) vm_host.ssh(['kill', '-KILL', pid.out.decode('utf-8')]) vm_service = test_utils.get_vm_service(engine, VM2_NAME) testlib.assert_true_within_long(
lambda:
vm_service.get().status == types.VmStatus.UP ) ../basic-suite-master/test-scenarios/004_basic_sanity_pytest.py:788: _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ /usr/lib/python2.7/site-packages/ovirtlago/testlib.py:286: in assert_true_within_long assert_equals_within_long(func, True, allowed_exceptions) /usr/lib/python2.7/site-packages/ovirtlago/testlib.py:273: in assert_equals_within_long func, value, LONG_TIMEOUT, allowed_exceptions=allowed_exceptions _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ func = <function <lambda> at 0x7f344c0ab578>, value = True, timeout = 600 allowed_exceptions = [], initial_wait = 0 def assert_equals_within( func, value, timeout, allowed_exceptions=None, initial_wait=10 ): allowed_exceptions = allowed_exceptions or [] with utils.EggTimer(timeout) as timer: while not timer.elapsed(): try: res = func() if res == value: return except Exception as exc: if _instance_of_any(exc, allowed_exceptions): time.sleep(3) continue LOGGER.exception("Unhandled exception in %s", func) raise if initial_wait == 0: time.sleep(3) else: time.sleep(initial_wait) initial_wait = 0 try: raise AssertionError(
'%s != %s after %s seconds' % (res, value, timeout)
) E AssertionError: False != True after 600 seconds /usr/lib/python2.7/site-packages/ovirtlago/testlib.py:252: AssertionError
Project: https://jenkins.ovirt.org/job/ovirt-system-tests_compat-4.3-suite-master/ Build: https://jenkins.ovirt.org/job/ovirt-system-tests_compat-4.3-suite-master/467... Build Number: 467 Build Status: Still Failing Triggered By: Started by timer ------------------------------------- Changes Since Last Success: ------------------------------------- Changes for Build #458 [Evgeny Slutsky] Missing script package in el7 repo Changes for Build #459 [Evgeny Slutsky] Missing script package in el7 repo Changes for Build #460 [Evgeny Slutsky] Missing script package in el7 repo Changes for Build #461 [Dominik Holler] basic-suite-master: remove skipping test_hotplug_nic [Ehud Yonasi] stdci_runner: add node label for templates. Changes for Build #462 [Dominik Holler] basic-suite-master: remove skipping test_hotplug_nic Changes for Build #463 [Dominik Holler] basic-suite-master: remove skipping test_hotplug_nic Changes for Build #464 [Dominik Holler] basic-suite-master: remove skipping test_hotplug_nic Changes for Build #465 [Yedidyah Bar David] he-node master: Search node images in el8 repo Changes for Build #466 [Yedidyah Bar David] he-node master: Search node images in el8 repo Changes for Build #467 [jenkins CI] Auto create ovirt-4.3 ----------------- Failed Tests: ----------------- 1 tests failed. FAILED: basic-suite-master.test-scenarios.004_basic_sanity_pytest.test_ha_recovery Error Message: AssertionError: False != True after 600 seconds Stack Trace: prefix = <ovirtlago.prefix.OvirtPrefix object at 0x7f3ef5182a10> @order_by(_TEST_LIST) def test_ha_recovery(prefix): engine = prefix.virt_env.engine_vm().get_api_v4().system_service() with test_utils.TestEvent(engine, [119, 9602, 506]): # VM_DOWN_ERROR event(119) # HA_VM_FAILED event event(9602) # VDS_INITIATED_RUN_VM event(506) vm_host = _vm_host(prefix, VM2_NAME) pid = vm_host.ssh(['pgrep', '-f', 'qemu.*guest=vm2']) vm_host.ssh(['kill', '-KILL', pid.out.decode('utf-8')]) vm_service = test_utils.get_vm_service(engine, VM2_NAME) testlib.assert_true_within_long(
lambda:
vm_service.get().status == types.VmStatus.UP ) ../basic-suite-master/test-scenarios/004_basic_sanity_pytest.py:788: _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ /usr/lib/python2.7/site-packages/ovirtlago/testlib.py:286: in assert_true_within_long assert_equals_within_long(func, True, allowed_exceptions) /usr/lib/python2.7/site-packages/ovirtlago/testlib.py:273: in assert_equals_within_long func, value, LONG_TIMEOUT, allowed_exceptions=allowed_exceptions _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ func = <function <lambda> at 0x7f3ef1d66758>, value = True, timeout = 600 allowed_exceptions = [], initial_wait = 0 def assert_equals_within( func, value, timeout, allowed_exceptions=None, initial_wait=10 ): allowed_exceptions = allowed_exceptions or [] with utils.EggTimer(timeout) as timer: while not timer.elapsed(): try: res = func() if res == value: return except Exception as exc: if _instance_of_any(exc, allowed_exceptions): time.sleep(3) continue LOGGER.exception("Unhandled exception in %s", func) raise if initial_wait == 0: time.sleep(3) else: time.sleep(initial_wait) initial_wait = 0 try: raise AssertionError(
'%s != %s after %s seconds' % (res, value, timeout)
) E AssertionError: False != True after 600 seconds /usr/lib/python2.7/site-packages/ovirtlago/testlib.py:252: AssertionError
Project: https://jenkins.ovirt.org/job/ovirt-system-tests_compat-4.3-suite-master/ Build: https://jenkins.ovirt.org/job/ovirt-system-tests_compat-4.3-suite-master/468... Build Number: 468 Build Status: Still Failing Triggered By: Started by timer ------------------------------------- Changes Since Last Success: ------------------------------------- Changes for Build #459 [Evgeny Slutsky] Missing script package in el7 repo Changes for Build #460 [Evgeny Slutsky] Missing script package in el7 repo Changes for Build #461 [Dominik Holler] basic-suite-master: remove skipping test_hotplug_nic [Ehud Yonasi] stdci_runner: add node label for templates. Changes for Build #462 [Dominik Holler] basic-suite-master: remove skipping test_hotplug_nic Changes for Build #463 [Dominik Holler] basic-suite-master: remove skipping test_hotplug_nic Changes for Build #464 [Dominik Holler] basic-suite-master: remove skipping test_hotplug_nic Changes for Build #465 [Yedidyah Bar David] he-node master: Search node images in el8 repo Changes for Build #466 [Yedidyah Bar David] he-node master: Search node images in el8 repo Changes for Build #467 [jenkins CI] Auto create ovirt-4.3 Changes for Build #468 [Marcin Sobczyk] ost-images: Drop rebasing of qcows [Ehud Yonasi] mock: fix yum repos injection. [Ehud Yonasi] onboard ost-images to stdci. ----------------- Failed Tests: ----------------- 1 tests failed. FAILED: basic-suite-master.test-scenarios.004_basic_sanity_pytest.test_ha_recovery Error Message: AssertionError: False != True after 600 seconds Stack Trace: prefix = <ovirtlago.prefix.OvirtPrefix object at 0x7fa0d5c86a10> @order_by(_TEST_LIST) def test_ha_recovery(prefix): engine = prefix.virt_env.engine_vm().get_api_v4().system_service() with test_utils.TestEvent(engine, [119, 9602, 506]): # VM_DOWN_ERROR event(119) # HA_VM_FAILED event event(9602) # VDS_INITIATED_RUN_VM event(506) vm_host = _vm_host(prefix, VM2_NAME) pid = vm_host.ssh(['pgrep', '-f', 'qemu.*guest=vm2']) vm_host.ssh(['kill', '-KILL', pid.out.decode('utf-8')]) vm_service = test_utils.get_vm_service(engine, VM2_NAME) testlib.assert_true_within_long(
lambda:
vm_service.get().status == types.VmStatus.UP ) ../basic-suite-master/test-scenarios/004_basic_sanity_pytest.py:788: _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ /usr/lib/python2.7/site-packages/ovirtlago/testlib.py:286: in assert_true_within_long assert_equals_within_long(func, True, allowed_exceptions) /usr/lib/python2.7/site-packages/ovirtlago/testlib.py:273: in assert_equals_within_long func, value, LONG_TIMEOUT, allowed_exceptions=allowed_exceptions _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ func = <function <lambda> at 0x7fa0d409a7d0>, value = True, timeout = 600 allowed_exceptions = [], initial_wait = 0 def assert_equals_within( func, value, timeout, allowed_exceptions=None, initial_wait=10 ): allowed_exceptions = allowed_exceptions or [] with utils.EggTimer(timeout) as timer: while not timer.elapsed(): try: res = func() if res == value: return except Exception as exc: if _instance_of_any(exc, allowed_exceptions): time.sleep(3) continue LOGGER.exception("Unhandled exception in %s", func) raise if initial_wait == 0: time.sleep(3) else: time.sleep(initial_wait) initial_wait = 0 try: raise AssertionError(
'%s != %s after %s seconds' % (res, value, timeout)
) E AssertionError: False != True after 600 seconds /usr/lib/python2.7/site-packages/ovirtlago/testlib.py:252: AssertionError
Project: https://jenkins.ovirt.org/job/ovirt-system-tests_compat-4.3-suite-master/ Build: https://jenkins.ovirt.org/job/ovirt-system-tests_compat-4.3-suite-master/469... Build Number: 469 Build Status: Still Failing Triggered By: Started by timer ------------------------------------- Changes Since Last Success: ------------------------------------- Changes for Build #460 [Evgeny Slutsky] Missing script package in el7 repo Changes for Build #461 [Dominik Holler] basic-suite-master: remove skipping test_hotplug_nic [Ehud Yonasi] stdci_runner: add node label for templates. Changes for Build #462 [Dominik Holler] basic-suite-master: remove skipping test_hotplug_nic Changes for Build #463 [Dominik Holler] basic-suite-master: remove skipping test_hotplug_nic Changes for Build #464 [Dominik Holler] basic-suite-master: remove skipping test_hotplug_nic Changes for Build #465 [Yedidyah Bar David] he-node master: Search node images in el8 repo Changes for Build #466 [Yedidyah Bar David] he-node master: Search node images in el8 repo Changes for Build #467 [jenkins CI] Auto create ovirt-4.3 Changes for Build #468 [Marcin Sobczyk] ost-images: Drop rebasing of qcows [Ehud Yonasi] mock: fix yum repos injection. [Ehud Yonasi] onboard ost-images to stdci. Changes for Build #469 [Marcin Sobczyk] ost-images: Drop rebasing of qcows ----------------- Failed Tests: ----------------- 1 tests failed. FAILED: basic-suite-master.test-scenarios.004_basic_sanity_pytest.test_ha_recovery Error Message: AssertionError: False != True after 600 seconds Stack Trace: prefix = <ovirtlago.prefix.OvirtPrefix object at 0x7f0ee9438a10> @order_by(_TEST_LIST) def test_ha_recovery(prefix): engine = prefix.virt_env.engine_vm().get_api_v4().system_service() with test_utils.TestEvent(engine, [119, 9602, 506]): # VM_DOWN_ERROR event(119) # HA_VM_FAILED event event(9602) # VDS_INITIATED_RUN_VM event(506) vm_host = _vm_host(prefix, VM2_NAME) pid = vm_host.ssh(['pgrep', '-f', 'qemu.*guest=vm2']) vm_host.ssh(['kill', '-KILL', pid.out.decode('utf-8')]) vm_service = test_utils.get_vm_service(engine, VM2_NAME) testlib.assert_true_within_long(
lambda:
vm_service.get().status == types.VmStatus.UP ) ../basic-suite-master/test-scenarios/004_basic_sanity_pytest.py:788: _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ /usr/lib/python2.7/site-packages/ovirtlago/testlib.py:286: in assert_true_within_long assert_equals_within_long(func, True, allowed_exceptions) /usr/lib/python2.7/site-packages/ovirtlago/testlib.py:273: in assert_equals_within_long func, value, LONG_TIMEOUT, allowed_exceptions=allowed_exceptions _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ func = <function <lambda> at 0x7f0ee92e1320>, value = True, timeout = 600 allowed_exceptions = [], initial_wait = 0 def assert_equals_within( func, value, timeout, allowed_exceptions=None, initial_wait=10 ): allowed_exceptions = allowed_exceptions or [] with utils.EggTimer(timeout) as timer: while not timer.elapsed(): try: res = func() if res == value: return except Exception as exc: if _instance_of_any(exc, allowed_exceptions): time.sleep(3) continue LOGGER.exception("Unhandled exception in %s", func) raise if initial_wait == 0: time.sleep(3) else: time.sleep(initial_wait) initial_wait = 0 try: raise AssertionError(
'%s != %s after %s seconds' % (res, value, timeout)
) E AssertionError: False != True after 600 seconds /usr/lib/python2.7/site-packages/ovirtlago/testlib.py:252: AssertionError
Project: https://jenkins.ovirt.org/job/ovirt-system-tests_compat-4.3-suite-master/ Build: https://jenkins.ovirt.org/job/ovirt-system-tests_compat-4.3-suite-master/470... Build Number: 470 Build Status: Still Failing Triggered By: Started by timer ------------------------------------- Changes Since Last Success: ------------------------------------- Changes for Build #461 [Dominik Holler] basic-suite-master: remove skipping test_hotplug_nic [Ehud Yonasi] stdci_runner: add node label for templates. Changes for Build #462 [Dominik Holler] basic-suite-master: remove skipping test_hotplug_nic Changes for Build #463 [Dominik Holler] basic-suite-master: remove skipping test_hotplug_nic Changes for Build #464 [Dominik Holler] basic-suite-master: remove skipping test_hotplug_nic Changes for Build #465 [Yedidyah Bar David] he-node master: Search node images in el8 repo Changes for Build #466 [Yedidyah Bar David] he-node master: Search node images in el8 repo Changes for Build #467 [jenkins CI] Auto create ovirt-4.3 Changes for Build #468 [Marcin Sobczyk] ost-images: Drop rebasing of qcows [Ehud Yonasi] mock: fix yum repos injection. [Ehud Yonasi] onboard ost-images to stdci. Changes for Build #469 [Marcin Sobczyk] ost-images: Drop rebasing of qcows Changes for Build #470 [Marcin Sobczyk] ost-images: Drop rebasing of qcows ----------------- Failed Tests: ----------------- 1 tests failed. FAILED: basic-suite-master.test-scenarios.004_basic_sanity_pytest.test_ha_recovery Error Message: AssertionError: False != True after 600 seconds Stack Trace: prefix = <ovirtlago.prefix.OvirtPrefix object at 0x7fa111964a10> @order_by(_TEST_LIST) def test_ha_recovery(prefix): engine = prefix.virt_env.engine_vm().get_api_v4().system_service() with test_utils.TestEvent(engine, [119, 9602, 506]): # VM_DOWN_ERROR event(119) # HA_VM_FAILED event event(9602) # VDS_INITIATED_RUN_VM event(506) vm_host = _vm_host(prefix, VM2_NAME) pid = vm_host.ssh(['pgrep', '-f', 'qemu.*guest=vm2']) vm_host.ssh(['kill', '-KILL', pid.out.decode('utf-8')]) vm_service = test_utils.get_vm_service(engine, VM2_NAME) testlib.assert_true_within_long(
lambda:
vm_service.get().status == types.VmStatus.UP ) ../basic-suite-master/test-scenarios/004_basic_sanity_pytest.py:788: _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ /usr/lib/python2.7/site-packages/ovirtlago/testlib.py:286: in assert_true_within_long assert_equals_within_long(func, True, allowed_exceptions) /usr/lib/python2.7/site-packages/ovirtlago/testlib.py:273: in assert_equals_within_long func, value, LONG_TIMEOUT, allowed_exceptions=allowed_exceptions _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ func = <function <lambda> at 0x7fa10a210cf8>, value = True, timeout = 600 allowed_exceptions = [], initial_wait = 0 def assert_equals_within( func, value, timeout, allowed_exceptions=None, initial_wait=10 ): allowed_exceptions = allowed_exceptions or [] with utils.EggTimer(timeout) as timer: while not timer.elapsed(): try: res = func() if res == value: return except Exception as exc: if _instance_of_any(exc, allowed_exceptions): time.sleep(3) continue LOGGER.exception("Unhandled exception in %s", func) raise if initial_wait == 0: time.sleep(3) else: time.sleep(initial_wait) initial_wait = 0 try: raise AssertionError(
'%s != %s after %s seconds' % (res, value, timeout)
) E AssertionError: False != True after 600 seconds /usr/lib/python2.7/site-packages/ovirtlago/testlib.py:252: AssertionError
Project: https://jenkins.ovirt.org/job/ovirt-system-tests_compat-4.3-suite-master/ Build: https://jenkins.ovirt.org/job/ovirt-system-tests_compat-4.3-suite-master/471... Build Number: 471 Build Status: Still Failing Triggered By: Started by timer ------------------------------------- Changes Since Last Success: ------------------------------------- Changes for Build #462 [Dominik Holler] basic-suite-master: remove skipping test_hotplug_nic Changes for Build #463 [Dominik Holler] basic-suite-master: remove skipping test_hotplug_nic Changes for Build #464 [Dominik Holler] basic-suite-master: remove skipping test_hotplug_nic Changes for Build #465 [Yedidyah Bar David] he-node master: Search node images in el8 repo Changes for Build #466 [Yedidyah Bar David] he-node master: Search node images in el8 repo Changes for Build #467 [jenkins CI] Auto create ovirt-4.3 Changes for Build #468 [Marcin Sobczyk] ost-images: Drop rebasing of qcows [Ehud Yonasi] mock: fix yum repos injection. [Ehud Yonasi] onboard ost-images to stdci. Changes for Build #469 [Marcin Sobczyk] ost-images: Drop rebasing of qcows Changes for Build #470 [Marcin Sobczyk] ost-images: Drop rebasing of qcows Changes for Build #471 [Marcin Sobczyk] ost-images: Drop rebasing of qcows ----------------- Failed Tests: ----------------- 1 tests failed. FAILED: basic-suite-master.test-scenarios.004_basic_sanity_pytest.test_ha_recovery Error Message: AssertionError: False != True after 600 seconds Stack Trace: prefix = <ovirtlago.prefix.OvirtPrefix object at 0x7fc2bd3c7a10> @order_by(_TEST_LIST) def test_ha_recovery(prefix): engine = prefix.virt_env.engine_vm().get_api_v4().system_service() with test_utils.TestEvent(engine, [119, 9602, 506]): # VM_DOWN_ERROR event(119) # HA_VM_FAILED event event(9602) # VDS_INITIATED_RUN_VM event(506) vm_host = _vm_host(prefix, VM2_NAME) pid = vm_host.ssh(['pgrep', '-f', 'qemu.*guest=vm2']) vm_host.ssh(['kill', '-KILL', pid.out.decode('utf-8')]) vm_service = test_utils.get_vm_service(engine, VM2_NAME) testlib.assert_true_within_long(
lambda:
vm_service.get().status == types.VmStatus.UP ) ../basic-suite-master/test-scenarios/004_basic_sanity_pytest.py:788: _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ /usr/lib/python2.7/site-packages/ovirtlago/testlib.py:286: in assert_true_within_long assert_equals_within_long(func, True, allowed_exceptions) /usr/lib/python2.7/site-packages/ovirtlago/testlib.py:273: in assert_equals_within_long func, value, LONG_TIMEOUT, allowed_exceptions=allowed_exceptions _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ func = <function <lambda> at 0x7fc2b9c77de8>, value = True, timeout = 600 allowed_exceptions = [], initial_wait = 0 def assert_equals_within( func, value, timeout, allowed_exceptions=None, initial_wait=10 ): allowed_exceptions = allowed_exceptions or [] with utils.EggTimer(timeout) as timer: while not timer.elapsed(): try: res = func() if res == value: return except Exception as exc: if _instance_of_any(exc, allowed_exceptions): time.sleep(3) continue LOGGER.exception("Unhandled exception in %s", func) raise if initial_wait == 0: time.sleep(3) else: time.sleep(initial_wait) initial_wait = 0 try: raise AssertionError(
'%s != %s after %s seconds' % (res, value, timeout)
) E AssertionError: False != True after 600 seconds /usr/lib/python2.7/site-packages/ovirtlago/testlib.py:252: AssertionError
Project: https://jenkins.ovirt.org/job/ovirt-system-tests_compat-4.3-suite-master/ Build: https://jenkins.ovirt.org/job/ovirt-system-tests_compat-4.3-suite-master/472... Build Number: 472 Build Status: Still Failing Triggered By: Started by timer ------------------------------------- Changes Since Last Success: ------------------------------------- Changes for Build #462 [Dominik Holler] basic-suite-master: remove skipping test_hotplug_nic Changes for Build #463 [Dominik Holler] basic-suite-master: remove skipping test_hotplug_nic Changes for Build #464 [Dominik Holler] basic-suite-master: remove skipping test_hotplug_nic Changes for Build #465 [Yedidyah Bar David] he-node master: Search node images in el8 repo Changes for Build #466 [Yedidyah Bar David] he-node master: Search node images in el8 repo Changes for Build #467 [jenkins CI] Auto create ovirt-4.3 Changes for Build #468 [Marcin Sobczyk] ost-images: Drop rebasing of qcows [Ehud Yonasi] mock: fix yum repos injection. [Ehud Yonasi] onboard ost-images to stdci. Changes for Build #469 [Marcin Sobczyk] ost-images: Drop rebasing of qcows Changes for Build #470 [Marcin Sobczyk] ost-images: Drop rebasing of qcows Changes for Build #471 [Marcin Sobczyk] ost-images: Drop rebasing of qcows Changes for Build #472 [Marcin Sobczyk] ost-images: Drop rebasing of qcows [Ehud Yonasi] stdci_runner: update templates node to ost-images. ----------------- Failed Tests: ----------------- No tests ran.
Project: https://jenkins.ovirt.org/job/ovirt-system-tests_compat-4.3-suite-master/ Build: https://jenkins.ovirt.org/job/ovirt-system-tests_compat-4.3-suite-master/473... Build Number: 473 Build Status: Still Failing Triggered By: Started by timer ------------------------------------- Changes Since Last Success: ------------------------------------- Changes for Build #464 [Dominik Holler] basic-suite-master: remove skipping test_hotplug_nic Changes for Build #465 [Yedidyah Bar David] he-node master: Search node images in el8 repo Changes for Build #466 [Yedidyah Bar David] he-node master: Search node images in el8 repo Changes for Build #467 [jenkins CI] Auto create ovirt-4.3 Changes for Build #468 [Marcin Sobczyk] ost-images: Drop rebasing of qcows [Ehud Yonasi] mock: fix yum repos injection. [Ehud Yonasi] onboard ost-images to stdci. Changes for Build #469 [Marcin Sobczyk] ost-images: Drop rebasing of qcows Changes for Build #470 [Marcin Sobczyk] ost-images: Drop rebasing of qcows Changes for Build #471 [Marcin Sobczyk] ost-images: Drop rebasing of qcows Changes for Build #472 [Marcin Sobczyk] ost-images: Drop rebasing of qcows [Ehud Yonasi] stdci_runner: update templates node to ost-images. Changes for Build #473 [Michal Skrivanek] test live storage migration again [Ehud Yonasi] poll: add ost-images to nightly. ----------------- Failed Tests: ----------------- 1 tests failed. FAILED: basic-suite-master.test-scenarios.004_basic_sanity_pytest.test_ha_recovery Error Message: AssertionError: False != True after 600 seconds Stack Trace: prefix = <ovirtlago.prefix.OvirtPrefix object at 0x7f59cbea5950> @order_by(_TEST_LIST) def test_ha_recovery(prefix): engine = prefix.virt_env.engine_vm().get_api_v4().system_service() with test_utils.TestEvent(engine, [119, 9602, 506]): # VM_DOWN_ERROR event(119) # HA_VM_FAILED event event(9602) # VDS_INITIATED_RUN_VM event(506) vm_host = _vm_host(prefix, VM2_NAME) pid = vm_host.ssh(['pgrep', '-f', 'qemu.*guest=vm2']) vm_host.ssh(['kill', '-KILL', pid.out.decode('utf-8')]) vm_service = test_utils.get_vm_service(engine, VM2_NAME) testlib.assert_true_within_long(
lambda:
vm_service.get().status == types.VmStatus.UP ) ../basic-suite-master/test-scenarios/004_basic_sanity_pytest.py:783: _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ /usr/lib/python2.7/site-packages/ovirtlago/testlib.py:286: in assert_true_within_long assert_equals_within_long(func, True, allowed_exceptions) /usr/lib/python2.7/site-packages/ovirtlago/testlib.py:273: in assert_equals_within_long func, value, LONG_TIMEOUT, allowed_exceptions=allowed_exceptions _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ func = <function <lambda> at 0x7f59c8a8a7d0>, value = True, timeout = 600 allowed_exceptions = [], initial_wait = 0 def assert_equals_within( func, value, timeout, allowed_exceptions=None, initial_wait=10 ): allowed_exceptions = allowed_exceptions or [] with utils.EggTimer(timeout) as timer: while not timer.elapsed(): try: res = func() if res == value: return except Exception as exc: if _instance_of_any(exc, allowed_exceptions): time.sleep(3) continue LOGGER.exception("Unhandled exception in %s", func) raise if initial_wait == 0: time.sleep(3) else: time.sleep(initial_wait) initial_wait = 0 try: raise AssertionError(
'%s != %s after %s seconds' % (res, value, timeout)
) E AssertionError: False != True after 600 seconds /usr/lib/python2.7/site-packages/ovirtlago/testlib.py:252: AssertionError
Project: https://jenkins.ovirt.org/job/ovirt-system-tests_compat-4.3-suite-master/ Build: https://jenkins.ovirt.org/job/ovirt-system-tests_compat-4.3-suite-master/474... Build Number: 474 Build Status: Still Failing Triggered By: Started by timer ------------------------------------- Changes Since Last Success: ------------------------------------- Changes for Build #465 [Yedidyah Bar David] he-node master: Search node images in el8 repo Changes for Build #466 [Yedidyah Bar David] he-node master: Search node images in el8 repo Changes for Build #467 [jenkins CI] Auto create ovirt-4.3 Changes for Build #468 [Marcin Sobczyk] ost-images: Drop rebasing of qcows [Ehud Yonasi] mock: fix yum repos injection. [Ehud Yonasi] onboard ost-images to stdci. Changes for Build #469 [Marcin Sobczyk] ost-images: Drop rebasing of qcows Changes for Build #470 [Marcin Sobczyk] ost-images: Drop rebasing of qcows Changes for Build #471 [Marcin Sobczyk] ost-images: Drop rebasing of qcows Changes for Build #472 [Marcin Sobczyk] ost-images: Drop rebasing of qcows [Ehud Yonasi] stdci_runner: update templates node to ost-images. Changes for Build #473 [Michal Skrivanek] test live storage migration again [Ehud Yonasi] poll: add ost-images to nightly. Changes for Build #474 [Artem Hrechanychenko] Move ansible installation to packages ----------------- Failed Tests: ----------------- 1 tests failed. FAILED: basic-suite-master.test-scenarios.004_basic_sanity_pytest.test_ha_recovery Error Message: AssertionError: False != True after 600 seconds Stack Trace: prefix = <ovirtlago.prefix.OvirtPrefix object at 0x7fb27681d0d0> @order_by(_TEST_LIST) def test_ha_recovery(prefix): engine = prefix.virt_env.engine_vm().get_api_v4().system_service() with test_utils.TestEvent(engine, [119, 9602, 506]): # VM_DOWN_ERROR event(119) # HA_VM_FAILED event event(9602) # VDS_INITIATED_RUN_VM event(506) vm_host = _vm_host(prefix, VM2_NAME) pid = vm_host.ssh(['pgrep', '-f', 'qemu.*guest=vm2']) vm_host.ssh(['kill', '-KILL', pid.out.decode('utf-8')]) vm_service = test_utils.get_vm_service(engine, VM2_NAME) testlib.assert_true_within_long(
lambda:
vm_service.get().status == types.VmStatus.UP ) ../basic-suite-master/test-scenarios/004_basic_sanity_pytest.py:783: _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ /usr/lib/python2.7/site-packages/ovirtlago/testlib.py:286: in assert_true_within_long assert_equals_within_long(func, True, allowed_exceptions) /usr/lib/python2.7/site-packages/ovirtlago/testlib.py:273: in assert_equals_within_long func, value, LONG_TIMEOUT, allowed_exceptions=allowed_exceptions _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ func = <function <lambda> at 0x7fb26f0c9b90>, value = True, timeout = 600 allowed_exceptions = [], initial_wait = 0 def assert_equals_within( func, value, timeout, allowed_exceptions=None, initial_wait=10 ): allowed_exceptions = allowed_exceptions or [] with utils.EggTimer(timeout) as timer: while not timer.elapsed(): try: res = func() if res == value: return except Exception as exc: if _instance_of_any(exc, allowed_exceptions): time.sleep(3) continue LOGGER.exception("Unhandled exception in %s", func) raise if initial_wait == 0: time.sleep(3) else: time.sleep(initial_wait) initial_wait = 0 try: raise AssertionError(
'%s != %s after %s seconds' % (res, value, timeout)
) E AssertionError: False != True after 600 seconds /usr/lib/python2.7/site-packages/ovirtlago/testlib.py:252: AssertionError
Project: https://jenkins.ovirt.org/job/ovirt-system-tests_compat-4.3-suite-master/ Build: https://jenkins.ovirt.org/job/ovirt-system-tests_compat-4.3-suite-master/475... Build Number: 475 Build Status: Still Failing Triggered By: Started by timer ------------------------------------- Changes Since Last Success: ------------------------------------- Changes for Build #466 [Yedidyah Bar David] he-node master: Search node images in el8 repo Changes for Build #467 [jenkins CI] Auto create ovirt-4.3 Changes for Build #468 [Marcin Sobczyk] ost-images: Drop rebasing of qcows [Ehud Yonasi] mock: fix yum repos injection. [Ehud Yonasi] onboard ost-images to stdci. Changes for Build #469 [Marcin Sobczyk] ost-images: Drop rebasing of qcows Changes for Build #470 [Marcin Sobczyk] ost-images: Drop rebasing of qcows Changes for Build #471 [Marcin Sobczyk] ost-images: Drop rebasing of qcows Changes for Build #472 [Marcin Sobczyk] ost-images: Drop rebasing of qcows [Ehud Yonasi] stdci_runner: update templates node to ost-images. Changes for Build #473 [Michal Skrivanek] test live storage migration again [Ehud Yonasi] poll: add ost-images to nightly. Changes for Build #474 [Artem Hrechanychenko] Move ansible installation to packages Changes for Build #475 [Galit] Add missing repo which has collectd-write_syslog [Sandro Bonazzola] Revert "ovirt-release: run only on fc29 nodes" ----------------- Failed Tests: ----------------- 1 tests failed. FAILED: basic-suite-master.test-scenarios.004_basic_sanity_pytest.test_ha_recovery Error Message: AssertionError: False != True after 600 seconds Stack Trace: prefix = <ovirtlago.prefix.OvirtPrefix object at 0x7f979edd7810> @order_by(_TEST_LIST) def test_ha_recovery(prefix): engine = prefix.virt_env.engine_vm().get_api_v4().system_service() with test_utils.TestEvent(engine, [119, 9602, 506]): # VM_DOWN_ERROR event(119) # HA_VM_FAILED event event(9602) # VDS_INITIATED_RUN_VM event(506) vm_host = _vm_host(prefix, VM2_NAME) pid = vm_host.ssh(['pgrep', '-f', 'qemu.*guest=vm2']) vm_host.ssh(['kill', '-KILL', pid.out.decode('utf-8')]) vm_service = test_utils.get_vm_service(engine, VM2_NAME) testlib.assert_true_within_long(
lambda:
vm_service.get().status == types.VmStatus.UP ) ../basic-suite-master/test-scenarios/004_basic_sanity_pytest.py:783: _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ /usr/lib/python2.7/site-packages/ovirtlago/testlib.py:286: in assert_true_within_long assert_equals_within_long(func, True, allowed_exceptions) /usr/lib/python2.7/site-packages/ovirtlago/testlib.py:273: in assert_equals_within_long func, value, LONG_TIMEOUT, allowed_exceptions=allowed_exceptions _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ func = <function <lambda> at 0x7f979c1e38c0>, value = True, timeout = 600 allowed_exceptions = [], initial_wait = 0 def assert_equals_within( func, value, timeout, allowed_exceptions=None, initial_wait=10 ): allowed_exceptions = allowed_exceptions or [] with utils.EggTimer(timeout) as timer: while not timer.elapsed(): try: res = func() if res == value: return except Exception as exc: if _instance_of_any(exc, allowed_exceptions): time.sleep(3) continue LOGGER.exception("Unhandled exception in %s", func) raise if initial_wait == 0: time.sleep(3) else: time.sleep(initial_wait) initial_wait = 0 try: raise AssertionError(
'%s != %s after %s seconds' % (res, value, timeout)
) E AssertionError: False != True after 600 seconds /usr/lib/python2.7/site-packages/ovirtlago/testlib.py:252: AssertionError
Project: https://jenkins.ovirt.org/job/ovirt-system-tests_compat-4.3-suite-master/ Build: https://jenkins.ovirt.org/job/ovirt-system-tests_compat-4.3-suite-master/476... Build Number: 476 Build Status: Still Failing Triggered By: Started by timer ------------------------------------- Changes Since Last Success: ------------------------------------- Changes for Build #467 [jenkins CI] Auto create ovirt-4.3 Changes for Build #468 [Marcin Sobczyk] ost-images: Drop rebasing of qcows [Ehud Yonasi] mock: fix yum repos injection. [Ehud Yonasi] onboard ost-images to stdci. Changes for Build #469 [Marcin Sobczyk] ost-images: Drop rebasing of qcows Changes for Build #470 [Marcin Sobczyk] ost-images: Drop rebasing of qcows Changes for Build #471 [Marcin Sobczyk] ost-images: Drop rebasing of qcows Changes for Build #472 [Marcin Sobczyk] ost-images: Drop rebasing of qcows [Ehud Yonasi] stdci_runner: update templates node to ost-images. Changes for Build #473 [Michal Skrivanek] test live storage migration again [Ehud Yonasi] poll: add ost-images to nightly. Changes for Build #474 [Artem Hrechanychenko] Move ansible installation to packages Changes for Build #475 [Galit] Add missing repo which has collectd-write_syslog [Sandro Bonazzola] Revert "ovirt-release: run only on fc29 nodes" Changes for Build #476 [Michal Skrivanek] make GLANCE failures fatal ----------------- Failed Tests: ----------------- 1 tests failed. FAILED: basic-suite-master.test-scenarios.004_basic_sanity_pytest.test_ha_recovery Error Message: AssertionError: False != True after 600 seconds Stack Trace: prefix = <ovirtlago.prefix.OvirtPrefix object at 0x7f368dfac810> @order_by(_TEST_LIST) def test_ha_recovery(prefix): engine = prefix.virt_env.engine_vm().get_api_v4().system_service() with test_utils.TestEvent(engine, [119, 9602, 506]): # VM_DOWN_ERROR event(119) # HA_VM_FAILED event event(9602) # VDS_INITIATED_RUN_VM event(506) vm_host = _vm_host(prefix, VM2_NAME) pid = vm_host.ssh(['pgrep', '-f', 'qemu.*guest=vm2']) vm_host.ssh(['kill', '-KILL', pid.out.decode('utf-8')]) vm_service = test_utils.get_vm_service(engine, VM2_NAME) testlib.assert_true_within_long(
lambda:
vm_service.get().status == types.VmStatus.UP ) ../basic-suite-master/test-scenarios/004_basic_sanity_pytest.py:783: _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ /usr/lib/python2.7/site-packages/ovirtlago/testlib.py:286: in assert_true_within_long assert_equals_within_long(func, True, allowed_exceptions) /usr/lib/python2.7/site-packages/ovirtlago/testlib.py:273: in assert_equals_within_long func, value, LONG_TIMEOUT, allowed_exceptions=allowed_exceptions _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ func = <function <lambda> at 0x7f368deb2320>, value = True, timeout = 600 allowed_exceptions = [], initial_wait = 0 def assert_equals_within( func, value, timeout, allowed_exceptions=None, initial_wait=10 ): allowed_exceptions = allowed_exceptions or [] with utils.EggTimer(timeout) as timer: while not timer.elapsed(): try: res = func() if res == value: return except Exception as exc: if _instance_of_any(exc, allowed_exceptions): time.sleep(3) continue LOGGER.exception("Unhandled exception in %s", func) raise if initial_wait == 0: time.sleep(3) else: time.sleep(initial_wait) initial_wait = 0 try: raise AssertionError(
'%s != %s after %s seconds' % (res, value, timeout)
) E AssertionError: False != True after 600 seconds /usr/lib/python2.7/site-packages/ovirtlago/testlib.py:252: AssertionError
Project: https://jenkins.ovirt.org/job/ovirt-system-tests_compat-4.3-suite-master/ Build: https://jenkins.ovirt.org/job/ovirt-system-tests_compat-4.3-suite-master/477... Build Number: 477 Build Status: Still Failing Triggered By: Started by timer ------------------------------------- Changes Since Last Success: ------------------------------------- Changes for Build #468 [Marcin Sobczyk] ost-images: Drop rebasing of qcows [Ehud Yonasi] mock: fix yum repos injection. [Ehud Yonasi] onboard ost-images to stdci. Changes for Build #469 [Marcin Sobczyk] ost-images: Drop rebasing of qcows Changes for Build #470 [Marcin Sobczyk] ost-images: Drop rebasing of qcows Changes for Build #471 [Marcin Sobczyk] ost-images: Drop rebasing of qcows Changes for Build #472 [Marcin Sobczyk] ost-images: Drop rebasing of qcows [Ehud Yonasi] stdci_runner: update templates node to ost-images. Changes for Build #473 [Michal Skrivanek] test live storage migration again [Ehud Yonasi] poll: add ost-images to nightly. Changes for Build #474 [Artem Hrechanychenko] Move ansible installation to packages Changes for Build #475 [Galit] Add missing repo which has collectd-write_syslog [Sandro Bonazzola] Revert "ovirt-release: run only on fc29 nodes" Changes for Build #476 [Michal Skrivanek] make GLANCE failures fatal Changes for Build #477 [Michal Skrivanek] use // in glance URL ----------------- Failed Tests: ----------------- 1 tests failed. FAILED: basic-suite-master.test-scenarios.002_bootstrap_pytest.test_download_engine_certs Error Message: AssertionError: False != True after 180 seconds Stack Trace: prefix = <ovirtlago.prefix.OvirtPrefix object at 0x7f7a951f85d0> @order_by(_TEST_LIST) def test_download_engine_certs(prefix): engine_ip = prefix.virt_env.engine_vm().ip() engine_base_url = '/ovirt-engine/services/pki-resource?resource=ca-certificate&format=' engine_ca_url = engine_base_url + 'X509-PEM-CA' engine_ssh_url = engine_base_url + 'OPENSSH-PUBKEY' # We use an unverified connection, as L0 host cannot resolve '...engine.lago.local' conn = http_client.HTTPSConnection(engine_ip, context=ssl._create_unverified_context()) def _download_file(url, path=None, compare_string=None): conn.request("GET", url) resp = conn.getresponse() if resp.status != 200: return False data = resp.read() if path: with open(path, 'wb') as outfile: outfile.write(data) if compare_string: assert data == compare_string return True testlib.assert_true_within_short( lambda: _download_file(engine_ca_url, 'engine-ca.pem') ) # TODO: verify certificate. Either use it, or run: # 'openssl x509 -in engine-ca.pem -text -noout' testlib.assert_true_within_short( lambda: _download_file(engine_ssh_url, 'engine-rsa.pub') ) # TODO: verify public key. Either use it, or run: # 'ssh-keygen -l -f engine-rsa.pub' healthy = b"DB Up!Welcome to Health Status!" testlib.assert_true_within_short(
lambda: _download_file('/ovirt-engine/services/health', path=None, compare_string=healthy)
) ../basic-suite-master/test-scenarios/002_bootstrap_pytest.py:1381: _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ /usr/lib/python2.7/site-packages/ovirtlago/testlib.py:282: in assert_true_within_short assert_equals_within_short(func, True, allowed_exceptions) /usr/lib/python2.7/site-packages/ovirtlago/testlib.py:266: in assert_equals_within_short func, value, SHORT_TIMEOUT, allowed_exceptions=allowed_exceptions _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ func = <function <lambda> at 0x7f7a950330c8>, value = True, timeout = 180 allowed_exceptions = [], initial_wait = 0 def assert_equals_within( func, value, timeout, allowed_exceptions=None, initial_wait=10 ): allowed_exceptions = allowed_exceptions or [] with utils.EggTimer(timeout) as timer: while not timer.elapsed(): try: res = func() if res == value: return except Exception as exc: if _instance_of_any(exc, allowed_exceptions): time.sleep(3) continue LOGGER.exception("Unhandled exception in %s", func) raise if initial_wait == 0: time.sleep(3) else: time.sleep(initial_wait) initial_wait = 0 try: raise AssertionError(
'%s != %s after %s seconds' % (res, value, timeout)
) E AssertionError: False != True after 180 seconds /usr/lib/python2.7/site-packages/ovirtlago/testlib.py:252: AssertionError
Project: https://jenkins.ovirt.org/job/ovirt-system-tests_compat-4.3-suite-master/ Build: https://jenkins.ovirt.org/job/ovirt-system-tests_compat-4.3-suite-master/478... Build Number: 478 Build Status: Still Failing Triggered By: Started by timer ------------------------------------- Changes Since Last Success: ------------------------------------- Changes for Build #469 [Marcin Sobczyk] ost-images: Drop rebasing of qcows Changes for Build #470 [Marcin Sobczyk] ost-images: Drop rebasing of qcows Changes for Build #471 [Marcin Sobczyk] ost-images: Drop rebasing of qcows Changes for Build #472 [Marcin Sobczyk] ost-images: Drop rebasing of qcows [Ehud Yonasi] stdci_runner: update templates node to ost-images. Changes for Build #473 [Michal Skrivanek] test live storage migration again [Ehud Yonasi] poll: add ost-images to nightly. Changes for Build #474 [Artem Hrechanychenko] Move ansible installation to packages Changes for Build #475 [Galit] Add missing repo which has collectd-write_syslog [Sandro Bonazzola] Revert "ovirt-release: run only on fc29 nodes" Changes for Build #476 [Michal Skrivanek] make GLANCE failures fatal Changes for Build #477 [Michal Skrivanek] use // in glance URL Changes for Build #478 [Galit] Fix: No module named ost_utils.memoized ----------------- Failed Tests: ----------------- 1 tests failed. FAILED: basic-suite-master.test-scenarios.004_basic_sanity_pytest.test_ha_recovery Error Message: AssertionError: False != True after 600 seconds Stack Trace: prefix = <ovirtlago.prefix.OvirtPrefix object at 0x7f393c4ce810> @order_by(_TEST_LIST) def test_ha_recovery(prefix): engine = prefix.virt_env.engine_vm().get_api_v4().system_service() with test_utils.TestEvent(engine, [119, 9602, 506]): # VM_DOWN_ERROR event(119) # HA_VM_FAILED event event(9602) # VDS_INITIATED_RUN_VM event(506) vm_host = _vm_host(prefix, VM2_NAME) pid = vm_host.ssh(['pgrep', '-f', 'qemu.*guest=vm2']) vm_host.ssh(['kill', '-KILL', pid.out.decode('utf-8')]) vm_service = test_utils.get_vm_service(engine, VM2_NAME) testlib.assert_true_within_long(
lambda:
vm_service.get().status == types.VmStatus.UP ) ../basic-suite-master/test-scenarios/004_basic_sanity_pytest.py:783: _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ /usr/lib/python2.7/site-packages/ovirtlago/testlib.py:286: in assert_true_within_long assert_equals_within_long(func, True, allowed_exceptions) /usr/lib/python2.7/site-packages/ovirtlago/testlib.py:273: in assert_equals_within_long func, value, LONG_TIMEOUT, allowed_exceptions=allowed_exceptions _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ func = <function <lambda> at 0x7f3938d93d70>, value = True, timeout = 600 allowed_exceptions = [], initial_wait = 0 def assert_equals_within( func, value, timeout, allowed_exceptions=None, initial_wait=10 ): allowed_exceptions = allowed_exceptions or [] with utils.EggTimer(timeout) as timer: while not timer.elapsed(): try: res = func() if res == value: return except Exception as exc: if _instance_of_any(exc, allowed_exceptions): time.sleep(3) continue LOGGER.exception("Unhandled exception in %s", func) raise if initial_wait == 0: time.sleep(3) else: time.sleep(initial_wait) initial_wait = 0 try: raise AssertionError(
'%s != %s after %s seconds' % (res, value, timeout)
) E AssertionError: False != True after 600 seconds /usr/lib/python2.7/site-packages/ovirtlago/testlib.py:252: AssertionError
Project: https://jenkins.ovirt.org/job/ovirt-system-tests_compat-4.3-suite-master/ Build: https://jenkins.ovirt.org/job/ovirt-system-tests_compat-4.3-suite-master/479... Build Number: 479 Build Status: Still Failing Triggered By: Started by timer ------------------------------------- Changes Since Last Success: ------------------------------------- Changes for Build #470 [Marcin Sobczyk] ost-images: Drop rebasing of qcows Changes for Build #471 [Marcin Sobczyk] ost-images: Drop rebasing of qcows Changes for Build #472 [Marcin Sobczyk] ost-images: Drop rebasing of qcows [Ehud Yonasi] stdci_runner: update templates node to ost-images. Changes for Build #473 [Michal Skrivanek] test live storage migration again [Ehud Yonasi] poll: add ost-images to nightly. Changes for Build #474 [Artem Hrechanychenko] Move ansible installation to packages Changes for Build #475 [Galit] Add missing repo which has collectd-write_syslog [Sandro Bonazzola] Revert "ovirt-release: run only on fc29 nodes" Changes for Build #476 [Michal Skrivanek] make GLANCE failures fatal Changes for Build #477 [Michal Skrivanek] use // in glance URL Changes for Build #478 [Galit] Fix: No module named ost_utils.memoized Changes for Build #479 [Martin Necas] ansible: fix deploy scripts to el8 [arachmani] Add arachman@redhat.com to jenkins recipient lists ----------------- Failed Tests: ----------------- 1 tests failed. FAILED: basic-suite-master.test-scenarios.004_basic_sanity_pytest.test_ha_recovery Error Message: AssertionError: False != True after 600 seconds Stack Trace: prefix = <ovirtlago.prefix.OvirtPrefix object at 0x7f1052bc2810> @order_by(_TEST_LIST) def test_ha_recovery(prefix): engine = prefix.virt_env.engine_vm().get_api_v4().system_service() with test_utils.TestEvent(engine, [119, 9602, 506]): # VM_DOWN_ERROR event(119) # HA_VM_FAILED event event(9602) # VDS_INITIATED_RUN_VM event(506) vm_host = _vm_host(prefix, VM2_NAME) pid = vm_host.ssh(['pgrep', '-f', 'qemu.*guest=vm2']) vm_host.ssh(['kill', '-KILL', pid.out.decode('utf-8')]) vm_service = test_utils.get_vm_service(engine, VM2_NAME) testlib.assert_true_within_long(
lambda:
vm_service.get().status == types.VmStatus.UP ) ../basic-suite-master/test-scenarios/004_basic_sanity_pytest.py:783: _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ /usr/lib/python2.7/site-packages/ovirtlago/testlib.py:286: in assert_true_within_long assert_equals_within_long(func, True, allowed_exceptions) /usr/lib/python2.7/site-packages/ovirtlago/testlib.py:273: in assert_equals_within_long func, value, LONG_TIMEOUT, allowed_exceptions=allowed_exceptions _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ func = <function <lambda> at 0x7f104b7ba7d0>, value = True, timeout = 600 allowed_exceptions = [], initial_wait = 0 def assert_equals_within( func, value, timeout, allowed_exceptions=None, initial_wait=10 ): allowed_exceptions = allowed_exceptions or [] with utils.EggTimer(timeout) as timer: while not timer.elapsed(): try: res = func() if res == value: return except Exception as exc: if _instance_of_any(exc, allowed_exceptions): time.sleep(3) continue LOGGER.exception("Unhandled exception in %s", func) raise if initial_wait == 0: time.sleep(3) else: time.sleep(initial_wait) initial_wait = 0 try: raise AssertionError(
'%s != %s after %s seconds' % (res, value, timeout)
) E AssertionError: False != True after 600 seconds /usr/lib/python2.7/site-packages/ovirtlago/testlib.py:252: AssertionError
Project: https://jenkins.ovirt.org/job/ovirt-system-tests_compat-4.3-suite-master/ Build: https://jenkins.ovirt.org/job/ovirt-system-tests_compat-4.3-suite-master/480... Build Number: 480 Build Status: Still Failing Triggered By: Started by timer ------------------------------------- Changes Since Last Success: ------------------------------------- Changes for Build #470 [Marcin Sobczyk] ost-images: Drop rebasing of qcows Changes for Build #471 [Marcin Sobczyk] ost-images: Drop rebasing of qcows Changes for Build #472 [Marcin Sobczyk] ost-images: Drop rebasing of qcows [Ehud Yonasi] stdci_runner: update templates node to ost-images. Changes for Build #473 [Michal Skrivanek] test live storage migration again [Ehud Yonasi] poll: add ost-images to nightly. Changes for Build #474 [Artem Hrechanychenko] Move ansible installation to packages Changes for Build #475 [Galit] Add missing repo which has collectd-write_syslog [Sandro Bonazzola] Revert "ovirt-release: run only on fc29 nodes" Changes for Build #476 [Michal Skrivanek] make GLANCE failures fatal Changes for Build #477 [Michal Skrivanek] use // in glance URL Changes for Build #478 [Galit] Fix: No module named ost_utils.memoized Changes for Build #479 [Martin Necas] ansible: fix deploy scripts to el8 [arachmani] Add arachman@redhat.com to jenkins recipient lists Changes for Build #480 [Martin Necas] ansible: fix deploy scripts to el8 [Evgheni Dereveanchin] Do not run jenkins check-patch job on fc29 ----------------- Failed Tests: ----------------- No tests ran.
Project: https://jenkins.ovirt.org/job/ovirt-system-tests_compat-4.3-suite-master/ Build: https://jenkins.ovirt.org/job/ovirt-system-tests_compat-4.3-suite-master/481... Build Number: 481 Build Status: Still Failing Triggered By: Started by timer ------------------------------------- Changes Since Last Success: ------------------------------------- Changes for Build #472 [Marcin Sobczyk] ost-images: Drop rebasing of qcows [Ehud Yonasi] stdci_runner: update templates node to ost-images. Changes for Build #473 [Michal Skrivanek] test live storage migration again [Ehud Yonasi] poll: add ost-images to nightly. Changes for Build #474 [Artem Hrechanychenko] Move ansible installation to packages Changes for Build #475 [Galit] Add missing repo which has collectd-write_syslog [Sandro Bonazzola] Revert "ovirt-release: run only on fc29 nodes" Changes for Build #476 [Michal Skrivanek] make GLANCE failures fatal Changes for Build #477 [Michal Skrivanek] use // in glance URL Changes for Build #478 [Galit] Fix: No module named ost_utils.memoized Changes for Build #479 [Martin Necas] ansible: fix deploy scripts to el8 [arachmani] Add arachman@redhat.com to jenkins recipient lists Changes for Build #480 [Martin Necas] ansible: fix deploy scripts to el8 [Evgheni Dereveanchin] Do not run jenkins check-patch job on fc29 Changes for Build #481 [Galit] Add centos8.2 official to templates ----------------- Failed Tests: ----------------- 1 tests failed. FAILED: basic-suite-master.test-scenarios.004_basic_sanity_pytest.test_ha_recovery Error Message: AssertionError: False != True after 600 seconds Stack Trace: prefix = <ovirtlago.prefix.OvirtPrefix object at 0x7f3bcb196810> @order_by(_TEST_LIST) def test_ha_recovery(prefix): engine = prefix.virt_env.engine_vm().get_api_v4().system_service() with test_utils.TestEvent(engine, [119, 9602, 506]): # VM_DOWN_ERROR event(119) # HA_VM_FAILED event event(9602) # VDS_INITIATED_RUN_VM event(506) vm_host = _vm_host(prefix, VM2_NAME) pid = vm_host.ssh(['pgrep', '-f', 'qemu.*guest=vm2']) vm_host.ssh(['kill', '-KILL', pid.out.decode('utf-8')]) vm_service = test_utils.get_vm_service(engine, VM2_NAME) testlib.assert_true_within_long(
lambda:
vm_service.get().status == types.VmStatus.UP ) ../basic-suite-master/test-scenarios/004_basic_sanity_pytest.py:783: _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ /usr/lib/python2.7/site-packages/ovirtlago/testlib.py:286: in assert_true_within_long assert_equals_within_long(func, True, allowed_exceptions) /usr/lib/python2.7/site-packages/ovirtlago/testlib.py:273: in assert_equals_within_long func, value, LONG_TIMEOUT, allowed_exceptions=allowed_exceptions _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ func = <function <lambda> at 0x7f3bbba00140>, value = True, timeout = 600 allowed_exceptions = [], initial_wait = 0 def assert_equals_within( func, value, timeout, allowed_exceptions=None, initial_wait=10 ): allowed_exceptions = allowed_exceptions or [] with utils.EggTimer(timeout) as timer: while not timer.elapsed(): try: res = func() if res == value: return except Exception as exc: if _instance_of_any(exc, allowed_exceptions): time.sleep(3) continue LOGGER.exception("Unhandled exception in %s", func) raise if initial_wait == 0: time.sleep(3) else: time.sleep(initial_wait) initial_wait = 0 try: raise AssertionError(
'%s != %s after %s seconds' % (res, value, timeout)
) E AssertionError: False != True after 600 seconds /usr/lib/python2.7/site-packages/ovirtlago/testlib.py:252: AssertionError
Project: https://jenkins.ovirt.org/job/ovirt-system-tests_compat-4.3-suite-master/ Build: https://jenkins.ovirt.org/job/ovirt-system-tests_compat-4.3-suite-master/482... Build Number: 482 Build Status: Still Failing Triggered By: Started by timer ------------------------------------- Changes Since Last Success: ------------------------------------- Changes for Build #473 [Michal Skrivanek] test live storage migration again [Ehud Yonasi] poll: add ost-images to nightly. Changes for Build #474 [Artem Hrechanychenko] Move ansible installation to packages Changes for Build #475 [Galit] Add missing repo which has collectd-write_syslog [Sandro Bonazzola] Revert "ovirt-release: run only on fc29 nodes" Changes for Build #476 [Michal Skrivanek] make GLANCE failures fatal Changes for Build #477 [Michal Skrivanek] use // in glance URL Changes for Build #478 [Galit] Fix: No module named ost_utils.memoized Changes for Build #479 [Martin Necas] ansible: fix deploy scripts to el8 [arachmani] Add arachman@redhat.com to jenkins recipient lists Changes for Build #480 [Martin Necas] ansible: fix deploy scripts to el8 [Evgheni Dereveanchin] Do not run jenkins check-patch job on fc29 Changes for Build #481 [Galit] Add centos8.2 official to templates Changes for Build #482 [Evgeny Slutsky] he-basic-suite-master: Add Domain_name to the host when vm_run [Gal Ben Haim] Add gcc to fcraw [Sandro Bonazzola] pipelines: add ovirt-dependencies ----------------- Failed Tests: ----------------- 1 tests failed. FAILED: basic-suite-master.test-scenarios.004_basic_sanity_pytest.test_ha_recovery Error Message: AssertionError: False != True after 600 seconds Stack Trace: prefix = <ovirtlago.prefix.OvirtPrefix object at 0x7fa79f96e810> @order_by(_TEST_LIST) def test_ha_recovery(prefix): engine = prefix.virt_env.engine_vm().get_api_v4().system_service() with test_utils.TestEvent(engine, [119, 9602, 506]): # VM_DOWN_ERROR event(119) # HA_VM_FAILED event event(9602) # VDS_INITIATED_RUN_VM event(506) vm_host = _vm_host(prefix, VM2_NAME) pid = vm_host.ssh(['pgrep', '-f', 'qemu.*guest=vm2']) vm_host.ssh(['kill', '-KILL', pid.out.decode('utf-8')]) vm_service = test_utils.get_vm_service(engine, VM2_NAME) testlib.assert_true_within_long(
lambda:
vm_service.get().status == types.VmStatus.UP ) ../basic-suite-master/test-scenarios/004_basic_sanity_pytest.py:783: _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ /usr/lib/python2.7/site-packages/ovirtlago/testlib.py:286: in assert_true_within_long assert_equals_within_long(func, True, allowed_exceptions) /usr/lib/python2.7/site-packages/ovirtlago/testlib.py:273: in assert_equals_within_long func, value, LONG_TIMEOUT, allowed_exceptions=allowed_exceptions _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ func = <function <lambda> at 0x7fa79c232ed8>, value = True, timeout = 600 allowed_exceptions = [], initial_wait = 0 def assert_equals_within( func, value, timeout, allowed_exceptions=None, initial_wait=10 ): allowed_exceptions = allowed_exceptions or [] with utils.EggTimer(timeout) as timer: while not timer.elapsed(): try: res = func() if res == value: return except Exception as exc: if _instance_of_any(exc, allowed_exceptions): time.sleep(3) continue LOGGER.exception("Unhandled exception in %s", func) raise if initial_wait == 0: time.sleep(3) else: time.sleep(initial_wait) initial_wait = 0 try: raise AssertionError(
'%s != %s after %s seconds' % (res, value, timeout)
) E AssertionError: False != True after 600 seconds /usr/lib/python2.7/site-packages/ovirtlago/testlib.py:252: AssertionError
Project: https://jenkins.ovirt.org/job/ovirt-system-tests_compat-4.3-suite-master/ Build: https://jenkins.ovirt.org/job/ovirt-system-tests_compat-4.3-suite-master/483... Build Number: 483 Build Status: Still Failing Triggered By: Started by timer ------------------------------------- Changes Since Last Success: ------------------------------------- Changes for Build #474 [Artem Hrechanychenko] Move ansible installation to packages Changes for Build #475 [Galit] Add missing repo which has collectd-write_syslog [Sandro Bonazzola] Revert "ovirt-release: run only on fc29 nodes" Changes for Build #476 [Michal Skrivanek] make GLANCE failures fatal Changes for Build #477 [Michal Skrivanek] use // in glance URL Changes for Build #478 [Galit] Fix: No module named ost_utils.memoized Changes for Build #479 [Martin Necas] ansible: fix deploy scripts to el8 [arachmani] Add arachman@redhat.com to jenkins recipient lists Changes for Build #480 [Martin Necas] ansible: fix deploy scripts to el8 [Evgheni Dereveanchin] Do not run jenkins check-patch job on fc29 Changes for Build #481 [Galit] Add centos8.2 official to templates Changes for Build #482 [Evgeny Slutsky] he-basic-suite-master: Add Domain_name to the host when vm_run [Gal Ben Haim] Add gcc to fcraw [Sandro Bonazzola] pipelines: add ovirt-dependencies Changes for Build #483 [Galit] Add centos 8.2 image: el8.2-base [Evgheni Dereveanchin] do not expand values in run_oc_playbook ----------------- Failed Tests: ----------------- 1 tests failed. FAILED: basic-suite-master.test-scenarios.004_basic_sanity_pytest.test_ha_recovery Error Message: AssertionError: False != True after 600 seconds Stack Trace: prefix = <ovirtlago.prefix.OvirtPrefix object at 0x7ff48a6b1390> @order_by(_TEST_LIST) def test_ha_recovery(prefix): engine = prefix.virt_env.engine_vm().get_api_v4().system_service() with test_utils.TestEvent(engine, [119, 9602, 506]): # VM_DOWN_ERROR event(119) # HA_VM_FAILED event event(9602) # VDS_INITIATED_RUN_VM event(506) vm_host = _vm_host(prefix, VM2_NAME) pid = vm_host.ssh(['pgrep', '-f', 'qemu.*guest=vm2']) vm_host.ssh(['kill', '-KILL', pid.out.decode('utf-8')]) vm_service = test_utils.get_vm_service(engine, VM2_NAME) testlib.assert_true_within_long(
lambda:
vm_service.get().status == types.VmStatus.UP ) ../basic-suite-master/test-scenarios/004_basic_sanity_pytest.py:783: _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ /usr/lib/python2.7/site-packages/ovirtlago/testlib.py:286: in assert_true_within_long assert_equals_within_long(func, True, allowed_exceptions) /usr/lib/python2.7/site-packages/ovirtlago/testlib.py:273: in assert_equals_within_long func, value, LONG_TIMEOUT, allowed_exceptions=allowed_exceptions _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ func = <function <lambda> at 0x7ff4882556e0>, value = True, timeout = 600 allowed_exceptions = [], initial_wait = 0 def assert_equals_within( func, value, timeout, allowed_exceptions=None, initial_wait=10 ): allowed_exceptions = allowed_exceptions or [] with utils.EggTimer(timeout) as timer: while not timer.elapsed(): try: res = func() if res == value: return except Exception as exc: if _instance_of_any(exc, allowed_exceptions): time.sleep(3) continue LOGGER.exception("Unhandled exception in %s", func) raise if initial_wait == 0: time.sleep(3) else: time.sleep(initial_wait) initial_wait = 0 try: raise AssertionError(
'%s != %s after %s seconds' % (res, value, timeout)
) E AssertionError: False != True after 600 seconds /usr/lib/python2.7/site-packages/ovirtlago/testlib.py:252: AssertionError
Project: https://jenkins.ovirt.org/job/ovirt-system-tests_compat-4.3-suite-master/ Build: https://jenkins.ovirt.org/job/ovirt-system-tests_compat-4.3-suite-master/484... Build Number: 484 Build Status: Still Failing Triggered By: Started by timer ------------------------------------- Changes Since Last Success: ------------------------------------- Changes for Build #475 [Galit] Add missing repo which has collectd-write_syslog [Sandro Bonazzola] Revert "ovirt-release: run only on fc29 nodes" Changes for Build #476 [Michal Skrivanek] make GLANCE failures fatal Changes for Build #477 [Michal Skrivanek] use // in glance URL Changes for Build #478 [Galit] Fix: No module named ost_utils.memoized Changes for Build #479 [Martin Necas] ansible: fix deploy scripts to el8 [arachmani] Add arachman@redhat.com to jenkins recipient lists Changes for Build #480 [Martin Necas] ansible: fix deploy scripts to el8 [Evgheni Dereveanchin] Do not run jenkins check-patch job on fc29 Changes for Build #481 [Galit] Add centos8.2 official to templates Changes for Build #482 [Evgeny Slutsky] he-basic-suite-master: Add Domain_name to the host when vm_run [Gal Ben Haim] Add gcc to fcraw [Sandro Bonazzola] pipelines: add ovirt-dependencies Changes for Build #483 [Galit] Add centos 8.2 image: el8.2-base [Evgheni Dereveanchin] do not expand values in run_oc_playbook Changes for Build #484 [Galit] Add centos 8.2 image: el8.2-base ----------------- Failed Tests: ----------------- 1 tests failed. FAILED: basic-suite-master.test-scenarios.004_basic_sanity_pytest.test_ha_recovery Error Message: AssertionError: False != True after 600 seconds Stack Trace: prefix = <ovirtlago.prefix.OvirtPrefix object at 0x7f89142d8390> @order_by(_TEST_LIST) def test_ha_recovery(prefix): engine = prefix.virt_env.engine_vm().get_api_v4().system_service() with test_utils.TestEvent(engine, [119, 9602, 506]): # VM_DOWN_ERROR event(119) # HA_VM_FAILED event event(9602) # VDS_INITIATED_RUN_VM event(506) vm_host = _vm_host(prefix, VM2_NAME) pid = vm_host.ssh(['pgrep', '-f', 'qemu.*guest=vm2']) vm_host.ssh(['kill', '-KILL', pid.out.decode('utf-8')]) vm_service = test_utils.get_vm_service(engine, VM2_NAME) testlib.assert_true_within_long(
lambda:
vm_service.get().status == types.VmStatus.UP ) ../basic-suite-master/test-scenarios/004_basic_sanity_pytest.py:783: _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ /usr/lib/python2.7/site-packages/ovirtlago/testlib.py:286: in assert_true_within_long assert_equals_within_long(func, True, allowed_exceptions) /usr/lib/python2.7/site-packages/ovirtlago/testlib.py:273: in assert_equals_within_long func, value, LONG_TIMEOUT, allowed_exceptions=allowed_exceptions _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ func = <function <lambda> at 0x7f891437ff50>, value = True, timeout = 600 allowed_exceptions = [], initial_wait = 0 def assert_equals_within( func, value, timeout, allowed_exceptions=None, initial_wait=10 ): allowed_exceptions = allowed_exceptions or [] with utils.EggTimer(timeout) as timer: while not timer.elapsed(): try: res = func() if res == value: return except Exception as exc: if _instance_of_any(exc, allowed_exceptions): time.sleep(3) continue LOGGER.exception("Unhandled exception in %s", func) raise if initial_wait == 0: time.sleep(3) else: time.sleep(initial_wait) initial_wait = 0 try: raise AssertionError(
'%s != %s after %s seconds' % (res, value, timeout)
) E AssertionError: False != True after 600 seconds /usr/lib/python2.7/site-packages/ovirtlago/testlib.py:252: AssertionError
Project: https://jenkins.ovirt.org/job/ovirt-system-tests_compat-4.3-suite-master/ Build: https://jenkins.ovirt.org/job/ovirt-system-tests_compat-4.3-suite-master/485... Build Number: 485 Build Status: Still Failing Triggered By: Started by timer ------------------------------------- Changes Since Last Success: ------------------------------------- Changes for Build #476 [Michal Skrivanek] make GLANCE failures fatal Changes for Build #477 [Michal Skrivanek] use // in glance URL Changes for Build #478 [Galit] Fix: No module named ost_utils.memoized Changes for Build #479 [Martin Necas] ansible: fix deploy scripts to el8 [arachmani] Add arachman@redhat.com to jenkins recipient lists Changes for Build #480 [Martin Necas] ansible: fix deploy scripts to el8 [Evgheni Dereveanchin] Do not run jenkins check-patch job on fc29 Changes for Build #481 [Galit] Add centos8.2 official to templates Changes for Build #482 [Evgeny Slutsky] he-basic-suite-master: Add Domain_name to the host when vm_run [Gal Ben Haim] Add gcc to fcraw [Sandro Bonazzola] pipelines: add ovirt-dependencies Changes for Build #483 [Galit] Add centos 8.2 image: el8.2-base [Evgheni Dereveanchin] do not expand values in run_oc_playbook Changes for Build #484 [Galit] Add centos 8.2 image: el8.2-base Changes for Build #485 [Galit] Add centos 8.2 image: el8.2-base [Gal Ben Haim] pusher: Fix 'if_not_exists' type [Gal Ben Haim] Add common_cli module [Gal Ben Haim] Use common CLI for the usrc updater [Gal Ben Haim] Add common CLI for updaters [Gal Ben Haim] Simplify usrc_updater [Gal Ben Haim] Share common tasks between updaters [Gal Ben Haim] dockerfile_utils: Fix error handling [Gal Ben Haim] dockerfile_utils: Inject SHA sums ----------------- Failed Tests: ----------------- 1 tests failed. FAILED: basic-suite-master.test-scenarios.004_basic_sanity_pytest.test_ha_recovery Error Message: AssertionError: False != True after 600 seconds Stack Trace: prefix = <ovirtlago.prefix.OvirtPrefix object at 0x7f4aa5ae6390> @order_by(_TEST_LIST) def test_ha_recovery(prefix): engine = prefix.virt_env.engine_vm().get_api_v4().system_service() with test_utils.TestEvent(engine, [119, 9602, 506]): # VM_DOWN_ERROR event(119) # HA_VM_FAILED event event(9602) # VDS_INITIATED_RUN_VM event(506) vm_host = _vm_host(prefix, VM2_NAME) pid = vm_host.ssh(['pgrep', '-f', 'qemu.*guest=vm2']) vm_host.ssh(['kill', '-KILL', pid.out.decode('utf-8')]) vm_service = test_utils.get_vm_service(engine, VM2_NAME) testlib.assert_true_within_long(
lambda:
vm_service.get().status == types.VmStatus.UP ) ../basic-suite-master/test-scenarios/004_basic_sanity_pytest.py:783: _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ /usr/lib/python2.7/site-packages/ovirtlago/testlib.py:286: in assert_true_within_long assert_equals_within_long(func, True, allowed_exceptions) /usr/lib/python2.7/site-packages/ovirtlago/testlib.py:273: in assert_equals_within_long func, value, LONG_TIMEOUT, allowed_exceptions=allowed_exceptions _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ func = <function <lambda> at 0x7f4a9e395230>, value = True, timeout = 600 allowed_exceptions = [], initial_wait = 0 def assert_equals_within( func, value, timeout, allowed_exceptions=None, initial_wait=10 ): allowed_exceptions = allowed_exceptions or [] with utils.EggTimer(timeout) as timer: while not timer.elapsed(): try: res = func() if res == value: return except Exception as exc: if _instance_of_any(exc, allowed_exceptions): time.sleep(3) continue LOGGER.exception("Unhandled exception in %s", func) raise if initial_wait == 0: time.sleep(3) else: time.sleep(initial_wait) initial_wait = 0 try: raise AssertionError(
'%s != %s after %s seconds' % (res, value, timeout)
) E AssertionError: False != True after 600 seconds /usr/lib/python2.7/site-packages/ovirtlago/testlib.py:252: AssertionError
Project: https://jenkins.ovirt.org/job/ovirt-system-tests_compat-4.3-suite-master/ Build: https://jenkins.ovirt.org/job/ovirt-system-tests_compat-4.3-suite-master/486... Build Number: 486 Build Status: Still Failing Triggered By: Started by timer ------------------------------------- Changes Since Last Success: ------------------------------------- Changes for Build #477 [Michal Skrivanek] use // in glance URL Changes for Build #478 [Galit] Fix: No module named ost_utils.memoized Changes for Build #479 [Martin Necas] ansible: fix deploy scripts to el8 [arachmani] Add arachman@redhat.com to jenkins recipient lists Changes for Build #480 [Martin Necas] ansible: fix deploy scripts to el8 [Evgheni Dereveanchin] Do not run jenkins check-patch job on fc29 Changes for Build #481 [Galit] Add centos8.2 official to templates Changes for Build #482 [Evgeny Slutsky] he-basic-suite-master: Add Domain_name to the host when vm_run [Gal Ben Haim] Add gcc to fcraw [Sandro Bonazzola] pipelines: add ovirt-dependencies Changes for Build #483 [Galit] Add centos 8.2 image: el8.2-base [Evgheni Dereveanchin] do not expand values in run_oc_playbook Changes for Build #484 [Galit] Add centos 8.2 image: el8.2-base Changes for Build #485 [Galit] Add centos 8.2 image: el8.2-base [Gal Ben Haim] pusher: Fix 'if_not_exists' type [Gal Ben Haim] Add common_cli module [Gal Ben Haim] Use common CLI for the usrc updater [Gal Ben Haim] Add common CLI for updaters [Gal Ben Haim] Simplify usrc_updater [Gal Ben Haim] Share common tasks between updaters [Gal Ben Haim] dockerfile_utils: Fix error handling [Gal Ben Haim] dockerfile_utils: Inject SHA sums Changes for Build #486 [Galit] Add centos 8.2 image: el8.2-base ----------------- Failed Tests: ----------------- 1 tests failed. FAILED: basic-suite-master.test-scenarios.004_basic_sanity_pytest.test_ha_recovery Error Message: AssertionError: False != True after 600 seconds Stack Trace: prefix = <ovirtlago.prefix.OvirtPrefix object at 0x7f71e0364390> @order_by(_TEST_LIST) def test_ha_recovery(prefix): engine = prefix.virt_env.engine_vm().get_api_v4().system_service() with test_utils.TestEvent(engine, [119, 9602, 506]): # VM_DOWN_ERROR event(119) # HA_VM_FAILED event event(9602) # VDS_INITIATED_RUN_VM event(506) vm_host = _vm_host(prefix, VM2_NAME) pid = vm_host.ssh(['pgrep', '-f', 'qemu.*guest=vm2']) vm_host.ssh(['kill', '-KILL', pid.out.decode('utf-8')]) vm_service = test_utils.get_vm_service(engine, VM2_NAME) testlib.assert_true_within_long(
lambda:
vm_service.get().status == types.VmStatus.UP ) ../basic-suite-master/test-scenarios/004_basic_sanity_pytest.py:783: _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ /usr/lib/python2.7/site-packages/ovirtlago/testlib.py:286: in assert_true_within_long assert_equals_within_long(func, True, allowed_exceptions) /usr/lib/python2.7/site-packages/ovirtlago/testlib.py:273: in assert_equals_within_long func, value, LONG_TIMEOUT, allowed_exceptions=allowed_exceptions _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ func = <function <lambda> at 0x7f71dcc928c0>, value = True, timeout = 600 allowed_exceptions = [], initial_wait = 0 def assert_equals_within( func, value, timeout, allowed_exceptions=None, initial_wait=10 ): allowed_exceptions = allowed_exceptions or [] with utils.EggTimer(timeout) as timer: while not timer.elapsed(): try: res = func() if res == value: return except Exception as exc: if _instance_of_any(exc, allowed_exceptions): time.sleep(3) continue LOGGER.exception("Unhandled exception in %s", func) raise if initial_wait == 0: time.sleep(3) else: time.sleep(initial_wait) initial_wait = 0 try: raise AssertionError(
'%s != %s after %s seconds' % (res, value, timeout)
) E AssertionError: False != True after 600 seconds /usr/lib/python2.7/site-packages/ovirtlago/testlib.py:252: AssertionError
Project: https://jenkins.ovirt.org/job/ovirt-system-tests_compat-4.3-suite-master/ Build: https://jenkins.ovirt.org/job/ovirt-system-tests_compat-4.3-suite-master/487... Build Number: 487 Build Status: Still Failing Triggered By: Started by timer ------------------------------------- Changes Since Last Success: ------------------------------------- Changes for Build #478 [Galit] Fix: No module named ost_utils.memoized Changes for Build #479 [Martin Necas] ansible: fix deploy scripts to el8 [arachmani] Add arachman@redhat.com to jenkins recipient lists Changes for Build #480 [Martin Necas] ansible: fix deploy scripts to el8 [Evgheni Dereveanchin] Do not run jenkins check-patch job on fc29 Changes for Build #481 [Galit] Add centos8.2 official to templates Changes for Build #482 [Evgeny Slutsky] he-basic-suite-master: Add Domain_name to the host when vm_run [Gal Ben Haim] Add gcc to fcraw [Sandro Bonazzola] pipelines: add ovirt-dependencies Changes for Build #483 [Galit] Add centos 8.2 image: el8.2-base [Evgheni Dereveanchin] do not expand values in run_oc_playbook Changes for Build #484 [Galit] Add centos 8.2 image: el8.2-base Changes for Build #485 [Galit] Add centos 8.2 image: el8.2-base [Gal Ben Haim] pusher: Fix 'if_not_exists' type [Gal Ben Haim] Add common_cli module [Gal Ben Haim] Use common CLI for the usrc updater [Gal Ben Haim] Add common CLI for updaters [Gal Ben Haim] Simplify usrc_updater [Gal Ben Haim] Share common tasks between updaters [Gal Ben Haim] dockerfile_utils: Fix error handling [Gal Ben Haim] dockerfile_utils: Inject SHA sums Changes for Build #486 [Galit] Add centos 8.2 image: el8.2-base Changes for Build #487 [Galit] Add centos 8.2 image: el8.2-base [Sandro Bonazzola] ovirt-engine_coverity-analysis: fix git_proto [Sandro Bonazzola] template: ovirt-engine_coverity-analysis: fix git_proto ----------------- Failed Tests: ----------------- 1 tests failed. FAILED: basic-suite-master.test-scenarios.004_basic_sanity_pytest.test_ha_recovery Error Message: AssertionError: False != True after 600 seconds Stack Trace: prefix = <ovirtlago.prefix.OvirtPrefix object at 0x7faa3d477390> @order_by(_TEST_LIST) def test_ha_recovery(prefix): engine = prefix.virt_env.engine_vm().get_api_v4().system_service() with test_utils.TestEvent(engine, [119, 9602, 506]): # VM_DOWN_ERROR event(119) # HA_VM_FAILED event event(9602) # VDS_INITIATED_RUN_VM event(506) vm_host = _vm_host(prefix, VM2_NAME) pid = vm_host.ssh(['pgrep', '-f', 'qemu.*guest=vm2']) vm_host.ssh(['kill', '-KILL', pid.out.decode('utf-8')]) vm_service = test_utils.get_vm_service(engine, VM2_NAME) testlib.assert_true_within_long(
lambda:
vm_service.get().status == types.VmStatus.UP ) ../basic-suite-master/test-scenarios/004_basic_sanity_pytest.py:783: _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ /usr/lib/python2.7/site-packages/ovirtlago/testlib.py:286: in assert_true_within_long assert_equals_within_long(func, True, allowed_exceptions) /usr/lib/python2.7/site-packages/ovirtlago/testlib.py:273: in assert_equals_within_long func, value, LONG_TIMEOUT, allowed_exceptions=allowed_exceptions _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ func = <function <lambda> at 0x7faa3c047c80>, value = True, timeout = 600 allowed_exceptions = [], initial_wait = 0 def assert_equals_within( func, value, timeout, allowed_exceptions=None, initial_wait=10 ): allowed_exceptions = allowed_exceptions or [] with utils.EggTimer(timeout) as timer: while not timer.elapsed(): try: res = func() if res == value: return except Exception as exc: if _instance_of_any(exc, allowed_exceptions): time.sleep(3) continue LOGGER.exception("Unhandled exception in %s", func) raise if initial_wait == 0: time.sleep(3) else: time.sleep(initial_wait) initial_wait = 0 try: raise AssertionError(
'%s != %s after %s seconds' % (res, value, timeout)
) E AssertionError: False != True after 600 seconds /usr/lib/python2.7/site-packages/ovirtlago/testlib.py:252: AssertionError
Project: https://jenkins.ovirt.org/job/ovirt-system-tests_compat-4.3-suite-master/ Build: https://jenkins.ovirt.org/job/ovirt-system-tests_compat-4.3-suite-master/488... Build Number: 488 Build Status: Still Failing Triggered By: Started by timer ------------------------------------- Changes Since Last Success: ------------------------------------- Changes for Build #479 [Martin Necas] ansible: fix deploy scripts to el8 [arachmani] Add arachman@redhat.com to jenkins recipient lists Changes for Build #480 [Martin Necas] ansible: fix deploy scripts to el8 [Evgheni Dereveanchin] Do not run jenkins check-patch job on fc29 Changes for Build #481 [Galit] Add centos8.2 official to templates Changes for Build #482 [Evgeny Slutsky] he-basic-suite-master: Add Domain_name to the host when vm_run [Gal Ben Haim] Add gcc to fcraw [Sandro Bonazzola] pipelines: add ovirt-dependencies Changes for Build #483 [Galit] Add centos 8.2 image: el8.2-base [Evgheni Dereveanchin] do not expand values in run_oc_playbook Changes for Build #484 [Galit] Add centos 8.2 image: el8.2-base Changes for Build #485 [Galit] Add centos 8.2 image: el8.2-base [Gal Ben Haim] pusher: Fix 'if_not_exists' type [Gal Ben Haim] Add common_cli module [Gal Ben Haim] Use common CLI for the usrc updater [Gal Ben Haim] Add common CLI for updaters [Gal Ben Haim] Simplify usrc_updater [Gal Ben Haim] Share common tasks between updaters [Gal Ben Haim] dockerfile_utils: Fix error handling [Gal Ben Haim] dockerfile_utils: Inject SHA sums Changes for Build #486 [Galit] Add centos 8.2 image: el8.2-base Changes for Build #487 [Galit] Add centos 8.2 image: el8.2-base [Sandro Bonazzola] ovirt-engine_coverity-analysis: fix git_proto [Sandro Bonazzola] template: ovirt-engine_coverity-analysis: fix git_proto Changes for Build #488 [Galit] Add centos 8.2 image: el8.2-base ----------------- Failed Tests: ----------------- 1 tests failed. FAILED: basic-suite-master.test-scenarios.004_basic_sanity_pytest.test_ha_recovery Error Message: AssertionError: False != True after 600 seconds Stack Trace: prefix = <ovirtlago.prefix.OvirtPrefix object at 0x7fe472d7fb10> @order_by(_TEST_LIST) def test_ha_recovery(prefix): engine = prefix.virt_env.engine_vm().get_api_v4().system_service() with test_utils.TestEvent(engine, [119, 9602, 506]): # VM_DOWN_ERROR event(119) # HA_VM_FAILED event event(9602) # VDS_INITIATED_RUN_VM event(506) vm_host = _vm_host(prefix, VM2_NAME) pid = vm_host.ssh(['pgrep', '-f', 'qemu.*guest=vm2']) vm_host.ssh(['kill', '-KILL', pid.out.decode('utf-8')]) vm_service = test_utils.get_vm_service(engine, VM2_NAME) testlib.assert_true_within_long(
lambda:
vm_service.get().status == types.VmStatus.UP ) ../basic-suite-master/test-scenarios/004_basic_sanity_pytest.py:783: _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ /usr/lib/python2.7/site-packages/ovirtlago/testlib.py:286: in assert_true_within_long assert_equals_within_long(func, True, allowed_exceptions) /usr/lib/python2.7/site-packages/ovirtlago/testlib.py:273: in assert_equals_within_long func, value, LONG_TIMEOUT, allowed_exceptions=allowed_exceptions _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ func = <function <lambda> at 0x7fe46b568a28>, value = True, timeout = 600 allowed_exceptions = [], initial_wait = 0 def assert_equals_within( func, value, timeout, allowed_exceptions=None, initial_wait=10 ): allowed_exceptions = allowed_exceptions or [] with utils.EggTimer(timeout) as timer: while not timer.elapsed(): try: res = func() if res == value: return except Exception as exc: if _instance_of_any(exc, allowed_exceptions): time.sleep(3) continue LOGGER.exception("Unhandled exception in %s", func) raise if initial_wait == 0: time.sleep(3) else: time.sleep(initial_wait) initial_wait = 0 try: raise AssertionError(
'%s != %s after %s seconds' % (res, value, timeout)
) E AssertionError: False != True after 600 seconds /usr/lib/python2.7/site-packages/ovirtlago/testlib.py:252: AssertionError
Project: https://jenkins.ovirt.org/job/ovirt-system-tests_compat-4.3-suite-master/ Build: https://jenkins.ovirt.org/job/ovirt-system-tests_compat-4.3-suite-master/489... Build Number: 489 Build Status: Still Failing Triggered By: Started by timer ------------------------------------- Changes Since Last Success: ------------------------------------- Changes for Build #480 [Martin Necas] ansible: fix deploy scripts to el8 [Evgheni Dereveanchin] Do not run jenkins check-patch job on fc29 Changes for Build #481 [Galit] Add centos8.2 official to templates Changes for Build #482 [Evgeny Slutsky] he-basic-suite-master: Add Domain_name to the host when vm_run [Gal Ben Haim] Add gcc to fcraw [Sandro Bonazzola] pipelines: add ovirt-dependencies Changes for Build #483 [Galit] Add centos 8.2 image: el8.2-base [Evgheni Dereveanchin] do not expand values in run_oc_playbook Changes for Build #484 [Galit] Add centos 8.2 image: el8.2-base Changes for Build #485 [Galit] Add centos 8.2 image: el8.2-base [Gal Ben Haim] pusher: Fix 'if_not_exists' type [Gal Ben Haim] Add common_cli module [Gal Ben Haim] Use common CLI for the usrc updater [Gal Ben Haim] Add common CLI for updaters [Gal Ben Haim] Simplify usrc_updater [Gal Ben Haim] Share common tasks between updaters [Gal Ben Haim] dockerfile_utils: Fix error handling [Gal Ben Haim] dockerfile_utils: Inject SHA sums Changes for Build #486 [Galit] Add centos 8.2 image: el8.2-base Changes for Build #487 [Galit] Add centos 8.2 image: el8.2-base [Sandro Bonazzola] ovirt-engine_coverity-analysis: fix git_proto [Sandro Bonazzola] template: ovirt-engine_coverity-analysis: fix git_proto Changes for Build #488 [Galit] Add centos 8.2 image: el8.2-base Changes for Build #489 [Marcin Sobczyk] basic: Add support for running on el8 [Ehud Yonasi] global_setup: enable el8 setup. ----------------- Failed Tests: ----------------- 1 tests failed. FAILED: basic-suite-master.test-scenarios.004_basic_sanity_pytest.test_ha_recovery Error Message: AssertionError: False != True after 600 seconds Stack Trace: prefix = <ovirtlago.prefix.OvirtPrefix object at 0x7f9dcf556b10> @order_by(_TEST_LIST) def test_ha_recovery(prefix): engine = prefix.virt_env.engine_vm().get_api_v4().system_service() with test_utils.TestEvent(engine, [119, 9602, 506]): # VM_DOWN_ERROR event(119) # HA_VM_FAILED event event(9602) # VDS_INITIATED_RUN_VM event(506) vm_host = _vm_host(prefix, VM2_NAME) pid = vm_host.ssh(['pgrep', '-f', 'qemu.*guest=vm2']) vm_host.ssh(['kill', '-KILL', pid.out.decode('utf-8')]) vm_service = test_utils.get_vm_service(engine, VM2_NAME) testlib.assert_true_within_long(
lambda:
vm_service.get().status == types.VmStatus.UP ) ../basic-suite-master/test-scenarios/004_basic_sanity_pytest.py:783: _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ /usr/lib/python2.7/site-packages/ovirtlago/testlib.py:286: in assert_true_within_long assert_equals_within_long(func, True, allowed_exceptions) /usr/lib/python2.7/site-packages/ovirtlago/testlib.py:273: in assert_equals_within_long func, value, LONG_TIMEOUT, allowed_exceptions=allowed_exceptions _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ func = <function <lambda> at 0x7f9dcc15ced8>, value = True, timeout = 600 allowed_exceptions = [], initial_wait = 0 def assert_equals_within( func, value, timeout, allowed_exceptions=None, initial_wait=10 ): allowed_exceptions = allowed_exceptions or [] with utils.EggTimer(timeout) as timer: while not timer.elapsed(): try: res = func() if res == value: return except Exception as exc: if _instance_of_any(exc, allowed_exceptions): time.sleep(3) continue LOGGER.exception("Unhandled exception in %s", func) raise if initial_wait == 0: time.sleep(3) else: time.sleep(initial_wait) initial_wait = 0 try: raise AssertionError(
'%s != %s after %s seconds' % (res, value, timeout)
) E AssertionError: False != True after 600 seconds /usr/lib/python2.7/site-packages/ovirtlago/testlib.py:252: AssertionError
Project: https://jenkins.ovirt.org/job/ovirt-system-tests_compat-4.3-suite-master/ Build: https://jenkins.ovirt.org/job/ovirt-system-tests_compat-4.3-suite-master/490... Build Number: 490 Build Status: Still Failing Triggered By: Started by timer ------------------------------------- Changes Since Last Success: ------------------------------------- Changes for Build #481 [Galit] Add centos8.2 official to templates Changes for Build #482 [Evgeny Slutsky] he-basic-suite-master: Add Domain_name to the host when vm_run [Gal Ben Haim] Add gcc to fcraw [Sandro Bonazzola] pipelines: add ovirt-dependencies Changes for Build #483 [Galit] Add centos 8.2 image: el8.2-base [Evgheni Dereveanchin] do not expand values in run_oc_playbook Changes for Build #484 [Galit] Add centos 8.2 image: el8.2-base Changes for Build #485 [Galit] Add centos 8.2 image: el8.2-base [Gal Ben Haim] pusher: Fix 'if_not_exists' type [Gal Ben Haim] Add common_cli module [Gal Ben Haim] Use common CLI for the usrc updater [Gal Ben Haim] Add common CLI for updaters [Gal Ben Haim] Simplify usrc_updater [Gal Ben Haim] Share common tasks between updaters [Gal Ben Haim] dockerfile_utils: Fix error handling [Gal Ben Haim] dockerfile_utils: Inject SHA sums Changes for Build #486 [Galit] Add centos 8.2 image: el8.2-base Changes for Build #487 [Galit] Add centos 8.2 image: el8.2-base [Sandro Bonazzola] ovirt-engine_coverity-analysis: fix git_proto [Sandro Bonazzola] template: ovirt-engine_coverity-analysis: fix git_proto Changes for Build #488 [Galit] Add centos 8.2 image: el8.2-base Changes for Build #489 [Marcin Sobczyk] basic: Add support for running on el8 [Ehud Yonasi] global_setup: enable el8 setup. Changes for Build #490 [Marcin Sobczyk] basic: Add support for running on el8 ----------------- Failed Tests: ----------------- 1 tests failed. FAILED: basic-suite-master.test-scenarios.004_basic_sanity_pytest.test_ha_recovery Error Message: AssertionError: False != True after 600 seconds Stack Trace: prefix = <ovirtlago.prefix.OvirtPrefix object at 0x7ff57be17b10> @order_by(_TEST_LIST) def test_ha_recovery(prefix): engine = prefix.virt_env.engine_vm().get_api_v4().system_service() with test_utils.TestEvent(engine, [119, 9602, 506]): # VM_DOWN_ERROR event(119) # HA_VM_FAILED event event(9602) # VDS_INITIATED_RUN_VM event(506) vm_host = _vm_host(prefix, VM2_NAME) pid = vm_host.ssh(['pgrep', '-f', 'qemu.*guest=vm2']) vm_host.ssh(['kill', '-KILL', pid.out.decode('utf-8')]) vm_service = test_utils.get_vm_service(engine, VM2_NAME) testlib.assert_true_within_long(
lambda:
vm_service.get().status == types.VmStatus.UP ) ../basic-suite-master/test-scenarios/004_basic_sanity_pytest.py:783: _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ /usr/lib/python2.7/site-packages/ovirtlago/testlib.py:286: in assert_true_within_long assert_equals_within_long(func, True, allowed_exceptions) /usr/lib/python2.7/site-packages/ovirtlago/testlib.py:273: in assert_equals_within_long func, value, LONG_TIMEOUT, allowed_exceptions=allowed_exceptions _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ func = <function <lambda> at 0x7ff57b945b18>, value = True, timeout = 600 allowed_exceptions = [], initial_wait = 0 def assert_equals_within( func, value, timeout, allowed_exceptions=None, initial_wait=10 ): allowed_exceptions = allowed_exceptions or [] with utils.EggTimer(timeout) as timer: while not timer.elapsed(): try: res = func() if res == value: return except Exception as exc: if _instance_of_any(exc, allowed_exceptions): time.sleep(3) continue LOGGER.exception("Unhandled exception in %s", func) raise if initial_wait == 0: time.sleep(3) else: time.sleep(initial_wait) initial_wait = 0 try: raise AssertionError(
'%s != %s after %s seconds' % (res, value, timeout)
) E AssertionError: False != True after 600 seconds /usr/lib/python2.7/site-packages/ovirtlago/testlib.py:252: AssertionError
Project: https://jenkins.ovirt.org/job/ovirt-system-tests_compat-4.3-suite-master/ Build: https://jenkins.ovirt.org/job/ovirt-system-tests_compat-4.3-suite-master/491... Build Number: 491 Build Status: Still Failing Triggered By: Started by timer ------------------------------------- Changes Since Last Success: ------------------------------------- Changes for Build #482 [Evgeny Slutsky] he-basic-suite-master: Add Domain_name to the host when vm_run [Gal Ben Haim] Add gcc to fcraw [Sandro Bonazzola] pipelines: add ovirt-dependencies Changes for Build #483 [Galit] Add centos 8.2 image: el8.2-base [Evgheni Dereveanchin] do not expand values in run_oc_playbook Changes for Build #484 [Galit] Add centos 8.2 image: el8.2-base Changes for Build #485 [Galit] Add centos 8.2 image: el8.2-base [Gal Ben Haim] pusher: Fix 'if_not_exists' type [Gal Ben Haim] Add common_cli module [Gal Ben Haim] Use common CLI for the usrc updater [Gal Ben Haim] Add common CLI for updaters [Gal Ben Haim] Simplify usrc_updater [Gal Ben Haim] Share common tasks between updaters [Gal Ben Haim] dockerfile_utils: Fix error handling [Gal Ben Haim] dockerfile_utils: Inject SHA sums Changes for Build #486 [Galit] Add centos 8.2 image: el8.2-base Changes for Build #487 [Galit] Add centos 8.2 image: el8.2-base [Sandro Bonazzola] ovirt-engine_coverity-analysis: fix git_proto [Sandro Bonazzola] template: ovirt-engine_coverity-analysis: fix git_proto Changes for Build #488 [Galit] Add centos 8.2 image: el8.2-base Changes for Build #489 [Marcin Sobczyk] basic: Add support for running on el8 [Ehud Yonasi] global_setup: enable el8 setup. Changes for Build #490 [Marcin Sobczyk] basic: Add support for running on el8 Changes for Build #491 [Marcin Sobczyk] basic: Add support for running on el8 ----------------- Failed Tests: ----------------- 1 tests failed. FAILED: basic-suite-master.test-scenarios.004_basic_sanity_pytest.test_ha_recovery Error Message: AssertionError: False != True after 600 seconds Stack Trace: prefix = <ovirtlago.prefix.OvirtPrefix object at 0x7ff68a145b10> @order_by(_TEST_LIST) def test_ha_recovery(prefix): engine = prefix.virt_env.engine_vm().get_api_v4().system_service() with test_utils.TestEvent(engine, [119, 9602, 506]): # VM_DOWN_ERROR event(119) # HA_VM_FAILED event event(9602) # VDS_INITIATED_RUN_VM event(506) vm_host = _vm_host(prefix, VM2_NAME) pid = vm_host.ssh(['pgrep', '-f', 'qemu.*guest=vm2']) vm_host.ssh(['kill', '-KILL', pid.out.decode('utf-8')]) vm_service = test_utils.get_vm_service(engine, VM2_NAME) testlib.assert_true_within_long(
lambda:
vm_service.get().status == types.VmStatus.UP ) ../basic-suite-master/test-scenarios/004_basic_sanity_pytest.py:783: _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ /usr/lib/python2.7/site-packages/ovirtlago/testlib.py:286: in assert_true_within_long assert_equals_within_long(func, True, allowed_exceptions) /usr/lib/python2.7/site-packages/ovirtlago/testlib.py:273: in assert_equals_within_long func, value, LONG_TIMEOUT, allowed_exceptions=allowed_exceptions _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ func = <function <lambda> at 0x7ff682626938>, value = True, timeout = 600 allowed_exceptions = [], initial_wait = 0 def assert_equals_within( func, value, timeout, allowed_exceptions=None, initial_wait=10 ): allowed_exceptions = allowed_exceptions or [] with utils.EggTimer(timeout) as timer: while not timer.elapsed(): try: res = func() if res == value: return except Exception as exc: if _instance_of_any(exc, allowed_exceptions): time.sleep(3) continue LOGGER.exception("Unhandled exception in %s", func) raise if initial_wait == 0: time.sleep(3) else: time.sleep(initial_wait) initial_wait = 0 try: raise AssertionError(
'%s != %s after %s seconds' % (res, value, timeout)
) E AssertionError: False != True after 600 seconds /usr/lib/python2.7/site-packages/ovirtlago/testlib.py:252: AssertionError
Project: https://jenkins.ovirt.org/job/ovirt-system-tests_compat-4.3-suite-master/ Build: https://jenkins.ovirt.org/job/ovirt-system-tests_compat-4.3-suite-master/492... Build Number: 492 Build Status: Still Failing Triggered By: Started by timer ------------------------------------- Changes Since Last Success: ------------------------------------- Changes for Build #483 [Galit] Add centos 8.2 image: el8.2-base [Evgheni Dereveanchin] do not expand values in run_oc_playbook Changes for Build #484 [Galit] Add centos 8.2 image: el8.2-base Changes for Build #485 [Galit] Add centos 8.2 image: el8.2-base [Gal Ben Haim] pusher: Fix 'if_not_exists' type [Gal Ben Haim] Add common_cli module [Gal Ben Haim] Use common CLI for the usrc updater [Gal Ben Haim] Add common CLI for updaters [Gal Ben Haim] Simplify usrc_updater [Gal Ben Haim] Share common tasks between updaters [Gal Ben Haim] dockerfile_utils: Fix error handling [Gal Ben Haim] dockerfile_utils: Inject SHA sums Changes for Build #486 [Galit] Add centos 8.2 image: el8.2-base Changes for Build #487 [Galit] Add centos 8.2 image: el8.2-base [Sandro Bonazzola] ovirt-engine_coverity-analysis: fix git_proto [Sandro Bonazzola] template: ovirt-engine_coverity-analysis: fix git_proto Changes for Build #488 [Galit] Add centos 8.2 image: el8.2-base Changes for Build #489 [Marcin Sobczyk] basic: Add support for running on el8 [Ehud Yonasi] global_setup: enable el8 setup. Changes for Build #490 [Marcin Sobczyk] basic: Add support for running on el8 Changes for Build #491 [Marcin Sobczyk] basic: Add support for running on el8 Changes for Build #492 [Marcin Sobczyk] basic: Add support for running on el8 ----------------- Failed Tests: ----------------- 1 tests failed. FAILED: basic-suite-master.test-scenarios.004_basic_sanity_pytest.test_ha_recovery Error Message: AssertionError: False != True after 600 seconds Stack Trace: prefix = <ovirtlago.prefix.OvirtPrefix object at 0x7fec5f0c5b10> @order_by(_TEST_LIST) def test_ha_recovery(prefix): engine = prefix.virt_env.engine_vm().get_api_v4().system_service() with test_utils.TestEvent(engine, [119, 9602, 506]): # VM_DOWN_ERROR event(119) # HA_VM_FAILED event event(9602) # VDS_INITIATED_RUN_VM event(506) vm_host = _vm_host(prefix, VM2_NAME) pid = vm_host.ssh(['pgrep', '-f', 'qemu.*guest=vm2']) vm_host.ssh(['kill', '-KILL', pid.out.decode('utf-8')]) vm_service = test_utils.get_vm_service(engine, VM2_NAME) testlib.assert_true_within_long(
lambda:
vm_service.get().status == types.VmStatus.UP ) ../basic-suite-master/test-scenarios/004_basic_sanity_pytest.py:783: _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ /usr/lib/python2.7/site-packages/ovirtlago/testlib.py:286: in assert_true_within_long assert_equals_within_long(func, True, allowed_exceptions) /usr/lib/python2.7/site-packages/ovirtlago/testlib.py:273: in assert_equals_within_long func, value, LONG_TIMEOUT, allowed_exceptions=allowed_exceptions _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ func = <function <lambda> at 0x7fec5c052140>, value = True, timeout = 600 allowed_exceptions = [], initial_wait = 0 def assert_equals_within( func, value, timeout, allowed_exceptions=None, initial_wait=10 ): allowed_exceptions = allowed_exceptions or [] with utils.EggTimer(timeout) as timer: while not timer.elapsed(): try: res = func() if res == value: return except Exception as exc: if _instance_of_any(exc, allowed_exceptions): time.sleep(3) continue LOGGER.exception("Unhandled exception in %s", func) raise if initial_wait == 0: time.sleep(3) else: time.sleep(initial_wait) initial_wait = 0 try: raise AssertionError(
'%s != %s after %s seconds' % (res, value, timeout)
) E AssertionError: False != True after 600 seconds /usr/lib/python2.7/site-packages/ovirtlago/testlib.py:252: AssertionError
Project: https://jenkins.ovirt.org/job/ovirt-system-tests_compat-4.3-suite-master/ Build: https://jenkins.ovirt.org/job/ovirt-system-tests_compat-4.3-suite-master/493... Build Number: 493 Build Status: Still Failing Triggered By: Started by timer ------------------------------------- Changes Since Last Success: ------------------------------------- Changes for Build #484 [Galit] Add centos 8.2 image: el8.2-base Changes for Build #485 [Galit] Add centos 8.2 image: el8.2-base [Gal Ben Haim] pusher: Fix 'if_not_exists' type [Gal Ben Haim] Add common_cli module [Gal Ben Haim] Use common CLI for the usrc updater [Gal Ben Haim] Add common CLI for updaters [Gal Ben Haim] Simplify usrc_updater [Gal Ben Haim] Share common tasks between updaters [Gal Ben Haim] dockerfile_utils: Fix error handling [Gal Ben Haim] dockerfile_utils: Inject SHA sums Changes for Build #486 [Galit] Add centos 8.2 image: el8.2-base Changes for Build #487 [Galit] Add centos 8.2 image: el8.2-base [Sandro Bonazzola] ovirt-engine_coverity-analysis: fix git_proto [Sandro Bonazzola] template: ovirt-engine_coverity-analysis: fix git_proto Changes for Build #488 [Galit] Add centos 8.2 image: el8.2-base Changes for Build #489 [Marcin Sobczyk] basic: Add support for running on el8 [Ehud Yonasi] global_setup: enable el8 setup. Changes for Build #490 [Marcin Sobczyk] basic: Add support for running on el8 Changes for Build #491 [Marcin Sobczyk] basic: Add support for running on el8 Changes for Build #492 [Marcin Sobczyk] basic: Add support for running on el8 Changes for Build #493 [jenkins CI] Auto create ovirt-4.2-release ----------------- Failed Tests: ----------------- 1 tests failed. FAILED: basic-suite-master.test-scenarios.004_basic_sanity_pytest.test_ha_recovery Error Message: AssertionError: False != True after 600 seconds Stack Trace: prefix = <ovirtlago.prefix.OvirtPrefix object at 0x7f098efb4b10> @order_by(_TEST_LIST) def test_ha_recovery(prefix): engine = prefix.virt_env.engine_vm().get_api_v4().system_service() with test_utils.TestEvent(engine, [119, 9602, 506]): # VM_DOWN_ERROR event(119) # HA_VM_FAILED event event(9602) # VDS_INITIATED_RUN_VM event(506) vm_host = _vm_host(prefix, VM2_NAME) pid = vm_host.ssh(['pgrep', '-f', 'qemu.*guest=vm2']) vm_host.ssh(['kill', '-KILL', pid.out.decode('utf-8')]) vm_service = test_utils.get_vm_service(engine, VM2_NAME) testlib.assert_true_within_long(
lambda:
vm_service.get().status == types.VmStatus.UP ) ../basic-suite-master/test-scenarios/004_basic_sanity_pytest.py:783: _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ /usr/lib/python2.7/site-packages/ovirtlago/testlib.py:286: in assert_true_within_long assert_equals_within_long(func, True, allowed_exceptions) /usr/lib/python2.7/site-packages/ovirtlago/testlib.py:273: in assert_equals_within_long func, value, LONG_TIMEOUT, allowed_exceptions=allowed_exceptions _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ func = <function <lambda> at 0x7f098eae2aa0>, value = True, timeout = 600 allowed_exceptions = [], initial_wait = 0 def assert_equals_within( func, value, timeout, allowed_exceptions=None, initial_wait=10 ): allowed_exceptions = allowed_exceptions or [] with utils.EggTimer(timeout) as timer: while not timer.elapsed(): try: res = func() if res == value: return except Exception as exc: if _instance_of_any(exc, allowed_exceptions): time.sleep(3) continue LOGGER.exception("Unhandled exception in %s", func) raise if initial_wait == 0: time.sleep(3) else: time.sleep(initial_wait) initial_wait = 0 try: raise AssertionError(
'%s != %s after %s seconds' % (res, value, timeout)
) E AssertionError: False != True after 600 seconds /usr/lib/python2.7/site-packages/ovirtlago/testlib.py:252: AssertionError
Project: https://jenkins.ovirt.org/job/ovirt-system-tests_compat-4.3-suite-master/ Build: https://jenkins.ovirt.org/job/ovirt-system-tests_compat-4.3-suite-master/494... Build Number: 494 Build Status: Still Failing Triggered By: Started by timer ------------------------------------- Changes Since Last Success: ------------------------------------- Changes for Build #485 [Galit] Add centos 8.2 image: el8.2-base [Gal Ben Haim] pusher: Fix 'if_not_exists' type [Gal Ben Haim] Add common_cli module [Gal Ben Haim] Use common CLI for the usrc updater [Gal Ben Haim] Add common CLI for updaters [Gal Ben Haim] Simplify usrc_updater [Gal Ben Haim] Share common tasks between updaters [Gal Ben Haim] dockerfile_utils: Fix error handling [Gal Ben Haim] dockerfile_utils: Inject SHA sums Changes for Build #486 [Galit] Add centos 8.2 image: el8.2-base Changes for Build #487 [Galit] Add centos 8.2 image: el8.2-base [Sandro Bonazzola] ovirt-engine_coverity-analysis: fix git_proto [Sandro Bonazzola] template: ovirt-engine_coverity-analysis: fix git_proto Changes for Build #488 [Galit] Add centos 8.2 image: el8.2-base Changes for Build #489 [Marcin Sobczyk] basic: Add support for running on el8 [Ehud Yonasi] global_setup: enable el8 setup. Changes for Build #490 [Marcin Sobczyk] basic: Add support for running on el8 Changes for Build #491 [Marcin Sobczyk] basic: Add support for running on el8 Changes for Build #492 [Marcin Sobczyk] basic: Add support for running on el8 Changes for Build #493 [jenkins CI] Auto create ovirt-4.2-release Changes for Build #494 [jenkins CI] Auto create ovirt-4.2 ----------------- Failed Tests: ----------------- 1 tests failed. FAILED: basic-suite-master.test-scenarios.004_basic_sanity_pytest.test_ha_recovery Error Message: AssertionError: False != True after 600 seconds Stack Trace: prefix = <ovirtlago.prefix.OvirtPrefix object at 0x7f00c447db10> @order_by(_TEST_LIST) def test_ha_recovery(prefix): engine = prefix.virt_env.engine_vm().get_api_v4().system_service() with test_utils.TestEvent(engine, [119, 9602, 506]): # VM_DOWN_ERROR event(119) # HA_VM_FAILED event event(9602) # VDS_INITIATED_RUN_VM event(506) vm_host = _vm_host(prefix, VM2_NAME) pid = vm_host.ssh(['pgrep', '-f', 'qemu.*guest=vm2']) vm_host.ssh(['kill', '-KILL', pid.out.decode('utf-8')]) vm_service = test_utils.get_vm_service(engine, VM2_NAME) testlib.assert_true_within_long(
lambda:
vm_service.get().status == types.VmStatus.UP ) ../basic-suite-master/test-scenarios/004_basic_sanity_pytest.py:783: _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ /usr/lib/python2.7/site-packages/ovirtlago/testlib.py:286: in assert_true_within_long assert_equals_within_long(func, True, allowed_exceptions) /usr/lib/python2.7/site-packages/ovirtlago/testlib.py:273: in assert_equals_within_long func, value, LONG_TIMEOUT, allowed_exceptions=allowed_exceptions _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ func = <function <lambda> at 0x7f00c5a5bc08>, value = True, timeout = 600 allowed_exceptions = [], initial_wait = 0 def assert_equals_within( func, value, timeout, allowed_exceptions=None, initial_wait=10 ): allowed_exceptions = allowed_exceptions or [] with utils.EggTimer(timeout) as timer: while not timer.elapsed(): try: res = func() if res == value: return except Exception as exc: if _instance_of_any(exc, allowed_exceptions): time.sleep(3) continue LOGGER.exception("Unhandled exception in %s", func) raise if initial_wait == 0: time.sleep(3) else: time.sleep(initial_wait) initial_wait = 0 try: raise AssertionError(
'%s != %s after %s seconds' % (res, value, timeout)
) E AssertionError: False != True after 600 seconds /usr/lib/python2.7/site-packages/ovirtlago/testlib.py:252: AssertionError
Project: https://jenkins.ovirt.org/job/ovirt-system-tests_compat-4.3-suite-master/ Build: https://jenkins.ovirt.org/job/ovirt-system-tests_compat-4.3-suite-master/495... Build Number: 495 Build Status: Still Failing Triggered By: Started by timer ------------------------------------- Changes Since Last Success: ------------------------------------- Changes for Build #486 [Galit] Add centos 8.2 image: el8.2-base Changes for Build #487 [Galit] Add centos 8.2 image: el8.2-base [Sandro Bonazzola] ovirt-engine_coverity-analysis: fix git_proto [Sandro Bonazzola] template: ovirt-engine_coverity-analysis: fix git_proto Changes for Build #488 [Galit] Add centos 8.2 image: el8.2-base Changes for Build #489 [Marcin Sobczyk] basic: Add support for running on el8 [Ehud Yonasi] global_setup: enable el8 setup. Changes for Build #490 [Marcin Sobczyk] basic: Add support for running on el8 Changes for Build #491 [Marcin Sobczyk] basic: Add support for running on el8 Changes for Build #492 [Marcin Sobczyk] basic: Add support for running on el8 Changes for Build #493 [jenkins CI] Auto create ovirt-4.2-release Changes for Build #494 [jenkins CI] Auto create ovirt-4.2 Changes for Build #495 [Marcin Sobczyk] network: Run network suite master on el8 ----------------- Failed Tests: ----------------- 1 tests failed. FAILED: basic-suite-master.test-scenarios.004_basic_sanity_pytest.test_ha_recovery Error Message: AssertionError: False != True after 600 seconds Stack Trace: prefix = <ovirtlago.prefix.OvirtPrefix object at 0x7fe932099b10> @order_by(_TEST_LIST) def test_ha_recovery(prefix): engine = prefix.virt_env.engine_vm().get_api_v4().system_service() with test_utils.TestEvent(engine, [119, 9602, 506]): # VM_DOWN_ERROR event(119) # HA_VM_FAILED event event(9602) # VDS_INITIATED_RUN_VM event(506) vm_host = _vm_host(prefix, VM2_NAME) pid = vm_host.ssh(['pgrep', '-f', 'qemu.*guest=vm2']) vm_host.ssh(['kill', '-KILL', pid.out.decode('utf-8')]) vm_service = test_utils.get_vm_service(engine, VM2_NAME) testlib.assert_true_within_long(
lambda:
vm_service.get().status == types.VmStatus.UP ) ../basic-suite-master/test-scenarios/004_basic_sanity_pytest.py:783: _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ /usr/lib/python2.7/site-packages/ovirtlago/testlib.py:286: in assert_true_within_long assert_equals_within_long(func, True, allowed_exceptions) /usr/lib/python2.7/site-packages/ovirtlago/testlib.py:273: in assert_equals_within_long func, value, LONG_TIMEOUT, allowed_exceptions=allowed_exceptions _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ func = <function <lambda> at 0x7fe930083140>, value = True, timeout = 600 allowed_exceptions = [], initial_wait = 0 def assert_equals_within( func, value, timeout, allowed_exceptions=None, initial_wait=10 ): allowed_exceptions = allowed_exceptions or [] with utils.EggTimer(timeout) as timer: while not timer.elapsed(): try: res = func() if res == value: return except Exception as exc: if _instance_of_any(exc, allowed_exceptions): time.sleep(3) continue LOGGER.exception("Unhandled exception in %s", func) raise if initial_wait == 0: time.sleep(3) else: time.sleep(initial_wait) initial_wait = 0 try: raise AssertionError(
'%s != %s after %s seconds' % (res, value, timeout)
) E AssertionError: False != True after 600 seconds /usr/lib/python2.7/site-packages/ovirtlago/testlib.py:252: AssertionError
Project: https://jenkins.ovirt.org/job/ovirt-system-tests_compat-4.3-suite-master/ Build: https://jenkins.ovirt.org/job/ovirt-system-tests_compat-4.3-suite-master/496... Build Number: 496 Build Status: Still Failing Triggered By: Started by timer ------------------------------------- Changes Since Last Success: ------------------------------------- Changes for Build #487 [Galit] Add centos 8.2 image: el8.2-base [Sandro Bonazzola] ovirt-engine_coverity-analysis: fix git_proto [Sandro Bonazzola] template: ovirt-engine_coverity-analysis: fix git_proto Changes for Build #488 [Galit] Add centos 8.2 image: el8.2-base Changes for Build #489 [Marcin Sobczyk] basic: Add support for running on el8 [Ehud Yonasi] global_setup: enable el8 setup. Changes for Build #490 [Marcin Sobczyk] basic: Add support for running on el8 Changes for Build #491 [Marcin Sobczyk] basic: Add support for running on el8 Changes for Build #492 [Marcin Sobczyk] basic: Add support for running on el8 Changes for Build #493 [jenkins CI] Auto create ovirt-4.2-release Changes for Build #494 [jenkins CI] Auto create ovirt-4.2 Changes for Build #495 [Marcin Sobczyk] network: Run network suite master on el8 Changes for Build #496 [Marcin Sobczyk] network: Run network suite master on el8 ----------------- Failed Tests: ----------------- 1 tests failed. FAILED: basic-suite-master.test-scenarios.004_basic_sanity_pytest.test_ha_recovery Error Message: AssertionError: False != True after 600 seconds Stack Trace: prefix = <ovirtlago.prefix.OvirtPrefix object at 0x7fcb28177b10> @order_by(_TEST_LIST) def test_ha_recovery(prefix): engine = prefix.virt_env.engine_vm().get_api_v4().system_service() with test_utils.TestEvent(engine, [119, 9602, 506]): # VM_DOWN_ERROR event(119) # HA_VM_FAILED event event(9602) # VDS_INITIATED_RUN_VM event(506) vm_host = _vm_host(prefix, VM2_NAME) pid = vm_host.ssh(['pgrep', '-f', 'qemu.*guest=vm2']) vm_host.ssh(['kill', '-KILL', pid.out.decode('utf-8')]) vm_service = test_utils.get_vm_service(engine, VM2_NAME) testlib.assert_true_within_long(
lambda:
vm_service.get().status == types.VmStatus.UP ) ../basic-suite-master/test-scenarios/004_basic_sanity_pytest.py:783: _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ /usr/lib/python2.7/site-packages/ovirtlago/testlib.py:286: in assert_true_within_long assert_equals_within_long(func, True, allowed_exceptions) /usr/lib/python2.7/site-packages/ovirtlago/testlib.py:273: in assert_equals_within_long func, value, LONG_TIMEOUT, allowed_exceptions=allowed_exceptions _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ func = <function <lambda> at 0x7fcb2466cf50>, value = True, timeout = 600 allowed_exceptions = [], initial_wait = 0 def assert_equals_within( func, value, timeout, allowed_exceptions=None, initial_wait=10 ): allowed_exceptions = allowed_exceptions or [] with utils.EggTimer(timeout) as timer: while not timer.elapsed(): try: res = func() if res == value: return except Exception as exc: if _instance_of_any(exc, allowed_exceptions): time.sleep(3) continue LOGGER.exception("Unhandled exception in %s", func) raise if initial_wait == 0: time.sleep(3) else: time.sleep(initial_wait) initial_wait = 0 try: raise AssertionError(
'%s != %s after %s seconds' % (res, value, timeout)
) E AssertionError: False != True after 600 seconds /usr/lib/python2.7/site-packages/ovirtlago/testlib.py:252: AssertionError
Project: https://jenkins.ovirt.org/job/ovirt-system-tests_compat-4.3-suite-master/ Build: https://jenkins.ovirt.org/job/ovirt-system-tests_compat-4.3-suite-master/497... Build Number: 497 Build Status: Still Failing Triggered By: Started by timer ------------------------------------- Changes Since Last Success: ------------------------------------- Changes for Build #488 [Galit] Add centos 8.2 image: el8.2-base Changes for Build #489 [Marcin Sobczyk] basic: Add support for running on el8 [Ehud Yonasi] global_setup: enable el8 setup. Changes for Build #490 [Marcin Sobczyk] basic: Add support for running on el8 Changes for Build #491 [Marcin Sobczyk] basic: Add support for running on el8 Changes for Build #492 [Marcin Sobczyk] basic: Add support for running on el8 Changes for Build #493 [jenkins CI] Auto create ovirt-4.2-release Changes for Build #494 [jenkins CI] Auto create ovirt-4.2 Changes for Build #495 [Marcin Sobczyk] network: Run network suite master on el8 Changes for Build #496 [Marcin Sobczyk] network: Run network suite master on el8 Changes for Build #497 [Marcin Sobczyk] network: Run network suite master on el8 ----------------- Failed Tests: ----------------- 1 tests failed. FAILED: basic-suite-master.test-scenarios.004_basic_sanity_pytest.test_ha_recovery Error Message: AssertionError: False != True after 600 seconds Stack Trace: prefix = <ovirtlago.prefix.OvirtPrefix object at 0x7fb0efaf7b10> @order_by(_TEST_LIST) def test_ha_recovery(prefix): engine = prefix.virt_env.engine_vm().get_api_v4().system_service() with test_utils.TestEvent(engine, [119, 9602, 506]): # VM_DOWN_ERROR event(119) # HA_VM_FAILED event event(9602) # VDS_INITIATED_RUN_VM event(506) vm_host = _vm_host(prefix, VM2_NAME) pid = vm_host.ssh(['pgrep', '-f', 'qemu.*guest=vm2']) vm_host.ssh(['kill', '-KILL', pid.out.decode('utf-8')]) vm_service = test_utils.get_vm_service(engine, VM2_NAME) testlib.assert_true_within_long(
lambda:
vm_service.get().status == types.VmStatus.UP ) ../basic-suite-master/test-scenarios/004_basic_sanity_pytest.py:783: _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ /usr/lib/python2.7/site-packages/ovirtlago/testlib.py:286: in assert_true_within_long assert_equals_within_long(func, True, allowed_exceptions) /usr/lib/python2.7/site-packages/ovirtlago/testlib.py:273: in assert_equals_within_long func, value, LONG_TIMEOUT, allowed_exceptions=allowed_exceptions _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ func = <function <lambda> at 0x7fb0dffaab18>, value = True, timeout = 600 allowed_exceptions = [], initial_wait = 0 def assert_equals_within( func, value, timeout, allowed_exceptions=None, initial_wait=10 ): allowed_exceptions = allowed_exceptions or [] with utils.EggTimer(timeout) as timer: while not timer.elapsed(): try: res = func() if res == value: return except Exception as exc: if _instance_of_any(exc, allowed_exceptions): time.sleep(3) continue LOGGER.exception("Unhandled exception in %s", func) raise if initial_wait == 0: time.sleep(3) else: time.sleep(initial_wait) initial_wait = 0 try: raise AssertionError(
'%s != %s after %s seconds' % (res, value, timeout)
) E AssertionError: False != True after 600 seconds /usr/lib/python2.7/site-packages/ovirtlago/testlib.py:252: AssertionError
Project: https://jenkins.ovirt.org/job/ovirt-system-tests_compat-4.3-suite-master/ Build: https://jenkins.ovirt.org/job/ovirt-system-tests_compat-4.3-suite-master/498... Build Number: 498 Build Status: Still Failing Triggered By: Started by timer ------------------------------------- Changes Since Last Success: ------------------------------------- Changes for Build #489 [Marcin Sobczyk] basic: Add support for running on el8 [Ehud Yonasi] global_setup: enable el8 setup. Changes for Build #490 [Marcin Sobczyk] basic: Add support for running on el8 Changes for Build #491 [Marcin Sobczyk] basic: Add support for running on el8 Changes for Build #492 [Marcin Sobczyk] basic: Add support for running on el8 Changes for Build #493 [jenkins CI] Auto create ovirt-4.2-release Changes for Build #494 [jenkins CI] Auto create ovirt-4.2 Changes for Build #495 [Marcin Sobczyk] network: Run network suite master on el8 Changes for Build #496 [Marcin Sobczyk] network: Run network suite master on el8 Changes for Build #497 [Marcin Sobczyk] network: Run network suite master on el8 Changes for Build #498 [Marcin Sobczyk] network: Run network suite master on el8 ----------------- Failed Tests: ----------------- 1 tests failed. FAILED: basic-suite-master.test-scenarios.004_basic_sanity_pytest.test_ha_recovery Error Message: AssertionError: False != True after 600 seconds Stack Trace: prefix = <ovirtlago.prefix.OvirtPrefix object at 0x7f2ed208eb10> @order_by(_TEST_LIST) def test_ha_recovery(prefix): engine = prefix.virt_env.engine_vm().get_api_v4().system_service() with test_utils.TestEvent(engine, [119, 9602, 506]): # VM_DOWN_ERROR event(119) # HA_VM_FAILED event event(9602) # VDS_INITIATED_RUN_VM event(506) vm_host = _vm_host(prefix, VM2_NAME) pid = vm_host.ssh(['pgrep', '-f', 'qemu.*guest=vm2']) vm_host.ssh(['kill', '-KILL', pid.out.decode('utf-8')]) vm_service = test_utils.get_vm_service(engine, VM2_NAME) testlib.assert_true_within_long(
lambda:
vm_service.get().status == types.VmStatus.UP ) ../basic-suite-master/test-scenarios/004_basic_sanity_pytest.py:783: _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ /usr/lib/python2.7/site-packages/ovirtlago/testlib.py:286: in assert_true_within_long assert_equals_within_long(func, True, allowed_exceptions) /usr/lib/python2.7/site-packages/ovirtlago/testlib.py:273: in assert_equals_within_long func, value, LONG_TIMEOUT, allowed_exceptions=allowed_exceptions _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ func = <function <lambda> at 0x7f2ed008b758>, value = True, timeout = 600 allowed_exceptions = [], initial_wait = 0 def assert_equals_within( func, value, timeout, allowed_exceptions=None, initial_wait=10 ): allowed_exceptions = allowed_exceptions or [] with utils.EggTimer(timeout) as timer: while not timer.elapsed(): try: res = func() if res == value: return except Exception as exc: if _instance_of_any(exc, allowed_exceptions): time.sleep(3) continue LOGGER.exception("Unhandled exception in %s", func) raise if initial_wait == 0: time.sleep(3) else: time.sleep(initial_wait) initial_wait = 0 try: raise AssertionError(
'%s != %s after %s seconds' % (res, value, timeout)
) E AssertionError: False != True after 600 seconds /usr/lib/python2.7/site-packages/ovirtlago/testlib.py:252: AssertionError
Project: https://jenkins.ovirt.org/job/ovirt-system-tests_compat-4.3-suite-master/ Build: https://jenkins.ovirt.org/job/ovirt-system-tests_compat-4.3-suite-master/499... Build Number: 499 Build Status: Still Failing Triggered By: Started by timer ------------------------------------- Changes Since Last Success: ------------------------------------- Changes for Build #490 [Marcin Sobczyk] basic: Add support for running on el8 Changes for Build #491 [Marcin Sobczyk] basic: Add support for running on el8 Changes for Build #492 [Marcin Sobczyk] basic: Add support for running on el8 Changes for Build #493 [jenkins CI] Auto create ovirt-4.2-release Changes for Build #494 [jenkins CI] Auto create ovirt-4.2 Changes for Build #495 [Marcin Sobczyk] network: Run network suite master on el8 Changes for Build #496 [Marcin Sobczyk] network: Run network suite master on el8 Changes for Build #497 [Marcin Sobczyk] network: Run network suite master on el8 Changes for Build #498 [Marcin Sobczyk] network: Run network suite master on el8 Changes for Build #499 [Andrej Cernek] network, hostlib: refactor setup_networks method [Barak Korren] usrc: Support *.yml files ----------------- Failed Tests: ----------------- 1 tests failed. FAILED: basic-suite-master.test-scenarios.004_basic_sanity_pytest.test_ha_recovery Error Message: AssertionError: False != True after 600 seconds Stack Trace: prefix = <ovirtlago.prefix.OvirtPrefix object at 0x7f9f7cd57b10> @order_by(_TEST_LIST) def test_ha_recovery(prefix): engine = prefix.virt_env.engine_vm().get_api_v4().system_service() with test_utils.TestEvent(engine, [119, 9602, 506]): # VM_DOWN_ERROR event(119) # HA_VM_FAILED event event(9602) # VDS_INITIATED_RUN_VM event(506) vm_host = _vm_host(prefix, VM2_NAME) pid = vm_host.ssh(['pgrep', '-f', 'qemu.*guest=vm2']) vm_host.ssh(['kill', '-KILL', pid.out.decode('utf-8')]) vm_service = test_utils.get_vm_service(engine, VM2_NAME) testlib.assert_true_within_long(
lambda:
vm_service.get().status == types.VmStatus.UP ) ../basic-suite-master/test-scenarios/004_basic_sanity_pytest.py:783: _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ /usr/lib/python2.7/site-packages/ovirtlago/testlib.py:286: in assert_true_within_long assert_equals_within_long(func, True, allowed_exceptions) /usr/lib/python2.7/site-packages/ovirtlago/testlib.py:273: in assert_equals_within_long func, value, LONG_TIMEOUT, allowed_exceptions=allowed_exceptions _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ func = <function <lambda> at 0x7f9f7924c848>, value = True, timeout = 600 allowed_exceptions = [], initial_wait = 0 def assert_equals_within( func, value, timeout, allowed_exceptions=None, initial_wait=10 ): allowed_exceptions = allowed_exceptions or [] with utils.EggTimer(timeout) as timer: while not timer.elapsed(): try: res = func() if res == value: return except Exception as exc: if _instance_of_any(exc, allowed_exceptions): time.sleep(3) continue LOGGER.exception("Unhandled exception in %s", func) raise if initial_wait == 0: time.sleep(3) else: time.sleep(initial_wait) initial_wait = 0 try: raise AssertionError(
'%s != %s after %s seconds' % (res, value, timeout)
) E AssertionError: False != True after 600 seconds /usr/lib/python2.7/site-packages/ovirtlago/testlib.py:252: AssertionError
Project: https://jenkins.ovirt.org/job/ovirt-system-tests_compat-4.3-suite-master/ Build: https://jenkins.ovirt.org/job/ovirt-system-tests_compat-4.3-suite-master/500... Build Number: 500 Build Status: Still Failing Triggered By: Started by timer ------------------------------------- Changes Since Last Success: ------------------------------------- Changes for Build #491 [Marcin Sobczyk] basic: Add support for running on el8 Changes for Build #492 [Marcin Sobczyk] basic: Add support for running on el8 Changes for Build #493 [jenkins CI] Auto create ovirt-4.2-release Changes for Build #494 [jenkins CI] Auto create ovirt-4.2 Changes for Build #495 [Marcin Sobczyk] network: Run network suite master on el8 Changes for Build #496 [Marcin Sobczyk] network: Run network suite master on el8 Changes for Build #497 [Marcin Sobczyk] network: Run network suite master on el8 Changes for Build #498 [Marcin Sobczyk] network: Run network suite master on el8 Changes for Build #499 [Andrej Cernek] network, hostlib: refactor setup_networks method [Barak Korren] usrc: Support *.yml files Changes for Build #500 [Galit] Remove basic-suite-4.2 suite ----------------- Failed Tests: ----------------- 1 tests failed. FAILED: basic-suite-master.test-scenarios.002_bootstrap_pytest.test_verify_add_hosts Error Message: RuntimeError: 1 hosts failed installation: lago-compat-4-3-suite-master-host-0: install_failed Stack Trace: api_v4 = <ovirtsdk4.Connection object at 0x7fa49c0e5250> @order_by(_TEST_LIST) def test_verify_add_hosts(api_v4): hosts_service = api_v4.system_service().hosts_service() hosts_status = hosts_service.list(search='datacenter={}'.format(DC_NAME)) total_hosts = len(hosts_status) dump_hosts = _host_status_to_print(hosts_service, hosts_status) LOGGER.debug('Host status, verify_add_hosts:\n {}'.format(dump_hosts)) testlib.assert_true_within( lambda: _single_host_up(hosts_service, total_hosts),
timeout=constants.ADD_HOST_TIMEOUT
) ../basic-suite-master/test-scenarios/002_bootstrap_pytest.py:432: _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ /usr/lib/python2.7/site-packages/ovirtlago/testlib.py:278: in assert_true_within assert_equals_within(func, True, timeout, allowed_exceptions) /usr/lib/python2.7/site-packages/ovirtlago/testlib.py:234: in assert_equals_within res = func() ../basic-suite-master/test-scenarios/002_bootstrap_pytest.py:431: in <lambda> lambda: _single_host_up(hosts_service, total_hosts), ../basic-suite-master/test-scenarios/002_bootstrap_pytest.py:228: in _single_host_up _check_problematic_hosts(hosts_service) _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ hosts_service = <ovirtsdk4.services.HostsService object at 0x7fa49c41ca10> def _check_problematic_hosts(hosts_service): problematic_hosts = hosts_service.list(search='datacenter={} AND status != installing and status != initializing and status != up'.format(DC_NAME)) if len(problematic_hosts): dump_hosts = '%s hosts failed installation:\n' % len(problematic_hosts) for host in problematic_hosts: host_service = hosts_service.host_service(host.id) dump_hosts += '%s: %s\n' % (host.name, host_service.get().status)
raise RuntimeError(dump_hosts)
E RuntimeError: 1 hosts failed installation: E lago-compat-4-3-suite-master-host-0: install_failed ../basic-suite-master/test-scenarios/002_bootstrap_pytest.py:237: RuntimeError
Project: https://jenkins.ovirt.org/job/ovirt-system-tests_compat-4.3-suite-master/ Build: https://jenkins.ovirt.org/job/ovirt-system-tests_compat-4.3-suite-master/501... Build Number: 501 Build Status: Still Failing Triggered By: Started by timer ------------------------------------- Changes Since Last Success: ------------------------------------- Changes for Build #492 [Marcin Sobczyk] basic: Add support for running on el8 Changes for Build #493 [jenkins CI] Auto create ovirt-4.2-release Changes for Build #494 [jenkins CI] Auto create ovirt-4.2 Changes for Build #495 [Marcin Sobczyk] network: Run network suite master on el8 Changes for Build #496 [Marcin Sobczyk] network: Run network suite master on el8 Changes for Build #497 [Marcin Sobczyk] network: Run network suite master on el8 Changes for Build #498 [Marcin Sobczyk] network: Run network suite master on el8 Changes for Build #499 [Andrej Cernek] network, hostlib: refactor setup_networks method [Barak Korren] usrc: Support *.yml files Changes for Build #500 [Galit] Remove basic-suite-4.2 suite Changes for Build #501 [Marcin Sobczyk] basic: Verify glance template creation before tearing down the engine ----------------- Failed Tests: ----------------- 1 tests failed. FAILED: basic-suite-master.test-scenarios.004_basic_sanity_pytest.test_ha_recovery Error Message: AssertionError: False != True after 600 seconds Stack Trace: prefix = <ovirtlago.prefix.OvirtPrefix object at 0x7f07129cc490> @order_by(_TEST_LIST) def test_ha_recovery(prefix): engine = prefix.virt_env.engine_vm().get_api_v4().system_service() with test_utils.TestEvent(engine, [119, 9602, 506]): # VM_DOWN_ERROR event(119) # HA_VM_FAILED event event(9602) # VDS_INITIATED_RUN_VM event(506) vm_host = _vm_host(prefix, VM2_NAME) pid = vm_host.ssh(['pgrep', '-f', 'qemu.*guest=vm2']) vm_host.ssh(['kill', '-KILL', pid.out.decode('utf-8')]) vm_service = test_utils.get_vm_service(engine, VM2_NAME) testlib.assert_true_within_long(
lambda:
vm_service.get().status == types.VmStatus.UP ) ../basic-suite-master/test-scenarios/004_basic_sanity_pytest.py:782: _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ /usr/lib/python2.7/site-packages/ovirtlago/testlib.py:286: in assert_true_within_long assert_equals_within_long(func, True, allowed_exceptions) /usr/lib/python2.7/site-packages/ovirtlago/testlib.py:273: in assert_equals_within_long func, value, LONG_TIMEOUT, allowed_exceptions=allowed_exceptions _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ func = <function <lambda> at 0x7f0712b12d70>, value = True, timeout = 600 allowed_exceptions = [], initial_wait = 0 def assert_equals_within( func, value, timeout, allowed_exceptions=None, initial_wait=10 ): allowed_exceptions = allowed_exceptions or [] with utils.EggTimer(timeout) as timer: while not timer.elapsed(): try: res = func() if res == value: return except Exception as exc: if _instance_of_any(exc, allowed_exceptions): time.sleep(3) continue LOGGER.exception("Unhandled exception in %s", func) raise if initial_wait == 0: time.sleep(3) else: time.sleep(initial_wait) initial_wait = 0 try: raise AssertionError(
'%s != %s after %s seconds' % (res, value, timeout)
) E AssertionError: False != True after 600 seconds /usr/lib/python2.7/site-packages/ovirtlago/testlib.py:252: AssertionError
Project: https://jenkins.ovirt.org/job/ovirt-system-tests_compat-4.3-suite-master/ Build: https://jenkins.ovirt.org/job/ovirt-system-tests_compat-4.3-suite-master/502... Build Number: 502 Build Status: Still Failing Triggered By: Started by timer ------------------------------------- Changes Since Last Success: ------------------------------------- Changes for Build #493 [jenkins CI] Auto create ovirt-4.2-release Changes for Build #494 [jenkins CI] Auto create ovirt-4.2 Changes for Build #495 [Marcin Sobczyk] network: Run network suite master on el8 Changes for Build #496 [Marcin Sobczyk] network: Run network suite master on el8 Changes for Build #497 [Marcin Sobczyk] network: Run network suite master on el8 Changes for Build #498 [Marcin Sobczyk] network: Run network suite master on el8 Changes for Build #499 [Andrej Cernek] network, hostlib: refactor setup_networks method [Barak Korren] usrc: Support *.yml files Changes for Build #500 [Galit] Remove basic-suite-4.2 suite Changes for Build #501 [Marcin Sobczyk] basic: Verify glance template creation before tearing down the engine Changes for Build #502 [Marcin Sobczyk] ost-images: Use OST images when available ----------------- Failed Tests: ----------------- 1 tests failed. FAILED: basic-suite-master.test-scenarios.004_basic_sanity_pytest.test_ha_recovery Error Message: AssertionError: False != True after 600 seconds Stack Trace: prefix = <ovirtlago.prefix.OvirtPrefix object at 0x7f8fecdb6490> @order_by(_TEST_LIST) def test_ha_recovery(prefix): engine = prefix.virt_env.engine_vm().get_api_v4().system_service() with test_utils.TestEvent(engine, [119, 9602, 506]): # VM_DOWN_ERROR event(119) # HA_VM_FAILED event event(9602) # VDS_INITIATED_RUN_VM event(506) vm_host = _vm_host(prefix, VM2_NAME) pid = vm_host.ssh(['pgrep', '-f', 'qemu.*guest=vm2']) vm_host.ssh(['kill', '-KILL', pid.out.decode('utf-8')]) vm_service = test_utils.get_vm_service(engine, VM2_NAME) testlib.assert_true_within_long(
lambda:
vm_service.get().status == types.VmStatus.UP ) ../basic-suite-master/test-scenarios/004_basic_sanity_pytest.py:782: _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ /usr/lib/python2.7/site-packages/ovirtlago/testlib.py:286: in assert_true_within_long assert_equals_within_long(func, True, allowed_exceptions) /usr/lib/python2.7/site-packages/ovirtlago/testlib.py:273: in assert_equals_within_long func, value, LONG_TIMEOUT, allowed_exceptions=allowed_exceptions _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ func = <function <lambda> at 0x7f8fecefc7d0>, value = True, timeout = 600 allowed_exceptions = [], initial_wait = 0 def assert_equals_within( func, value, timeout, allowed_exceptions=None, initial_wait=10 ): allowed_exceptions = allowed_exceptions or [] with utils.EggTimer(timeout) as timer: while not timer.elapsed(): try: res = func() if res == value: return except Exception as exc: if _instance_of_any(exc, allowed_exceptions): time.sleep(3) continue LOGGER.exception("Unhandled exception in %s", func) raise if initial_wait == 0: time.sleep(3) else: time.sleep(initial_wait) initial_wait = 0 try: raise AssertionError(
'%s != %s after %s seconds' % (res, value, timeout)
) E AssertionError: False != True after 600 seconds /usr/lib/python2.7/site-packages/ovirtlago/testlib.py:252: AssertionError
Project: https://jenkins.ovirt.org/job/ovirt-system-tests_compat-4.3-suite-master/ Build: https://jenkins.ovirt.org/job/ovirt-system-tests_compat-4.3-suite-master/503... Build Number: 503 Build Status: Still Failing Triggered By: Started by timer ------------------------------------- Changes Since Last Success: ------------------------------------- Changes for Build #494 [jenkins CI] Auto create ovirt-4.2 Changes for Build #495 [Marcin Sobczyk] network: Run network suite master on el8 Changes for Build #496 [Marcin Sobczyk] network: Run network suite master on el8 Changes for Build #497 [Marcin Sobczyk] network: Run network suite master on el8 Changes for Build #498 [Marcin Sobczyk] network: Run network suite master on el8 Changes for Build #499 [Andrej Cernek] network, hostlib: refactor setup_networks method [Barak Korren] usrc: Support *.yml files Changes for Build #500 [Galit] Remove basic-suite-4.2 suite Changes for Build #501 [Marcin Sobczyk] basic: Verify glance template creation before tearing down the engine Changes for Build #502 [Marcin Sobczyk] ost-images: Use OST images when available Changes for Build #503 [Ehud Yonasi] fix el7 container label. [Roy Golan] Removing rgolan@redhat.com from email notifs ----------------- Failed Tests: ----------------- 1 tests failed. FAILED: basic-suite-master.test-scenarios.004_basic_sanity_pytest.test_ha_recovery Error Message: AssertionError: False != True after 600 seconds Stack Trace: prefix = <ovirtlago.prefix.OvirtPrefix object at 0x7f26be6fb490> @order_by(_TEST_LIST) def test_ha_recovery(prefix): engine = prefix.virt_env.engine_vm().get_api_v4().system_service() with test_utils.TestEvent(engine, [119, 9602, 506]): # VM_DOWN_ERROR event(119) # HA_VM_FAILED event event(9602) # VDS_INITIATED_RUN_VM event(506) vm_host = _vm_host(prefix, VM2_NAME) pid = vm_host.ssh(['pgrep', '-f', 'qemu.*guest=vm2']) vm_host.ssh(['kill', '-KILL', pid.out.decode('utf-8')]) vm_service = test_utils.get_vm_service(engine, VM2_NAME) testlib.assert_true_within_long(
lambda:
vm_service.get().status == types.VmStatus.UP ) ../basic-suite-master/test-scenarios/004_basic_sanity_pytest.py:782: _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ /usr/lib/python2.7/site-packages/ovirtlago/testlib.py:286: in assert_true_within_long assert_equals_within_long(func, True, allowed_exceptions) /usr/lib/python2.7/site-packages/ovirtlago/testlib.py:273: in assert_equals_within_long func, value, LONG_TIMEOUT, allowed_exceptions=allowed_exceptions _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ func = <function <lambda> at 0x7f26c00e0f50>, value = True, timeout = 600 allowed_exceptions = [], initial_wait = 0 def assert_equals_within( func, value, timeout, allowed_exceptions=None, initial_wait=10 ): allowed_exceptions = allowed_exceptions or [] with utils.EggTimer(timeout) as timer: while not timer.elapsed(): try: res = func() if res == value: return except Exception as exc: if _instance_of_any(exc, allowed_exceptions): time.sleep(3) continue LOGGER.exception("Unhandled exception in %s", func) raise if initial_wait == 0: time.sleep(3) else: time.sleep(initial_wait) initial_wait = 0 try: raise AssertionError(
'%s != %s after %s seconds' % (res, value, timeout)
) E AssertionError: False != True after 600 seconds /usr/lib/python2.7/site-packages/ovirtlago/testlib.py:252: AssertionError
Project: https://jenkins.ovirt.org/job/ovirt-system-tests_compat-4.3-suite-master/ Build: https://jenkins.ovirt.org/job/ovirt-system-tests_compat-4.3-suite-master/504... Build Number: 504 Build Status: Still Failing Triggered By: Started by timer ------------------------------------- Changes Since Last Success: ------------------------------------- Changes for Build #494 [jenkins CI] Auto create ovirt-4.2 Changes for Build #495 [Marcin Sobczyk] network: Run network suite master on el8 Changes for Build #496 [Marcin Sobczyk] network: Run network suite master on el8 Changes for Build #497 [Marcin Sobczyk] network: Run network suite master on el8 Changes for Build #498 [Marcin Sobczyk] network: Run network suite master on el8 Changes for Build #499 [Andrej Cernek] network, hostlib: refactor setup_networks method [Barak Korren] usrc: Support *.yml files Changes for Build #500 [Galit] Remove basic-suite-4.2 suite Changes for Build #501 [Marcin Sobczyk] basic: Verify glance template creation before tearing down the engine Changes for Build #502 [Marcin Sobczyk] ost-images: Use OST images when available Changes for Build #503 [Ehud Yonasi] fix el7 container label. [Roy Golan] Removing rgolan@redhat.com from email notifs Changes for Build #504 [Marcin Sobczyk] ost_utils: Add ansible fixtures ----------------- Failed Tests: ----------------- No tests ran.
Project: https://jenkins.ovirt.org/job/ovirt-system-tests_compat-4.3-suite-master/ Build: https://jenkins.ovirt.org/job/ovirt-system-tests_compat-4.3-suite-master/505... Build Number: 505 Build Status: Still Failing Triggered By: Started by timer ------------------------------------- Changes Since Last Success: ------------------------------------- Changes for Build #496 [Marcin Sobczyk] network: Run network suite master on el8 Changes for Build #497 [Marcin Sobczyk] network: Run network suite master on el8 Changes for Build #498 [Marcin Sobczyk] network: Run network suite master on el8 Changes for Build #499 [Andrej Cernek] network, hostlib: refactor setup_networks method [Barak Korren] usrc: Support *.yml files Changes for Build #500 [Galit] Remove basic-suite-4.2 suite Changes for Build #501 [Marcin Sobczyk] basic: Verify glance template creation before tearing down the engine Changes for Build #502 [Marcin Sobczyk] ost-images: Use OST images when available Changes for Build #503 [Ehud Yonasi] fix el7 container label. [Roy Golan] Removing rgolan@redhat.com from email notifs Changes for Build #504 [Marcin Sobczyk] ost_utils: Add ansible fixtures Changes for Build #505 [Marcin Sobczyk] ost_utils: Add ansible fixtures ----------------- Failed Tests: ----------------- 1 tests failed. FAILED: basic-suite-master.test-scenarios.004_basic_sanity_pytest.test_ha_recovery Error Message: AssertionError: False != True after 600 seconds Stack Trace: prefix = <ovirtlago.prefix.OvirtPrefix object at 0x7f6c1ef917d0> @order_by(_TEST_LIST) def test_ha_recovery(prefix): engine = prefix.virt_env.engine_vm().get_api_v4().system_service() with test_utils.TestEvent(engine, [119, 9602, 506]): # VM_DOWN_ERROR event(119) # HA_VM_FAILED event event(9602) # VDS_INITIATED_RUN_VM event(506) vm_host = _vm_host(prefix, VM2_NAME) pid = vm_host.ssh(['pgrep', '-f', 'qemu.*guest=vm2']) vm_host.ssh(['kill', '-KILL', pid.out.decode('utf-8')]) vm_service = test_utils.get_vm_service(engine, VM2_NAME) testlib.assert_true_within_long(
lambda:
vm_service.get().status == types.VmStatus.UP ) ../basic-suite-master/test-scenarios/004_basic_sanity_pytest.py:782: _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ /usr/lib/python2.7/site-packages/ovirtlago/testlib.py:286: in assert_true_within_long assert_equals_within_long(func, True, allowed_exceptions) /usr/lib/python2.7/site-packages/ovirtlago/testlib.py:273: in assert_equals_within_long func, value, LONG_TIMEOUT, allowed_exceptions=allowed_exceptions _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ func = <function <lambda> at 0x7f6c0f8a1320>, value = True, timeout = 600 allowed_exceptions = [], initial_wait = 0 def assert_equals_within( func, value, timeout, allowed_exceptions=None, initial_wait=10 ): allowed_exceptions = allowed_exceptions or [] with utils.EggTimer(timeout) as timer: while not timer.elapsed(): try: res = func() if res == value: return except Exception as exc: if _instance_of_any(exc, allowed_exceptions): time.sleep(3) continue LOGGER.exception("Unhandled exception in %s", func) raise if initial_wait == 0: time.sleep(3) else: time.sleep(initial_wait) initial_wait = 0 try: raise AssertionError(
'%s != %s after %s seconds' % (res, value, timeout)
) E AssertionError: False != True after 600 seconds /usr/lib/python2.7/site-packages/ovirtlago/testlib.py:252: AssertionError
Project: https://jenkins.ovirt.org/job/ovirt-system-tests_compat-4.3-suite-master/ Build: https://jenkins.ovirt.org/job/ovirt-system-tests_compat-4.3-suite-master/506... Build Number: 506 Build Status: Still Failing Triggered By: Started by timer ------------------------------------- Changes Since Last Success: ------------------------------------- Changes for Build #497 [Marcin Sobczyk] network: Run network suite master on el8 Changes for Build #498 [Marcin Sobczyk] network: Run network suite master on el8 Changes for Build #499 [Andrej Cernek] network, hostlib: refactor setup_networks method [Barak Korren] usrc: Support *.yml files Changes for Build #500 [Galit] Remove basic-suite-4.2 suite Changes for Build #501 [Marcin Sobczyk] basic: Verify glance template creation before tearing down the engine Changes for Build #502 [Marcin Sobczyk] ost-images: Use OST images when available Changes for Build #503 [Ehud Yonasi] fix el7 container label. [Roy Golan] Removing rgolan@redhat.com from email notifs Changes for Build #504 [Marcin Sobczyk] ost_utils: Add ansible fixtures Changes for Build #505 [Marcin Sobczyk] ost_utils: Add ansible fixtures Changes for Build #506 [Marcin Sobczyk] ost_utils: Add ansible fixtures ----------------- Failed Tests: ----------------- 1 tests failed. FAILED: basic-suite-master.test-scenarios.004_basic_sanity_pytest.test_ha_recovery Error Message: AssertionError: False != True after 600 seconds Stack Trace: prefix = <ovirtlago.prefix.OvirtPrefix object at 0x7ff6796d37d0> @order_by(_TEST_LIST) def test_ha_recovery(prefix): engine = prefix.virt_env.engine_vm().get_api_v4().system_service() with test_utils.TestEvent(engine, [119, 9602, 506]): # VM_DOWN_ERROR event(119) # HA_VM_FAILED event event(9602) # VDS_INITIATED_RUN_VM event(506) vm_host = _vm_host(prefix, VM2_NAME) pid = vm_host.ssh(['pgrep', '-f', 'qemu.*guest=vm2']) vm_host.ssh(['kill', '-KILL', pid.out.decode('utf-8')]) vm_service = test_utils.get_vm_service(engine, VM2_NAME) testlib.assert_true_within_long(
lambda:
vm_service.get().status == types.VmStatus.UP ) ../basic-suite-master/test-scenarios/004_basic_sanity_pytest.py:782: _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ /usr/lib/python2.7/site-packages/ovirtlago/testlib.py:286: in assert_true_within_long assert_equals_within_long(func, True, allowed_exceptions) /usr/lib/python2.7/site-packages/ovirtlago/testlib.py:273: in assert_equals_within_long func, value, LONG_TIMEOUT, allowed_exceptions=allowed_exceptions _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ func = <function <lambda> at 0x7ff678058e60>, value = True, timeout = 600 allowed_exceptions = [], initial_wait = 0 def assert_equals_within( func, value, timeout, allowed_exceptions=None, initial_wait=10 ): allowed_exceptions = allowed_exceptions or [] with utils.EggTimer(timeout) as timer: while not timer.elapsed(): try: res = func() if res == value: return except Exception as exc: if _instance_of_any(exc, allowed_exceptions): time.sleep(3) continue LOGGER.exception("Unhandled exception in %s", func) raise if initial_wait == 0: time.sleep(3) else: time.sleep(initial_wait) initial_wait = 0 try: raise AssertionError(
'%s != %s after %s seconds' % (res, value, timeout)
) E AssertionError: False != True after 600 seconds /usr/lib/python2.7/site-packages/ovirtlago/testlib.py:252: AssertionError
Project: https://jenkins.ovirt.org/job/ovirt-system-tests_compat-4.3-suite-master/ Build: https://jenkins.ovirt.org/job/ovirt-system-tests_compat-4.3-suite-master/507... Build Number: 507 Build Status: Still Failing Triggered By: Started by timer ------------------------------------- Changes Since Last Success: ------------------------------------- Changes for Build #498 [Marcin Sobczyk] network: Run network suite master on el8 Changes for Build #499 [Andrej Cernek] network, hostlib: refactor setup_networks method [Barak Korren] usrc: Support *.yml files Changes for Build #500 [Galit] Remove basic-suite-4.2 suite Changes for Build #501 [Marcin Sobczyk] basic: Verify glance template creation before tearing down the engine Changes for Build #502 [Marcin Sobczyk] ost-images: Use OST images when available Changes for Build #503 [Ehud Yonasi] fix el7 container label. [Roy Golan] Removing rgolan@redhat.com from email notifs Changes for Build #504 [Marcin Sobczyk] ost_utils: Add ansible fixtures Changes for Build #505 [Marcin Sobczyk] ost_utils: Add ansible fixtures Changes for Build #506 [Marcin Sobczyk] ost_utils: Add ansible fixtures Changes for Build #507 [Marcin Sobczyk] performance: Fix broken links for pgdg repos ----------------- Failed Tests: ----------------- 1 tests failed. FAILED: basic-suite-master.test-scenarios.004_basic_sanity_pytest.test_ha_recovery Error Message: AssertionError: False != True after 600 seconds Stack Trace: prefix = <ovirtlago.prefix.OvirtPrefix object at 0x7fdc8347b7d0> @order_by(_TEST_LIST) def test_ha_recovery(prefix): engine = prefix.virt_env.engine_vm().get_api_v4().system_service() with test_utils.TestEvent(engine, [119, 9602, 506]): # VM_DOWN_ERROR event(119) # HA_VM_FAILED event event(9602) # VDS_INITIATED_RUN_VM event(506) vm_host = _vm_host(prefix, VM2_NAME) pid = vm_host.ssh(['pgrep', '-f', 'qemu.*guest=vm2']) vm_host.ssh(['kill', '-KILL', pid.out.decode('utf-8')]) vm_service = test_utils.get_vm_service(engine, VM2_NAME) testlib.assert_true_within_long(
lambda:
vm_service.get().status == types.VmStatus.UP ) ../basic-suite-master/test-scenarios/004_basic_sanity_pytest.py:782: _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ /usr/lib/python2.7/site-packages/ovirtlago/testlib.py:286: in assert_true_within_long assert_equals_within_long(func, True, allowed_exceptions) /usr/lib/python2.7/site-packages/ovirtlago/testlib.py:273: in assert_equals_within_long func, value, LONG_TIMEOUT, allowed_exceptions=allowed_exceptions _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ func = <function <lambda> at 0x7fdc80026aa0>, value = True, timeout = 600 allowed_exceptions = [], initial_wait = 0 def assert_equals_within( func, value, timeout, allowed_exceptions=None, initial_wait=10 ): allowed_exceptions = allowed_exceptions or [] with utils.EggTimer(timeout) as timer: while not timer.elapsed(): try: res = func() if res == value: return except Exception as exc: if _instance_of_any(exc, allowed_exceptions): time.sleep(3) continue LOGGER.exception("Unhandled exception in %s", func) raise if initial_wait == 0: time.sleep(3) else: time.sleep(initial_wait) initial_wait = 0 try: raise AssertionError(
'%s != %s after %s seconds' % (res, value, timeout)
) E AssertionError: False != True after 600 seconds /usr/lib/python2.7/site-packages/ovirtlago/testlib.py:252: AssertionError
Project: https://jenkins.ovirt.org/job/ovirt-system-tests_compat-4.3-suite-master/ Build: https://jenkins.ovirt.org/job/ovirt-system-tests_compat-4.3-suite-master/508... Build Number: 508 Build Status: Still Failing Triggered By: Started by timer ------------------------------------- Changes Since Last Success: ------------------------------------- Changes for Build #499 [Andrej Cernek] network, hostlib: refactor setup_networks method [Barak Korren] usrc: Support *.yml files Changes for Build #500 [Galit] Remove basic-suite-4.2 suite Changes for Build #501 [Marcin Sobczyk] basic: Verify glance template creation before tearing down the engine Changes for Build #502 [Marcin Sobczyk] ost-images: Use OST images when available Changes for Build #503 [Ehud Yonasi] fix el7 container label. [Roy Golan] Removing rgolan@redhat.com from email notifs Changes for Build #504 [Marcin Sobczyk] ost_utils: Add ansible fixtures Changes for Build #505 [Marcin Sobczyk] ost_utils: Add ansible fixtures Changes for Build #506 [Marcin Sobczyk] ost_utils: Add ansible fixtures Changes for Build #507 [Marcin Sobczyk] performance: Fix broken links for pgdg repos Changes for Build #508 [Marcin Sobczyk] lago: Skip sysprep when ssh key is available ----------------- Failed Tests: ----------------- 1 tests failed. FAILED: basic-suite-master.test-scenarios.004_basic_sanity_pytest.test_ha_recovery Error Message: AssertionError: False != True after 600 seconds Stack Trace: prefix = <ovirtlago.prefix.OvirtPrefix object at 0x7f8055b397d0> @order_by(_TEST_LIST) def test_ha_recovery(prefix): engine = prefix.virt_env.engine_vm().get_api_v4().system_service() with test_utils.TestEvent(engine, [119, 9602, 506]): # VM_DOWN_ERROR event(119) # HA_VM_FAILED event event(9602) # VDS_INITIATED_RUN_VM event(506) vm_host = _vm_host(prefix, VM2_NAME) pid = vm_host.ssh(['pgrep', '-f', 'qemu.*guest=vm2']) vm_host.ssh(['kill', '-KILL', pid.out.decode('utf-8')]) vm_service = test_utils.get_vm_service(engine, VM2_NAME) testlib.assert_true_within_long(
lambda:
vm_service.get().status == types.VmStatus.UP ) ../basic-suite-master/test-scenarios/004_basic_sanity_pytest.py:782: _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ /usr/lib/python2.7/site-packages/ovirtlago/testlib.py:286: in assert_true_within_long assert_equals_within_long(func, True, allowed_exceptions) /usr/lib/python2.7/site-packages/ovirtlago/testlib.py:273: in assert_equals_within_long func, value, LONG_TIMEOUT, allowed_exceptions=allowed_exceptions _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ func = <function <lambda> at 0x7f804e42f758>, value = True, timeout = 600 allowed_exceptions = [], initial_wait = 0 def assert_equals_within( func, value, timeout, allowed_exceptions=None, initial_wait=10 ): allowed_exceptions = allowed_exceptions or [] with utils.EggTimer(timeout) as timer: while not timer.elapsed(): try: res = func() if res == value: return except Exception as exc: if _instance_of_any(exc, allowed_exceptions): time.sleep(3) continue LOGGER.exception("Unhandled exception in %s", func) raise if initial_wait == 0: time.sleep(3) else: time.sleep(initial_wait) initial_wait = 0 try: raise AssertionError(
'%s != %s after %s seconds' % (res, value, timeout)
) E AssertionError: False != True after 600 seconds /usr/lib/python2.7/site-packages/ovirtlago/testlib.py:252: AssertionError
Project: https://jenkins.ovirt.org/job/ovirt-system-tests_compat-4.3-suite-master/ Build: https://jenkins.ovirt.org/job/ovirt-system-tests_compat-4.3-suite-master/509... Build Number: 509 Build Status: Still Failing Triggered By: Started by timer ------------------------------------- Changes Since Last Success: ------------------------------------- Changes for Build #499 [Andrej Cernek] network, hostlib: refactor setup_networks method [Barak Korren] usrc: Support *.yml files Changes for Build #500 [Galit] Remove basic-suite-4.2 suite Changes for Build #501 [Marcin Sobczyk] basic: Verify glance template creation before tearing down the engine Changes for Build #502 [Marcin Sobczyk] ost-images: Use OST images when available Changes for Build #503 [Ehud Yonasi] fix el7 container label. [Roy Golan] Removing rgolan@redhat.com from email notifs Changes for Build #504 [Marcin Sobczyk] ost_utils: Add ansible fixtures Changes for Build #505 [Marcin Sobczyk] ost_utils: Add ansible fixtures Changes for Build #506 [Marcin Sobczyk] ost_utils: Add ansible fixtures Changes for Build #507 [Marcin Sobczyk] performance: Fix broken links for pgdg repos Changes for Build #508 [Marcin Sobczyk] lago: Skip sysprep when ssh key is available Changes for Build #509 [Eyal Shenitzky] Add incremental backup VM test ----------------- Failed Tests: ----------------- 1 tests failed. FAILED: basic-suite-master.test-scenarios.002_bootstrap_pytest.test_verify_add_hosts Error Message: RuntimeError: 1 hosts failed installation: lago-compat-4-3-suite-master-host-1: install_failed Stack Trace: api_v4 = <ovirtsdk4.Connection object at 0x7f48525dc910> @order_by(_TEST_LIST) def test_verify_add_hosts(api_v4): hosts_service = api_v4.system_service().hosts_service() hosts_status = hosts_service.list(search='datacenter={}'.format(DC_NAME)) total_hosts = len(hosts_status) dump_hosts = _host_status_to_print(hosts_service, hosts_status) LOGGER.debug('Host status, verify_add_hosts:\n {}'.format(dump_hosts)) testlib.assert_true_within( lambda: _single_host_up(hosts_service, total_hosts),
timeout=constants.ADD_HOST_TIMEOUT
) ../basic-suite-master/test-scenarios/002_bootstrap_pytest.py:433: _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ /usr/lib/python2.7/site-packages/ovirtlago/testlib.py:278: in assert_true_within assert_equals_within(func, True, timeout, allowed_exceptions) /usr/lib/python2.7/site-packages/ovirtlago/testlib.py:234: in assert_equals_within res = func() ../basic-suite-master/test-scenarios/002_bootstrap_pytest.py:432: in <lambda> lambda: _single_host_up(hosts_service, total_hosts), ../basic-suite-master/test-scenarios/002_bootstrap_pytest.py:229: in _single_host_up _check_problematic_hosts(hosts_service) _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ hosts_service = <ovirtsdk4.services.HostsService object at 0x7f48525021d0> def _check_problematic_hosts(hosts_service): problematic_hosts = hosts_service.list(search='datacenter={} AND status != installing and status != initializing and status != up'.format(DC_NAME)) if len(problematic_hosts): dump_hosts = '%s hosts failed installation:\n' % len(problematic_hosts) for host in problematic_hosts: host_service = hosts_service.host_service(host.id) dump_hosts += '%s: %s\n' % (host.name, host_service.get().status)
raise RuntimeError(dump_hosts)
E RuntimeError: 1 hosts failed installation: E lago-compat-4-3-suite-master-host-1: install_failed ../basic-suite-master/test-scenarios/002_bootstrap_pytest.py:238: RuntimeError
Project: https://jenkins.ovirt.org/job/ovirt-system-tests_compat-4.3-suite-master/ Build: https://jenkins.ovirt.org/job/ovirt-system-tests_compat-4.3-suite-master/510... Build Number: 510 Build Status: Still Failing Triggered By: Started by timer ------------------------------------- Changes Since Last Success: ------------------------------------- Changes for Build #501 [Marcin Sobczyk] basic: Verify glance template creation before tearing down the engine Changes for Build #502 [Marcin Sobczyk] ost-images: Use OST images when available Changes for Build #503 [Ehud Yonasi] fix el7 container label. [Roy Golan] Removing rgolan@redhat.com from email notifs Changes for Build #504 [Marcin Sobczyk] ost_utils: Add ansible fixtures Changes for Build #505 [Marcin Sobczyk] ost_utils: Add ansible fixtures Changes for Build #506 [Marcin Sobczyk] ost_utils: Add ansible fixtures Changes for Build #507 [Marcin Sobczyk] performance: Fix broken links for pgdg repos Changes for Build #508 [Marcin Sobczyk] lago: Skip sysprep when ssh key is available Changes for Build #509 [Eyal Shenitzky] Add incremental backup VM test Changes for Build #510 [Nir Soffer] setup: Do not hide real issue by reserving ports ----------------- Failed Tests: ----------------- 1 tests failed. FAILED: basic-suite-master.test-scenarios.004_basic_sanity_pytest.test_ha_recovery Error Message: AssertionError: False != True after 600 seconds Stack Trace: prefix = <ovirtlago.prefix.OvirtPrefix object at 0x7fa8dd9dca50> @order_by(_TEST_LIST) def test_ha_recovery(prefix): engine = prefix.virt_env.engine_vm().get_api_v4().system_service() with test_utils.TestEvent(engine, [119, 9602, 506]): # VM_DOWN_ERROR event(119) # HA_VM_FAILED event event(9602) # VDS_INITIATED_RUN_VM event(506) vm_host = _vm_host(prefix, VM2_NAME) pid = vm_host.ssh(['pgrep', '-f', 'qemu.*guest=vm2']) vm_host.ssh(['kill', '-KILL', pid.out.decode('utf-8')]) vm_service = test_utils.get_vm_service(engine, VM2_NAME) testlib.assert_true_within_long(
lambda:
vm_service.get().status == types.VmStatus.UP ) ../basic-suite-master/test-scenarios/004_basic_sanity_pytest.py:824: _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ /usr/lib/python2.7/site-packages/ovirtlago/testlib.py:286: in assert_true_within_long assert_equals_within_long(func, True, allowed_exceptions) /usr/lib/python2.7/site-packages/ovirtlago/testlib.py:273: in assert_equals_within_long func, value, LONG_TIMEOUT, allowed_exceptions=allowed_exceptions _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ func = <function <lambda> at 0x7fa8dc0dd7d0>, value = True, timeout = 600 allowed_exceptions = [], initial_wait = 0 def assert_equals_within( func, value, timeout, allowed_exceptions=None, initial_wait=10 ): allowed_exceptions = allowed_exceptions or [] with utils.EggTimer(timeout) as timer: while not timer.elapsed(): try: res = func() if res == value: return except Exception as exc: if _instance_of_any(exc, allowed_exceptions): time.sleep(3) continue LOGGER.exception("Unhandled exception in %s", func) raise if initial_wait == 0: time.sleep(3) else: time.sleep(initial_wait) initial_wait = 0 try: raise AssertionError(
'%s != %s after %s seconds' % (res, value, timeout)
) E AssertionError: False != True after 600 seconds /usr/lib/python2.7/site-packages/ovirtlago/testlib.py:252: AssertionError
Project: https://jenkins.ovirt.org/job/ovirt-system-tests_compat-4.3-suite-master/ Build: https://jenkins.ovirt.org/job/ovirt-system-tests_compat-4.3-suite-master/511... Build Number: 511 Build Status: Still Failing Triggered By: Started by timer ------------------------------------- Changes Since Last Success: ------------------------------------- Changes for Build #502 [Marcin Sobczyk] ost-images: Use OST images when available Changes for Build #503 [Ehud Yonasi] fix el7 container label. [Roy Golan] Removing rgolan@redhat.com from email notifs Changes for Build #504 [Marcin Sobczyk] ost_utils: Add ansible fixtures Changes for Build #505 [Marcin Sobczyk] ost_utils: Add ansible fixtures Changes for Build #506 [Marcin Sobczyk] ost_utils: Add ansible fixtures Changes for Build #507 [Marcin Sobczyk] performance: Fix broken links for pgdg repos Changes for Build #508 [Marcin Sobczyk] lago: Skip sysprep when ssh key is available Changes for Build #509 [Eyal Shenitzky] Add incremental backup VM test Changes for Build #510 [Nir Soffer] setup: Do not hide real issue by reserving ports Changes for Build #511 [Nir Soffer] setup: Do not hide real issue by reserving ports ----------------- Failed Tests: ----------------- 1 tests failed. FAILED: basic-suite-master.test-scenarios.004_basic_sanity_pytest.test_ha_recovery Error Message: AssertionError: False != True after 600 seconds Stack Trace: prefix = <ovirtlago.prefix.OvirtPrefix object at 0x7f98fbb93a50> @order_by(_TEST_LIST) def test_ha_recovery(prefix): engine = prefix.virt_env.engine_vm().get_api_v4().system_service() with test_utils.TestEvent(engine, [119, 9602, 506]): # VM_DOWN_ERROR event(119) # HA_VM_FAILED event event(9602) # VDS_INITIATED_RUN_VM event(506) vm_host = _vm_host(prefix, VM2_NAME) pid = vm_host.ssh(['pgrep', '-f', 'qemu.*guest=vm2']) vm_host.ssh(['kill', '-KILL', pid.out.decode('utf-8')]) vm_service = test_utils.get_vm_service(engine, VM2_NAME) testlib.assert_true_within_long(
lambda:
vm_service.get().status == types.VmStatus.UP ) ../basic-suite-master/test-scenarios/004_basic_sanity_pytest.py:824: _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ /usr/lib/python2.7/site-packages/ovirtlago/testlib.py:286: in assert_true_within_long assert_equals_within_long(func, True, allowed_exceptions) /usr/lib/python2.7/site-packages/ovirtlago/testlib.py:273: in assert_equals_within_long func, value, LONG_TIMEOUT, allowed_exceptions=allowed_exceptions _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ func = <function <lambda> at 0x7f98f84c8c80>, value = True, timeout = 600 allowed_exceptions = [], initial_wait = 0 def assert_equals_within( func, value, timeout, allowed_exceptions=None, initial_wait=10 ): allowed_exceptions = allowed_exceptions or [] with utils.EggTimer(timeout) as timer: while not timer.elapsed(): try: res = func() if res == value: return except Exception as exc: if _instance_of_any(exc, allowed_exceptions): time.sleep(3) continue LOGGER.exception("Unhandled exception in %s", func) raise if initial_wait == 0: time.sleep(3) else: time.sleep(initial_wait) initial_wait = 0 try: raise AssertionError(
'%s != %s after %s seconds' % (res, value, timeout)
) E AssertionError: False != True after 600 seconds /usr/lib/python2.7/site-packages/ovirtlago/testlib.py:252: AssertionError
Project: https://jenkins.ovirt.org/job/ovirt-system-tests_compat-4.3-suite-master/ Build: https://jenkins.ovirt.org/job/ovirt-system-tests_compat-4.3-suite-master/512... Build Number: 512 Build Status: Still Failing Triggered By: Started by timer ------------------------------------- Changes Since Last Success: ------------------------------------- Changes for Build #503 [Ehud Yonasi] fix el7 container label. [Roy Golan] Removing rgolan@redhat.com from email notifs Changes for Build #504 [Marcin Sobczyk] ost_utils: Add ansible fixtures Changes for Build #505 [Marcin Sobczyk] ost_utils: Add ansible fixtures Changes for Build #506 [Marcin Sobczyk] ost_utils: Add ansible fixtures Changes for Build #507 [Marcin Sobczyk] performance: Fix broken links for pgdg repos Changes for Build #508 [Marcin Sobczyk] lago: Skip sysprep when ssh key is available Changes for Build #509 [Eyal Shenitzky] Add incremental backup VM test Changes for Build #510 [Nir Soffer] setup: Do not hide real issue by reserving ports Changes for Build #511 [Nir Soffer] setup: Do not hide real issue by reserving ports Changes for Build #512 [Nir Soffer] setup: Do not hide real issue by reserving ports ----------------- Failed Tests: ----------------- 1 tests failed. FAILED: basic-suite-master.test-scenarios.004_basic_sanity_pytest.test_ha_recovery Error Message: AssertionError: False != True after 600 seconds Stack Trace: prefix = <ovirtlago.prefix.OvirtPrefix object at 0x7f3ccd6f5a50> @order_by(_TEST_LIST) def test_ha_recovery(prefix): engine = prefix.virt_env.engine_vm().get_api_v4().system_service() with test_utils.TestEvent(engine, [119, 9602, 506]): # VM_DOWN_ERROR event(119) # HA_VM_FAILED event event(9602) # VDS_INITIATED_RUN_VM event(506) vm_host = _vm_host(prefix, VM2_NAME) pid = vm_host.ssh(['pgrep', '-f', 'qemu.*guest=vm2']) vm_host.ssh(['kill', '-KILL', pid.out.decode('utf-8')]) vm_service = test_utils.get_vm_service(engine, VM2_NAME) testlib.assert_true_within_long(
lambda:
vm_service.get().status == types.VmStatus.UP ) ../basic-suite-master/test-scenarios/004_basic_sanity_pytest.py:824: _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ /usr/lib/python2.7/site-packages/ovirtlago/testlib.py:286: in assert_true_within_long assert_equals_within_long(func, True, allowed_exceptions) /usr/lib/python2.7/site-packages/ovirtlago/testlib.py:273: in assert_equals_within_long func, value, LONG_TIMEOUT, allowed_exceptions=allowed_exceptions _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ func = <function <lambda> at 0x7f3ccc0aa410>, value = True, timeout = 600 allowed_exceptions = [], initial_wait = 0 def assert_equals_within( func, value, timeout, allowed_exceptions=None, initial_wait=10 ): allowed_exceptions = allowed_exceptions or [] with utils.EggTimer(timeout) as timer: while not timer.elapsed(): try: res = func() if res == value: return except Exception as exc: if _instance_of_any(exc, allowed_exceptions): time.sleep(3) continue LOGGER.exception("Unhandled exception in %s", func) raise if initial_wait == 0: time.sleep(3) else: time.sleep(initial_wait) initial_wait = 0 try: raise AssertionError(
'%s != %s after %s seconds' % (res, value, timeout)
) E AssertionError: False != True after 600 seconds /usr/lib/python2.7/site-packages/ovirtlago/testlib.py:252: AssertionError
Project: https://jenkins.ovirt.org/job/ovirt-system-tests_compat-4.3-suite-master/ Build: https://jenkins.ovirt.org/job/ovirt-system-tests_compat-4.3-suite-master/513... Build Number: 513 Build Status: Still Failing Triggered By: Started by timer ------------------------------------- Changes Since Last Success: ------------------------------------- Changes for Build #504 [Marcin Sobczyk] ost_utils: Add ansible fixtures Changes for Build #505 [Marcin Sobczyk] ost_utils: Add ansible fixtures Changes for Build #506 [Marcin Sobczyk] ost_utils: Add ansible fixtures Changes for Build #507 [Marcin Sobczyk] performance: Fix broken links for pgdg repos Changes for Build #508 [Marcin Sobczyk] lago: Skip sysprep when ssh key is available Changes for Build #509 [Eyal Shenitzky] Add incremental backup VM test Changes for Build #510 [Nir Soffer] setup: Do not hide real issue by reserving ports Changes for Build #511 [Nir Soffer] setup: Do not hide real issue by reserving ports Changes for Build #512 [Nir Soffer] setup: Do not hide real issue by reserving ports Changes for Build #513 [Nir Soffer] setup: Do not hide real issue by reserving ports ----------------- Failed Tests: ----------------- 1 tests failed. FAILED: basic-suite-master.test-scenarios.004_basic_sanity_pytest.test_ha_recovery Error Message: AssertionError: False != True after 600 seconds Stack Trace: prefix = <ovirtlago.prefix.OvirtPrefix object at 0x7fe76b4b3a50> @order_by(_TEST_LIST) def test_ha_recovery(prefix): engine = prefix.virt_env.engine_vm().get_api_v4().system_service() with test_utils.TestEvent(engine, [119, 9602, 506]): # VM_DOWN_ERROR event(119) # HA_VM_FAILED event event(9602) # VDS_INITIATED_RUN_VM event(506) vm_host = _vm_host(prefix, VM2_NAME) pid = vm_host.ssh(['pgrep', '-f', 'qemu.*guest=vm2']) vm_host.ssh(['kill', '-KILL', pid.out.decode('utf-8')]) vm_service = test_utils.get_vm_service(engine, VM2_NAME) testlib.assert_true_within_long(
lambda:
vm_service.get().status == types.VmStatus.UP ) ../basic-suite-master/test-scenarios/004_basic_sanity_pytest.py:824: _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ /usr/lib/python2.7/site-packages/ovirtlago/testlib.py:286: in assert_true_within_long assert_equals_within_long(func, True, allowed_exceptions) /usr/lib/python2.7/site-packages/ovirtlago/testlib.py:273: in assert_equals_within_long func, value, LONG_TIMEOUT, allowed_exceptions=allowed_exceptions _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ func = <function <lambda> at 0x7fe7680ce6e0>, value = True, timeout = 600 allowed_exceptions = [], initial_wait = 0 def assert_equals_within( func, value, timeout, allowed_exceptions=None, initial_wait=10 ): allowed_exceptions = allowed_exceptions or [] with utils.EggTimer(timeout) as timer: while not timer.elapsed(): try: res = func() if res == value: return except Exception as exc: if _instance_of_any(exc, allowed_exceptions): time.sleep(3) continue LOGGER.exception("Unhandled exception in %s", func) raise if initial_wait == 0: time.sleep(3) else: time.sleep(initial_wait) initial_wait = 0 try: raise AssertionError(
'%s != %s after %s seconds' % (res, value, timeout)
) E AssertionError: False != True after 600 seconds /usr/lib/python2.7/site-packages/ovirtlago/testlib.py:252: AssertionError
Project: https://jenkins.ovirt.org/job/ovirt-system-tests_compat-4.3-suite-master/ Build: https://jenkins.ovirt.org/job/ovirt-system-tests_compat-4.3-suite-master/514... Build Number: 514 Build Status: Still Failing Triggered By: Started by timer ------------------------------------- Changes Since Last Success: ------------------------------------- Changes for Build #505 [Marcin Sobczyk] ost_utils: Add ansible fixtures Changes for Build #506 [Marcin Sobczyk] ost_utils: Add ansible fixtures Changes for Build #507 [Marcin Sobczyk] performance: Fix broken links for pgdg repos Changes for Build #508 [Marcin Sobczyk] lago: Skip sysprep when ssh key is available Changes for Build #509 [Eyal Shenitzky] Add incremental backup VM test Changes for Build #510 [Nir Soffer] setup: Do not hide real issue by reserving ports Changes for Build #511 [Nir Soffer] setup: Do not hide real issue by reserving ports Changes for Build #512 [Nir Soffer] setup: Do not hide real issue by reserving ports Changes for Build #513 [Nir Soffer] setup: Do not hide real issue by reserving ports Changes for Build #514 [Nir Soffer] setup: Do not hide real issue by reserving ports [Sandro Bonazzola] ovirt-guest-agent: drop fc30 [Ehud Yonasi] mock_configs: added rhel configs. ----------------- Failed Tests: ----------------- 1 tests failed. FAILED: basic-suite-master.test-scenarios.004_basic_sanity_pytest.test_ha_recovery Error Message: AssertionError: False != True after 600 seconds Stack Trace: prefix = <ovirtlago.prefix.OvirtPrefix object at 0x7f56b66eca50> @order_by(_TEST_LIST) def test_ha_recovery(prefix): engine = prefix.virt_env.engine_vm().get_api_v4().system_service() with test_utils.TestEvent(engine, [119, 9602, 506]): # VM_DOWN_ERROR event(119) # HA_VM_FAILED event event(9602) # VDS_INITIATED_RUN_VM event(506) vm_host = _vm_host(prefix, VM2_NAME) pid = vm_host.ssh(['pgrep', '-f', 'qemu.*guest=vm2']) vm_host.ssh(['kill', '-KILL', pid.out.decode('utf-8')]) vm_service = test_utils.get_vm_service(engine, VM2_NAME) testlib.assert_true_within_long(
lambda:
vm_service.get().status == types.VmStatus.UP ) ../basic-suite-master/test-scenarios/004_basic_sanity_pytest.py:824: _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ /usr/lib/python2.7/site-packages/ovirtlago/testlib.py:286: in assert_true_within_long assert_equals_within_long(func, True, allowed_exceptions) /usr/lib/python2.7/site-packages/ovirtlago/testlib.py:273: in assert_equals_within_long func, value, LONG_TIMEOUT, allowed_exceptions=allowed_exceptions _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ func = <function <lambda> at 0x7f56b67b7320>, value = True, timeout = 600 allowed_exceptions = [], initial_wait = 0 def assert_equals_within( func, value, timeout, allowed_exceptions=None, initial_wait=10 ): allowed_exceptions = allowed_exceptions or [] with utils.EggTimer(timeout) as timer: while not timer.elapsed(): try: res = func() if res == value: return except Exception as exc: if _instance_of_any(exc, allowed_exceptions): time.sleep(3) continue LOGGER.exception("Unhandled exception in %s", func) raise if initial_wait == 0: time.sleep(3) else: time.sleep(initial_wait) initial_wait = 0 try: raise AssertionError(
'%s != %s after %s seconds' % (res, value, timeout)
) E AssertionError: False != True after 600 seconds /usr/lib/python2.7/site-packages/ovirtlago/testlib.py:252: AssertionError
Project: https://jenkins.ovirt.org/job/ovirt-system-tests_compat-4.3-suite-master/ Build: https://jenkins.ovirt.org/job/ovirt-system-tests_compat-4.3-suite-master/515... Build Number: 515 Build Status: Still Failing Triggered By: Started by timer ------------------------------------- Changes Since Last Success: ------------------------------------- Changes for Build #506 [Marcin Sobczyk] ost_utils: Add ansible fixtures Changes for Build #507 [Marcin Sobczyk] performance: Fix broken links for pgdg repos Changes for Build #508 [Marcin Sobczyk] lago: Skip sysprep when ssh key is available Changes for Build #509 [Eyal Shenitzky] Add incremental backup VM test Changes for Build #510 [Nir Soffer] setup: Do not hide real issue by reserving ports Changes for Build #511 [Nir Soffer] setup: Do not hide real issue by reserving ports Changes for Build #512 [Nir Soffer] setup: Do not hide real issue by reserving ports Changes for Build #513 [Nir Soffer] setup: Do not hide real issue by reserving ports Changes for Build #514 [Nir Soffer] setup: Do not hide real issue by reserving ports [Sandro Bonazzola] ovirt-guest-agent: drop fc30 [Ehud Yonasi] mock_configs: added rhel configs. Changes for Build #515 [Nir Soffer] setup: Do not hide real issue by reserving ports ----------------- Failed Tests: ----------------- 1 tests failed. FAILED: basic-suite-master.test-scenarios.004_basic_sanity_pytest.test_ha_recovery Error Message: AssertionError: False != True after 600 seconds Stack Trace: prefix = <ovirtlago.prefix.OvirtPrefix object at 0x7f0839d12a50> @order_by(_TEST_LIST) def test_ha_recovery(prefix): engine = prefix.virt_env.engine_vm().get_api_v4().system_service() with test_utils.TestEvent(engine, [119, 9602, 506]): # VM_DOWN_ERROR event(119) # HA_VM_FAILED event event(9602) # VDS_INITIATED_RUN_VM event(506) vm_host = _vm_host(prefix, VM2_NAME) pid = vm_host.ssh(['pgrep', '-f', 'qemu.*guest=vm2']) vm_host.ssh(['kill', '-KILL', pid.out.decode('utf-8')]) vm_service = test_utils.get_vm_service(engine, VM2_NAME) testlib.assert_true_within_long(
lambda:
vm_service.get().status == types.VmStatus.UP ) ../basic-suite-master/test-scenarios/004_basic_sanity_pytest.py:824: _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ /usr/lib/python2.7/site-packages/ovirtlago/testlib.py:286: in assert_true_within_long assert_equals_within_long(func, True, allowed_exceptions) /usr/lib/python2.7/site-packages/ovirtlago/testlib.py:273: in assert_equals_within_long func, value, LONG_TIMEOUT, allowed_exceptions=allowed_exceptions _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ func = <function <lambda> at 0x7f0832636f50>, value = True, timeout = 600 allowed_exceptions = [], initial_wait = 0 def assert_equals_within( func, value, timeout, allowed_exceptions=None, initial_wait=10 ): allowed_exceptions = allowed_exceptions or [] with utils.EggTimer(timeout) as timer: while not timer.elapsed(): try: res = func() if res == value: return except Exception as exc: if _instance_of_any(exc, allowed_exceptions): time.sleep(3) continue LOGGER.exception("Unhandled exception in %s", func) raise if initial_wait == 0: time.sleep(3) else: time.sleep(initial_wait) initial_wait = 0 try: raise AssertionError(
'%s != %s after %s seconds' % (res, value, timeout)
) E AssertionError: False != True after 600 seconds /usr/lib/python2.7/site-packages/ovirtlago/testlib.py:252: AssertionError
Project: https://jenkins.ovirt.org/job/ovirt-system-tests_compat-4.3-suite-master/ Build: https://jenkins.ovirt.org/job/ovirt-system-tests_compat-4.3-suite-master/516... Build Number: 516 Build Status: Still Failing Triggered By: Started by timer ------------------------------------- Changes Since Last Success: ------------------------------------- Changes for Build #507 [Marcin Sobczyk] performance: Fix broken links for pgdg repos Changes for Build #508 [Marcin Sobczyk] lago: Skip sysprep when ssh key is available Changes for Build #509 [Eyal Shenitzky] Add incremental backup VM test Changes for Build #510 [Nir Soffer] setup: Do not hide real issue by reserving ports Changes for Build #511 [Nir Soffer] setup: Do not hide real issue by reserving ports Changes for Build #512 [Nir Soffer] setup: Do not hide real issue by reserving ports Changes for Build #513 [Nir Soffer] setup: Do not hide real issue by reserving ports Changes for Build #514 [Nir Soffer] setup: Do not hide real issue by reserving ports [Sandro Bonazzola] ovirt-guest-agent: drop fc30 [Ehud Yonasi] mock_configs: added rhel configs. Changes for Build #515 [Nir Soffer] setup: Do not hide real issue by reserving ports Changes for Build #516 [Michal Skrivanek] verify hotplug memory and test unplug [Tristan Cacqueray] zuul-jobs-gate: repparent to base-gate-job [Ehud Yonasi] stdci_runner: added the rhel labels selection. ----------------- Failed Tests: ----------------- 1 tests failed. FAILED: basic-suite-master.test-scenarios.002_bootstrap_pytest.test_verify_glance_import Error Message: AssertionError: False != True after 600 seconds Stack Trace: api_v4 = <ovirtsdk4.Connection object at 0x7fed132a6a50> @order_by(_TEST_LIST) def test_verify_glance_import(api_v4): # If we go with the engine backup before the glance template # creation is complete, we'll fail the creation of 'vm1' later, # which is based on that template. templates_service = api_v4.system_service().templates_service() testlib.assert_true_within_long(
lambda: TEMPLATE_GUEST in [t.name for t in templates_service.list()]
) ../basic-suite-master/test-scenarios/002_bootstrap_pytest.py:1289: _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ /usr/lib/python2.7/site-packages/ovirtlago/testlib.py:286: in assert_true_within_long assert_equals_within_long(func, True, allowed_exceptions) /usr/lib/python2.7/site-packages/ovirtlago/testlib.py:273: in assert_equals_within_long func, value, LONG_TIMEOUT, allowed_exceptions=allowed_exceptions _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ func = <function <lambda> at 0x7fed10057230>, value = True, timeout = 600 allowed_exceptions = [], initial_wait = 0 def assert_equals_within( func, value, timeout, allowed_exceptions=None, initial_wait=10 ): allowed_exceptions = allowed_exceptions or [] with utils.EggTimer(timeout) as timer: while not timer.elapsed(): try: res = func() if res == value: return except Exception as exc: if _instance_of_any(exc, allowed_exceptions): time.sleep(3) continue LOGGER.exception("Unhandled exception in %s", func) raise if initial_wait == 0: time.sleep(3) else: time.sleep(initial_wait) initial_wait = 0 try: raise AssertionError(
'%s != %s after %s seconds' % (res, value, timeout)
) E AssertionError: False != True after 600 seconds /usr/lib/python2.7/site-packages/ovirtlago/testlib.py:252: AssertionError
Project: https://jenkins.ovirt.org/job/ovirt-system-tests_compat-4.3-suite-master/ Build: https://jenkins.ovirt.org/job/ovirt-system-tests_compat-4.3-suite-master/517... Build Number: 517 Build Status: Still Failing Triggered By: Started by timer ------------------------------------- Changes Since Last Success: ------------------------------------- Changes for Build #508 [Marcin Sobczyk] lago: Skip sysprep when ssh key is available Changes for Build #509 [Eyal Shenitzky] Add incremental backup VM test Changes for Build #510 [Nir Soffer] setup: Do not hide real issue by reserving ports Changes for Build #511 [Nir Soffer] setup: Do not hide real issue by reserving ports Changes for Build #512 [Nir Soffer] setup: Do not hide real issue by reserving ports Changes for Build #513 [Nir Soffer] setup: Do not hide real issue by reserving ports Changes for Build #514 [Nir Soffer] setup: Do not hide real issue by reserving ports [Sandro Bonazzola] ovirt-guest-agent: drop fc30 [Ehud Yonasi] mock_configs: added rhel configs. Changes for Build #515 [Nir Soffer] setup: Do not hide real issue by reserving ports Changes for Build #516 [Michal Skrivanek] verify hotplug memory and test unplug [Tristan Cacqueray] zuul-jobs-gate: repparent to base-gate-job [Ehud Yonasi] stdci_runner: added the rhel labels selection. Changes for Build #517 [Galit Rosenthal] Remove the test configure_vdsm_sos_plugin not needed ----------------- Failed Tests: ----------------- 1 tests failed. FAILED: basic-suite-master.test-scenarios.004_basic_sanity_pytest.test_ha_recovery Error Message: AssertionError: False != True after 600 seconds Stack Trace: prefix = <ovirtlago.prefix.OvirtPrefix object at 0x7f7e46046f90> @order_by(_TEST_LIST) def test_ha_recovery(prefix): engine = prefix.virt_env.engine_vm().get_api_v4().system_service() with test_utils.TestEvent(engine, [119, 9602, 506]): # VM_DOWN_ERROR event(119) # HA_VM_FAILED event event(9602) # VDS_INITIATED_RUN_VM event(506) vm_host = _vm_host(prefix, VM2_NAME) pid = vm_host.ssh(['pgrep', '-f', 'qemu.*guest=vm2']) vm_host.ssh(['kill', '-KILL', pid.out.decode('utf-8')]) vm_service = test_utils.get_vm_service(engine, VM2_NAME) testlib.assert_true_within_long(
lambda:
vm_service.get().status == types.VmStatus.UP ) ../basic-suite-master/test-scenarios/004_basic_sanity_pytest.py:819: _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ /usr/lib/python2.7/site-packages/ovirtlago/testlib.py:286: in assert_true_within_long assert_equals_within_long(func, True, allowed_exceptions) /usr/lib/python2.7/site-packages/ovirtlago/testlib.py:273: in assert_equals_within_long func, value, LONG_TIMEOUT, allowed_exceptions=allowed_exceptions _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ func = <function <lambda> at 0x7f7e440730c8>, value = True, timeout = 600 allowed_exceptions = [], initial_wait = 0 def assert_equals_within( func, value, timeout, allowed_exceptions=None, initial_wait=10 ): allowed_exceptions = allowed_exceptions or [] with utils.EggTimer(timeout) as timer: while not timer.elapsed(): try: res = func() if res == value: return except Exception as exc: if _instance_of_any(exc, allowed_exceptions): time.sleep(3) continue LOGGER.exception("Unhandled exception in %s", func) raise if initial_wait == 0: time.sleep(3) else: time.sleep(initial_wait) initial_wait = 0 try: raise AssertionError(
'%s != %s after %s seconds' % (res, value, timeout)
) E AssertionError: False != True after 600 seconds /usr/lib/python2.7/site-packages/ovirtlago/testlib.py:252: AssertionError
Project: https://jenkins.ovirt.org/job/ovirt-system-tests_compat-4.3-suite-master/ Build: https://jenkins.ovirt.org/job/ovirt-system-tests_compat-4.3-suite-master/518... Build Number: 518 Build Status: Still Failing Triggered By: Started by timer ------------------------------------- Changes Since Last Success: ------------------------------------- Changes for Build #509 [Eyal Shenitzky] Add incremental backup VM test Changes for Build #510 [Nir Soffer] setup: Do not hide real issue by reserving ports Changes for Build #511 [Nir Soffer] setup: Do not hide real issue by reserving ports Changes for Build #512 [Nir Soffer] setup: Do not hide real issue by reserving ports Changes for Build #513 [Nir Soffer] setup: Do not hide real issue by reserving ports Changes for Build #514 [Nir Soffer] setup: Do not hide real issue by reserving ports [Sandro Bonazzola] ovirt-guest-agent: drop fc30 [Ehud Yonasi] mock_configs: added rhel configs. Changes for Build #515 [Nir Soffer] setup: Do not hide real issue by reserving ports Changes for Build #516 [Michal Skrivanek] verify hotplug memory and test unplug [Tristan Cacqueray] zuul-jobs-gate: repparent to base-gate-job [Ehud Yonasi] stdci_runner: added the rhel labels selection. Changes for Build #517 [Galit Rosenthal] Remove the test configure_vdsm_sos_plugin not needed Changes for Build #518 [Marcin Sobczyk] basic: ansible: Port '003_00_metrics_bootstrap_pytest.py' to ansible ----------------- Failed Tests: ----------------- 1 tests failed. FAILED: basic-suite-master.test-scenarios.004_basic_sanity_pytest.test_ha_recovery Error Message: AssertionError: False != True after 600 seconds Stack Trace: prefix = <ovirtlago.prefix.OvirtPrefix object at 0x7f1caae3a710> @order_by(_TEST_LIST) def test_ha_recovery(prefix): engine = prefix.virt_env.engine_vm().get_api_v4().system_service() with test_utils.TestEvent(engine, [119, 9602, 506]): # VM_DOWN_ERROR event(119) # HA_VM_FAILED event event(9602) # VDS_INITIATED_RUN_VM event(506) vm_host = _vm_host(prefix, VM2_NAME) pid = vm_host.ssh(['pgrep', '-f', 'qemu.*guest=vm2']) vm_host.ssh(['kill', '-KILL', pid.out.decode('utf-8')]) vm_service = test_utils.get_vm_service(engine, VM2_NAME) testlib.assert_true_within_long(
lambda:
vm_service.get().status == types.VmStatus.UP ) ../basic-suite-master/test-scenarios/004_basic_sanity_pytest.py:819: _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ /usr/lib/python2.7/site-packages/ovirtlago/testlib.py:286: in assert_true_within_long assert_equals_within_long(func, True, allowed_exceptions) /usr/lib/python2.7/site-packages/ovirtlago/testlib.py:273: in assert_equals_within_long func, value, LONG_TIMEOUT, allowed_exceptions=allowed_exceptions _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ func = <function <lambda> at 0x7f1caac710c8>, value = True, timeout = 600 allowed_exceptions = [], initial_wait = 0 def assert_equals_within( func, value, timeout, allowed_exceptions=None, initial_wait=10 ): allowed_exceptions = allowed_exceptions or [] with utils.EggTimer(timeout) as timer: while not timer.elapsed(): try: res = func() if res == value: return except Exception as exc: if _instance_of_any(exc, allowed_exceptions): time.sleep(3) continue LOGGER.exception("Unhandled exception in %s", func) raise if initial_wait == 0: time.sleep(3) else: time.sleep(initial_wait) initial_wait = 0 try: raise AssertionError(
'%s != %s after %s seconds' % (res, value, timeout)
) E AssertionError: False != True after 600 seconds /usr/lib/python2.7/site-packages/ovirtlago/testlib.py:252: AssertionError
Project: https://jenkins.ovirt.org/job/ovirt-system-tests_compat-4.3-suite-master/ Build: https://jenkins.ovirt.org/job/ovirt-system-tests_compat-4.3-suite-master/519... Build Number: 519 Build Status: Still Failing Triggered By: Started by timer ------------------------------------- Changes Since Last Success: ------------------------------------- Changes for Build #510 [Nir Soffer] setup: Do not hide real issue by reserving ports Changes for Build #511 [Nir Soffer] setup: Do not hide real issue by reserving ports Changes for Build #512 [Nir Soffer] setup: Do not hide real issue by reserving ports Changes for Build #513 [Nir Soffer] setup: Do not hide real issue by reserving ports Changes for Build #514 [Nir Soffer] setup: Do not hide real issue by reserving ports [Sandro Bonazzola] ovirt-guest-agent: drop fc30 [Ehud Yonasi] mock_configs: added rhel configs. Changes for Build #515 [Nir Soffer] setup: Do not hide real issue by reserving ports Changes for Build #516 [Michal Skrivanek] verify hotplug memory and test unplug [Tristan Cacqueray] zuul-jobs-gate: repparent to base-gate-job [Ehud Yonasi] stdci_runner: added the rhel labels selection. Changes for Build #517 [Galit Rosenthal] Remove the test configure_vdsm_sos_plugin not needed Changes for Build #518 [Marcin Sobczyk] basic: ansible: Port '003_00_metrics_bootstrap_pytest.py' to ansible Changes for Build #519 [Marcin Sobczyk] basic: ansible: Port '003_00_metrics_bootstrap_pytest.py' to ansible ----------------- Failed Tests: ----------------- 1 tests failed. FAILED: basic-suite-master.test-scenarios.004_basic_sanity_pytest.test_ha_recovery Error Message: AssertionError: False != True after 600 seconds Stack Trace: prefix = <ovirtlago.prefix.OvirtPrefix object at 0x7f151a85f710> @order_by(_TEST_LIST) def test_ha_recovery(prefix): engine = prefix.virt_env.engine_vm().get_api_v4().system_service() with test_utils.TestEvent(engine, [119, 9602, 506]): # VM_DOWN_ERROR event(119) # HA_VM_FAILED event event(9602) # VDS_INITIATED_RUN_VM event(506) vm_host = _vm_host(prefix, VM2_NAME) pid = vm_host.ssh(['pgrep', '-f', 'qemu.*guest=vm2']) vm_host.ssh(['kill', '-KILL', pid.out.decode('utf-8')]) vm_service = test_utils.get_vm_service(engine, VM2_NAME) testlib.assert_true_within_long(
lambda:
vm_service.get().status == types.VmStatus.UP ) ../basic-suite-master/test-scenarios/004_basic_sanity_pytest.py:819: _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ /usr/lib/python2.7/site-packages/ovirtlago/testlib.py:286: in assert_true_within_long assert_equals_within_long(func, True, allowed_exceptions) /usr/lib/python2.7/site-packages/ovirtlago/testlib.py:273: in assert_equals_within_long func, value, LONG_TIMEOUT, allowed_exceptions=allowed_exceptions _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ func = <function <lambda> at 0x7f151a6960c8>, value = True, timeout = 600 allowed_exceptions = [], initial_wait = 0 def assert_equals_within( func, value, timeout, allowed_exceptions=None, initial_wait=10 ): allowed_exceptions = allowed_exceptions or [] with utils.EggTimer(timeout) as timer: while not timer.elapsed(): try: res = func() if res == value: return except Exception as exc: if _instance_of_any(exc, allowed_exceptions): time.sleep(3) continue LOGGER.exception("Unhandled exception in %s", func) raise if initial_wait == 0: time.sleep(3) else: time.sleep(initial_wait) initial_wait = 0 try: raise AssertionError(
'%s != %s after %s seconds' % (res, value, timeout)
) E AssertionError: False != True after 600 seconds /usr/lib/python2.7/site-packages/ovirtlago/testlib.py:252: AssertionError
Project: https://jenkins.ovirt.org/job/ovirt-system-tests_compat-4.3-suite-master/ Build: https://jenkins.ovirt.org/job/ovirt-system-tests_compat-4.3-suite-master/520... Build Number: 520 Build Status: Still Failing Triggered By: Started by timer ------------------------------------- Changes Since Last Success: ------------------------------------- Changes for Build #511 [Nir Soffer] setup: Do not hide real issue by reserving ports Changes for Build #512 [Nir Soffer] setup: Do not hide real issue by reserving ports Changes for Build #513 [Nir Soffer] setup: Do not hide real issue by reserving ports Changes for Build #514 [Nir Soffer] setup: Do not hide real issue by reserving ports [Sandro Bonazzola] ovirt-guest-agent: drop fc30 [Ehud Yonasi] mock_configs: added rhel configs. Changes for Build #515 [Nir Soffer] setup: Do not hide real issue by reserving ports Changes for Build #516 [Michal Skrivanek] verify hotplug memory and test unplug [Tristan Cacqueray] zuul-jobs-gate: repparent to base-gate-job [Ehud Yonasi] stdci_runner: added the rhel labels selection. Changes for Build #517 [Galit Rosenthal] Remove the test configure_vdsm_sos_plugin not needed Changes for Build #518 [Marcin Sobczyk] basic: ansible: Port '003_00_metrics_bootstrap_pytest.py' to ansible Changes for Build #519 [Marcin Sobczyk] basic: ansible: Port '003_00_metrics_bootstrap_pytest.py' to ansible Changes for Build #520 [Marcin Sobczyk] basic: ansible: Port '003_00_metrics_bootstrap_pytest.py' to ansible ----------------- Failed Tests: ----------------- 1 tests failed. FAILED: basic-suite-master.test-scenarios.004_basic_sanity_pytest.test_ha_recovery Error Message: AssertionError: False != True after 600 seconds Stack Trace: prefix = <ovirtlago.prefix.OvirtPrefix object at 0x7f9b5f62b710> @order_by(_TEST_LIST) def test_ha_recovery(prefix): engine = prefix.virt_env.engine_vm().get_api_v4().system_service() with test_utils.TestEvent(engine, [119, 9602, 506]): # VM_DOWN_ERROR event(119) # HA_VM_FAILED event event(9602) # VDS_INITIATED_RUN_VM event(506) vm_host = _vm_host(prefix, VM2_NAME) pid = vm_host.ssh(['pgrep', '-f', 'qemu.*guest=vm2']) vm_host.ssh(['kill', '-KILL', pid.out.decode('utf-8')]) vm_service = test_utils.get_vm_service(engine, VM2_NAME) testlib.assert_true_within_long(
lambda:
vm_service.get().status == types.VmStatus.UP ) ../basic-suite-master/test-scenarios/004_basic_sanity_pytest.py:819: _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ /usr/lib/python2.7/site-packages/ovirtlago/testlib.py:286: in assert_true_within_long assert_equals_within_long(func, True, allowed_exceptions) /usr/lib/python2.7/site-packages/ovirtlago/testlib.py:273: in assert_equals_within_long func, value, LONG_TIMEOUT, allowed_exceptions=allowed_exceptions _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ func = <function <lambda> at 0x7f9b5f462aa0>, value = True, timeout = 600 allowed_exceptions = [], initial_wait = 0 def assert_equals_within( func, value, timeout, allowed_exceptions=None, initial_wait=10 ): allowed_exceptions = allowed_exceptions or [] with utils.EggTimer(timeout) as timer: while not timer.elapsed(): try: res = func() if res == value: return except Exception as exc: if _instance_of_any(exc, allowed_exceptions): time.sleep(3) continue LOGGER.exception("Unhandled exception in %s", func) raise if initial_wait == 0: time.sleep(3) else: time.sleep(initial_wait) initial_wait = 0 try: raise AssertionError(
'%s != %s after %s seconds' % (res, value, timeout)
) E AssertionError: False != True after 600 seconds /usr/lib/python2.7/site-packages/ovirtlago/testlib.py:252: AssertionError
Project: https://jenkins.ovirt.org/job/ovirt-system-tests_compat-4.3-suite-master/ Build: https://jenkins.ovirt.org/job/ovirt-system-tests_compat-4.3-suite-master/521... Build Number: 521 Build Status: Still Failing Triggered By: Started by timer ------------------------------------- Changes Since Last Success: ------------------------------------- Changes for Build #512 [Nir Soffer] setup: Do not hide real issue by reserving ports Changes for Build #513 [Nir Soffer] setup: Do not hide real issue by reserving ports Changes for Build #514 [Nir Soffer] setup: Do not hide real issue by reserving ports [Sandro Bonazzola] ovirt-guest-agent: drop fc30 [Ehud Yonasi] mock_configs: added rhel configs. Changes for Build #515 [Nir Soffer] setup: Do not hide real issue by reserving ports Changes for Build #516 [Michal Skrivanek] verify hotplug memory and test unplug [Tristan Cacqueray] zuul-jobs-gate: repparent to base-gate-job [Ehud Yonasi] stdci_runner: added the rhel labels selection. Changes for Build #517 [Galit Rosenthal] Remove the test configure_vdsm_sos_plugin not needed Changes for Build #518 [Marcin Sobczyk] basic: ansible: Port '003_00_metrics_bootstrap_pytest.py' to ansible Changes for Build #519 [Marcin Sobczyk] basic: ansible: Port '003_00_metrics_bootstrap_pytest.py' to ansible Changes for Build #520 [Marcin Sobczyk] basic: ansible: Port '003_00_metrics_bootstrap_pytest.py' to ansible Changes for Build #521 [Galit Rosenthal] Vagrant: Move vagrant to version 2.2.9 ----------------- Failed Tests: ----------------- 1 tests failed. FAILED: basic-suite-master.test-scenarios.004_basic_sanity_pytest.test_hotplug_memory Error Message: AttributeError: 'NoneType' object has no attribute 'group' Stack Trace: prefix = <ovirtlago.prefix.OvirtPrefix object at 0x7f237e32d710> hotplug_mem_amount = 268435456 @order_by(_TEST_LIST) def test_hotplug_memory(prefix, hotplug_mem_amount): api_v4 = prefix.virt_env.engine_vm().get_api_v4() engine = api_v4.system_service() vm_service = test_utils.get_vm_service(engine, VM0_NAME) new_memory = vm_service.get().memory + hotplug_mem_amount with test_utils.TestEvent(engine, 2039): # HOT_SET_MEMORY(2,039) vm_service.update( vm=types.Vm( memory=new_memory ) ) assert vm_service.get().memory == new_memory assert_vm0_is_alive(prefix)
assert _vm_libvirt_memory_amount(prefix, VM0_NAME) // KB == new_memory // MB
../basic-suite-master/test-scenarios/004_basic_sanity_pytest.py:1050: _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ prefix = <ovirtlago.prefix.OvirtPrefix object at 0x7f237e32d710> vm_name = 'vm0' def _vm_libvirt_memory_amount(prefix, vm_name): vm_host = _vm_host(prefix, vm_name) ret = vm_host.ssh(['virsh', '-r', 'dumpxml', vm_name]) assert ret.code == 0 match = re.search(r'<currentMemory unit=\'KiB\'>(?P<mem>[0-9]+)', ret.out.decode('utf-8'))
return int(match.group('mem'))
E AttributeError: 'NoneType' object has no attribute 'group' ../basic-suite-master/test-scenarios/004_basic_sanity_pytest.py:1032: AttributeError
Project: https://jenkins.ovirt.org/job/ovirt-system-tests_compat-4.3-suite-master/ Build: https://jenkins.ovirt.org/job/ovirt-system-tests_compat-4.3-suite-master/522... Build Number: 522 Build Status: Still Failing Triggered By: Started by timer ------------------------------------- Changes Since Last Success: ------------------------------------- Changes for Build #513 [Nir Soffer] setup: Do not hide real issue by reserving ports Changes for Build #514 [Nir Soffer] setup: Do not hide real issue by reserving ports [Sandro Bonazzola] ovirt-guest-agent: drop fc30 [Ehud Yonasi] mock_configs: added rhel configs. Changes for Build #515 [Nir Soffer] setup: Do not hide real issue by reserving ports Changes for Build #516 [Michal Skrivanek] verify hotplug memory and test unplug [Tristan Cacqueray] zuul-jobs-gate: repparent to base-gate-job [Ehud Yonasi] stdci_runner: added the rhel labels selection. Changes for Build #517 [Galit Rosenthal] Remove the test configure_vdsm_sos_plugin not needed Changes for Build #518 [Marcin Sobczyk] basic: ansible: Port '003_00_metrics_bootstrap_pytest.py' to ansible Changes for Build #519 [Marcin Sobczyk] basic: ansible: Port '003_00_metrics_bootstrap_pytest.py' to ansible Changes for Build #520 [Marcin Sobczyk] basic: ansible: Port '003_00_metrics_bootstrap_pytest.py' to ansible Changes for Build #521 [Galit Rosenthal] Vagrant: Move vagrant to version 2.2.9 Changes for Build #522 [Martin Necas] ansible: disable iscsi ----------------- Failed Tests: ----------------- 1 tests failed. FAILED: basic-suite-master.test-scenarios.002_bootstrap_pytest.test_verify_add_hosts Error Message: RuntimeError: 1 hosts failed installation: lago-compat-4-3-suite-master-host-0: install_failed Stack Trace: engine_api = <ovirtsdk4.Connection object at 0x7f2a63f45610> @order_by(_TEST_LIST) def test_verify_add_hosts(engine_api): hosts_service = engine_api.system_service().hosts_service() hosts_status = hosts_service.list(search='datacenter={}'.format(DC_NAME)) total_hosts = len(hosts_status) dump_hosts = _host_status_to_print(hosts_service, hosts_status) LOGGER.debug('Host status, verify_add_hosts:\n {}'.format(dump_hosts)) testlib.assert_true_within( lambda: _single_host_up(hosts_service, total_hosts),
timeout=constants.ADD_HOST_TIMEOUT
) ../basic-suite-master/test-scenarios/002_bootstrap_pytest.py:434: _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ /usr/lib/python2.7/site-packages/ovirtlago/testlib.py:278: in assert_true_within assert_equals_within(func, True, timeout, allowed_exceptions) /usr/lib/python2.7/site-packages/ovirtlago/testlib.py:234: in assert_equals_within res = func() ../basic-suite-master/test-scenarios/002_bootstrap_pytest.py:433: in <lambda> lambda: _single_host_up(hosts_service, total_hosts), ../basic-suite-master/test-scenarios/002_bootstrap_pytest.py:230: in _single_host_up _check_problematic_hosts(hosts_service) _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ hosts_service = <ovirtsdk4.services.HostsService object at 0x7f2a641531d0> def _check_problematic_hosts(hosts_service): problematic_hosts = hosts_service.list(search='datacenter={} AND status != installing and status != initializing and status != up'.format(DC_NAME)) if len(problematic_hosts): dump_hosts = '%s hosts failed installation:\n' % len(problematic_hosts) for host in problematic_hosts: host_service = hosts_service.host_service(host.id) dump_hosts += '%s: %s\n' % (host.name, host_service.get().status)
raise RuntimeError(dump_hosts)
E RuntimeError: 1 hosts failed installation: E lago-compat-4-3-suite-master-host-0: install_failed ../basic-suite-master/test-scenarios/002_bootstrap_pytest.py:239: RuntimeError
Project: https://jenkins.ovirt.org/job/ovirt-system-tests_compat-4.3-suite-master/ Build: https://jenkins.ovirt.org/job/ovirt-system-tests_compat-4.3-suite-master/523... Build Number: 523 Build Status: Still Failing Triggered By: Started by timer ------------------------------------- Changes Since Last Success: ------------------------------------- Changes for Build #514 [Nir Soffer] setup: Do not hide real issue by reserving ports [Sandro Bonazzola] ovirt-guest-agent: drop fc30 [Ehud Yonasi] mock_configs: added rhel configs. Changes for Build #515 [Nir Soffer] setup: Do not hide real issue by reserving ports Changes for Build #516 [Michal Skrivanek] verify hotplug memory and test unplug [Tristan Cacqueray] zuul-jobs-gate: repparent to base-gate-job [Ehud Yonasi] stdci_runner: added the rhel labels selection. Changes for Build #517 [Galit Rosenthal] Remove the test configure_vdsm_sos_plugin not needed Changes for Build #518 [Marcin Sobczyk] basic: ansible: Port '003_00_metrics_bootstrap_pytest.py' to ansible Changes for Build #519 [Marcin Sobczyk] basic: ansible: Port '003_00_metrics_bootstrap_pytest.py' to ansible Changes for Build #520 [Marcin Sobczyk] basic: ansible: Port '003_00_metrics_bootstrap_pytest.py' to ansible Changes for Build #521 [Galit Rosenthal] Vagrant: Move vagrant to version 2.2.9 Changes for Build #522 [Martin Necas] ansible: disable iscsi Changes for Build #523 [Martin Necas] ansible: disable iscsi ----------------- Failed Tests: ----------------- 1 tests failed. FAILED: basic-suite-master.test-scenarios.002_bootstrap_pytest.test_verify_add_hosts Error Message: RuntimeError: 1 hosts failed installation: lago-compat-4-3-suite-master-host-0: install_failed Stack Trace: engine_api = <ovirtsdk4.Connection object at 0x7fc46ec92610> @order_by(_TEST_LIST) def test_verify_add_hosts(engine_api): hosts_service = engine_api.system_service().hosts_service() hosts_status = hosts_service.list(search='datacenter={}'.format(DC_NAME)) total_hosts = len(hosts_status) dump_hosts = _host_status_to_print(hosts_service, hosts_status) LOGGER.debug('Host status, verify_add_hosts:\n {}'.format(dump_hosts)) testlib.assert_true_within( lambda: _single_host_up(hosts_service, total_hosts),
timeout=constants.ADD_HOST_TIMEOUT
) ../basic-suite-master/test-scenarios/002_bootstrap_pytest.py:434: _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ /usr/lib/python2.7/site-packages/ovirtlago/testlib.py:278: in assert_true_within assert_equals_within(func, True, timeout, allowed_exceptions) /usr/lib/python2.7/site-packages/ovirtlago/testlib.py:234: in assert_equals_within res = func() ../basic-suite-master/test-scenarios/002_bootstrap_pytest.py:433: in <lambda> lambda: _single_host_up(hosts_service, total_hosts), ../basic-suite-master/test-scenarios/002_bootstrap_pytest.py:230: in _single_host_up _check_problematic_hosts(hosts_service) _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ hosts_service = <ovirtsdk4.services.HostsService object at 0x7fc46eea01d0> def _check_problematic_hosts(hosts_service): problematic_hosts = hosts_service.list(search='datacenter={} AND status != installing and status != initializing and status != up'.format(DC_NAME)) if len(problematic_hosts): dump_hosts = '%s hosts failed installation:\n' % len(problematic_hosts) for host in problematic_hosts: host_service = hosts_service.host_service(host.id) dump_hosts += '%s: %s\n' % (host.name, host_service.get().status)
raise RuntimeError(dump_hosts)
E RuntimeError: 1 hosts failed installation: E lago-compat-4-3-suite-master-host-0: install_failed ../basic-suite-master/test-scenarios/002_bootstrap_pytest.py:239: RuntimeError
Project: https://jenkins.ovirt.org/job/ovirt-system-tests_compat-4.3-suite-master/ Build: https://jenkins.ovirt.org/job/ovirt-system-tests_compat-4.3-suite-master/524... Build Number: 524 Build Status: Still Failing Triggered By: Started by timer ------------------------------------- Changes Since Last Success: ------------------------------------- Changes for Build #515 [Nir Soffer] setup: Do not hide real issue by reserving ports Changes for Build #516 [Michal Skrivanek] verify hotplug memory and test unplug [Tristan Cacqueray] zuul-jobs-gate: repparent to base-gate-job [Ehud Yonasi] stdci_runner: added the rhel labels selection. Changes for Build #517 [Galit Rosenthal] Remove the test configure_vdsm_sos_plugin not needed Changes for Build #518 [Marcin Sobczyk] basic: ansible: Port '003_00_metrics_bootstrap_pytest.py' to ansible Changes for Build #519 [Marcin Sobczyk] basic: ansible: Port '003_00_metrics_bootstrap_pytest.py' to ansible Changes for Build #520 [Marcin Sobczyk] basic: ansible: Port '003_00_metrics_bootstrap_pytest.py' to ansible Changes for Build #521 [Galit Rosenthal] Vagrant: Move vagrant to version 2.2.9 Changes for Build #522 [Martin Necas] ansible: disable iscsi Changes for Build #523 [Martin Necas] ansible: disable iscsi Changes for Build #524 [Michal Skrivanek] smaller OST VMs ----------------- Failed Tests: ----------------- 1 tests failed. FAILED: basic-suite-master.test-scenarios.004_basic_sanity_pytest.test_ha_recovery Error Message: AssertionError: False != True after 600 seconds Stack Trace: prefix = <ovirtlago.prefix.OvirtPrefix object at 0x7f2ad0fca710> @order_by(_TEST_LIST) def test_ha_recovery(prefix): engine = prefix.virt_env.engine_vm().get_api_v4().system_service() with test_utils.TestEvent(engine, [119, 9602, 506]): # VM_DOWN_ERROR event(119) # HA_VM_FAILED event event(9602) # VDS_INITIATED_RUN_VM event(506) vm_host = _vm_host(prefix, VM2_NAME) pid = vm_host.ssh(['pgrep', '-f', 'qemu.*guest=vm2']) vm_host.ssh(['kill', '-KILL', pid.out.decode('utf-8')]) vm_service = test_utils.get_vm_service(engine, VM2_NAME) testlib.assert_true_within_long(
lambda:
vm_service.get().status == types.VmStatus.UP ) ../basic-suite-master/test-scenarios/004_basic_sanity_pytest.py:819: _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ /usr/lib/python2.7/site-packages/ovirtlago/testlib.py:286: in assert_true_within_long assert_equals_within_long(func, True, allowed_exceptions) /usr/lib/python2.7/site-packages/ovirtlago/testlib.py:273: in assert_equals_within_long func, value, LONG_TIMEOUT, allowed_exceptions=allowed_exceptions _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ func = <function <lambda> at 0x7f2ad0e32aa0>, value = True, timeout = 600 allowed_exceptions = [], initial_wait = 0 def assert_equals_within( func, value, timeout, allowed_exceptions=None, initial_wait=10 ): allowed_exceptions = allowed_exceptions or [] with utils.EggTimer(timeout) as timer: while not timer.elapsed(): try: res = func() if res == value: return except Exception as exc: if _instance_of_any(exc, allowed_exceptions): time.sleep(3) continue LOGGER.exception("Unhandled exception in %s", func) raise if initial_wait == 0: time.sleep(3) else: time.sleep(initial_wait) initial_wait = 0 try: raise AssertionError(
'%s != %s after %s seconds' % (res, value, timeout)
) E AssertionError: False != True after 600 seconds /usr/lib/python2.7/site-packages/ovirtlago/testlib.py:252: AssertionError
Project: https://jenkins.ovirt.org/job/ovirt-system-tests_compat-4.3-suite-master/ Build: https://jenkins.ovirt.org/job/ovirt-system-tests_compat-4.3-suite-master/525... Build Number: 525 Build Status: Still Failing Triggered By: Started by timer ------------------------------------- Changes Since Last Success: ------------------------------------- Changes for Build #516 [Michal Skrivanek] verify hotplug memory and test unplug [Tristan Cacqueray] zuul-jobs-gate: repparent to base-gate-job [Ehud Yonasi] stdci_runner: added the rhel labels selection. Changes for Build #517 [Galit Rosenthal] Remove the test configure_vdsm_sos_plugin not needed Changes for Build #518 [Marcin Sobczyk] basic: ansible: Port '003_00_metrics_bootstrap_pytest.py' to ansible Changes for Build #519 [Marcin Sobczyk] basic: ansible: Port '003_00_metrics_bootstrap_pytest.py' to ansible Changes for Build #520 [Marcin Sobczyk] basic: ansible: Port '003_00_metrics_bootstrap_pytest.py' to ansible Changes for Build #521 [Galit Rosenthal] Vagrant: Move vagrant to version 2.2.9 Changes for Build #522 [Martin Necas] ansible: disable iscsi Changes for Build #523 [Martin Necas] ansible: disable iscsi Changes for Build #524 [Michal Skrivanek] smaller OST VMs Changes for Build #525 [Michal Skrivanek] smaller OST VMs ----------------- Failed Tests: ----------------- 1 tests failed. FAILED: basic-suite-master.test-scenarios.004_basic_sanity_pytest.test_suspend_resume_vm0 Error Message: LagoSSHTimeoutException: Timed out (in 116 s) trying to ssh to 192.168.201.163 Stack Trace: prefix = <ovirtlago.prefix.OvirtPrefix object at 0x7fd8ac468710> @order_by(_TEST_LIST) def test_suspend_resume_vm0(prefix): vm_host = _vm_host(prefix, VM0_NAME) ret = vm_host.ssh(['tail', '-1', VDSM_LOG]) assert ret.code == EX_OK log_items = ret.out.decode('utf-8').split() global _log_time_before_suspend _log_time_before_suspend = log_items[0] + ' ' + log_items[1] # date + time
assert_vm0_is_alive(prefix)
../basic-suite-master/test-scenarios/004_basic_sanity_pytest.py:1210: _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ ../basic-suite-master/test-scenarios/004_basic_sanity_pytest.py:191: in assert_vm0_is_alive assert_vm_is_alive(prefix, VM0_NAME) ../basic-suite-master/test-scenarios/004_basic_sanity_pytest.py:199: in assert_vm_is_alive assert _vm_ssh(prefix, vm_hostname, ['true']).code == EX_OK ../basic-suite-master/test-scenarios/004_basic_sanity_pytest.py:186: in _vm_ssh tries=tries, /usr/lib/python2.7/site-packages/lago/ssh.py:46: in ssh password=password, _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ ip_addr = '192.168.201.163', ssh_key = None, host_name = '192.168.201.163' ssh_tries = 0, propagate_fail = True, username = 'cirros', password = 'gocubsgo' def get_ssh_client( ip_addr, ssh_key=None, host_name=None, ssh_tries=None, propagate_fail=True, username='root', password='123456', ): """ Get a connected SSH client Args: ip_addr(str): IP address of the endpoint ssh_key(str or list of str): Path to a file which contains the private key hotname(str): The hostname of the endpoint ssh_tries(int): The number of attempts to connect to the endpoint propagate_fail(bool): If set to true, this event will be in the log and fail the outer stage. Otherwise, it will be discarded. username(str): The username to authenticate with password(str): Used for password authentication or for private key decryption Raises: :exc:`~LagoSSHTimeoutException`: If the client failed to connect after "ssh_tries" """ host_name = host_name or ip_addr with LogTask( 'Get ssh client for %s' % host_name, level='debug', propagate_fail=propagate_fail, ): ssh_timeout = int(config.get('ssh_timeout')) if ssh_tries is None: ssh_tries = int(config.get('ssh_tries', 10)) start_time = time.time() client = paramiko.SSHClient() client.set_missing_host_key_policy(paramiko.AutoAddPolicy(), ) while ssh_tries > 0: try: client.connect( ip_addr, username=username, password=password, key_filename=ssh_key, timeout=ssh_timeout, ) break except (socket.error, socket.timeout) as err: LOGGER.debug( 'Socket error connecting to %s: %s', host_name, err, ) except paramiko.ssh_exception.SSHException as err: LOGGER.debug( 'SSH error connecting to %s: %s', host_name, err, ) except EOFError as err: LOGGER.debug('EOFError connecting to %s: %s', host_name, err) ssh_tries -= 1 LOGGER.debug( 'Still got %d tries for %s', ssh_tries, host_name, ) time.sleep(1) else: end_time = time.time() raise LagoSSHTimeoutException( 'Timed out (in %d s) trying to ssh to %s' %
(end_time - start_time, host_name)
) E LagoSSHTimeoutException: Timed out (in 116 s) trying to ssh to 192.168.201.163 /usr/lib/python2.7/site-packages/lago/ssh.py:388: LagoSSHTimeoutException
Project: https://jenkins.ovirt.org/job/ovirt-system-tests_compat-4.3-suite-master/ Build: https://jenkins.ovirt.org/job/ovirt-system-tests_compat-4.3-suite-master/526... Build Number: 526 Build Status: Still Failing Triggered By: Started by timer ------------------------------------- Changes Since Last Success: ------------------------------------- Changes for Build #517 [Galit Rosenthal] Remove the test configure_vdsm_sos_plugin not needed Changes for Build #518 [Marcin Sobczyk] basic: ansible: Port '003_00_metrics_bootstrap_pytest.py' to ansible Changes for Build #519 [Marcin Sobczyk] basic: ansible: Port '003_00_metrics_bootstrap_pytest.py' to ansible Changes for Build #520 [Marcin Sobczyk] basic: ansible: Port '003_00_metrics_bootstrap_pytest.py' to ansible Changes for Build #521 [Galit Rosenthal] Vagrant: Move vagrant to version 2.2.9 Changes for Build #522 [Martin Necas] ansible: disable iscsi Changes for Build #523 [Martin Necas] ansible: disable iscsi Changes for Build #524 [Michal Skrivanek] smaller OST VMs Changes for Build #525 [Michal Skrivanek] smaller OST VMs Changes for Build #526 [Michal Skrivanek] smaller OST VMs ----------------- Failed Tests: ----------------- 1 tests failed. FAILED: basic-suite-master.test-scenarios.004_basic_sanity_pytest.test_ha_recovery Error Message: AssertionError: False != True after 600 seconds Stack Trace: prefix = <ovirtlago.prefix.OvirtPrefix object at 0x7ff1823ef710> @order_by(_TEST_LIST) def test_ha_recovery(prefix): engine = prefix.virt_env.engine_vm().get_api_v4().system_service() with test_utils.TestEvent(engine, [119, 9602, 506]): # VM_DOWN_ERROR event(119) # HA_VM_FAILED event event(9602) # VDS_INITIATED_RUN_VM event(506) vm_host = _vm_host(prefix, VM2_NAME) pid = vm_host.ssh(['pgrep', '-f', 'qemu.*guest=vm2']) vm_host.ssh(['kill', '-KILL', pid.out.decode('utf-8')]) vm_service = test_utils.get_vm_service(engine, VM2_NAME) testlib.assert_true_within_long(
lambda:
vm_service.get().status == types.VmStatus.UP ) ../basic-suite-master/test-scenarios/004_basic_sanity_pytest.py:819: _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ /usr/lib/python2.7/site-packages/ovirtlago/testlib.py:286: in assert_true_within_long assert_equals_within_long(func, True, allowed_exceptions) /usr/lib/python2.7/site-packages/ovirtlago/testlib.py:273: in assert_equals_within_long func, value, LONG_TIMEOUT, allowed_exceptions=allowed_exceptions _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ func = <function <lambda> at 0x7ff182261aa0>, value = True, timeout = 600 allowed_exceptions = [], initial_wait = 0 def assert_equals_within( func, value, timeout, allowed_exceptions=None, initial_wait=10 ): allowed_exceptions = allowed_exceptions or [] with utils.EggTimer(timeout) as timer: while not timer.elapsed(): try: res = func() if res == value: return except Exception as exc: if _instance_of_any(exc, allowed_exceptions): time.sleep(3) continue LOGGER.exception("Unhandled exception in %s", func) raise if initial_wait == 0: time.sleep(3) else: time.sleep(initial_wait) initial_wait = 0 try: raise AssertionError(
'%s != %s after %s seconds' % (res, value, timeout)
) E AssertionError: False != True after 600 seconds /usr/lib/python2.7/site-packages/ovirtlago/testlib.py:252: AssertionError
Project: https://jenkins.ovirt.org/job/ovirt-system-tests_compat-4.3-suite-master/ Build: https://jenkins.ovirt.org/job/ovirt-system-tests_compat-4.3-suite-master/527... Build Number: 527 Build Status: Still Failing Triggered By: Started by timer ------------------------------------- Changes Since Last Success: ------------------------------------- Changes for Build #518 [Marcin Sobczyk] basic: ansible: Port '003_00_metrics_bootstrap_pytest.py' to ansible Changes for Build #519 [Marcin Sobczyk] basic: ansible: Port '003_00_metrics_bootstrap_pytest.py' to ansible Changes for Build #520 [Marcin Sobczyk] basic: ansible: Port '003_00_metrics_bootstrap_pytest.py' to ansible Changes for Build #521 [Galit Rosenthal] Vagrant: Move vagrant to version 2.2.9 Changes for Build #522 [Martin Necas] ansible: disable iscsi Changes for Build #523 [Martin Necas] ansible: disable iscsi Changes for Build #524 [Michal Skrivanek] smaller OST VMs Changes for Build #525 [Michal Skrivanek] smaller OST VMs Changes for Build #526 [Michal Skrivanek] smaller OST VMs Changes for Build #527 [Michal Skrivanek] smaller OST VMs ----------------- Failed Tests: ----------------- 1 tests failed. FAILED: basic-suite-master.test-scenarios.004_basic_sanity_pytest.test_suspend_resume_vm0 Error Message: LagoSSHTimeoutException: Timed out (in 117 s) trying to ssh to 192.168.201.226 Stack Trace: prefix = <ovirtlago.prefix.OvirtPrefix object at 0x7ff14b590710> @order_by(_TEST_LIST) def test_suspend_resume_vm0(prefix): vm_host = _vm_host(prefix, VM0_NAME) ret = vm_host.ssh(['tail', '-1', VDSM_LOG]) assert ret.code == EX_OK log_items = ret.out.decode('utf-8').split() global _log_time_before_suspend _log_time_before_suspend = log_items[0] + ' ' + log_items[1] # date + time
assert_vm0_is_alive(prefix)
../basic-suite-master/test-scenarios/004_basic_sanity_pytest.py:1210: _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ ../basic-suite-master/test-scenarios/004_basic_sanity_pytest.py:191: in assert_vm0_is_alive assert_vm_is_alive(prefix, VM0_NAME) ../basic-suite-master/test-scenarios/004_basic_sanity_pytest.py:199: in assert_vm_is_alive assert _vm_ssh(prefix, vm_hostname, ['true']).code == EX_OK ../basic-suite-master/test-scenarios/004_basic_sanity_pytest.py:186: in _vm_ssh tries=tries, /usr/lib/python2.7/site-packages/lago/ssh.py:46: in ssh password=password, _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ ip_addr = '192.168.201.226', ssh_key = None, host_name = '192.168.201.226' ssh_tries = 0, propagate_fail = True, username = 'cirros', password = 'gocubsgo' def get_ssh_client( ip_addr, ssh_key=None, host_name=None, ssh_tries=None, propagate_fail=True, username='root', password='123456', ): """ Get a connected SSH client Args: ip_addr(str): IP address of the endpoint ssh_key(str or list of str): Path to a file which contains the private key hotname(str): The hostname of the endpoint ssh_tries(int): The number of attempts to connect to the endpoint propagate_fail(bool): If set to true, this event will be in the log and fail the outer stage. Otherwise, it will be discarded. username(str): The username to authenticate with password(str): Used for password authentication or for private key decryption Raises: :exc:`~LagoSSHTimeoutException`: If the client failed to connect after "ssh_tries" """ host_name = host_name or ip_addr with LogTask( 'Get ssh client for %s' % host_name, level='debug', propagate_fail=propagate_fail, ): ssh_timeout = int(config.get('ssh_timeout')) if ssh_tries is None: ssh_tries = int(config.get('ssh_tries', 10)) start_time = time.time() client = paramiko.SSHClient() client.set_missing_host_key_policy(paramiko.AutoAddPolicy(), ) while ssh_tries > 0: try: client.connect( ip_addr, username=username, password=password, key_filename=ssh_key, timeout=ssh_timeout, ) break except (socket.error, socket.timeout) as err: LOGGER.debug( 'Socket error connecting to %s: %s', host_name, err, ) except paramiko.ssh_exception.SSHException as err: LOGGER.debug( 'SSH error connecting to %s: %s', host_name, err, ) except EOFError as err: LOGGER.debug('EOFError connecting to %s: %s', host_name, err) ssh_tries -= 1 LOGGER.debug( 'Still got %d tries for %s', ssh_tries, host_name, ) time.sleep(1) else: end_time = time.time() raise LagoSSHTimeoutException( 'Timed out (in %d s) trying to ssh to %s' %
(end_time - start_time, host_name)
) E LagoSSHTimeoutException: Timed out (in 117 s) trying to ssh to 192.168.201.226 /usr/lib/python2.7/site-packages/lago/ssh.py:388: LagoSSHTimeoutException
Project: https://jenkins.ovirt.org/job/ovirt-system-tests_compat-4.3-suite-master/ Build: https://jenkins.ovirt.org/job/ovirt-system-tests_compat-4.3-suite-master/528... Build Number: 528 Build Status: Still Failing Triggered By: Started by timer ------------------------------------- Changes Since Last Success: ------------------------------------- Changes for Build #519 [Marcin Sobczyk] basic: ansible: Port '003_00_metrics_bootstrap_pytest.py' to ansible Changes for Build #520 [Marcin Sobczyk] basic: ansible: Port '003_00_metrics_bootstrap_pytest.py' to ansible Changes for Build #521 [Galit Rosenthal] Vagrant: Move vagrant to version 2.2.9 Changes for Build #522 [Martin Necas] ansible: disable iscsi Changes for Build #523 [Martin Necas] ansible: disable iscsi Changes for Build #524 [Michal Skrivanek] smaller OST VMs Changes for Build #525 [Michal Skrivanek] smaller OST VMs Changes for Build #526 [Michal Skrivanek] smaller OST VMs Changes for Build #527 [Michal Skrivanek] smaller OST VMs Changes for Build #528 [Michal Skrivanek] smaller OST VMs [Barak Korren] Dropped FCRAW tests for STDCI [Barak Korren] usrc: expand test for 'files' provider ----------------- Failed Tests: ----------------- 1 tests failed. FAILED: basic-suite-master.test-scenarios.004_basic_sanity_pytest.test_ha_recovery Error Message: AssertionError: False != True after 600 seconds Stack Trace: prefix = <ovirtlago.prefix.OvirtPrefix object at 0x7f301fc18710> @order_by(_TEST_LIST) def test_ha_recovery(prefix): engine = prefix.virt_env.engine_vm().get_api_v4().system_service() with test_utils.TestEvent(engine, [119, 9602, 506]): # VM_DOWN_ERROR event(119) # HA_VM_FAILED event event(9602) # VDS_INITIATED_RUN_VM event(506) vm_host = _vm_host(prefix, VM2_NAME) pid = vm_host.ssh(['pgrep', '-f', 'qemu.*guest=vm2']) vm_host.ssh(['kill', '-KILL', pid.out.decode('utf-8')]) vm_service = test_utils.get_vm_service(engine, VM2_NAME) testlib.assert_true_within_long(
lambda:
vm_service.get().status == types.VmStatus.UP ) ../basic-suite-master/test-scenarios/004_basic_sanity_pytest.py:819: _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ /usr/lib/python2.7/site-packages/ovirtlago/testlib.py:286: in assert_true_within_long assert_equals_within_long(func, True, allowed_exceptions) /usr/lib/python2.7/site-packages/ovirtlago/testlib.py:273: in assert_equals_within_long func, value, LONG_TIMEOUT, allowed_exceptions=allowed_exceptions _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ func = <function <lambda> at 0x7f301fa8a0c8>, value = True, timeout = 600 allowed_exceptions = [], initial_wait = 0 def assert_equals_within( func, value, timeout, allowed_exceptions=None, initial_wait=10 ): allowed_exceptions = allowed_exceptions or [] with utils.EggTimer(timeout) as timer: while not timer.elapsed(): try: res = func() if res == value: return except Exception as exc: if _instance_of_any(exc, allowed_exceptions): time.sleep(3) continue LOGGER.exception("Unhandled exception in %s", func) raise if initial_wait == 0: time.sleep(3) else: time.sleep(initial_wait) initial_wait = 0 try: raise AssertionError(
'%s != %s after %s seconds' % (res, value, timeout)
) E AssertionError: False != True after 600 seconds /usr/lib/python2.7/site-packages/ovirtlago/testlib.py:252: AssertionError
Project: https://jenkins.ovirt.org/job/ovirt-system-tests_compat-4.3-suite-master/ Build: https://jenkins.ovirt.org/job/ovirt-system-tests_compat-4.3-suite-master/529... Build Number: 529 Build Status: Still Failing Triggered By: Started by timer ------------------------------------- Changes Since Last Success: ------------------------------------- Changes for Build #520 [Marcin Sobczyk] basic: ansible: Port '003_00_metrics_bootstrap_pytest.py' to ansible Changes for Build #521 [Galit Rosenthal] Vagrant: Move vagrant to version 2.2.9 Changes for Build #522 [Martin Necas] ansible: disable iscsi Changes for Build #523 [Martin Necas] ansible: disable iscsi Changes for Build #524 [Michal Skrivanek] smaller OST VMs Changes for Build #525 [Michal Skrivanek] smaller OST VMs Changes for Build #526 [Michal Skrivanek] smaller OST VMs Changes for Build #527 [Michal Skrivanek] smaller OST VMs Changes for Build #528 [Michal Skrivanek] smaller OST VMs [Barak Korren] Dropped FCRAW tests for STDCI [Barak Korren] usrc: expand test for 'files' provider Changes for Build #529 [Michal Skrivanek] smaller OST VMs [Barak Korren] usrc: Filtering for the `files` provider ----------------- Failed Tests: ----------------- 1 tests failed. FAILED: basic-suite-master.test-scenarios.004_basic_sanity_pytest.test_suspend_resume_vm0 Error Message: LagoSSHTimeoutException: Timed out (in 113 s) trying to ssh to 192.168.201.221 Stack Trace: prefix = <ovirtlago.prefix.OvirtPrefix object at 0x7fa139bbe710> @order_by(_TEST_LIST) def test_suspend_resume_vm0(prefix): vm_host = _vm_host(prefix, VM0_NAME) ret = vm_host.ssh(['tail', '-1', VDSM_LOG]) assert ret.code == EX_OK log_items = ret.out.decode('utf-8').split() global _log_time_before_suspend _log_time_before_suspend = log_items[0] + ' ' + log_items[1] # date + time
assert_vm0_is_alive(prefix)
../basic-suite-master/test-scenarios/004_basic_sanity_pytest.py:1210: _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ ../basic-suite-master/test-scenarios/004_basic_sanity_pytest.py:191: in assert_vm0_is_alive assert_vm_is_alive(prefix, VM0_NAME) ../basic-suite-master/test-scenarios/004_basic_sanity_pytest.py:199: in assert_vm_is_alive assert _vm_ssh(prefix, vm_hostname, ['true']).code == EX_OK ../basic-suite-master/test-scenarios/004_basic_sanity_pytest.py:186: in _vm_ssh tries=tries, /usr/lib/python2.7/site-packages/lago/ssh.py:46: in ssh password=password, _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ ip_addr = '192.168.201.221', ssh_key = None, host_name = '192.168.201.221' ssh_tries = 0, propagate_fail = True, username = 'cirros', password = 'gocubsgo' def get_ssh_client( ip_addr, ssh_key=None, host_name=None, ssh_tries=None, propagate_fail=True, username='root', password='123456', ): """ Get a connected SSH client Args: ip_addr(str): IP address of the endpoint ssh_key(str or list of str): Path to a file which contains the private key hotname(str): The hostname of the endpoint ssh_tries(int): The number of attempts to connect to the endpoint propagate_fail(bool): If set to true, this event will be in the log and fail the outer stage. Otherwise, it will be discarded. username(str): The username to authenticate with password(str): Used for password authentication or for private key decryption Raises: :exc:`~LagoSSHTimeoutException`: If the client failed to connect after "ssh_tries" """ host_name = host_name or ip_addr with LogTask( 'Get ssh client for %s' % host_name, level='debug', propagate_fail=propagate_fail, ): ssh_timeout = int(config.get('ssh_timeout')) if ssh_tries is None: ssh_tries = int(config.get('ssh_tries', 10)) start_time = time.time() client = paramiko.SSHClient() client.set_missing_host_key_policy(paramiko.AutoAddPolicy(), ) while ssh_tries > 0: try: client.connect( ip_addr, username=username, password=password, key_filename=ssh_key, timeout=ssh_timeout, ) break except (socket.error, socket.timeout) as err: LOGGER.debug( 'Socket error connecting to %s: %s', host_name, err, ) except paramiko.ssh_exception.SSHException as err: LOGGER.debug( 'SSH error connecting to %s: %s', host_name, err, ) except EOFError as err: LOGGER.debug('EOFError connecting to %s: %s', host_name, err) ssh_tries -= 1 LOGGER.debug( 'Still got %d tries for %s', ssh_tries, host_name, ) time.sleep(1) else: end_time = time.time() raise LagoSSHTimeoutException( 'Timed out (in %d s) trying to ssh to %s' %
(end_time - start_time, host_name)
) E LagoSSHTimeoutException: Timed out (in 113 s) trying to ssh to 192.168.201.221 /usr/lib/python2.7/site-packages/lago/ssh.py:388: LagoSSHTimeoutException
Project: https://jenkins.ovirt.org/job/ovirt-system-tests_compat-4.3-suite-master/ Build: https://jenkins.ovirt.org/job/ovirt-system-tests_compat-4.3-suite-master/530... Build Number: 530 Build Status: Still Failing Triggered By: Started by timer ------------------------------------- Changes Since Last Success: ------------------------------------- Changes for Build #520 [Marcin Sobczyk] basic: ansible: Port '003_00_metrics_bootstrap_pytest.py' to ansible Changes for Build #521 [Galit Rosenthal] Vagrant: Move vagrant to version 2.2.9 Changes for Build #522 [Martin Necas] ansible: disable iscsi Changes for Build #523 [Martin Necas] ansible: disable iscsi Changes for Build #524 [Michal Skrivanek] smaller OST VMs Changes for Build #525 [Michal Skrivanek] smaller OST VMs Changes for Build #526 [Michal Skrivanek] smaller OST VMs Changes for Build #527 [Michal Skrivanek] smaller OST VMs Changes for Build #528 [Michal Skrivanek] smaller OST VMs [Barak Korren] Dropped FCRAW tests for STDCI [Barak Korren] usrc: expand test for 'files' provider Changes for Build #529 [Michal Skrivanek] smaller OST VMs [Barak Korren] usrc: Filtering for the `files` provider Changes for Build #530 [Marcin Sobczyk] ost_utils: ansible: Sort events by creation date ----------------- Failed Tests: ----------------- No tests ran.
Project: https://jenkins.ovirt.org/job/ovirt-system-tests_compat-4.3-suite-master/ Build: https://jenkins.ovirt.org/job/ovirt-system-tests_compat-4.3-suite-master/531... Build Number: 531 Build Status: Still Failing Triggered By: Started by timer ------------------------------------- Changes Since Last Success: ------------------------------------- Changes for Build #522 [Martin Necas] ansible: disable iscsi Changes for Build #523 [Martin Necas] ansible: disable iscsi Changes for Build #524 [Michal Skrivanek] smaller OST VMs Changes for Build #525 [Michal Skrivanek] smaller OST VMs Changes for Build #526 [Michal Skrivanek] smaller OST VMs Changes for Build #527 [Michal Skrivanek] smaller OST VMs Changes for Build #528 [Michal Skrivanek] smaller OST VMs [Barak Korren] Dropped FCRAW tests for STDCI [Barak Korren] usrc: expand test for 'files' provider Changes for Build #529 [Michal Skrivanek] smaller OST VMs [Barak Korren] usrc: Filtering for the `files` provider Changes for Build #530 [Marcin Sobczyk] ost_utils: ansible: Sort events by creation date Changes for Build #531 [Michal Skrivanek] avoid ballooning crash in vm0 after memory hotplug ----------------- Failed Tests: ----------------- 1 tests failed. FAILED: basic-suite-master.test-scenarios.002_bootstrap_pytest.test_verify_engine_backup Error Message: AssertionError: False != True after 600 seconds Stack Trace: ansible_engine = <ost_utils.ansible._AnsibleModuleMapper object at 0x7fe2b4d331d0> engine_api = <ovirtsdk4.Connection object at 0x7fe2b8699a50> @order_by(_TEST_LIST) def test_verify_engine_backup(ansible_engine, engine_api): ansible_engine.file( path='/var/log/ost-engine-backup', state='directory', mode='0755' ) engine = engine_api.system_service() with test_utils.TestEvent(engine, [9024, 9025]): #backup started event, completed ansible_engine.shell(
'engine-backup '
'--mode=backup ' '--file=/var/log/ost-engine-backup/backup.tgz ' '--log=/var/log/ost-engine-backup/log.txt' ) ../basic-suite-master/test-scenarios/002_bootstrap_pytest.py:1341: _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ /usr/lib64/python2.7/contextlib.py:35: in __exit__ self.gen.throw(type, value, traceback) ../compat-4.3-suite-master/test_utils/__init__.py:252: in TestEvent lambda: /usr/lib/python2.7/site-packages/ovirtlago/testlib.py:286: in assert_true_within_long assert_equals_within_long(func, True, allowed_exceptions) /usr/lib/python2.7/site-packages/ovirtlago/testlib.py:273: in assert_equals_within_long func, value, LONG_TIMEOUT, allowed_exceptions=allowed_exceptions _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ func = <function <lambda> at 0x7fe2b4cfc938>, value = True, timeout = 600 allowed_exceptions = [], initial_wait = 0 def assert_equals_within( func, value, timeout, allowed_exceptions=None, initial_wait=10 ): allowed_exceptions = allowed_exceptions or [] with utils.EggTimer(timeout) as timer: while not timer.elapsed(): try: res = func() if res == value: return except Exception as exc: if _instance_of_any(exc, allowed_exceptions): time.sleep(3) continue LOGGER.exception("Unhandled exception in %s", func) raise if initial_wait == 0: time.sleep(3) else: time.sleep(initial_wait) initial_wait = 0 try: raise AssertionError(
'%s != %s after %s seconds' % (res, value, timeout)
) E AssertionError: False != True after 600 seconds /usr/lib/python2.7/site-packages/ovirtlago/testlib.py:252: AssertionError
Project: https://jenkins.ovirt.org/job/ovirt-system-tests_compat-4.3-suite-master/ Build: https://jenkins.ovirt.org/job/ovirt-system-tests_compat-4.3-suite-master/532... Build Number: 532 Build Status: Still Failing Triggered By: Started by timer ------------------------------------- Changes Since Last Success: ------------------------------------- Changes for Build #523 [Martin Necas] ansible: disable iscsi Changes for Build #524 [Michal Skrivanek] smaller OST VMs Changes for Build #525 [Michal Skrivanek] smaller OST VMs Changes for Build #526 [Michal Skrivanek] smaller OST VMs Changes for Build #527 [Michal Skrivanek] smaller OST VMs Changes for Build #528 [Michal Skrivanek] smaller OST VMs [Barak Korren] Dropped FCRAW tests for STDCI [Barak Korren] usrc: expand test for 'files' provider Changes for Build #529 [Michal Skrivanek] smaller OST VMs [Barak Korren] usrc: Filtering for the `files` provider Changes for Build #530 [Marcin Sobczyk] ost_utils: ansible: Sort events by creation date Changes for Build #531 [Michal Skrivanek] avoid ballooning crash in vm0 after memory hotplug Changes for Build #532 [Michal Skrivanek] avoid ballooning crash in vm0 after memory hotplug ----------------- Failed Tests: ----------------- 1 tests failed. FAILED: basic-suite-master.test-scenarios.004_basic_sanity_pytest.test_run_vms Error Message: AssertionError: False != True after 600 seconds Stack Trace: prefix = <ovirtlago.prefix.OvirtPrefix object at 0x7f35f13b32d0> @order_by(_TEST_LIST) def test_run_vms(prefix): engine = prefix.virt_env.engine_vm().get_api_v4().system_service() vm_params = types.Vm( initialization=types.Initialization( user_name=VM_USER_NAME, root_password=VM_PASSWORD ) ) vm_params.initialization.host_name = BACKUP_VM_NAME backup_vm_service = test_utils.get_vm_service(engine, BACKUP_VM_NAME) backup_vm_service.start(use_cloud_init=True, vm=vm_params) vm_params.initialization.host_name = VM2_NAME vm2_service = test_utils.get_vm_service(engine, VM2_NAME) vm2_service.start(use_cloud_init=True, vm=vm_params) gw_ip = test_utils.get_management_net(prefix).gw() # CirrOS cloud-init is different, networking doesn't work since it doesn't support the format oVirt is using vm_params.initialization.host_name = VM0_NAME # hostname seems to work, the others not vm_params.initialization.dns_search = 'lago.local' vm_params.initialization.domain = 'lago.local' vm_params.initialization.dns_servers = gw_ip vm0_service = test_utils.get_vm_service(engine, VM0_NAME) vm0_service.start(use_cloud_init=True, vm=vm_params) for vm_name in [VM0_NAME, BACKUP_VM_NAME]:
_verify_vm_state(engine, vm_name, types.VmStatus.UP)
../basic-suite-master/test-scenarios/004_basic_sanity_pytest.py:752: _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ ../basic-suite-master/test-scenarios/004_basic_sanity_pytest.py:170: in _verify_vm_state lambda: /usr/lib/python2.7/site-packages/ovirtlago/testlib.py:286: in assert_true_within_long assert_equals_within_long(func, True, allowed_exceptions) /usr/lib/python2.7/site-packages/ovirtlago/testlib.py:273: in assert_equals_within_long func, value, LONG_TIMEOUT, allowed_exceptions=allowed_exceptions _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ func = <function <lambda> at 0x7f35f1246e60>, value = True, timeout = 600 allowed_exceptions = [], initial_wait = 0 def assert_equals_within( func, value, timeout, allowed_exceptions=None, initial_wait=10 ): allowed_exceptions = allowed_exceptions or [] with utils.EggTimer(timeout) as timer: while not timer.elapsed(): try: res = func() if res == value: return except Exception as exc: if _instance_of_any(exc, allowed_exceptions): time.sleep(3) continue LOGGER.exception("Unhandled exception in %s", func) raise if initial_wait == 0: time.sleep(3) else: time.sleep(initial_wait) initial_wait = 0 try: raise AssertionError(
'%s != %s after %s seconds' % (res, value, timeout)
) E AssertionError: False != True after 600 seconds /usr/lib/python2.7/site-packages/ovirtlago/testlib.py:252: AssertionError
Project: https://jenkins.ovirt.org/job/ovirt-system-tests_compat-4.3-suite-master/ Build: https://jenkins.ovirt.org/job/ovirt-system-tests_compat-4.3-suite-master/533... Build Number: 533 Build Status: Still Failing Triggered By: Started by timer ------------------------------------- Changes Since Last Success: ------------------------------------- Changes for Build #524 [Michal Skrivanek] smaller OST VMs Changes for Build #525 [Michal Skrivanek] smaller OST VMs Changes for Build #526 [Michal Skrivanek] smaller OST VMs Changes for Build #527 [Michal Skrivanek] smaller OST VMs Changes for Build #528 [Michal Skrivanek] smaller OST VMs [Barak Korren] Dropped FCRAW tests for STDCI [Barak Korren] usrc: expand test for 'files' provider Changes for Build #529 [Michal Skrivanek] smaller OST VMs [Barak Korren] usrc: Filtering for the `files` provider Changes for Build #530 [Marcin Sobczyk] ost_utils: ansible: Sort events by creation date Changes for Build #531 [Michal Skrivanek] avoid ballooning crash in vm0 after memory hotplug Changes for Build #532 [Michal Skrivanek] avoid ballooning crash in vm0 after memory hotplug Changes for Build #533 [Michal Skrivanek] avoid ballooning crash in vm0 after memory hotplug ----------------- Failed Tests: ----------------- 1 tests failed. FAILED: basic-suite-master.test-scenarios.004_basic_sanity_pytest.test_run_vms Error Message: AssertionError: False != True after 600 seconds Stack Trace: prefix = <ovirtlago.prefix.OvirtPrefix object at 0x7f9926a1f2d0> @order_by(_TEST_LIST) def test_run_vms(prefix): engine = prefix.virt_env.engine_vm().get_api_v4().system_service() vm_params = types.Vm( initialization=types.Initialization( user_name=VM_USER_NAME, root_password=VM_PASSWORD ) ) vm_params.initialization.host_name = BACKUP_VM_NAME backup_vm_service = test_utils.get_vm_service(engine, BACKUP_VM_NAME) backup_vm_service.start(use_cloud_init=True, vm=vm_params) vm_params.initialization.host_name = VM2_NAME vm2_service = test_utils.get_vm_service(engine, VM2_NAME) vm2_service.start(use_cloud_init=True, vm=vm_params) gw_ip = test_utils.get_management_net(prefix).gw() # CirrOS cloud-init is different, networking doesn't work since it doesn't support the format oVirt is using vm_params.initialization.host_name = VM0_NAME # hostname seems to work, the others not vm_params.initialization.dns_search = 'lago.local' vm_params.initialization.domain = 'lago.local' vm_params.initialization.dns_servers = gw_ip vm0_service = test_utils.get_vm_service(engine, VM0_NAME) vm0_service.start(use_cloud_init=True, vm=vm_params) for vm_name in [VM0_NAME, BACKUP_VM_NAME]:
_verify_vm_state(engine, vm_name, types.VmStatus.UP)
../basic-suite-master/test-scenarios/004_basic_sanity_pytest.py:752: _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ ../basic-suite-master/test-scenarios/004_basic_sanity_pytest.py:170: in _verify_vm_state lambda: /usr/lib/python2.7/site-packages/ovirtlago/testlib.py:286: in assert_true_within_long assert_equals_within_long(func, True, allowed_exceptions) /usr/lib/python2.7/site-packages/ovirtlago/testlib.py:273: in assert_equals_within_long func, value, LONG_TIMEOUT, allowed_exceptions=allowed_exceptions _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ func = <function <lambda> at 0x7f99268b2e60>, value = True, timeout = 600 allowed_exceptions = [], initial_wait = 0 def assert_equals_within( func, value, timeout, allowed_exceptions=None, initial_wait=10 ): allowed_exceptions = allowed_exceptions or [] with utils.EggTimer(timeout) as timer: while not timer.elapsed(): try: res = func() if res == value: return except Exception as exc: if _instance_of_any(exc, allowed_exceptions): time.sleep(3) continue LOGGER.exception("Unhandled exception in %s", func) raise if initial_wait == 0: time.sleep(3) else: time.sleep(initial_wait) initial_wait = 0 try: raise AssertionError(
'%s != %s after %s seconds' % (res, value, timeout)
) E AssertionError: False != True after 600 seconds /usr/lib/python2.7/site-packages/ovirtlago/testlib.py:252: AssertionError
Project: https://jenkins.ovirt.org/job/ovirt-system-tests_compat-4.3-suite-master/ Build: https://jenkins.ovirt.org/job/ovirt-system-tests_compat-4.3-suite-master/534... Build Number: 534 Build Status: Still Failing Triggered By: Started by timer ------------------------------------- Changes Since Last Success: ------------------------------------- Changes for Build #525 [Michal Skrivanek] smaller OST VMs Changes for Build #526 [Michal Skrivanek] smaller OST VMs Changes for Build #527 [Michal Skrivanek] smaller OST VMs Changes for Build #528 [Michal Skrivanek] smaller OST VMs [Barak Korren] Dropped FCRAW tests for STDCI [Barak Korren] usrc: expand test for 'files' provider Changes for Build #529 [Michal Skrivanek] smaller OST VMs [Barak Korren] usrc: Filtering for the `files` provider Changes for Build #530 [Marcin Sobczyk] ost_utils: ansible: Sort events by creation date Changes for Build #531 [Michal Skrivanek] avoid ballooning crash in vm0 after memory hotplug Changes for Build #532 [Michal Skrivanek] avoid ballooning crash in vm0 after memory hotplug Changes for Build #533 [Michal Skrivanek] avoid ballooning crash in vm0 after memory hotplug Changes for Build #534 [Michal Skrivanek] avoid ballooning crash in vm0 after memory hotplug ----------------- Failed Tests: ----------------- 1 tests failed. FAILED: basic-suite-master.test-scenarios.004_basic_sanity_pytest.test_run_vms Error Message: AssertionError: False != True after 600 seconds Stack Trace: prefix = <ovirtlago.prefix.OvirtPrefix object at 0x7f11630b52d0> @order_by(_TEST_LIST) def test_run_vms(prefix): engine = prefix.virt_env.engine_vm().get_api_v4().system_service() vm_params = types.Vm( initialization=types.Initialization( user_name=VM_USER_NAME, root_password=VM_PASSWORD ) ) vm_params.initialization.host_name = BACKUP_VM_NAME backup_vm_service = test_utils.get_vm_service(engine, BACKUP_VM_NAME) backup_vm_service.start(use_cloud_init=True, vm=vm_params) vm_params.initialization.host_name = VM2_NAME vm2_service = test_utils.get_vm_service(engine, VM2_NAME) vm2_service.start(use_cloud_init=True, vm=vm_params) gw_ip = test_utils.get_management_net(prefix).gw() # CirrOS cloud-init is different, networking doesn't work since it doesn't support the format oVirt is using vm_params.initialization.host_name = VM0_NAME # hostname seems to work, the others not vm_params.initialization.dns_search = 'lago.local' vm_params.initialization.domain = 'lago.local' vm_params.initialization.dns_servers = gw_ip vm0_service = test_utils.get_vm_service(engine, VM0_NAME) vm0_service.start(use_cloud_init=True, vm=vm_params) for vm_name in [VM0_NAME, BACKUP_VM_NAME]:
_verify_vm_state(engine, vm_name, types.VmStatus.UP)
../basic-suite-master/test-scenarios/004_basic_sanity_pytest.py:752: _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ ../basic-suite-master/test-scenarios/004_basic_sanity_pytest.py:170: in _verify_vm_state lambda: /usr/lib/python2.7/site-packages/ovirtlago/testlib.py:286: in assert_true_within_long assert_equals_within_long(func, True, allowed_exceptions) /usr/lib/python2.7/site-packages/ovirtlago/testlib.py:273: in assert_equals_within_long func, value, LONG_TIMEOUT, allowed_exceptions=allowed_exceptions _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ func = <function <lambda> at 0x7f1162f78e60>, value = True, timeout = 600 allowed_exceptions = [], initial_wait = 0 def assert_equals_within( func, value, timeout, allowed_exceptions=None, initial_wait=10 ): allowed_exceptions = allowed_exceptions or [] with utils.EggTimer(timeout) as timer: while not timer.elapsed(): try: res = func() if res == value: return except Exception as exc: if _instance_of_any(exc, allowed_exceptions): time.sleep(3) continue LOGGER.exception("Unhandled exception in %s", func) raise if initial_wait == 0: time.sleep(3) else: time.sleep(initial_wait) initial_wait = 0 try: raise AssertionError(
'%s != %s after %s seconds' % (res, value, timeout)
) E AssertionError: False != True after 600 seconds /usr/lib/python2.7/site-packages/ovirtlago/testlib.py:252: AssertionError
Project: https://jenkins.ovirt.org/job/ovirt-system-tests_compat-4.3-suite-master/ Build: https://jenkins.ovirt.org/job/ovirt-system-tests_compat-4.3-suite-master/535... Build Number: 535 Build Status: Still Failing Triggered By: Started by timer ------------------------------------- Changes Since Last Success: ------------------------------------- Changes for Build #526 [Michal Skrivanek] smaller OST VMs Changes for Build #527 [Michal Skrivanek] smaller OST VMs Changes for Build #528 [Michal Skrivanek] smaller OST VMs [Barak Korren] Dropped FCRAW tests for STDCI [Barak Korren] usrc: expand test for 'files' provider Changes for Build #529 [Michal Skrivanek] smaller OST VMs [Barak Korren] usrc: Filtering for the `files` provider Changes for Build #530 [Marcin Sobczyk] ost_utils: ansible: Sort events by creation date Changes for Build #531 [Michal Skrivanek] avoid ballooning crash in vm0 after memory hotplug Changes for Build #532 [Michal Skrivanek] avoid ballooning crash in vm0 after memory hotplug Changes for Build #533 [Michal Skrivanek] avoid ballooning crash in vm0 after memory hotplug Changes for Build #534 [Michal Skrivanek] avoid ballooning crash in vm0 after memory hotplug Changes for Build #535 [Michal Skrivanek] fix using ost-images without lagofy ----------------- Failed Tests: ----------------- 1 tests failed. FAILED: basic-suite-master.test-scenarios.004_basic_sanity_pytest.test_ha_recovery Error Message: AssertionError: False != True after 600 seconds Stack Trace: prefix = <ovirtlago.prefix.OvirtPrefix object at 0x7fb6268b2750> @order_by(_TEST_LIST) def test_ha_recovery(prefix): engine = prefix.virt_env.engine_vm().get_api_v4().system_service() with test_utils.TestEvent(engine, [119, 9602, 506]): # VM_DOWN_ERROR event(119) # HA_VM_FAILED event event(9602) # VDS_INITIATED_RUN_VM event(506) vm_host = _vm_host(prefix, VM2_NAME) pid = vm_host.ssh(['pgrep', '-f', 'qemu.*guest=vm2']) vm_host.ssh(['kill', '-KILL', pid.out.decode('utf-8')]) vm_service = test_utils.get_vm_service(engine, VM2_NAME) testlib.assert_true_within_long(
lambda:
vm_service.get().status == types.VmStatus.UP ) ../basic-suite-master/test-scenarios/004_basic_sanity_pytest.py:819: _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ /usr/lib/python2.7/site-packages/ovirtlago/testlib.py:286: in assert_true_within_long assert_equals_within_long(func, True, allowed_exceptions) /usr/lib/python2.7/site-packages/ovirtlago/testlib.py:273: in assert_equals_within_long func, value, LONG_TIMEOUT, allowed_exceptions=allowed_exceptions _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ func = <function <lambda> at 0x7fb6240559b0>, value = True, timeout = 600 allowed_exceptions = [], initial_wait = 0 def assert_equals_within( func, value, timeout, allowed_exceptions=None, initial_wait=10 ): allowed_exceptions = allowed_exceptions or [] with utils.EggTimer(timeout) as timer: while not timer.elapsed(): try: res = func() if res == value: return except Exception as exc: if _instance_of_any(exc, allowed_exceptions): time.sleep(3) continue LOGGER.exception("Unhandled exception in %s", func) raise if initial_wait == 0: time.sleep(3) else: time.sleep(initial_wait) initial_wait = 0 try: raise AssertionError(
'%s != %s after %s seconds' % (res, value, timeout)
) E AssertionError: False != True after 600 seconds /usr/lib/python2.7/site-packages/ovirtlago/testlib.py:252: AssertionError
Project: https://jenkins.ovirt.org/job/ovirt-system-tests_compat-4.3-suite-master/ Build: https://jenkins.ovirt.org/job/ovirt-system-tests_compat-4.3-suite-master/536... Build Number: 536 Build Status: Still Failing Triggered By: Started by timer ------------------------------------- Changes Since Last Success: ------------------------------------- Changes for Build #527 [Michal Skrivanek] smaller OST VMs Changes for Build #528 [Michal Skrivanek] smaller OST VMs [Barak Korren] Dropped FCRAW tests for STDCI [Barak Korren] usrc: expand test for 'files' provider Changes for Build #529 [Michal Skrivanek] smaller OST VMs [Barak Korren] usrc: Filtering for the `files` provider Changes for Build #530 [Marcin Sobczyk] ost_utils: ansible: Sort events by creation date Changes for Build #531 [Michal Skrivanek] avoid ballooning crash in vm0 after memory hotplug Changes for Build #532 [Michal Skrivanek] avoid ballooning crash in vm0 after memory hotplug Changes for Build #533 [Michal Skrivanek] avoid ballooning crash in vm0 after memory hotplug Changes for Build #534 [Michal Skrivanek] avoid ballooning crash in vm0 after memory hotplug Changes for Build #535 [Michal Skrivanek] fix using ost-images without lagofy Changes for Build #536 [Michal Skrivanek] fix using ost-images without lagofy ----------------- Failed Tests: ----------------- 1 tests failed. FAILED: basic-suite-master.test-scenarios.004_basic_sanity_pytest.test_sparsify_disk1 Error Message: AssertionError: False != True after 600 seconds Stack Trace: api_v4 = <ovirtsdk4.Connection object at 0x7fd8fe9bb690> @order_by(_TEST_LIST) def test_sparsify_disk1(api_v4): engine = api_v4.system_service() disk_service = test_utils.get_disk_service(engine, DISK1_NAME) with test_utils.TestEvent(engine, 1325): # USER_SPARSIFY_IMAGE_START event disk_service.sparsify() with test_utils.TestEvent(engine, 1326): # USER_SPARSIFY_IMAGE_FINISH_SUCCESS
pass
../basic-suite-master/test-scenarios/004_basic_sanity_pytest.py:295: _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ /usr/lib64/python2.7/contextlib.py:24: in __exit__ self.gen.next() ../compat-4.3-suite-master/test_utils/__init__.py:252: in TestEvent lambda: /usr/lib/python2.7/site-packages/ovirtlago/testlib.py:286: in assert_true_within_long assert_equals_within_long(func, True, allowed_exceptions) /usr/lib/python2.7/site-packages/ovirtlago/testlib.py:273: in assert_equals_within_long func, value, LONG_TIMEOUT, allowed_exceptions=allowed_exceptions _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ func = <function <lambda> at 0x7fd8f729a320>, value = True, timeout = 600 allowed_exceptions = [], initial_wait = 0 def assert_equals_within( func, value, timeout, allowed_exceptions=None, initial_wait=10 ): allowed_exceptions = allowed_exceptions or [] with utils.EggTimer(timeout) as timer: while not timer.elapsed(): try: res = func() if res == value: return except Exception as exc: if _instance_of_any(exc, allowed_exceptions): time.sleep(3) continue LOGGER.exception("Unhandled exception in %s", func) raise if initial_wait == 0: time.sleep(3) else: time.sleep(initial_wait) initial_wait = 0 try: raise AssertionError(
'%s != %s after %s seconds' % (res, value, timeout)
) E AssertionError: False != True after 600 seconds /usr/lib/python2.7/site-packages/ovirtlago/testlib.py:252: AssertionError
Project: https://jenkins.ovirt.org/job/ovirt-system-tests_compat-4.3-suite-master/ Build: https://jenkins.ovirt.org/job/ovirt-system-tests_compat-4.3-suite-master/537... Build Number: 537 Build Status: Still Failing Triggered By: Started by timer ------------------------------------- Changes Since Last Success: ------------------------------------- Changes for Build #528 [Michal Skrivanek] smaller OST VMs [Barak Korren] Dropped FCRAW tests for STDCI [Barak Korren] usrc: expand test for 'files' provider Changes for Build #529 [Michal Skrivanek] smaller OST VMs [Barak Korren] usrc: Filtering for the `files` provider Changes for Build #530 [Marcin Sobczyk] ost_utils: ansible: Sort events by creation date Changes for Build #531 [Michal Skrivanek] avoid ballooning crash in vm0 after memory hotplug Changes for Build #532 [Michal Skrivanek] avoid ballooning crash in vm0 after memory hotplug Changes for Build #533 [Michal Skrivanek] avoid ballooning crash in vm0 after memory hotplug Changes for Build #534 [Michal Skrivanek] avoid ballooning crash in vm0 after memory hotplug Changes for Build #535 [Michal Skrivanek] fix using ost-images without lagofy Changes for Build #536 [Michal Skrivanek] fix using ost-images without lagofy Changes for Build #537 [Michal Skrivanek] fix using ost-images without lagofy ----------------- Failed Tests: ----------------- 1 tests failed. FAILED: basic-suite-master.test-scenarios.004_basic_sanity_pytest.test_ha_recovery Error Message: AssertionError: False != True after 600 seconds Stack Trace: prefix = <ovirtlago.prefix.OvirtPrefix object at 0x7f0c4a566750> @order_by(_TEST_LIST) def test_ha_recovery(prefix): engine = prefix.virt_env.engine_vm().get_api_v4().system_service() with test_utils.TestEvent(engine, [119, 9602, 506]): # VM_DOWN_ERROR event(119) # HA_VM_FAILED event event(9602) # VDS_INITIATED_RUN_VM event(506) vm_host = _vm_host(prefix, VM2_NAME) pid = vm_host.ssh(['pgrep', '-f', 'qemu.*guest=vm2']) vm_host.ssh(['kill', '-KILL', pid.out.decode('utf-8')]) vm_service = test_utils.get_vm_service(engine, VM2_NAME) testlib.assert_true_within_long(
lambda:
vm_service.get().status == types.VmStatus.UP ) ../basic-suite-master/test-scenarios/004_basic_sanity_pytest.py:819: _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ /usr/lib/python2.7/site-packages/ovirtlago/testlib.py:286: in assert_true_within_long assert_equals_within_long(func, True, allowed_exceptions) /usr/lib/python2.7/site-packages/ovirtlago/testlib.py:273: in assert_equals_within_long func, value, LONG_TIMEOUT, allowed_exceptions=allowed_exceptions _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ func = <function <lambda> at 0x7f0c42cd39b0>, value = True, timeout = 600 allowed_exceptions = [], initial_wait = 0 def assert_equals_within( func, value, timeout, allowed_exceptions=None, initial_wait=10 ): allowed_exceptions = allowed_exceptions or [] with utils.EggTimer(timeout) as timer: while not timer.elapsed(): try: res = func() if res == value: return except Exception as exc: if _instance_of_any(exc, allowed_exceptions): time.sleep(3) continue LOGGER.exception("Unhandled exception in %s", func) raise if initial_wait == 0: time.sleep(3) else: time.sleep(initial_wait) initial_wait = 0 try: raise AssertionError(
'%s != %s after %s seconds' % (res, value, timeout)
) E AssertionError: False != True after 600 seconds /usr/lib/python2.7/site-packages/ovirtlago/testlib.py:252: AssertionError
Project: https://jenkins.ovirt.org/job/ovirt-system-tests_compat-4.3-suite-master/ Build: https://jenkins.ovirt.org/job/ovirt-system-tests_compat-4.3-suite-master/538... Build Number: 538 Build Status: Still Failing Triggered By: Started by timer ------------------------------------- Changes Since Last Success: ------------------------------------- Changes for Build #529 [Michal Skrivanek] smaller OST VMs [Barak Korren] usrc: Filtering for the `files` provider Changes for Build #530 [Marcin Sobczyk] ost_utils: ansible: Sort events by creation date Changes for Build #531 [Michal Skrivanek] avoid ballooning crash in vm0 after memory hotplug Changes for Build #532 [Michal Skrivanek] avoid ballooning crash in vm0 after memory hotplug Changes for Build #533 [Michal Skrivanek] avoid ballooning crash in vm0 after memory hotplug Changes for Build #534 [Michal Skrivanek] avoid ballooning crash in vm0 after memory hotplug Changes for Build #535 [Michal Skrivanek] fix using ost-images without lagofy Changes for Build #536 [Michal Skrivanek] fix using ost-images without lagofy Changes for Build #537 [Michal Skrivanek] fix using ost-images without lagofy Changes for Build #538 [Michal Skrivanek] add a log of used images and their ovirt-engine and vdsm ----------------- Failed Tests: ----------------- 1 tests failed. FAILED: basic-suite-master.test-scenarios.004_basic_sanity_pytest.test_ha_recovery Error Message: AssertionError: False != True after 600 seconds Stack Trace: prefix = <ovirtlago.prefix.OvirtPrefix object at 0x7f06bc470f90> @order_by(_TEST_LIST) def test_ha_recovery(prefix): engine = prefix.virt_env.engine_vm().get_api_v4().system_service() with test_utils.TestEvent(engine, [119, 9602, 506]): # VM_DOWN_ERROR event(119) # HA_VM_FAILED event event(9602) # VDS_INITIATED_RUN_VM event(506) vm_host = _vm_host(prefix, VM2_NAME) pid = vm_host.ssh(['pgrep', '-f', 'qemu.*guest=vm2']) vm_host.ssh(['kill', '-KILL', pid.out.decode('utf-8')]) vm_service = test_utils.get_vm_service(engine, VM2_NAME) testlib.assert_true_within_long(
lambda:
vm_service.get().status == types.VmStatus.UP ) ../basic-suite-master/test-scenarios/004_basic_sanity_pytest.py:819: _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ /usr/lib/python2.7/site-packages/ovirtlago/testlib.py:286: in assert_true_within_long assert_equals_within_long(func, True, allowed_exceptions) /usr/lib/python2.7/site-packages/ovirtlago/testlib.py:273: in assert_equals_within_long func, value, LONG_TIMEOUT, allowed_exceptions=allowed_exceptions _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ func = <function <lambda> at 0x7f06bc578de8>, value = True, timeout = 600 allowed_exceptions = [], initial_wait = 0 def assert_equals_within( func, value, timeout, allowed_exceptions=None, initial_wait=10 ): allowed_exceptions = allowed_exceptions or [] with utils.EggTimer(timeout) as timer: while not timer.elapsed(): try: res = func() if res == value: return except Exception as exc: if _instance_of_any(exc, allowed_exceptions): time.sleep(3) continue LOGGER.exception("Unhandled exception in %s", func) raise if initial_wait == 0: time.sleep(3) else: time.sleep(initial_wait) initial_wait = 0 try: raise AssertionError(
'%s != %s after %s seconds' % (res, value, timeout)
) E AssertionError: False != True after 600 seconds /usr/lib/python2.7/site-packages/ovirtlago/testlib.py:252: AssertionError
Project: https://jenkins.ovirt.org/job/ovirt-system-tests_compat-4.3-suite-master/ Build: https://jenkins.ovirt.org/job/ovirt-system-tests_compat-4.3-suite-master/539... Build Number: 539 Build Status: Still Failing Triggered By: Started by timer ------------------------------------- Changes Since Last Success: ------------------------------------- Changes for Build #530 [Marcin Sobczyk] ost_utils: ansible: Sort events by creation date Changes for Build #531 [Michal Skrivanek] avoid ballooning crash in vm0 after memory hotplug Changes for Build #532 [Michal Skrivanek] avoid ballooning crash in vm0 after memory hotplug Changes for Build #533 [Michal Skrivanek] avoid ballooning crash in vm0 after memory hotplug Changes for Build #534 [Michal Skrivanek] avoid ballooning crash in vm0 after memory hotplug Changes for Build #535 [Michal Skrivanek] fix using ost-images without lagofy Changes for Build #536 [Michal Skrivanek] fix using ost-images without lagofy Changes for Build #537 [Michal Skrivanek] fix using ost-images without lagofy Changes for Build #538 [Michal Skrivanek] add a log of used images and their ovirt-engine and vdsm Changes for Build #539 [Michal Skrivanek] add a log of used images and their ovirt-engine and vdsm ----------------- Failed Tests: ----------------- 1 tests failed. FAILED: basic-suite-master.test-scenarios.004_basic_sanity_pytest.test_ha_recovery Error Message: AssertionError: False != True after 600 seconds Stack Trace: prefix = <ovirtlago.prefix.OvirtPrefix object at 0x7fafd5cbef90> @order_by(_TEST_LIST) def test_ha_recovery(prefix): engine = prefix.virt_env.engine_vm().get_api_v4().system_service() with test_utils.TestEvent(engine, [119, 9602, 506]): # VM_DOWN_ERROR event(119) # HA_VM_FAILED event event(9602) # VDS_INITIATED_RUN_VM event(506) vm_host = _vm_host(prefix, VM2_NAME) pid = vm_host.ssh(['pgrep', '-f', 'qemu.*guest=vm2']) vm_host.ssh(['kill', '-KILL', pid.out.decode('utf-8')]) vm_service = test_utils.get_vm_service(engine, VM2_NAME) testlib.assert_true_within_long(
lambda:
vm_service.get().status == types.VmStatus.UP ) ../basic-suite-master/test-scenarios/004_basic_sanity_pytest.py:819: _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ /usr/lib/python2.7/site-packages/ovirtlago/testlib.py:286: in assert_true_within_long assert_equals_within_long(func, True, allowed_exceptions) /usr/lib/python2.7/site-packages/ovirtlago/testlib.py:273: in assert_equals_within_long func, value, LONG_TIMEOUT, allowed_exceptions=allowed_exceptions _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ func = <function <lambda> at 0x7fafd5dc6de8>, value = True, timeout = 600 allowed_exceptions = [], initial_wait = 0 def assert_equals_within( func, value, timeout, allowed_exceptions=None, initial_wait=10 ): allowed_exceptions = allowed_exceptions or [] with utils.EggTimer(timeout) as timer: while not timer.elapsed(): try: res = func() if res == value: return except Exception as exc: if _instance_of_any(exc, allowed_exceptions): time.sleep(3) continue LOGGER.exception("Unhandled exception in %s", func) raise if initial_wait == 0: time.sleep(3) else: time.sleep(initial_wait) initial_wait = 0 try: raise AssertionError(
'%s != %s after %s seconds' % (res, value, timeout)
) E AssertionError: False != True after 600 seconds /usr/lib/python2.7/site-packages/ovirtlago/testlib.py:252: AssertionError
Project: https://jenkins.ovirt.org/job/ovirt-system-tests_compat-4.3-suite-master/ Build: https://jenkins.ovirt.org/job/ovirt-system-tests_compat-4.3-suite-master/540... Build Number: 540 Build Status: Still Failing Triggered By: Started by timer ------------------------------------- Changes Since Last Success: ------------------------------------- Changes for Build #531 [Michal Skrivanek] avoid ballooning crash in vm0 after memory hotplug Changes for Build #532 [Michal Skrivanek] avoid ballooning crash in vm0 after memory hotplug Changes for Build #533 [Michal Skrivanek] avoid ballooning crash in vm0 after memory hotplug Changes for Build #534 [Michal Skrivanek] avoid ballooning crash in vm0 after memory hotplug Changes for Build #535 [Michal Skrivanek] fix using ost-images without lagofy Changes for Build #536 [Michal Skrivanek] fix using ost-images without lagofy Changes for Build #537 [Michal Skrivanek] fix using ost-images without lagofy Changes for Build #538 [Michal Skrivanek] add a log of used images and their ovirt-engine and vdsm Changes for Build #539 [Michal Skrivanek] add a log of used images and their ovirt-engine and vdsm Changes for Build #540 [Michal Skrivanek] add a log of used images and their ovirt-engine and vdsm ----------------- Failed Tests: ----------------- 1 tests failed. FAILED: basic-suite-master.test-scenarios.004_basic_sanity_pytest.test_ha_recovery Error Message: AssertionError: False != True after 600 seconds Stack Trace: prefix = <ovirtlago.prefix.OvirtPrefix object at 0x7fa00ebf4f90> @order_by(_TEST_LIST) def test_ha_recovery(prefix): engine = prefix.virt_env.engine_vm().get_api_v4().system_service() with test_utils.TestEvent(engine, [119, 9602, 506]): # VM_DOWN_ERROR event(119) # HA_VM_FAILED event event(9602) # VDS_INITIATED_RUN_VM event(506) vm_host = _vm_host(prefix, VM2_NAME) pid = vm_host.ssh(['pgrep', '-f', 'qemu.*guest=vm2']) vm_host.ssh(['kill', '-KILL', pid.out.decode('utf-8')]) vm_service = test_utils.get_vm_service(engine, VM2_NAME) testlib.assert_true_within_long(
lambda:
vm_service.get().status == types.VmStatus.UP ) ../basic-suite-master/test-scenarios/004_basic_sanity_pytest.py:819: _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ /usr/lib/python2.7/site-packages/ovirtlago/testlib.py:286: in assert_true_within_long assert_equals_within_long(func, True, allowed_exceptions) /usr/lib/python2.7/site-packages/ovirtlago/testlib.py:273: in assert_equals_within_long func, value, LONG_TIMEOUT, allowed_exceptions=allowed_exceptions _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ func = <function <lambda> at 0x7fa00c1cc398>, value = True, timeout = 600 allowed_exceptions = [], initial_wait = 0 def assert_equals_within( func, value, timeout, allowed_exceptions=None, initial_wait=10 ): allowed_exceptions = allowed_exceptions or [] with utils.EggTimer(timeout) as timer: while not timer.elapsed(): try: res = func() if res == value: return except Exception as exc: if _instance_of_any(exc, allowed_exceptions): time.sleep(3) continue LOGGER.exception("Unhandled exception in %s", func) raise if initial_wait == 0: time.sleep(3) else: time.sleep(initial_wait) initial_wait = 0 try: raise AssertionError(
'%s != %s after %s seconds' % (res, value, timeout)
) E AssertionError: False != True after 600 seconds /usr/lib/python2.7/site-packages/ovirtlago/testlib.py:252: AssertionError
Project: https://jenkins.ovirt.org/job/ovirt-system-tests_compat-4.3-suite-master/ Build: https://jenkins.ovirt.org/job/ovirt-system-tests_compat-4.3-suite-master/541... Build Number: 541 Build Status: Still Failing Triggered By: Started by timer ------------------------------------- Changes Since Last Success: ------------------------------------- Changes for Build #532 [Michal Skrivanek] avoid ballooning crash in vm0 after memory hotplug Changes for Build #533 [Michal Skrivanek] avoid ballooning crash in vm0 after memory hotplug Changes for Build #534 [Michal Skrivanek] avoid ballooning crash in vm0 after memory hotplug Changes for Build #535 [Michal Skrivanek] fix using ost-images without lagofy Changes for Build #536 [Michal Skrivanek] fix using ost-images without lagofy Changes for Build #537 [Michal Skrivanek] fix using ost-images without lagofy Changes for Build #538 [Michal Skrivanek] add a log of used images and their ovirt-engine and vdsm Changes for Build #539 [Michal Skrivanek] add a log of used images and their ovirt-engine and vdsm Changes for Build #540 [Michal Skrivanek] add a log of used images and their ovirt-engine and vdsm Changes for Build #541 [jenkins CI] Auto create ovirt-4.3 ----------------- Failed Tests: ----------------- 1 tests failed. FAILED: basic-suite-master.test-scenarios.004_basic_sanity_pytest.test_ha_recovery Error Message: AssertionError: False != True after 600 seconds Stack Trace: prefix = <ovirtlago.prefix.OvirtPrefix object at 0x7fded839af90> @order_by(_TEST_LIST) def test_ha_recovery(prefix): engine = prefix.virt_env.engine_vm().get_api_v4().system_service() with test_utils.TestEvent(engine, [119, 9602, 506]): # VM_DOWN_ERROR event(119) # HA_VM_FAILED event event(9602) # VDS_INITIATED_RUN_VM event(506) vm_host = _vm_host(prefix, VM2_NAME) pid = vm_host.ssh(['pgrep', '-f', 'qemu.*guest=vm2']) vm_host.ssh(['kill', '-KILL', pid.out.decode('utf-8')]) vm_service = test_utils.get_vm_service(engine, VM2_NAME) testlib.assert_true_within_long(
lambda:
vm_service.get().status == types.VmStatus.UP ) ../basic-suite-master/test-scenarios/004_basic_sanity_pytest.py:819: _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ /usr/lib/python2.7/site-packages/ovirtlago/testlib.py:286: in assert_true_within_long assert_equals_within_long(func, True, allowed_exceptions) /usr/lib/python2.7/site-packages/ovirtlago/testlib.py:273: in assert_equals_within_long func, value, LONG_TIMEOUT, allowed_exceptions=allowed_exceptions _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ func = <function <lambda> at 0x7fded84a2de8>, value = True, timeout = 600 allowed_exceptions = [], initial_wait = 0 def assert_equals_within( func, value, timeout, allowed_exceptions=None, initial_wait=10 ): allowed_exceptions = allowed_exceptions or [] with utils.EggTimer(timeout) as timer: while not timer.elapsed(): try: res = func() if res == value: return except Exception as exc: if _instance_of_any(exc, allowed_exceptions): time.sleep(3) continue LOGGER.exception("Unhandled exception in %s", func) raise if initial_wait == 0: time.sleep(3) else: time.sleep(initial_wait) initial_wait = 0 try: raise AssertionError(
'%s != %s after %s seconds' % (res, value, timeout)
) E AssertionError: False != True after 600 seconds /usr/lib/python2.7/site-packages/ovirtlago/testlib.py:252: AssertionError
Project: https://jenkins.ovirt.org/job/ovirt-system-tests_compat-4.3-suite-master/ Build: https://jenkins.ovirt.org/job/ovirt-system-tests_compat-4.3-suite-master/542... Build Number: 542 Build Status: Still Failing Triggered By: Started by timer ------------------------------------- Changes Since Last Success: ------------------------------------- Changes for Build #533 [Michal Skrivanek] avoid ballooning crash in vm0 after memory hotplug Changes for Build #534 [Michal Skrivanek] avoid ballooning crash in vm0 after memory hotplug Changes for Build #535 [Michal Skrivanek] fix using ost-images without lagofy Changes for Build #536 [Michal Skrivanek] fix using ost-images without lagofy Changes for Build #537 [Michal Skrivanek] fix using ost-images without lagofy Changes for Build #538 [Michal Skrivanek] add a log of used images and their ovirt-engine and vdsm Changes for Build #539 [Michal Skrivanek] add a log of used images and their ovirt-engine and vdsm Changes for Build #540 [Michal Skrivanek] add a log of used images and their ovirt-engine and vdsm Changes for Build #541 [jenkins CI] Auto create ovirt-4.3 Changes for Build #542 [jenkins CI] Auto create ovirt-4.3 ----------------- Failed Tests: ----------------- 1 tests failed. FAILED: basic-suite-master.test-scenarios.004_basic_sanity_pytest.test_ha_recovery Error Message: AssertionError: False != True after 600 seconds Stack Trace: prefix = <ovirtlago.prefix.OvirtPrefix object at 0x7f81174cef90> @order_by(_TEST_LIST) def test_ha_recovery(prefix): engine = prefix.virt_env.engine_vm().get_api_v4().system_service() with test_utils.TestEvent(engine, [119, 9602, 506]): # VM_DOWN_ERROR event(119) # HA_VM_FAILED event event(9602) # VDS_INITIATED_RUN_VM event(506) vm_host = _vm_host(prefix, VM2_NAME) pid = vm_host.ssh(['pgrep', '-f', 'qemu.*guest=vm2']) vm_host.ssh(['kill', '-KILL', pid.out.decode('utf-8')]) vm_service = test_utils.get_vm_service(engine, VM2_NAME) testlib.assert_true_within_long(
lambda:
vm_service.get().status == types.VmStatus.UP ) ../basic-suite-master/test-scenarios/004_basic_sanity_pytest.py:819: _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ /usr/lib/python2.7/site-packages/ovirtlago/testlib.py:286: in assert_true_within_long assert_equals_within_long(func, True, allowed_exceptions) /usr/lib/python2.7/site-packages/ovirtlago/testlib.py:273: in assert_equals_within_long func, value, LONG_TIMEOUT, allowed_exceptions=allowed_exceptions _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ func = <function <lambda> at 0x7f8107fbc398>, value = True, timeout = 600 allowed_exceptions = [], initial_wait = 0 def assert_equals_within( func, value, timeout, allowed_exceptions=None, initial_wait=10 ): allowed_exceptions = allowed_exceptions or [] with utils.EggTimer(timeout) as timer: while not timer.elapsed(): try: res = func() if res == value: return except Exception as exc: if _instance_of_any(exc, allowed_exceptions): time.sleep(3) continue LOGGER.exception("Unhandled exception in %s", func) raise if initial_wait == 0: time.sleep(3) else: time.sleep(initial_wait) initial_wait = 0 try: raise AssertionError(
'%s != %s after %s seconds' % (res, value, timeout)
) E AssertionError: False != True after 600 seconds /usr/lib/python2.7/site-packages/ovirtlago/testlib.py:252: AssertionError
Project: https://jenkins.ovirt.org/job/ovirt-system-tests_compat-4.3-suite-master/ Build: https://jenkins.ovirt.org/job/ovirt-system-tests_compat-4.3-suite-master/543... Build Number: 543 Build Status: Still Failing Triggered By: Started by timer ------------------------------------- Changes Since Last Success: ------------------------------------- Changes for Build #534 [Michal Skrivanek] avoid ballooning crash in vm0 after memory hotplug Changes for Build #535 [Michal Skrivanek] fix using ost-images without lagofy Changes for Build #536 [Michal Skrivanek] fix using ost-images without lagofy Changes for Build #537 [Michal Skrivanek] fix using ost-images without lagofy Changes for Build #538 [Michal Skrivanek] add a log of used images and their ovirt-engine and vdsm Changes for Build #539 [Michal Skrivanek] add a log of used images and their ovirt-engine and vdsm Changes for Build #540 [Michal Skrivanek] add a log of used images and their ovirt-engine and vdsm Changes for Build #541 [jenkins CI] Auto create ovirt-4.3 Changes for Build #542 [jenkins CI] Auto create ovirt-4.3 Changes for Build #543 [Galit Rosenthal] For he master:Chg ssh to host ----------------- Failed Tests: ----------------- 1 tests failed. FAILED: basic-suite-master.test-scenarios.004_basic_sanity_pytest.test_ha_recovery Error Message: AssertionError: False != True after 600 seconds Stack Trace: prefix = <ovirtlago.prefix.OvirtPrefix object at 0x7ff1440eef90> @order_by(_TEST_LIST) def test_ha_recovery(prefix): engine = prefix.virt_env.engine_vm().get_api_v4().system_service() with test_utils.TestEvent(engine, [119, 9602, 506]): # VM_DOWN_ERROR event(119) # HA_VM_FAILED event event(9602) # VDS_INITIATED_RUN_VM event(506) vm_host = _vm_host(prefix, VM2_NAME) pid = vm_host.ssh(['pgrep', '-f', 'qemu.*guest=vm2']) vm_host.ssh(['kill', '-KILL', pid.out.decode('utf-8')]) vm_service = test_utils.get_vm_service(engine, VM2_NAME) testlib.assert_true_within_long(
lambda:
vm_service.get().status == types.VmStatus.UP ) ../basic-suite-master/test-scenarios/004_basic_sanity_pytest.py:819: _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ /usr/lib/python2.7/site-packages/ovirtlago/testlib.py:286: in assert_true_within_long assert_equals_within_long(func, True, allowed_exceptions) /usr/lib/python2.7/site-packages/ovirtlago/testlib.py:273: in assert_equals_within_long func, value, LONG_TIMEOUT, allowed_exceptions=allowed_exceptions _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ func = <function <lambda> at 0x7ff1441f6de8>, value = True, timeout = 600 allowed_exceptions = [], initial_wait = 0 def assert_equals_within( func, value, timeout, allowed_exceptions=None, initial_wait=10 ): allowed_exceptions = allowed_exceptions or [] with utils.EggTimer(timeout) as timer: while not timer.elapsed(): try: res = func() if res == value: return except Exception as exc: if _instance_of_any(exc, allowed_exceptions): time.sleep(3) continue LOGGER.exception("Unhandled exception in %s", func) raise if initial_wait == 0: time.sleep(3) else: time.sleep(initial_wait) initial_wait = 0 try: raise AssertionError(
'%s != %s after %s seconds' % (res, value, timeout)
) E AssertionError: False != True after 600 seconds /usr/lib/python2.7/site-packages/ovirtlago/testlib.py:252: AssertionError
Project: https://jenkins.ovirt.org/job/ovirt-system-tests_compat-4.3-suite-master/ Build: https://jenkins.ovirt.org/job/ovirt-system-tests_compat-4.3-suite-master/544... Build Number: 544 Build Status: Still Failing Triggered By: Started by timer ------------------------------------- Changes Since Last Success: ------------------------------------- Changes for Build #535 [Michal Skrivanek] fix using ost-images without lagofy Changes for Build #536 [Michal Skrivanek] fix using ost-images without lagofy Changes for Build #537 [Michal Skrivanek] fix using ost-images without lagofy Changes for Build #538 [Michal Skrivanek] add a log of used images and their ovirt-engine and vdsm Changes for Build #539 [Michal Skrivanek] add a log of used images and their ovirt-engine and vdsm Changes for Build #540 [Michal Skrivanek] add a log of used images and their ovirt-engine and vdsm Changes for Build #541 [jenkins CI] Auto create ovirt-4.3 Changes for Build #542 [jenkins CI] Auto create ovirt-4.3 Changes for Build #543 [Galit Rosenthal] For he master:Chg ssh to host Changes for Build #544 [Martin Necas] Ansible: fix collection path and use sed to rename [Ehud Yonasi] poll: remove metrics job. ----------------- Failed Tests: ----------------- 1 tests failed. FAILED: basic-suite-master.test-scenarios.004_basic_sanity_pytest.test_ha_recovery Error Message: AssertionError: False != True after 600 seconds Stack Trace: prefix = <ovirtlago.prefix.OvirtPrefix object at 0x7fb5cb07df90> @order_by(_TEST_LIST) def test_ha_recovery(prefix): engine = prefix.virt_env.engine_vm().get_api_v4().system_service() with test_utils.TestEvent(engine, [119, 9602, 506]): # VM_DOWN_ERROR event(119) # HA_VM_FAILED event event(9602) # VDS_INITIATED_RUN_VM event(506) vm_host = _vm_host(prefix, VM2_NAME) pid = vm_host.ssh(['pgrep', '-f', 'qemu.*guest=vm2']) vm_host.ssh(['kill', '-KILL', pid.out.decode('utf-8')]) vm_service = test_utils.get_vm_service(engine, VM2_NAME) testlib.assert_true_within_long(
lambda:
vm_service.get().status == types.VmStatus.UP ) ../basic-suite-master/test-scenarios/004_basic_sanity_pytest.py:819: _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ /usr/lib/python2.7/site-packages/ovirtlago/testlib.py:286: in assert_true_within_long assert_equals_within_long(func, True, allowed_exceptions) /usr/lib/python2.7/site-packages/ovirtlago/testlib.py:273: in assert_equals_within_long func, value, LONG_TIMEOUT, allowed_exceptions=allowed_exceptions _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ func = <function <lambda> at 0x7fb5c8069398>, value = True, timeout = 600 allowed_exceptions = [], initial_wait = 0 def assert_equals_within( func, value, timeout, allowed_exceptions=None, initial_wait=10 ): allowed_exceptions = allowed_exceptions or [] with utils.EggTimer(timeout) as timer: while not timer.elapsed(): try: res = func() if res == value: return except Exception as exc: if _instance_of_any(exc, allowed_exceptions): time.sleep(3) continue LOGGER.exception("Unhandled exception in %s", func) raise if initial_wait == 0: time.sleep(3) else: time.sleep(initial_wait) initial_wait = 0 try: raise AssertionError(
'%s != %s after %s seconds' % (res, value, timeout)
) E AssertionError: False != True after 600 seconds /usr/lib/python2.7/site-packages/ovirtlago/testlib.py:252: AssertionError
Project: https://jenkins.ovirt.org/job/ovirt-system-tests_compat-4.3-suite-master/ Build: https://jenkins.ovirt.org/job/ovirt-system-tests_compat-4.3-suite-master/545... Build Number: 545 Build Status: Still Failing Triggered By: Started by timer ------------------------------------- Changes Since Last Success: ------------------------------------- Changes for Build #536 [Michal Skrivanek] fix using ost-images without lagofy Changes for Build #537 [Michal Skrivanek] fix using ost-images without lagofy Changes for Build #538 [Michal Skrivanek] add a log of used images and their ovirt-engine and vdsm Changes for Build #539 [Michal Skrivanek] add a log of used images and their ovirt-engine and vdsm Changes for Build #540 [Michal Skrivanek] add a log of used images and their ovirt-engine and vdsm Changes for Build #541 [jenkins CI] Auto create ovirt-4.3 Changes for Build #542 [jenkins CI] Auto create ovirt-4.3 Changes for Build #543 [Galit Rosenthal] For he master:Chg ssh to host Changes for Build #544 [Martin Necas] Ansible: fix collection path and use sed to rename [Ehud Yonasi] poll: remove metrics job. Changes for Build #545 [Galit Rosenthal] Remove he-basic-ipv6_suite_4.3 from check-patch ----------------- Failed Tests: ----------------- 1 tests failed. FAILED: basic-suite-master.test-scenarios.004_basic_sanity_pytest.test_ha_recovery Error Message: AssertionError: False != True after 600 seconds Stack Trace: prefix = <ovirtlago.prefix.OvirtPrefix object at 0x7f9e24de4f90> @order_by(_TEST_LIST) def test_ha_recovery(prefix): engine = prefix.virt_env.engine_vm().get_api_v4().system_service() with test_utils.TestEvent(engine, [119, 9602, 506]): # VM_DOWN_ERROR event(119) # HA_VM_FAILED event event(9602) # VDS_INITIATED_RUN_VM event(506) vm_host = _vm_host(prefix, VM2_NAME) pid = vm_host.ssh(['pgrep', '-f', 'qemu.*guest=vm2']) vm_host.ssh(['kill', '-KILL', pid.out.decode('utf-8')]) vm_service = test_utils.get_vm_service(engine, VM2_NAME) testlib.assert_true_within_long(
lambda:
vm_service.get().status == types.VmStatus.UP ) ../basic-suite-master/test-scenarios/004_basic_sanity_pytest.py:819: _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ /usr/lib/python2.7/site-packages/ovirtlago/testlib.py:286: in assert_true_within_long assert_equals_within_long(func, True, allowed_exceptions) /usr/lib/python2.7/site-packages/ovirtlago/testlib.py:273: in assert_equals_within_long func, value, LONG_TIMEOUT, allowed_exceptions=allowed_exceptions _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ func = <function <lambda> at 0x7f9e24eecde8>, value = True, timeout = 600 allowed_exceptions = [], initial_wait = 0 def assert_equals_within( func, value, timeout, allowed_exceptions=None, initial_wait=10 ): allowed_exceptions = allowed_exceptions or [] with utils.EggTimer(timeout) as timer: while not timer.elapsed(): try: res = func() if res == value: return except Exception as exc: if _instance_of_any(exc, allowed_exceptions): time.sleep(3) continue LOGGER.exception("Unhandled exception in %s", func) raise if initial_wait == 0: time.sleep(3) else: time.sleep(initial_wait) initial_wait = 0 try: raise AssertionError(
'%s != %s after %s seconds' % (res, value, timeout)
) E AssertionError: False != True after 600 seconds /usr/lib/python2.7/site-packages/ovirtlago/testlib.py:252: AssertionError
Project: https://jenkins.ovirt.org/job/ovirt-system-tests_compat-4.3-suite-master/ Build: https://jenkins.ovirt.org/job/ovirt-system-tests_compat-4.3-suite-master/546... Build Number: 546 Build Status: Still Failing Triggered By: Started by timer ------------------------------------- Changes Since Last Success: ------------------------------------- Changes for Build #537 [Michal Skrivanek] fix using ost-images without lagofy Changes for Build #538 [Michal Skrivanek] add a log of used images and their ovirt-engine and vdsm Changes for Build #539 [Michal Skrivanek] add a log of used images and their ovirt-engine and vdsm Changes for Build #540 [Michal Skrivanek] add a log of used images and their ovirt-engine and vdsm Changes for Build #541 [jenkins CI] Auto create ovirt-4.3 Changes for Build #542 [jenkins CI] Auto create ovirt-4.3 Changes for Build #543 [Galit Rosenthal] For he master:Chg ssh to host Changes for Build #544 [Martin Necas] Ansible: fix collection path and use sed to rename [Ehud Yonasi] poll: remove metrics job. Changes for Build #545 [Galit Rosenthal] Remove he-basic-ipv6_suite_4.3 from check-patch Changes for Build #546 [Michal Skrivanek] silence new line output ----------------- Failed Tests: ----------------- 1 tests failed. FAILED: basic-suite-master.test-scenarios.004_basic_sanity_pytest.test_ha_recovery Error Message: AssertionError: False != True after 600 seconds Stack Trace: prefix = <ovirtlago.prefix.OvirtPrefix object at 0x7f76c3117f90> @order_by(_TEST_LIST) def test_ha_recovery(prefix): engine = prefix.virt_env.engine_vm().get_api_v4().system_service() with test_utils.TestEvent(engine, [119, 9602, 506]): # VM_DOWN_ERROR event(119) # HA_VM_FAILED event event(9602) # VDS_INITIATED_RUN_VM event(506) vm_host = _vm_host(prefix, VM2_NAME) pid = vm_host.ssh(['pgrep', '-f', 'qemu.*guest=vm2']) vm_host.ssh(['kill', '-KILL', pid.out.decode('utf-8')]) vm_service = test_utils.get_vm_service(engine, VM2_NAME) testlib.assert_true_within_long(
lambda:
vm_service.get().status == types.VmStatus.UP ) ../basic-suite-master/test-scenarios/004_basic_sanity_pytest.py:819: _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ /usr/lib/python2.7/site-packages/ovirtlago/testlib.py:286: in assert_true_within_long assert_equals_within_long(func, True, allowed_exceptions) /usr/lib/python2.7/site-packages/ovirtlago/testlib.py:273: in assert_equals_within_long func, value, LONG_TIMEOUT, allowed_exceptions=allowed_exceptions _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ func = <function <lambda> at 0x7f76c011a398>, value = True, timeout = 600 allowed_exceptions = [], initial_wait = 0 def assert_equals_within( func, value, timeout, allowed_exceptions=None, initial_wait=10 ): allowed_exceptions = allowed_exceptions or [] with utils.EggTimer(timeout) as timer: while not timer.elapsed(): try: res = func() if res == value: return except Exception as exc: if _instance_of_any(exc, allowed_exceptions): time.sleep(3) continue LOGGER.exception("Unhandled exception in %s", func) raise if initial_wait == 0: time.sleep(3) else: time.sleep(initial_wait) initial_wait = 0 try: raise AssertionError(
'%s != %s after %s seconds' % (res, value, timeout)
) E AssertionError: False != True after 600 seconds /usr/lib/python2.7/site-packages/ovirtlago/testlib.py:252: AssertionError
Project: https://jenkins.ovirt.org/job/ovirt-system-tests_compat-4.3-suite-master/ Build: https://jenkins.ovirt.org/job/ovirt-system-tests_compat-4.3-suite-master/547... Build Number: 547 Build Status: Still Failing Triggered By: Started by timer ------------------------------------- Changes Since Last Success: ------------------------------------- Changes for Build #538 [Michal Skrivanek] add a log of used images and their ovirt-engine and vdsm Changes for Build #539 [Michal Skrivanek] add a log of used images and their ovirt-engine and vdsm Changes for Build #540 [Michal Skrivanek] add a log of used images and their ovirt-engine and vdsm Changes for Build #541 [jenkins CI] Auto create ovirt-4.3 Changes for Build #542 [jenkins CI] Auto create ovirt-4.3 Changes for Build #543 [Galit Rosenthal] For he master:Chg ssh to host Changes for Build #544 [Martin Necas] Ansible: fix collection path and use sed to rename [Ehud Yonasi] poll: remove metrics job. Changes for Build #545 [Galit Rosenthal] Remove he-basic-ipv6_suite_4.3 from check-patch Changes for Build #546 [Michal Skrivanek] silence new line output Changes for Build #547 [Michal Skrivanek] silence new line output ----------------- Failed Tests: ----------------- 1 tests failed. FAILED: basic-suite-master.test-scenarios.002_bootstrap_pytest.test_verify_add_hosts Error Message: RuntimeError: 1 hosts failed installation: lago-compat-4-3-suite-master-host-0: install_failed Stack Trace: engine_api = <ovirtsdk4.Connection object at 0x7fc0b6dc8210> @order_by(_TEST_LIST) def test_verify_add_hosts(engine_api): hosts_service = engine_api.system_service().hosts_service() hosts_status = hosts_service.list(search='datacenter={}'.format(DC_NAME)) total_hosts = len(hosts_status) dump_hosts = _host_status_to_print(hosts_service, hosts_status) LOGGER.debug('Host status, verify_add_hosts:\n {}'.format(dump_hosts)) testlib.assert_true_within( lambda: _single_host_up(hosts_service, total_hosts),
timeout=constants.ADD_HOST_TIMEOUT
) ../basic-suite-master/test-scenarios/002_bootstrap_pytest.py:465: _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ /usr/lib/python2.7/site-packages/ovirtlago/testlib.py:278: in assert_true_within assert_equals_within(func, True, timeout, allowed_exceptions) /usr/lib/python2.7/site-packages/ovirtlago/testlib.py:234: in assert_equals_within res = func() ../basic-suite-master/test-scenarios/002_bootstrap_pytest.py:464: in <lambda> lambda: _single_host_up(hosts_service, total_hosts), ../basic-suite-master/test-scenarios/002_bootstrap_pytest.py:236: in _single_host_up _check_problematic_hosts(hosts_service) _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ hosts_service = <ovirtsdk4.services.HostsService object at 0x7fc0b6c37f50> def _check_problematic_hosts(hosts_service): problematic_hosts = hosts_service.list(search='datacenter={} AND status != installing and status != initializing and status != up'.format(DC_NAME)) if len(problematic_hosts): dump_hosts = '%s hosts failed installation:\n' % len(problematic_hosts) for host in problematic_hosts: host_service = hosts_service.host_service(host.id) dump_hosts += '%s: %s\n' % (host.name, host_service.get().status)
raise RuntimeError(dump_hosts)
E RuntimeError: 1 hosts failed installation: E lago-compat-4-3-suite-master-host-0: install_failed ../basic-suite-master/test-scenarios/002_bootstrap_pytest.py:245: RuntimeError
Project: https://jenkins.ovirt.org/job/ovirt-system-tests_compat-4.3-suite-master/ Build: https://jenkins.ovirt.org/job/ovirt-system-tests_compat-4.3-suite-master/548... Build Number: 548 Build Status: Still Failing Triggered By: Started by timer ------------------------------------- Changes Since Last Success: ------------------------------------- Changes for Build #539 [Michal Skrivanek] add a log of used images and their ovirt-engine and vdsm Changes for Build #540 [Michal Skrivanek] add a log of used images and their ovirt-engine and vdsm Changes for Build #541 [jenkins CI] Auto create ovirt-4.3 Changes for Build #542 [jenkins CI] Auto create ovirt-4.3 Changes for Build #543 [Galit Rosenthal] For he master:Chg ssh to host Changes for Build #544 [Martin Necas] Ansible: fix collection path and use sed to rename [Ehud Yonasi] poll: remove metrics job. Changes for Build #545 [Galit Rosenthal] Remove he-basic-ipv6_suite_4.3 from check-patch Changes for Build #546 [Michal Skrivanek] silence new line output Changes for Build #547 [Michal Skrivanek] silence new line output Changes for Build #548 [Michal Skrivanek] silence new line output ----------------- Failed Tests: ----------------- 1 tests failed. FAILED: basic-suite-master.test-scenarios.002_bootstrap_pytest.test_verify_add_hosts Error Message: RuntimeError: 1 hosts failed installation: lago-compat-4-3-suite-master-host-0: install_failed Stack Trace: engine_api = <ovirtsdk4.Connection object at 0x7fc3e7eef210> @order_by(_TEST_LIST) def test_verify_add_hosts(engine_api): hosts_service = engine_api.system_service().hosts_service() hosts_status = hosts_service.list(search='datacenter={}'.format(DC_NAME)) total_hosts = len(hosts_status) dump_hosts = _host_status_to_print(hosts_service, hosts_status) LOGGER.debug('Host status, verify_add_hosts:\n {}'.format(dump_hosts)) testlib.assert_true_within( lambda: _single_host_up(hosts_service, total_hosts),
timeout=constants.ADD_HOST_TIMEOUT
) ../basic-suite-master/test-scenarios/002_bootstrap_pytest.py:465: _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ /usr/lib/python2.7/site-packages/ovirtlago/testlib.py:278: in assert_true_within assert_equals_within(func, True, timeout, allowed_exceptions) /usr/lib/python2.7/site-packages/ovirtlago/testlib.py:234: in assert_equals_within res = func() ../basic-suite-master/test-scenarios/002_bootstrap_pytest.py:464: in <lambda> lambda: _single_host_up(hosts_service, total_hosts), ../basic-suite-master/test-scenarios/002_bootstrap_pytest.py:236: in _single_host_up _check_problematic_hosts(hosts_service) _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ hosts_service = <ovirtsdk4.services.HostsService object at 0x7fc3e7d8bf50> def _check_problematic_hosts(hosts_service): problematic_hosts = hosts_service.list(search='datacenter={} AND status != installing and status != initializing and status != up'.format(DC_NAME)) if len(problematic_hosts): dump_hosts = '%s hosts failed installation:\n' % len(problematic_hosts) for host in problematic_hosts: host_service = hosts_service.host_service(host.id) dump_hosts += '%s: %s\n' % (host.name, host_service.get().status)
raise RuntimeError(dump_hosts)
E RuntimeError: 1 hosts failed installation: E lago-compat-4-3-suite-master-host-0: install_failed ../basic-suite-master/test-scenarios/002_bootstrap_pytest.py:245: RuntimeError
Project: https://jenkins.ovirt.org/job/ovirt-system-tests_compat-4.3-suite-master/ Build: https://jenkins.ovirt.org/job/ovirt-system-tests_compat-4.3-suite-master/549... Build Number: 549 Build Status: Still Failing Triggered By: Started by timer ------------------------------------- Changes Since Last Success: ------------------------------------- Changes for Build #540 [Michal Skrivanek] add a log of used images and their ovirt-engine and vdsm Changes for Build #541 [jenkins CI] Auto create ovirt-4.3 Changes for Build #542 [jenkins CI] Auto create ovirt-4.3 Changes for Build #543 [Galit Rosenthal] For he master:Chg ssh to host Changes for Build #544 [Martin Necas] Ansible: fix collection path and use sed to rename [Ehud Yonasi] poll: remove metrics job. Changes for Build #545 [Galit Rosenthal] Remove he-basic-ipv6_suite_4.3 from check-patch Changes for Build #546 [Michal Skrivanek] silence new line output Changes for Build #547 [Michal Skrivanek] silence new line output Changes for Build #548 [Michal Skrivanek] silence new line output Changes for Build #549 [Michal Skrivanek] silence new line output ----------------- Failed Tests: ----------------- 1 tests failed. FAILED: basic-suite-master.test-scenarios.004_basic_sanity_pytest.test_ha_recovery Error Message: AssertionError: False != True after 600 seconds Stack Trace: prefix = <ovirtlago.prefix.OvirtPrefix object at 0x7f81df4f7f90> @order_by(_TEST_LIST) def test_ha_recovery(prefix): engine = prefix.virt_env.engine_vm().get_api_v4().system_service() with test_utils.TestEvent(engine, [119, 9602, 506]): # VM_DOWN_ERROR event(119) # HA_VM_FAILED event event(9602) # VDS_INITIATED_RUN_VM event(506) vm_host = _vm_host(prefix, VM2_NAME) pid = vm_host.ssh(['pgrep', '-f', 'qemu.*guest=vm2']) vm_host.ssh(['kill', '-KILL', pid.out.decode('utf-8')]) vm_service = test_utils.get_vm_service(engine, VM2_NAME) testlib.assert_true_within_long(
lambda:
vm_service.get().status == types.VmStatus.UP ) ../basic-suite-master/test-scenarios/004_basic_sanity_pytest.py:819: _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ /usr/lib/python2.7/site-packages/ovirtlago/testlib.py:286: in assert_true_within_long assert_equals_within_long(func, True, allowed_exceptions) /usr/lib/python2.7/site-packages/ovirtlago/testlib.py:273: in assert_equals_within_long func, value, LONG_TIMEOUT, allowed_exceptions=allowed_exceptions _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ func = <function <lambda> at 0x7f81cffa7398>, value = True, timeout = 600 allowed_exceptions = [], initial_wait = 0 def assert_equals_within( func, value, timeout, allowed_exceptions=None, initial_wait=10 ): allowed_exceptions = allowed_exceptions or [] with utils.EggTimer(timeout) as timer: while not timer.elapsed(): try: res = func() if res == value: return except Exception as exc: if _instance_of_any(exc, allowed_exceptions): time.sleep(3) continue LOGGER.exception("Unhandled exception in %s", func) raise if initial_wait == 0: time.sleep(3) else: time.sleep(initial_wait) initial_wait = 0 try: raise AssertionError(
'%s != %s after %s seconds' % (res, value, timeout)
) E AssertionError: False != True after 600 seconds /usr/lib/python2.7/site-packages/ovirtlago/testlib.py:252: AssertionError
Project: https://jenkins.ovirt.org/job/ovirt-system-tests_compat-4.3-suite-master/ Build: https://jenkins.ovirt.org/job/ovirt-system-tests_compat-4.3-suite-master/550... Build Number: 550 Build Status: Still Failing Triggered By: Started by timer ------------------------------------- Changes Since Last Success: ------------------------------------- Changes for Build #541 [jenkins CI] Auto create ovirt-4.3 Changes for Build #542 [jenkins CI] Auto create ovirt-4.3 Changes for Build #543 [Galit Rosenthal] For he master:Chg ssh to host Changes for Build #544 [Martin Necas] Ansible: fix collection path and use sed to rename [Ehud Yonasi] poll: remove metrics job. Changes for Build #545 [Galit Rosenthal] Remove he-basic-ipv6_suite_4.3 from check-patch Changes for Build #546 [Michal Skrivanek] silence new line output Changes for Build #547 [Michal Skrivanek] silence new line output Changes for Build #548 [Michal Skrivanek] silence new line output Changes for Build #549 [Michal Skrivanek] silence new line output Changes for Build #550 [Marcin Sobczyk] ost_utils: ansible: Always return the result ----------------- Failed Tests: ----------------- 1 tests failed. FAILED: basic-suite-master.test-scenarios.004_basic_sanity_pytest.test_ha_recovery Error Message: AssertionError: False != True after 600 seconds Stack Trace: prefix = <ovirtlago.prefix.OvirtPrefix object at 0x7f6a5133af90> @order_by(_TEST_LIST) def test_ha_recovery(prefix): engine = prefix.virt_env.engine_vm().get_api_v4().system_service() with test_utils.TestEvent(engine, [119, 9602, 506]): # VM_DOWN_ERROR event(119) # HA_VM_FAILED event event(9602) # VDS_INITIATED_RUN_VM event(506) vm_host = _vm_host(prefix, VM2_NAME) pid = vm_host.ssh(['pgrep', '-f', 'qemu.*guest=vm2']) vm_host.ssh(['kill', '-KILL', pid.out.decode('utf-8')]) vm_service = test_utils.get_vm_service(engine, VM2_NAME) testlib.assert_true_within_long(
lambda:
vm_service.get().status == types.VmStatus.UP ) ../basic-suite-master/test-scenarios/004_basic_sanity_pytest.py:819: _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ /usr/lib/python2.7/site-packages/ovirtlago/testlib.py:286: in assert_true_within_long assert_equals_within_long(func, True, allowed_exceptions) /usr/lib/python2.7/site-packages/ovirtlago/testlib.py:273: in assert_equals_within_long func, value, LONG_TIMEOUT, allowed_exceptions=allowed_exceptions _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ func = <function <lambda> at 0x7f6a500f5320>, value = True, timeout = 600 allowed_exceptions = [], initial_wait = 0 def assert_equals_within( func, value, timeout, allowed_exceptions=None, initial_wait=10 ): allowed_exceptions = allowed_exceptions or [] with utils.EggTimer(timeout) as timer: while not timer.elapsed(): try: res = func() if res == value: return except Exception as exc: if _instance_of_any(exc, allowed_exceptions): time.sleep(3) continue LOGGER.exception("Unhandled exception in %s", func) raise if initial_wait == 0: time.sleep(3) else: time.sleep(initial_wait) initial_wait = 0 try: raise AssertionError(
'%s != %s after %s seconds' % (res, value, timeout)
) E AssertionError: False != True after 600 seconds /usr/lib/python2.7/site-packages/ovirtlago/testlib.py:252: AssertionError
Project: https://jenkins.ovirt.org/job/ovirt-system-tests_compat-4.3-suite-master/ Build: https://jenkins.ovirt.org/job/ovirt-system-tests_compat-4.3-suite-master/551... Build Number: 551 Build Status: Still Failing Triggered By: Started by timer ------------------------------------- Changes Since Last Success: ------------------------------------- Changes for Build #542 [jenkins CI] Auto create ovirt-4.3 Changes for Build #543 [Galit Rosenthal] For he master:Chg ssh to host Changes for Build #544 [Martin Necas] Ansible: fix collection path and use sed to rename [Ehud Yonasi] poll: remove metrics job. Changes for Build #545 [Galit Rosenthal] Remove he-basic-ipv6_suite_4.3 from check-patch Changes for Build #546 [Michal Skrivanek] silence new line output Changes for Build #547 [Michal Skrivanek] silence new line output Changes for Build #548 [Michal Skrivanek] silence new line output Changes for Build #549 [Michal Skrivanek] silence new line output Changes for Build #550 [Marcin Sobczyk] ost_utils: ansible: Always return the result Changes for Build #551 [Marcin Sobczyk] ost_utils: Extract assertions to a separate module ----------------- Failed Tests: ----------------- 1 tests failed. FAILED: basic-suite-master.test-scenarios.004_basic_sanity_pytest.test_ha_recovery Error Message: AssertionError: False != True after 600 seconds Stack Trace: prefix = <ovirtlago.prefix.OvirtPrefix object at 0x7fb98c429f90> @order_by(_TEST_LIST) def test_ha_recovery(prefix): engine = prefix.virt_env.engine_vm().get_api_v4().system_service() with test_utils.TestEvent(engine, [119, 9602, 506]): # VM_DOWN_ERROR event(119) # HA_VM_FAILED event event(9602) # VDS_INITIATED_RUN_VM event(506) vm_host = _vm_host(prefix, VM2_NAME) pid = vm_host.ssh(['pgrep', '-f', 'qemu.*guest=vm2']) vm_host.ssh(['kill', '-KILL', pid.out.decode('utf-8')]) vm_service = test_utils.get_vm_service(engine, VM2_NAME) testlib.assert_true_within_long(
lambda:
vm_service.get().status == types.VmStatus.UP ) ../basic-suite-master/test-scenarios/004_basic_sanity_pytest.py:819: _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ /usr/lib/python2.7/site-packages/ovirtlago/testlib.py:286: in assert_true_within_long assert_equals_within_long(func, True, allowed_exceptions) /usr/lib/python2.7/site-packages/ovirtlago/testlib.py:273: in assert_equals_within_long func, value, LONG_TIMEOUT, allowed_exceptions=allowed_exceptions _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ func = <function <lambda> at 0x7fb988f111b8>, value = True, timeout = 600 allowed_exceptions = [], initial_wait = 0 def assert_equals_within( func, value, timeout, allowed_exceptions=None, initial_wait=10 ): allowed_exceptions = allowed_exceptions or [] with utils.EggTimer(timeout) as timer: while not timer.elapsed(): try: res = func() if res == value: return except Exception as exc: if _instance_of_any(exc, allowed_exceptions): time.sleep(3) continue LOGGER.exception("Unhandled exception in %s", func) raise if initial_wait == 0: time.sleep(3) else: time.sleep(initial_wait) initial_wait = 0 try: raise AssertionError(
'%s != %s after %s seconds' % (res, value, timeout)
) E AssertionError: False != True after 600 seconds /usr/lib/python2.7/site-packages/ovirtlago/testlib.py:252: AssertionError
Project: https://jenkins.ovirt.org/job/ovirt-system-tests_compat-4.3-suite-master/ Build: https://jenkins.ovirt.org/job/ovirt-system-tests_compat-4.3-suite-master/552... Build Number: 552 Build Status: Still Failing Triggered By: Started by timer ------------------------------------- Changes Since Last Success: ------------------------------------- Changes for Build #543 [Galit Rosenthal] For he master:Chg ssh to host Changes for Build #544 [Martin Necas] Ansible: fix collection path and use sed to rename [Ehud Yonasi] poll: remove metrics job. Changes for Build #545 [Galit Rosenthal] Remove he-basic-ipv6_suite_4.3 from check-patch Changes for Build #546 [Michal Skrivanek] silence new line output Changes for Build #547 [Michal Skrivanek] silence new line output Changes for Build #548 [Michal Skrivanek] silence new line output Changes for Build #549 [Michal Skrivanek] silence new line output Changes for Build #550 [Marcin Sobczyk] ost_utils: ansible: Always return the result Changes for Build #551 [Marcin Sobczyk] ost_utils: Extract assertions to a separate module Changes for Build #552 [Milan Zamazal] lagofy.sh: Fix check for accept_ra ----------------- Failed Tests: ----------------- 1 tests failed. FAILED: basic-suite-master.test-scenarios.004_basic_sanity_pytest.test_ha_recovery Error Message: AssertionError: False != True after 600 seconds Stack Trace: prefix = <ovirtlago.prefix.OvirtPrefix object at 0x7fe19b6c1f90> @order_by(_TEST_LIST) def test_ha_recovery(prefix): engine = prefix.virt_env.engine_vm().get_api_v4().system_service() with test_utils.TestEvent(engine, [119, 9602, 506]): # VM_DOWN_ERROR event(119) # HA_VM_FAILED event event(9602) # VDS_INITIATED_RUN_VM event(506) vm_host = _vm_host(prefix, VM2_NAME) pid = vm_host.ssh(['pgrep', '-f', 'qemu.*guest=vm2']) vm_host.ssh(['kill', '-KILL', pid.out.decode('utf-8')]) vm_service = test_utils.get_vm_service(engine, VM2_NAME) testlib.assert_true_within_long(
lambda:
vm_service.get().status == types.VmStatus.UP ) ../basic-suite-master/test-scenarios/004_basic_sanity_pytest.py:819: _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ /usr/lib/python2.7/site-packages/ovirtlago/testlib.py:286: in assert_true_within_long assert_equals_within_long(func, True, allowed_exceptions) /usr/lib/python2.7/site-packages/ovirtlago/testlib.py:273: in assert_equals_within_long func, value, LONG_TIMEOUT, allowed_exceptions=allowed_exceptions _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ func = <function <lambda> at 0x7fe19b9d3f50>, value = True, timeout = 600 allowed_exceptions = [], initial_wait = 0 def assert_equals_within( func, value, timeout, allowed_exceptions=None, initial_wait=10 ): allowed_exceptions = allowed_exceptions or [] with utils.EggTimer(timeout) as timer: while not timer.elapsed(): try: res = func() if res == value: return except Exception as exc: if _instance_of_any(exc, allowed_exceptions): time.sleep(3) continue LOGGER.exception("Unhandled exception in %s", func) raise if initial_wait == 0: time.sleep(3) else: time.sleep(initial_wait) initial_wait = 0 try: raise AssertionError(
'%s != %s after %s seconds' % (res, value, timeout)
) E AssertionError: False != True after 600 seconds /usr/lib/python2.7/site-packages/ovirtlago/testlib.py:252: AssertionError
Project: https://jenkins.ovirt.org/job/ovirt-system-tests_compat-4.3-suite-master/ Build: https://jenkins.ovirt.org/job/ovirt-system-tests_compat-4.3-suite-master/553... Build Number: 553 Build Status: Still Failing Triggered By: Started by timer ------------------------------------- Changes Since Last Success: ------------------------------------- Changes for Build #544 [Martin Necas] Ansible: fix collection path and use sed to rename [Ehud Yonasi] poll: remove metrics job. Changes for Build #545 [Galit Rosenthal] Remove he-basic-ipv6_suite_4.3 from check-patch Changes for Build #546 [Michal Skrivanek] silence new line output Changes for Build #547 [Michal Skrivanek] silence new line output Changes for Build #548 [Michal Skrivanek] silence new line output Changes for Build #549 [Michal Skrivanek] silence new line output Changes for Build #550 [Marcin Sobczyk] ost_utils: ansible: Always return the result Changes for Build #551 [Marcin Sobczyk] ost_utils: Extract assertions to a separate module Changes for Build #552 [Milan Zamazal] lagofy.sh: Fix check for accept_ra Changes for Build #553 [Marcin Sobczyk] ost_utils: network_utils: Add 'get_ips[6]' functions ----------------- Failed Tests: ----------------- 1 tests failed. FAILED: basic-suite-master.test-scenarios.004_basic_sanity_pytest.test_ha_recovery Error Message: AssertionError: False != True after 600 seconds Stack Trace: prefix = <ovirtlago.prefix.OvirtPrefix object at 0x7fb60050cc10> @order_by(_TEST_LIST) def test_ha_recovery(prefix): engine = prefix.virt_env.engine_vm().get_api_v4().system_service() with test_utils.TestEvent(engine, [119, 9602, 506]): # VM_DOWN_ERROR event(119) # HA_VM_FAILED event event(9602) # VDS_INITIATED_RUN_VM event(506) vm_host = _vm_host(prefix, VM2_NAME) pid = vm_host.ssh(['pgrep', '-f', 'qemu.*guest=vm2']) vm_host.ssh(['kill', '-KILL', pid.out.decode('utf-8')]) vm_service = test_utils.get_vm_service(engine, VM2_NAME) testlib.assert_true_within_long(
lambda:
vm_service.get().status == types.VmStatus.UP ) ../basic-suite-master/test-scenarios/004_basic_sanity_pytest.py:819: _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ /usr/lib/python2.7/site-packages/ovirtlago/testlib.py:286: in assert_true_within_long assert_equals_within_long(func, True, allowed_exceptions) /usr/lib/python2.7/site-packages/ovirtlago/testlib.py:273: in assert_equals_within_long func, value, LONG_TIMEOUT, allowed_exceptions=allowed_exceptions _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ func = <function <lambda> at 0x7fb6007fc398>, value = True, timeout = 600 allowed_exceptions = [], initial_wait = 0 def assert_equals_within( func, value, timeout, allowed_exceptions=None, initial_wait=10 ): allowed_exceptions = allowed_exceptions or [] with utils.EggTimer(timeout) as timer: while not timer.elapsed(): try: res = func() if res == value: return except Exception as exc: if _instance_of_any(exc, allowed_exceptions): time.sleep(3) continue LOGGER.exception("Unhandled exception in %s", func) raise if initial_wait == 0: time.sleep(3) else: time.sleep(initial_wait) initial_wait = 0 try: raise AssertionError(
'%s != %s after %s seconds' % (res, value, timeout)
) E AssertionError: False != True after 600 seconds /usr/lib/python2.7/site-packages/ovirtlago/testlib.py:252: AssertionError
Project: https://jenkins.ovirt.org/job/ovirt-system-tests_compat-4.3-suite-master/ Build: https://jenkins.ovirt.org/job/ovirt-system-tests_compat-4.3-suite-master/554... Build Number: 554 Build Status: Still Failing Triggered By: Started by timer ------------------------------------- Changes Since Last Success: ------------------------------------- Changes for Build #545 [Galit Rosenthal] Remove he-basic-ipv6_suite_4.3 from check-patch Changes for Build #546 [Michal Skrivanek] silence new line output Changes for Build #547 [Michal Skrivanek] silence new line output Changes for Build #548 [Michal Skrivanek] silence new line output Changes for Build #549 [Michal Skrivanek] silence new line output Changes for Build #550 [Marcin Sobczyk] ost_utils: ansible: Always return the result Changes for Build #551 [Marcin Sobczyk] ost_utils: Extract assertions to a separate module Changes for Build #552 [Milan Zamazal] lagofy.sh: Fix check for accept_ra Changes for Build #553 [Marcin Sobczyk] ost_utils: network_utils: Add 'get_ips[6]' functions Changes for Build #554 [Marcin Sobczyk] ost_utils: network_utils: Add 'get_ips[6]' functions ----------------- Failed Tests: ----------------- 1 tests failed. FAILED: basic-suite-master.test-scenarios.004_basic_sanity_pytest.test_ha_recovery Error Message: AssertionError: False != True after 600 seconds Stack Trace: prefix = <ovirtlago.prefix.OvirtPrefix object at 0x7f391d7a4c10> @order_by(_TEST_LIST) def test_ha_recovery(prefix): engine = prefix.virt_env.engine_vm().get_api_v4().system_service() with test_utils.TestEvent(engine, [119, 9602, 506]): # VM_DOWN_ERROR event(119) # HA_VM_FAILED event event(9602) # VDS_INITIATED_RUN_VM event(506) vm_host = _vm_host(prefix, VM2_NAME) pid = vm_host.ssh(['pgrep', '-f', 'qemu.*guest=vm2']) vm_host.ssh(['kill', '-KILL', pid.out.decode('utf-8')]) vm_service = test_utils.get_vm_service(engine, VM2_NAME) testlib.assert_true_within_long(
lambda:
vm_service.get().status == types.VmStatus.UP ) ../basic-suite-master/test-scenarios/004_basic_sanity_pytest.py:819: _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ /usr/lib/python2.7/site-packages/ovirtlago/testlib.py:286: in assert_true_within_long assert_equals_within_long(func, True, allowed_exceptions) /usr/lib/python2.7/site-packages/ovirtlago/testlib.py:273: in assert_equals_within_long func, value, LONG_TIMEOUT, allowed_exceptions=allowed_exceptions _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ func = <function <lambda> at 0x7f391d762320>, value = True, timeout = 600 allowed_exceptions = [], initial_wait = 0 def assert_equals_within( func, value, timeout, allowed_exceptions=None, initial_wait=10 ): allowed_exceptions = allowed_exceptions or [] with utils.EggTimer(timeout) as timer: while not timer.elapsed(): try: res = func() if res == value: return except Exception as exc: if _instance_of_any(exc, allowed_exceptions): time.sleep(3) continue LOGGER.exception("Unhandled exception in %s", func) raise if initial_wait == 0: time.sleep(3) else: time.sleep(initial_wait) initial_wait = 0 try: raise AssertionError(
'%s != %s after %s seconds' % (res, value, timeout)
) E AssertionError: False != True after 600 seconds /usr/lib/python2.7/site-packages/ovirtlago/testlib.py:252: AssertionError
Project: https://jenkins.ovirt.org/job/ovirt-system-tests_compat-4.3-suite-master/ Build: https://jenkins.ovirt.org/job/ovirt-system-tests_compat-4.3-suite-master/555... Build Number: 555 Build Status: Still Failing Triggered By: Started by timer ------------------------------------- Changes Since Last Success: ------------------------------------- Changes for Build #546 [Michal Skrivanek] silence new line output Changes for Build #547 [Michal Skrivanek] silence new line output Changes for Build #548 [Michal Skrivanek] silence new line output Changes for Build #549 [Michal Skrivanek] silence new line output Changes for Build #550 [Marcin Sobczyk] ost_utils: ansible: Always return the result Changes for Build #551 [Marcin Sobczyk] ost_utils: Extract assertions to a separate module Changes for Build #552 [Milan Zamazal] lagofy.sh: Fix check for accept_ra Changes for Build #553 [Marcin Sobczyk] ost_utils: network_utils: Add 'get_ips[6]' functions Changes for Build #554 [Marcin Sobczyk] ost_utils: network_utils: Add 'get_ips[6]' functions Changes for Build #555 [Marcin Sobczyk] ost_utils: network_utils: Add 'get_ips[6]' functions [Gal Ben Haim] dockerfile-utils: Better handling for nested repos ----------------- Failed Tests: ----------------- 1 tests failed. FAILED: basic-suite-master.test-scenarios.004_basic_sanity_pytest.test_ha_recovery Error Message: AssertionError: False != True after 600 seconds Stack Trace: prefix = <ovirtlago.prefix.OvirtPrefix object at 0x7fd2fe4c4c10> @order_by(_TEST_LIST) def test_ha_recovery(prefix): engine = prefix.virt_env.engine_vm().get_api_v4().system_service() with test_utils.TestEvent(engine, [119, 9602, 506]): # VM_DOWN_ERROR event(119) # HA_VM_FAILED event event(9602) # VDS_INITIATED_RUN_VM event(506) vm_host = _vm_host(prefix, VM2_NAME) pid = vm_host.ssh(['pgrep', '-f', 'qemu.*guest=vm2']) vm_host.ssh(['kill', '-KILL', pid.out.decode('utf-8')]) vm_service = test_utils.get_vm_service(engine, VM2_NAME) testlib.assert_true_within_long(
lambda:
vm_service.get().status == types.VmStatus.UP ) ../basic-suite-master/test-scenarios/004_basic_sanity_pytest.py:819: _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ /usr/lib/python2.7/site-packages/ovirtlago/testlib.py:286: in assert_true_within_long assert_equals_within_long(func, True, allowed_exceptions) /usr/lib/python2.7/site-packages/ovirtlago/testlib.py:273: in assert_equals_within_long func, value, LONG_TIMEOUT, allowed_exceptions=allowed_exceptions _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ func = <function <lambda> at 0x7fd2fc16b398>, value = True, timeout = 600 allowed_exceptions = [], initial_wait = 0 def assert_equals_within( func, value, timeout, allowed_exceptions=None, initial_wait=10 ): allowed_exceptions = allowed_exceptions or [] with utils.EggTimer(timeout) as timer: while not timer.elapsed(): try: res = func() if res == value: return except Exception as exc: if _instance_of_any(exc, allowed_exceptions): time.sleep(3) continue LOGGER.exception("Unhandled exception in %s", func) raise if initial_wait == 0: time.sleep(3) else: time.sleep(initial_wait) initial_wait = 0 try: raise AssertionError(
'%s != %s after %s seconds' % (res, value, timeout)
) E AssertionError: False != True after 600 seconds /usr/lib/python2.7/site-packages/ovirtlago/testlib.py:252: AssertionError
Project: https://jenkins.ovirt.org/job/ovirt-system-tests_compat-4.3-suite-master/ Build: https://jenkins.ovirt.org/job/ovirt-system-tests_compat-4.3-suite-master/556... Build Number: 556 Build Status: Still Failing Triggered By: Started by timer ------------------------------------- Changes Since Last Success: ------------------------------------- Changes for Build #547 [Michal Skrivanek] silence new line output Changes for Build #548 [Michal Skrivanek] silence new line output Changes for Build #549 [Michal Skrivanek] silence new line output Changes for Build #550 [Marcin Sobczyk] ost_utils: ansible: Always return the result Changes for Build #551 [Marcin Sobczyk] ost_utils: Extract assertions to a separate module Changes for Build #552 [Milan Zamazal] lagofy.sh: Fix check for accept_ra Changes for Build #553 [Marcin Sobczyk] ost_utils: network_utils: Add 'get_ips[6]' functions Changes for Build #554 [Marcin Sobczyk] ost_utils: network_utils: Add 'get_ips[6]' functions Changes for Build #555 [Marcin Sobczyk] ost_utils: network_utils: Add 'get_ips[6]' functions [Gal Ben Haim] dockerfile-utils: Better handling for nested repos Changes for Build #556 [Marcin Sobczyk] ost_utils: network_utils: Add 'get_ips[6]' functions ----------------- Failed Tests: ----------------- 1 tests failed. FAILED: basic-suite-master.test-scenarios.004_basic_sanity_pytest.test_ha_recovery Error Message: AssertionError: False != True after 600 seconds Stack Trace: prefix = <ovirtlago.prefix.OvirtPrefix object at 0x7f2e9ce4bc10> @order_by(_TEST_LIST) def test_ha_recovery(prefix): engine = prefix.virt_env.engine_vm().get_api_v4().system_service() with test_utils.TestEvent(engine, [119, 9602, 506]): # VM_DOWN_ERROR event(119) # HA_VM_FAILED event event(9602) # VDS_INITIATED_RUN_VM event(506) vm_host = _vm_host(prefix, VM2_NAME) pid = vm_host.ssh(['pgrep', '-f', 'qemu.*guest=vm2']) vm_host.ssh(['kill', '-KILL', pid.out.decode('utf-8')]) vm_service = test_utils.get_vm_service(engine, VM2_NAME) testlib.assert_true_within_long(
lambda:
vm_service.get().status == types.VmStatus.UP ) ../basic-suite-master/test-scenarios/004_basic_sanity_pytest.py:819: _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ /usr/lib/python2.7/site-packages/ovirtlago/testlib.py:286: in assert_true_within_long assert_equals_within_long(func, True, allowed_exceptions) /usr/lib/python2.7/site-packages/ovirtlago/testlib.py:273: in assert_equals_within_long func, value, LONG_TIMEOUT, allowed_exceptions=allowed_exceptions _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ func = <function <lambda> at 0x7f2e995dd488>, value = True, timeout = 600 allowed_exceptions = [], initial_wait = 0 def assert_equals_within( func, value, timeout, allowed_exceptions=None, initial_wait=10 ): allowed_exceptions = allowed_exceptions or [] with utils.EggTimer(timeout) as timer: while not timer.elapsed(): try: res = func() if res == value: return except Exception as exc: if _instance_of_any(exc, allowed_exceptions): time.sleep(3) continue LOGGER.exception("Unhandled exception in %s", func) raise if initial_wait == 0: time.sleep(3) else: time.sleep(initial_wait) initial_wait = 0 try: raise AssertionError(
'%s != %s after %s seconds' % (res, value, timeout)
) E AssertionError: False != True after 600 seconds /usr/lib/python2.7/site-packages/ovirtlago/testlib.py:252: AssertionError
Project: https://jenkins.ovirt.org/job/ovirt-system-tests_compat-4.3-suite-master/ Build: https://jenkins.ovirt.org/job/ovirt-system-tests_compat-4.3-suite-master/557... Build Number: 557 Build Status: Still Failing Triggered By: Started by timer ------------------------------------- Changes Since Last Success: ------------------------------------- Changes for Build #548 [Michal Skrivanek] silence new line output Changes for Build #549 [Michal Skrivanek] silence new line output Changes for Build #550 [Marcin Sobczyk] ost_utils: ansible: Always return the result Changes for Build #551 [Marcin Sobczyk] ost_utils: Extract assertions to a separate module Changes for Build #552 [Milan Zamazal] lagofy.sh: Fix check for accept_ra Changes for Build #553 [Marcin Sobczyk] ost_utils: network_utils: Add 'get_ips[6]' functions Changes for Build #554 [Marcin Sobczyk] ost_utils: network_utils: Add 'get_ips[6]' functions Changes for Build #555 [Marcin Sobczyk] ost_utils: network_utils: Add 'get_ips[6]' functions [Gal Ben Haim] dockerfile-utils: Better handling for nested repos Changes for Build #556 [Marcin Sobczyk] ost_utils: network_utils: Add 'get_ips[6]' functions Changes for Build #557 [Marcin Sobczyk] basic: Drop outdated code ----------------- Failed Tests: ----------------- 1 tests failed. FAILED: basic-suite-master.test-scenarios.004_basic_sanity_pytest.test_ha_recovery Error Message: AssertionError: False != True after 600 seconds Stack Trace: prefix = <ovirtlago.prefix.OvirtPrefix object at 0x7fabd34b0c10> @order_by(_TEST_LIST) def test_ha_recovery(prefix): engine = prefix.virt_env.engine_vm().get_api_v4().system_service() with test_utils.TestEvent(engine, [119, 9602, 506]): # VM_DOWN_ERROR event(119) # HA_VM_FAILED event event(9602) # VDS_INITIATED_RUN_VM event(506) vm_host = _vm_host(prefix, VM2_NAME) pid = vm_host.ssh(['pgrep', '-f', 'qemu.*guest=vm2']) vm_host.ssh(['kill', '-KILL', pid.out.decode('utf-8')]) vm_service = test_utils.get_vm_service(engine, VM2_NAME) testlib.assert_true_within_long(
lambda:
vm_service.get().status == types.VmStatus.UP ) ../basic-suite-master/test-scenarios/004_basic_sanity_pytest.py:819: _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ /usr/lib/python2.7/site-packages/ovirtlago/testlib.py:286: in assert_true_within_long assert_equals_within_long(func, True, allowed_exceptions) /usr/lib/python2.7/site-packages/ovirtlago/testlib.py:273: in assert_equals_within_long func, value, LONG_TIMEOUT, allowed_exceptions=allowed_exceptions _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ func = <function <lambda> at 0x7fabd3475578>, value = True, timeout = 600 allowed_exceptions = [], initial_wait = 0 def assert_equals_within( func, value, timeout, allowed_exceptions=None, initial_wait=10 ): allowed_exceptions = allowed_exceptions or [] with utils.EggTimer(timeout) as timer: while not timer.elapsed(): try: res = func() if res == value: return except Exception as exc: if _instance_of_any(exc, allowed_exceptions): time.sleep(3) continue LOGGER.exception("Unhandled exception in %s", func) raise if initial_wait == 0: time.sleep(3) else: time.sleep(initial_wait) initial_wait = 0 try: raise AssertionError(
'%s != %s after %s seconds' % (res, value, timeout)
) E AssertionError: False != True after 600 seconds /usr/lib/python2.7/site-packages/ovirtlago/testlib.py:252: AssertionError
Project: https://jenkins.ovirt.org/job/ovirt-system-tests_compat-4.3-suite-master/ Build: https://jenkins.ovirt.org/job/ovirt-system-tests_compat-4.3-suite-master/558... Build Number: 558 Build Status: Still Failing Triggered By: Started by timer ------------------------------------- Changes Since Last Success: ------------------------------------- Changes for Build #549 [Michal Skrivanek] silence new line output Changes for Build #550 [Marcin Sobczyk] ost_utils: ansible: Always return the result Changes for Build #551 [Marcin Sobczyk] ost_utils: Extract assertions to a separate module Changes for Build #552 [Milan Zamazal] lagofy.sh: Fix check for accept_ra Changes for Build #553 [Marcin Sobczyk] ost_utils: network_utils: Add 'get_ips[6]' functions Changes for Build #554 [Marcin Sobczyk] ost_utils: network_utils: Add 'get_ips[6]' functions Changes for Build #555 [Marcin Sobczyk] ost_utils: network_utils: Add 'get_ips[6]' functions [Gal Ben Haim] dockerfile-utils: Better handling for nested repos Changes for Build #556 [Marcin Sobczyk] ost_utils: network_utils: Add 'get_ips[6]' functions Changes for Build #557 [Marcin Sobczyk] basic: Drop outdated code Changes for Build #558 [Martin Necas] Ansible: remove nic_on_boot from cloud-init ----------------- Failed Tests: ----------------- 1 tests failed. FAILED: basic-suite-master.test-scenarios.004_basic_sanity_pytest.test_ha_recovery Error Message: AssertionError: False != True after 600 seconds Stack Trace: prefix = <ovirtlago.prefix.OvirtPrefix object at 0x7f9899dbac10> @order_by(_TEST_LIST) def test_ha_recovery(prefix): engine = prefix.virt_env.engine_vm().get_api_v4().system_service() with test_utils.TestEvent(engine, [119, 9602, 506]): # VM_DOWN_ERROR event(119) # HA_VM_FAILED event event(9602) # VDS_INITIATED_RUN_VM event(506) vm_host = _vm_host(prefix, VM2_NAME) pid = vm_host.ssh(['pgrep', '-f', 'qemu.*guest=vm2']) vm_host.ssh(['kill', '-KILL', pid.out.decode('utf-8')]) vm_service = test_utils.get_vm_service(engine, VM2_NAME) testlib.assert_true_within_long(
lambda:
vm_service.get().status == types.VmStatus.UP ) ../basic-suite-master/test-scenarios/004_basic_sanity_pytest.py:819: _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ /usr/lib/python2.7/site-packages/ovirtlago/testlib.py:286: in assert_true_within_long assert_equals_within_long(func, True, allowed_exceptions) /usr/lib/python2.7/site-packages/ovirtlago/testlib.py:273: in assert_equals_within_long func, value, LONG_TIMEOUT, allowed_exceptions=allowed_exceptions _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ func = <function <lambda> at 0x7f9899d78320>, value = True, timeout = 600 allowed_exceptions = [], initial_wait = 0 def assert_equals_within( func, value, timeout, allowed_exceptions=None, initial_wait=10 ): allowed_exceptions = allowed_exceptions or [] with utils.EggTimer(timeout) as timer: while not timer.elapsed(): try: res = func() if res == value: return except Exception as exc: if _instance_of_any(exc, allowed_exceptions): time.sleep(3) continue LOGGER.exception("Unhandled exception in %s", func) raise if initial_wait == 0: time.sleep(3) else: time.sleep(initial_wait) initial_wait = 0 try: raise AssertionError(
'%s != %s after %s seconds' % (res, value, timeout)
) E AssertionError: False != True after 600 seconds /usr/lib/python2.7/site-packages/ovirtlago/testlib.py:252: AssertionError
Project: https://jenkins.ovirt.org/job/ovirt-system-tests_compat-4.3-suite-master/ Build: https://jenkins.ovirt.org/job/ovirt-system-tests_compat-4.3-suite-master/559... Build Number: 559 Build Status: Still Failing Triggered By: Started by timer ------------------------------------- Changes Since Last Success: ------------------------------------- Changes for Build #550 [Marcin Sobczyk] ost_utils: ansible: Always return the result Changes for Build #551 [Marcin Sobczyk] ost_utils: Extract assertions to a separate module Changes for Build #552 [Milan Zamazal] lagofy.sh: Fix check for accept_ra Changes for Build #553 [Marcin Sobczyk] ost_utils: network_utils: Add 'get_ips[6]' functions Changes for Build #554 [Marcin Sobczyk] ost_utils: network_utils: Add 'get_ips[6]' functions Changes for Build #555 [Marcin Sobczyk] ost_utils: network_utils: Add 'get_ips[6]' functions [Gal Ben Haim] dockerfile-utils: Better handling for nested repos Changes for Build #556 [Marcin Sobczyk] ost_utils: network_utils: Add 'get_ips[6]' functions Changes for Build #557 [Marcin Sobczyk] basic: Drop outdated code Changes for Build #558 [Martin Necas] Ansible: remove nic_on_boot from cloud-init Changes for Build #559 [Milan Zamazal] basic: Disable smartcard temporarily [Ori Liel] Migrate model, metamodel projects to STDCIv2 ----------------- Failed Tests: ----------------- 1 tests failed. FAILED: basic-suite-master.test-scenarios.004_basic_sanity_pytest.test_ha_recovery Error Message: AssertionError: False != True after 600 seconds Stack Trace: prefix = <ovirtlago.prefix.OvirtPrefix object at 0x7fe7fb913c10> @order_by(_TEST_LIST) def test_ha_recovery(prefix): engine = prefix.virt_env.engine_vm().get_api_v4().system_service() with test_utils.TestEvent(engine, [119, 9602, 506]): # VM_DOWN_ERROR event(119) # HA_VM_FAILED event event(9602) # VDS_INITIATED_RUN_VM event(506) vm_host = _vm_host(prefix, VM2_NAME) pid = vm_host.ssh(['pgrep', '-f', 'qemu.*guest=vm2']) vm_host.ssh(['kill', '-KILL', pid.out.decode('utf-8')]) vm_service = test_utils.get_vm_service(engine, VM2_NAME) testlib.assert_true_within_long(
lambda:
vm_service.get().status == types.VmStatus.UP ) ../basic-suite-master/test-scenarios/004_basic_sanity_pytest.py:819: _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ /usr/lib/python2.7/site-packages/ovirtlago/testlib.py:286: in assert_true_within_long assert_equals_within_long(func, True, allowed_exceptions) /usr/lib/python2.7/site-packages/ovirtlago/testlib.py:273: in assert_equals_within_long func, value, LONG_TIMEOUT, allowed_exceptions=allowed_exceptions _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ func = <function <lambda> at 0x7fe7f83d0488>, value = True, timeout = 600 allowed_exceptions = [], initial_wait = 0 def assert_equals_within( func, value, timeout, allowed_exceptions=None, initial_wait=10 ): allowed_exceptions = allowed_exceptions or [] with utils.EggTimer(timeout) as timer: while not timer.elapsed(): try: res = func() if res == value: return except Exception as exc: if _instance_of_any(exc, allowed_exceptions): time.sleep(3) continue LOGGER.exception("Unhandled exception in %s", func) raise if initial_wait == 0: time.sleep(3) else: time.sleep(initial_wait) initial_wait = 0 try: raise AssertionError(
'%s != %s after %s seconds' % (res, value, timeout)
) E AssertionError: False != True after 600 seconds /usr/lib/python2.7/site-packages/ovirtlago/testlib.py:252: AssertionError
Project: https://jenkins.ovirt.org/job/ovirt-system-tests_compat-4.3-suite-master/ Build: https://jenkins.ovirt.org/job/ovirt-system-tests_compat-4.3-suite-master/560... Build Number: 560 Build Status: Still Failing Triggered By: Started by timer ------------------------------------- Changes Since Last Success: ------------------------------------- Changes for Build #551 [Marcin Sobczyk] ost_utils: Extract assertions to a separate module Changes for Build #552 [Milan Zamazal] lagofy.sh: Fix check for accept_ra Changes for Build #553 [Marcin Sobczyk] ost_utils: network_utils: Add 'get_ips[6]' functions Changes for Build #554 [Marcin Sobczyk] ost_utils: network_utils: Add 'get_ips[6]' functions Changes for Build #555 [Marcin Sobczyk] ost_utils: network_utils: Add 'get_ips[6]' functions [Gal Ben Haim] dockerfile-utils: Better handling for nested repos Changes for Build #556 [Marcin Sobczyk] ost_utils: network_utils: Add 'get_ips[6]' functions Changes for Build #557 [Marcin Sobczyk] basic: Drop outdated code Changes for Build #558 [Martin Necas] Ansible: remove nic_on_boot from cloud-init Changes for Build #559 [Milan Zamazal] basic: Disable smartcard temporarily [Ori Liel] Migrate model, metamodel projects to STDCIv2 Changes for Build #560 [Milan Zamazal] basic: Disable smartcard temporarily ----------------- Failed Tests: ----------------- 1 tests failed. FAILED: basic-suite-master.test-scenarios.004_basic_sanity_pytest.test_ha_recovery Error Message: AssertionError: False != True after 600 seconds Stack Trace: prefix = <ovirtlago.prefix.OvirtPrefix object at 0x7f5a4a546c10> @order_by(_TEST_LIST) def test_ha_recovery(prefix): engine = prefix.virt_env.engine_vm().get_api_v4().system_service() with test_utils.TestEvent(engine, [119, 9602, 506]): # VM_DOWN_ERROR event(119) # HA_VM_FAILED event event(9602) # VDS_INITIATED_RUN_VM event(506) vm_host = _vm_host(prefix, VM2_NAME) pid = vm_host.ssh(['pgrep', '-f', 'qemu.*guest=vm2']) vm_host.ssh(['kill', '-KILL', pid.out.decode('utf-8')]) vm_service = test_utils.get_vm_service(engine, VM2_NAME) testlib.assert_true_within_long(
lambda:
vm_service.get().status == types.VmStatus.UP ) ../basic-suite-master/test-scenarios/004_basic_sanity_pytest.py:819: _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ /usr/lib/python2.7/site-packages/ovirtlago/testlib.py:286: in assert_true_within_long assert_equals_within_long(func, True, allowed_exceptions) /usr/lib/python2.7/site-packages/ovirtlago/testlib.py:273: in assert_equals_within_long func, value, LONG_TIMEOUT, allowed_exceptions=allowed_exceptions _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ func = <function <lambda> at 0x7f5a42cc9398>, value = True, timeout = 600 allowed_exceptions = [], initial_wait = 0 def assert_equals_within( func, value, timeout, allowed_exceptions=None, initial_wait=10 ): allowed_exceptions = allowed_exceptions or [] with utils.EggTimer(timeout) as timer: while not timer.elapsed(): try: res = func() if res == value: return except Exception as exc: if _instance_of_any(exc, allowed_exceptions): time.sleep(3) continue LOGGER.exception("Unhandled exception in %s", func) raise if initial_wait == 0: time.sleep(3) else: time.sleep(initial_wait) initial_wait = 0 try: raise AssertionError(
'%s != %s after %s seconds' % (res, value, timeout)
) E AssertionError: False != True after 600 seconds /usr/lib/python2.7/site-packages/ovirtlago/testlib.py:252: AssertionError
Project: https://jenkins.ovirt.org/job/ovirt-system-tests_compat-4.3-suite-master/ Build: https://jenkins.ovirt.org/job/ovirt-system-tests_compat-4.3-suite-master/561... Build Number: 561 Build Status: Still Failing Triggered By: Started by timer ------------------------------------- Changes Since Last Success: ------------------------------------- Changes for Build #552 [Milan Zamazal] lagofy.sh: Fix check for accept_ra Changes for Build #553 [Marcin Sobczyk] ost_utils: network_utils: Add 'get_ips[6]' functions Changes for Build #554 [Marcin Sobczyk] ost_utils: network_utils: Add 'get_ips[6]' functions Changes for Build #555 [Marcin Sobczyk] ost_utils: network_utils: Add 'get_ips[6]' functions [Gal Ben Haim] dockerfile-utils: Better handling for nested repos Changes for Build #556 [Marcin Sobczyk] ost_utils: network_utils: Add 'get_ips[6]' functions Changes for Build #557 [Marcin Sobczyk] basic: Drop outdated code Changes for Build #558 [Martin Necas] Ansible: remove nic_on_boot from cloud-init Changes for Build #559 [Milan Zamazal] basic: Disable smartcard temporarily [Ori Liel] Migrate model, metamodel projects to STDCIv2 Changes for Build #560 [Milan Zamazal] basic: Disable smartcard temporarily Changes for Build #561 [Milan Zamazal] basic: Disable smartcard temporarily ----------------- Failed Tests: ----------------- 1 tests failed. FAILED: basic-suite-master.test-scenarios.004_basic_sanity_pytest.test_ha_recovery Error Message: AssertionError: False != True after 600 seconds Stack Trace: prefix = <ovirtlago.prefix.OvirtPrefix object at 0x7fec6a714c10> @order_by(_TEST_LIST) def test_ha_recovery(prefix): engine = prefix.virt_env.engine_vm().get_api_v4().system_service() with test_utils.TestEvent(engine, [119, 9602, 506]): # VM_DOWN_ERROR event(119) # HA_VM_FAILED event event(9602) # VDS_INITIATED_RUN_VM event(506) vm_host = _vm_host(prefix, VM2_NAME) pid = vm_host.ssh(['pgrep', '-f', 'qemu.*guest=vm2']) vm_host.ssh(['kill', '-KILL', pid.out.decode('utf-8')]) vm_service = test_utils.get_vm_service(engine, VM2_NAME) testlib.assert_true_within_long(
lambda:
vm_service.get().status == types.VmStatus.UP ) ../basic-suite-master/test-scenarios/004_basic_sanity_pytest.py:819: _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ /usr/lib/python2.7/site-packages/ovirtlago/testlib.py:286: in assert_true_within_long assert_equals_within_long(func, True, allowed_exceptions) /usr/lib/python2.7/site-packages/ovirtlago/testlib.py:273: in assert_equals_within_long func, value, LONG_TIMEOUT, allowed_exceptions=allowed_exceptions _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ func = <function <lambda> at 0x7fec6a6d2320>, value = True, timeout = 600 allowed_exceptions = [], initial_wait = 0 def assert_equals_within( func, value, timeout, allowed_exceptions=None, initial_wait=10 ): allowed_exceptions = allowed_exceptions or [] with utils.EggTimer(timeout) as timer: while not timer.elapsed(): try: res = func() if res == value: return except Exception as exc: if _instance_of_any(exc, allowed_exceptions): time.sleep(3) continue LOGGER.exception("Unhandled exception in %s", func) raise if initial_wait == 0: time.sleep(3) else: time.sleep(initial_wait) initial_wait = 0 try: raise AssertionError(
'%s != %s after %s seconds' % (res, value, timeout)
) E AssertionError: False != True after 600 seconds /usr/lib/python2.7/site-packages/ovirtlago/testlib.py:252: AssertionError
Project: https://jenkins.ovirt.org/job/ovirt-system-tests_compat-4.3-suite-master/ Build: https://jenkins.ovirt.org/job/ovirt-system-tests_compat-4.3-suite-master/562... Build Number: 562 Build Status: Still Failing Triggered By: Started by timer ------------------------------------- Changes Since Last Success: ------------------------------------- Changes for Build #553 [Marcin Sobczyk] ost_utils: network_utils: Add 'get_ips[6]' functions Changes for Build #554 [Marcin Sobczyk] ost_utils: network_utils: Add 'get_ips[6]' functions Changes for Build #555 [Marcin Sobczyk] ost_utils: network_utils: Add 'get_ips[6]' functions [Gal Ben Haim] dockerfile-utils: Better handling for nested repos Changes for Build #556 [Marcin Sobczyk] ost_utils: network_utils: Add 'get_ips[6]' functions Changes for Build #557 [Marcin Sobczyk] basic: Drop outdated code Changes for Build #558 [Martin Necas] Ansible: remove nic_on_boot from cloud-init Changes for Build #559 [Milan Zamazal] basic: Disable smartcard temporarily [Ori Liel] Migrate model, metamodel projects to STDCIv2 Changes for Build #560 [Milan Zamazal] basic: Disable smartcard temporarily Changes for Build #561 [Milan Zamazal] basic: Disable smartcard temporarily Changes for Build #562 [Milan Zamazal] basic: Disable smartcard temporarily ----------------- Failed Tests: ----------------- 1 tests failed. FAILED: basic-suite-master.test-scenarios.004_basic_sanity_pytest.test_ha_recovery Error Message: AssertionError: False != True after 600 seconds Stack Trace: prefix = <ovirtlago.prefix.OvirtPrefix object at 0x7f3361e0dc10> @order_by(_TEST_LIST) def test_ha_recovery(prefix): engine = prefix.virt_env.engine_vm().get_api_v4().system_service() with test_utils.TestEvent(engine, [119, 9602, 506]): # VM_DOWN_ERROR event(119) # HA_VM_FAILED event event(9602) # VDS_INITIATED_RUN_VM event(506) vm_host = _vm_host(prefix, VM2_NAME) pid = vm_host.ssh(['pgrep', '-f', 'qemu.*guest=vm2']) vm_host.ssh(['kill', '-KILL', pid.out.decode('utf-8')]) vm_service = test_utils.get_vm_service(engine, VM2_NAME) testlib.assert_true_within_long(
lambda:
vm_service.get().status == types.VmStatus.UP ) ../basic-suite-master/test-scenarios/004_basic_sanity_pytest.py:819: _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ /usr/lib/python2.7/site-packages/ovirtlago/testlib.py:286: in assert_true_within_long assert_equals_within_long(func, True, allowed_exceptions) /usr/lib/python2.7/site-packages/ovirtlago/testlib.py:273: in assert_equals_within_long func, value, LONG_TIMEOUT, allowed_exceptions=allowed_exceptions _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ func = <function <lambda> at 0x7f3361dcb500>, value = True, timeout = 600 allowed_exceptions = [], initial_wait = 0 def assert_equals_within( func, value, timeout, allowed_exceptions=None, initial_wait=10 ): allowed_exceptions = allowed_exceptions or [] with utils.EggTimer(timeout) as timer: while not timer.elapsed(): try: res = func() if res == value: return except Exception as exc: if _instance_of_any(exc, allowed_exceptions): time.sleep(3) continue LOGGER.exception("Unhandled exception in %s", func) raise if initial_wait == 0: time.sleep(3) else: time.sleep(initial_wait) initial_wait = 0 try: raise AssertionError(
'%s != %s after %s seconds' % (res, value, timeout)
) E AssertionError: False != True after 600 seconds /usr/lib/python2.7/site-packages/ovirtlago/testlib.py:252: AssertionError
Project: https://jenkins.ovirt.org/job/ovirt-system-tests_compat-4.3-suite-master/ Build: https://jenkins.ovirt.org/job/ovirt-system-tests_compat-4.3-suite-master/563... Build Number: 563 Build Status: Still Failing Triggered By: Started by timer ------------------------------------- Changes Since Last Success: ------------------------------------- Changes for Build #554 [Marcin Sobczyk] ost_utils: network_utils: Add 'get_ips[6]' functions Changes for Build #555 [Marcin Sobczyk] ost_utils: network_utils: Add 'get_ips[6]' functions [Gal Ben Haim] dockerfile-utils: Better handling for nested repos Changes for Build #556 [Marcin Sobczyk] ost_utils: network_utils: Add 'get_ips[6]' functions Changes for Build #557 [Marcin Sobczyk] basic: Drop outdated code Changes for Build #558 [Martin Necas] Ansible: remove nic_on_boot from cloud-init Changes for Build #559 [Milan Zamazal] basic: Disable smartcard temporarily [Ori Liel] Migrate model, metamodel projects to STDCIv2 Changes for Build #560 [Milan Zamazal] basic: Disable smartcard temporarily Changes for Build #561 [Milan Zamazal] basic: Disable smartcard temporarily Changes for Build #562 [Milan Zamazal] basic: Disable smartcard temporarily Changes for Build #563 [Milan Zamazal] basic: Disable smartcard temporarily ----------------- Failed Tests: ----------------- 1 tests failed. FAILED: basic-suite-master.test-scenarios.004_basic_sanity_pytest.test_ha_recovery Error Message: AssertionError: False != True after 600 seconds Stack Trace: prefix = <ovirtlago.prefix.OvirtPrefix object at 0x7fd67530fc10> @order_by(_TEST_LIST) def test_ha_recovery(prefix): engine = prefix.virt_env.engine_vm().get_api_v4().system_service() with test_utils.TestEvent(engine, [119, 9602, 506]): # VM_DOWN_ERROR event(119) # HA_VM_FAILED event event(9602) # VDS_INITIATED_RUN_VM event(506) vm_host = _vm_host(prefix, VM2_NAME) pid = vm_host.ssh(['pgrep', '-f', 'qemu.*guest=vm2']) vm_host.ssh(['kill', '-KILL', pid.out.decode('utf-8')]) vm_service = test_utils.get_vm_service(engine, VM2_NAME) testlib.assert_true_within_long(
lambda:
vm_service.get().status == types.VmStatus.UP ) ../basic-suite-master/test-scenarios/004_basic_sanity_pytest.py:819: _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ /usr/lib/python2.7/site-packages/ovirtlago/testlib.py:286: in assert_true_within_long assert_equals_within_long(func, True, allowed_exceptions) /usr/lib/python2.7/site-packages/ovirtlago/testlib.py:273: in assert_equals_within_long func, value, LONG_TIMEOUT, allowed_exceptions=allowed_exceptions _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ func = <function <lambda> at 0x7fd6752cd320>, value = True, timeout = 600 allowed_exceptions = [], initial_wait = 0 def assert_equals_within( func, value, timeout, allowed_exceptions=None, initial_wait=10 ): allowed_exceptions = allowed_exceptions or [] with utils.EggTimer(timeout) as timer: while not timer.elapsed(): try: res = func() if res == value: return except Exception as exc: if _instance_of_any(exc, allowed_exceptions): time.sleep(3) continue LOGGER.exception("Unhandled exception in %s", func) raise if initial_wait == 0: time.sleep(3) else: time.sleep(initial_wait) initial_wait = 0 try: raise AssertionError(
'%s != %s after %s seconds' % (res, value, timeout)
) E AssertionError: False != True after 600 seconds /usr/lib/python2.7/site-packages/ovirtlago/testlib.py:252: AssertionError
Project: https://jenkins.ovirt.org/job/ovirt-system-tests_compat-4.3-suite-master/ Build: https://jenkins.ovirt.org/job/ovirt-system-tests_compat-4.3-suite-master/564... Build Number: 564 Build Status: Still Failing Triggered By: Started by timer ------------------------------------- Changes Since Last Success: ------------------------------------- Changes for Build #555 [Marcin Sobczyk] ost_utils: network_utils: Add 'get_ips[6]' functions [Gal Ben Haim] dockerfile-utils: Better handling for nested repos Changes for Build #556 [Marcin Sobczyk] ost_utils: network_utils: Add 'get_ips[6]' functions Changes for Build #557 [Marcin Sobczyk] basic: Drop outdated code Changes for Build #558 [Martin Necas] Ansible: remove nic_on_boot from cloud-init Changes for Build #559 [Milan Zamazal] basic: Disable smartcard temporarily [Ori Liel] Migrate model, metamodel projects to STDCIv2 Changes for Build #560 [Milan Zamazal] basic: Disable smartcard temporarily Changes for Build #561 [Milan Zamazal] basic: Disable smartcard temporarily Changes for Build #562 [Milan Zamazal] basic: Disable smartcard temporarily Changes for Build #563 [Milan Zamazal] basic: Disable smartcard temporarily Changes for Build #564 [Galit Rosenthal] Rm he basic iscsi4.3 he node ng 4.3 from check_patch ----------------- Failed Tests: ----------------- 1 tests failed. FAILED: basic-suite-master.test-scenarios.004_basic_sanity_pytest.test_ha_recovery Error Message: AssertionError: False != True after 600 seconds Stack Trace: prefix = <ovirtlago.prefix.OvirtPrefix object at 0x7f0ddf066c10> @order_by(_TEST_LIST) def test_ha_recovery(prefix): engine = prefix.virt_env.engine_vm().get_api_v4().system_service() with test_utils.TestEvent(engine, [119, 9602, 506]): # VM_DOWN_ERROR event(119) # HA_VM_FAILED event event(9602) # VDS_INITIATED_RUN_VM event(506) vm_host = _vm_host(prefix, VM2_NAME) pid = vm_host.ssh(['pgrep', '-f', 'qemu.*guest=vm2']) vm_host.ssh(['kill', '-KILL', pid.out.decode('utf-8')]) vm_service = test_utils.get_vm_service(engine, VM2_NAME) testlib.assert_true_within_long(
lambda:
vm_service.get().status == types.VmStatus.UP ) ../basic-suite-master/test-scenarios/004_basic_sanity_pytest.py:819: _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ /usr/lib/python2.7/site-packages/ovirtlago/testlib.py:286: in assert_true_within_long assert_equals_within_long(func, True, allowed_exceptions) /usr/lib/python2.7/site-packages/ovirtlago/testlib.py:273: in assert_equals_within_long func, value, LONG_TIMEOUT, allowed_exceptions=allowed_exceptions _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ func = <function <lambda> at 0x7f0dd77f1488>, value = True, timeout = 600 allowed_exceptions = [], initial_wait = 0 def assert_equals_within( func, value, timeout, allowed_exceptions=None, initial_wait=10 ): allowed_exceptions = allowed_exceptions or [] with utils.EggTimer(timeout) as timer: while not timer.elapsed(): try: res = func() if res == value: return except Exception as exc: if _instance_of_any(exc, allowed_exceptions): time.sleep(3) continue LOGGER.exception("Unhandled exception in %s", func) raise if initial_wait == 0: time.sleep(3) else: time.sleep(initial_wait) initial_wait = 0 try: raise AssertionError(
'%s != %s after %s seconds' % (res, value, timeout)
) E AssertionError: False != True after 600 seconds /usr/lib/python2.7/site-packages/ovirtlago/testlib.py:252: AssertionError
Project: https://jenkins.ovirt.org/job/ovirt-system-tests_compat-4.3-suite-master/ Build: https://jenkins.ovirt.org/job/ovirt-system-tests_compat-4.3-suite-master/565... Build Number: 565 Build Status: Still Failing Triggered By: Started by timer ------------------------------------- Changes Since Last Success: ------------------------------------- Changes for Build #556 [Marcin Sobczyk] ost_utils: network_utils: Add 'get_ips[6]' functions Changes for Build #557 [Marcin Sobczyk] basic: Drop outdated code Changes for Build #558 [Martin Necas] Ansible: remove nic_on_boot from cloud-init Changes for Build #559 [Milan Zamazal] basic: Disable smartcard temporarily [Ori Liel] Migrate model, metamodel projects to STDCIv2 Changes for Build #560 [Milan Zamazal] basic: Disable smartcard temporarily Changes for Build #561 [Milan Zamazal] basic: Disable smartcard temporarily Changes for Build #562 [Milan Zamazal] basic: Disable smartcard temporarily Changes for Build #563 [Milan Zamazal] basic: Disable smartcard temporarily Changes for Build #564 [Galit Rosenthal] Rm he basic iscsi4.3 he node ng 4.3 from check_patch Changes for Build #565 [Sandro Bonazzola] yum-repos: cbs.centos.org repos doesn't exist anymore ----------------- Failed Tests: ----------------- 1 tests failed. FAILED: basic-suite-master.test-scenarios.004_basic_sanity_pytest.test_ha_recovery Error Message: AssertionError: False != True after 600 seconds Stack Trace: prefix = <ovirtlago.prefix.OvirtPrefix object at 0x7fb543f75c10> @order_by(_TEST_LIST) def test_ha_recovery(prefix): engine = prefix.virt_env.engine_vm().get_api_v4().system_service() with test_utils.TestEvent(engine, [119, 9602, 506]): # VM_DOWN_ERROR event(119) # HA_VM_FAILED event event(9602) # VDS_INITIATED_RUN_VM event(506) vm_host = _vm_host(prefix, VM2_NAME) pid = vm_host.ssh(['pgrep', '-f', 'qemu.*guest=vm2']) vm_host.ssh(['kill', '-KILL', pid.out.decode('utf-8')]) vm_service = test_utils.get_vm_service(engine, VM2_NAME) testlib.assert_true_within_long(
lambda:
vm_service.get().status == types.VmStatus.UP ) ../basic-suite-master/test-scenarios/004_basic_sanity_pytest.py:819: _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ /usr/lib/python2.7/site-packages/ovirtlago/testlib.py:286: in assert_true_within_long assert_equals_within_long(func, True, allowed_exceptions) /usr/lib/python2.7/site-packages/ovirtlago/testlib.py:273: in assert_equals_within_long func, value, LONG_TIMEOUT, allowed_exceptions=allowed_exceptions _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ func = <function <lambda> at 0x7fb54070a398>, value = True, timeout = 600 allowed_exceptions = [], initial_wait = 0 def assert_equals_within( func, value, timeout, allowed_exceptions=None, initial_wait=10 ): allowed_exceptions = allowed_exceptions or [] with utils.EggTimer(timeout) as timer: while not timer.elapsed(): try: res = func() if res == value: return except Exception as exc: if _instance_of_any(exc, allowed_exceptions): time.sleep(3) continue LOGGER.exception("Unhandled exception in %s", func) raise if initial_wait == 0: time.sleep(3) else: time.sleep(initial_wait) initial_wait = 0 try: raise AssertionError(
'%s != %s after %s seconds' % (res, value, timeout)
) E AssertionError: False != True after 600 seconds /usr/lib/python2.7/site-packages/ovirtlago/testlib.py:252: AssertionError
Project: https://jenkins.ovirt.org/job/ovirt-system-tests_compat-4.3-suite-master/ Build: https://jenkins.ovirt.org/job/ovirt-system-tests_compat-4.3-suite-master/566... Build Number: 566 Build Status: Still Failing Triggered By: Started by timer ------------------------------------- Changes Since Last Success: ------------------------------------- Changes for Build #557 [Marcin Sobczyk] basic: Drop outdated code Changes for Build #558 [Martin Necas] Ansible: remove nic_on_boot from cloud-init Changes for Build #559 [Milan Zamazal] basic: Disable smartcard temporarily [Ori Liel] Migrate model, metamodel projects to STDCIv2 Changes for Build #560 [Milan Zamazal] basic: Disable smartcard temporarily Changes for Build #561 [Milan Zamazal] basic: Disable smartcard temporarily Changes for Build #562 [Milan Zamazal] basic: Disable smartcard temporarily Changes for Build #563 [Milan Zamazal] basic: Disable smartcard temporarily Changes for Build #564 [Galit Rosenthal] Rm he basic iscsi4.3 he node ng 4.3 from check_patch Changes for Build #565 [Sandro Bonazzola] yum-repos: cbs.centos.org repos doesn't exist anymore Changes for Build #566 [Sandro Bonazzola] yum-repos: cbs.centos.org repos doesn't exist anymore [Gal Ben Haim] dockerfile-utils: Specify authfile [Gal Ben Haim] dockerfile-utils: Skopeo change args order ----------------- Failed Tests: ----------------- No tests ran.
Project: https://jenkins.ovirt.org/job/ovirt-system-tests_compat-4.3-suite-master/ Build: https://jenkins.ovirt.org/job/ovirt-system-tests_compat-4.3-suite-master/567... Build Number: 567 Build Status: Still Failing Triggered By: Started by timer ------------------------------------- Changes Since Last Success: ------------------------------------- Changes for Build #558 [Martin Necas] Ansible: remove nic_on_boot from cloud-init Changes for Build #559 [Milan Zamazal] basic: Disable smartcard temporarily [Ori Liel] Migrate model, metamodel projects to STDCIv2 Changes for Build #560 [Milan Zamazal] basic: Disable smartcard temporarily Changes for Build #561 [Milan Zamazal] basic: Disable smartcard temporarily Changes for Build #562 [Milan Zamazal] basic: Disable smartcard temporarily Changes for Build #563 [Milan Zamazal] basic: Disable smartcard temporarily Changes for Build #564 [Galit Rosenthal] Rm he basic iscsi4.3 he node ng 4.3 from check_patch Changes for Build #565 [Sandro Bonazzola] yum-repos: cbs.centos.org repos doesn't exist anymore Changes for Build #566 [Sandro Bonazzola] yum-repos: cbs.centos.org repos doesn't exist anymore [Gal Ben Haim] dockerfile-utils: Specify authfile [Gal Ben Haim] dockerfile-utils: Skopeo change args order Changes for Build #567 [Marcin Sobczyk] basic: Remove remaining lago deps from '002_bootstrap_pytest' ----------------- Failed Tests: ----------------- 1 tests failed. FAILED: basic-suite-master.test-scenarios.004_basic_sanity_pytest.test_ha_recovery Error Message: AssertionError: False != True after 600 seconds Stack Trace: prefix = <ovirtlago.prefix.OvirtPrefix object at 0x7f5623a3ac10> @order_by(_TEST_LIST) def test_ha_recovery(prefix): engine = prefix.virt_env.engine_vm().get_api_v4().system_service() with test_utils.TestEvent(engine, [119, 9602, 506]): # VM_DOWN_ERROR event(119) # HA_VM_FAILED event event(9602) # VDS_INITIATED_RUN_VM event(506) vm_host = _vm_host(prefix, VM2_NAME) pid = vm_host.ssh(['pgrep', '-f', 'qemu.*guest=vm2']) vm_host.ssh(['kill', '-KILL', pid.out.decode('utf-8')]) vm_service = test_utils.get_vm_service(engine, VM2_NAME) testlib.assert_true_within_long(
lambda:
vm_service.get().status == types.VmStatus.UP ) ../basic-suite-master/test-scenarios/004_basic_sanity_pytest.py:819: _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ /usr/lib/python2.7/site-packages/ovirtlago/testlib.py:286: in assert_true_within_long assert_equals_within_long(func, True, allowed_exceptions) /usr/lib/python2.7/site-packages/ovirtlago/testlib.py:273: in assert_equals_within_long func, value, LONG_TIMEOUT, allowed_exceptions=allowed_exceptions _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ func = <function <lambda> at 0x7f56201cf2a8>, value = True, timeout = 600 allowed_exceptions = [], initial_wait = 0 def assert_equals_within( func, value, timeout, allowed_exceptions=None, initial_wait=10 ): allowed_exceptions = allowed_exceptions or [] with utils.EggTimer(timeout) as timer: while not timer.elapsed(): try: res = func() if res == value: return except Exception as exc: if _instance_of_any(exc, allowed_exceptions): time.sleep(3) continue LOGGER.exception("Unhandled exception in %s", func) raise if initial_wait == 0: time.sleep(3) else: time.sleep(initial_wait) initial_wait = 0 try: raise AssertionError(
'%s != %s after %s seconds' % (res, value, timeout)
) E AssertionError: False != True after 600 seconds /usr/lib/python2.7/site-packages/ovirtlago/testlib.py:252: AssertionError
Project: https://jenkins.ovirt.org/job/ovirt-system-tests_compat-4.3-suite-master/ Build: https://jenkins.ovirt.org/job/ovirt-system-tests_compat-4.3-suite-master/568... Build Number: 568 Build Status: Still Failing Triggered By: Started by timer ------------------------------------- Changes Since Last Success: ------------------------------------- Changes for Build #559 [Milan Zamazal] basic: Disable smartcard temporarily [Ori Liel] Migrate model, metamodel projects to STDCIv2 Changes for Build #560 [Milan Zamazal] basic: Disable smartcard temporarily Changes for Build #561 [Milan Zamazal] basic: Disable smartcard temporarily Changes for Build #562 [Milan Zamazal] basic: Disable smartcard temporarily Changes for Build #563 [Milan Zamazal] basic: Disable smartcard temporarily Changes for Build #564 [Galit Rosenthal] Rm he basic iscsi4.3 he node ng 4.3 from check_patch Changes for Build #565 [Sandro Bonazzola] yum-repos: cbs.centos.org repos doesn't exist anymore Changes for Build #566 [Sandro Bonazzola] yum-repos: cbs.centos.org repos doesn't exist anymore [Gal Ben Haim] dockerfile-utils: Specify authfile [Gal Ben Haim] dockerfile-utils: Skopeo change args order Changes for Build #567 [Marcin Sobczyk] basic: Remove remaining lago deps from '002_bootstrap_pytest' Changes for Build #568 [Marcin Sobczyk] basic: Remove remaining lago deps from '002_bootstrap_pytest' ----------------- Failed Tests: ----------------- 1 tests failed. FAILED: basic-suite-master.test-scenarios.004_basic_sanity_pytest.test_ha_recovery Error Message: AssertionError: False != True after 600 seconds Stack Trace: prefix = <ovirtlago.prefix.OvirtPrefix object at 0x7f2fe3c90c10> @order_by(_TEST_LIST) def test_ha_recovery(prefix): engine = prefix.virt_env.engine_vm().get_api_v4().system_service() with test_utils.TestEvent(engine, [119, 9602, 506]): # VM_DOWN_ERROR event(119) # HA_VM_FAILED event event(9602) # VDS_INITIATED_RUN_VM event(506) vm_host = _vm_host(prefix, VM2_NAME) pid = vm_host.ssh(['pgrep', '-f', 'qemu.*guest=vm2']) vm_host.ssh(['kill', '-KILL', pid.out.decode('utf-8')]) vm_service = test_utils.get_vm_service(engine, VM2_NAME) testlib.assert_true_within_long(
lambda:
vm_service.get().status == types.VmStatus.UP ) ../basic-suite-master/test-scenarios/004_basic_sanity_pytest.py:819: _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ /usr/lib/python2.7/site-packages/ovirtlago/testlib.py:286: in assert_true_within_long assert_equals_within_long(func, True, allowed_exceptions) /usr/lib/python2.7/site-packages/ovirtlago/testlib.py:273: in assert_equals_within_long func, value, LONG_TIMEOUT, allowed_exceptions=allowed_exceptions _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ func = <function <lambda> at 0x7f2fe0750398>, value = True, timeout = 600 allowed_exceptions = [], initial_wait = 0 def assert_equals_within( func, value, timeout, allowed_exceptions=None, initial_wait=10 ): allowed_exceptions = allowed_exceptions or [] with utils.EggTimer(timeout) as timer: while not timer.elapsed(): try: res = func() if res == value: return except Exception as exc: if _instance_of_any(exc, allowed_exceptions): time.sleep(3) continue LOGGER.exception("Unhandled exception in %s", func) raise if initial_wait == 0: time.sleep(3) else: time.sleep(initial_wait) initial_wait = 0 try: raise AssertionError(
'%s != %s after %s seconds' % (res, value, timeout)
) E AssertionError: False != True after 600 seconds /usr/lib/python2.7/site-packages/ovirtlago/testlib.py:252: AssertionError
Project: https://jenkins.ovirt.org/job/ovirt-system-tests_compat-4.3-suite-master/ Build: https://jenkins.ovirt.org/job/ovirt-system-tests_compat-4.3-suite-master/569... Build Number: 569 Build Status: Still Failing Triggered By: Started by timer ------------------------------------- Changes Since Last Success: ------------------------------------- Changes for Build #560 [Milan Zamazal] basic: Disable smartcard temporarily Changes for Build #561 [Milan Zamazal] basic: Disable smartcard temporarily Changes for Build #562 [Milan Zamazal] basic: Disable smartcard temporarily Changes for Build #563 [Milan Zamazal] basic: Disable smartcard temporarily Changes for Build #564 [Galit Rosenthal] Rm he basic iscsi4.3 he node ng 4.3 from check_patch Changes for Build #565 [Sandro Bonazzola] yum-repos: cbs.centos.org repos doesn't exist anymore Changes for Build #566 [Sandro Bonazzola] yum-repos: cbs.centos.org repos doesn't exist anymore [Gal Ben Haim] dockerfile-utils: Specify authfile [Gal Ben Haim] dockerfile-utils: Skopeo change args order Changes for Build #567 [Marcin Sobczyk] basic: Remove remaining lago deps from '002_bootstrap_pytest' Changes for Build #568 [Marcin Sobczyk] basic: Remove remaining lago deps from '002_bootstrap_pytest' Changes for Build #569 [Marcin Sobczyk] basic: Remove remaining lago deps from '002_bootstrap_pytest' ----------------- Failed Tests: ----------------- 1 tests failed. FAILED: basic-suite-master.test-scenarios.004_basic_sanity_pytest.test_ha_recovery Error Message: AssertionError: False != True after 600 seconds Stack Trace: prefix = <ovirtlago.prefix.OvirtPrefix object at 0x7fa131614c10> @order_by(_TEST_LIST) def test_ha_recovery(prefix): engine = prefix.virt_env.engine_vm().get_api_v4().system_service() with test_utils.TestEvent(engine, [119, 9602, 506]): # VM_DOWN_ERROR event(119) # HA_VM_FAILED event event(9602) # VDS_INITIATED_RUN_VM event(506) vm_host = _vm_host(prefix, VM2_NAME) pid = vm_host.ssh(['pgrep', '-f', 'qemu.*guest=vm2']) vm_host.ssh(['kill', '-KILL', pid.out.decode('utf-8')]) vm_service = test_utils.get_vm_service(engine, VM2_NAME) testlib.assert_true_within_long(
lambda:
vm_service.get().status == types.VmStatus.UP ) ../basic-suite-master/test-scenarios/004_basic_sanity_pytest.py:819: _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ /usr/lib/python2.7/site-packages/ovirtlago/testlib.py:286: in assert_true_within_long assert_equals_within_long(func, True, allowed_exceptions) /usr/lib/python2.7/site-packages/ovirtlago/testlib.py:273: in assert_equals_within_long func, value, LONG_TIMEOUT, allowed_exceptions=allowed_exceptions _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ func = <function <lambda> at 0x7fa13006d488>, value = True, timeout = 600 allowed_exceptions = [], initial_wait = 0 def assert_equals_within( func, value, timeout, allowed_exceptions=None, initial_wait=10 ): allowed_exceptions = allowed_exceptions or [] with utils.EggTimer(timeout) as timer: while not timer.elapsed(): try: res = func() if res == value: return except Exception as exc: if _instance_of_any(exc, allowed_exceptions): time.sleep(3) continue LOGGER.exception("Unhandled exception in %s", func) raise if initial_wait == 0: time.sleep(3) else: time.sleep(initial_wait) initial_wait = 0 try: raise AssertionError(
'%s != %s after %s seconds' % (res, value, timeout)
) E AssertionError: False != True after 600 seconds /usr/lib/python2.7/site-packages/ovirtlago/testlib.py:252: AssertionError
Project: https://jenkins.ovirt.org/job/ovirt-system-tests_compat-4.3-suite-master/ Build: https://jenkins.ovirt.org/job/ovirt-system-tests_compat-4.3-suite-master/570... Build Number: 570 Build Status: Still Failing Triggered By: Started by timer ------------------------------------- Changes Since Last Success: ------------------------------------- Changes for Build #561 [Milan Zamazal] basic: Disable smartcard temporarily Changes for Build #562 [Milan Zamazal] basic: Disable smartcard temporarily Changes for Build #563 [Milan Zamazal] basic: Disable smartcard temporarily Changes for Build #564 [Galit Rosenthal] Rm he basic iscsi4.3 he node ng 4.3 from check_patch Changes for Build #565 [Sandro Bonazzola] yum-repos: cbs.centos.org repos doesn't exist anymore Changes for Build #566 [Sandro Bonazzola] yum-repos: cbs.centos.org repos doesn't exist anymore [Gal Ben Haim] dockerfile-utils: Specify authfile [Gal Ben Haim] dockerfile-utils: Skopeo change args order Changes for Build #567 [Marcin Sobczyk] basic: Remove remaining lago deps from '002_bootstrap_pytest' Changes for Build #568 [Marcin Sobczyk] basic: Remove remaining lago deps from '002_bootstrap_pytest' Changes for Build #569 [Marcin Sobczyk] basic: Remove remaining lago deps from '002_bootstrap_pytest' Changes for Build #570 [Marcin Sobczyk] basic: Remove remaining lago deps from '002_bootstrap_pytest' ----------------- Failed Tests: ----------------- 1 tests failed. FAILED: basic-suite-master.test-scenarios.004_basic_sanity_pytest.test_ha_recovery Error Message: AssertionError: False != True after 600 seconds Stack Trace: prefix = <ovirtlago.prefix.OvirtPrefix object at 0x7f07bf474c10> @order_by(_TEST_LIST) def test_ha_recovery(prefix): engine = prefix.virt_env.engine_vm().get_api_v4().system_service() with test_utils.TestEvent(engine, [119, 9602, 506]): # VM_DOWN_ERROR event(119) # HA_VM_FAILED event event(9602) # VDS_INITIATED_RUN_VM event(506) vm_host = _vm_host(prefix, VM2_NAME) pid = vm_host.ssh(['pgrep', '-f', 'qemu.*guest=vm2']) vm_host.ssh(['kill', '-KILL', pid.out.decode('utf-8')]) vm_service = test_utils.get_vm_service(engine, VM2_NAME) testlib.assert_true_within_long(
lambda:
vm_service.get().status == types.VmStatus.UP ) ../basic-suite-master/test-scenarios/004_basic_sanity_pytest.py:819: _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ /usr/lib/python2.7/site-packages/ovirtlago/testlib.py:286: in assert_true_within_long assert_equals_within_long(func, True, allowed_exceptions) /usr/lib/python2.7/site-packages/ovirtlago/testlib.py:273: in assert_equals_within_long func, value, LONG_TIMEOUT, allowed_exceptions=allowed_exceptions _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ func = <function <lambda> at 0x7f07bf75b500>, value = True, timeout = 600 allowed_exceptions = [], initial_wait = 0 def assert_equals_within( func, value, timeout, allowed_exceptions=None, initial_wait=10 ): allowed_exceptions = allowed_exceptions or [] with utils.EggTimer(timeout) as timer: while not timer.elapsed(): try: res = func() if res == value: return except Exception as exc: if _instance_of_any(exc, allowed_exceptions): time.sleep(3) continue LOGGER.exception("Unhandled exception in %s", func) raise if initial_wait == 0: time.sleep(3) else: time.sleep(initial_wait) initial_wait = 0 try: raise AssertionError(
'%s != %s after %s seconds' % (res, value, timeout)
) E AssertionError: False != True after 600 seconds /usr/lib/python2.7/site-packages/ovirtlago/testlib.py:252: AssertionError
Project: https://jenkins.ovirt.org/job/ovirt-system-tests_compat-4.3-suite-master/ Build: https://jenkins.ovirt.org/job/ovirt-system-tests_compat-4.3-suite-master/571... Build Number: 571 Build Status: Still Failing Triggered By: Started by timer ------------------------------------- Changes Since Last Success: ------------------------------------- Changes for Build #562 [Milan Zamazal] basic: Disable smartcard temporarily Changes for Build #563 [Milan Zamazal] basic: Disable smartcard temporarily Changes for Build #564 [Galit Rosenthal] Rm he basic iscsi4.3 he node ng 4.3 from check_patch Changes for Build #565 [Sandro Bonazzola] yum-repos: cbs.centos.org repos doesn't exist anymore Changes for Build #566 [Sandro Bonazzola] yum-repos: cbs.centos.org repos doesn't exist anymore [Gal Ben Haim] dockerfile-utils: Specify authfile [Gal Ben Haim] dockerfile-utils: Skopeo change args order Changes for Build #567 [Marcin Sobczyk] basic: Remove remaining lago deps from '002_bootstrap_pytest' Changes for Build #568 [Marcin Sobczyk] basic: Remove remaining lago deps from '002_bootstrap_pytest' Changes for Build #569 [Marcin Sobczyk] basic: Remove remaining lago deps from '002_bootstrap_pytest' Changes for Build #570 [Marcin Sobczyk] basic: Remove remaining lago deps from '002_bootstrap_pytest' Changes for Build #571 [Marcin Sobczyk] basic: Remove remaining lago deps from '002_bootstrap_pytest' ----------------- Failed Tests: ----------------- 1 tests failed. FAILED: basic-suite-master.test-scenarios.002_bootstrap_pytest.test_verify_engine_backup Error Message: AssertionError: False != True after 600 seconds Stack Trace: ansible_engine = <ost_utils.ansible._AnsibleModuleMapper object at 0x7f90b4988790> engine_api = <ovirtsdk4.Connection object at 0x7f90b48d43d0> @order_by(_TEST_LIST) def test_verify_engine_backup(ansible_engine, engine_api): ansible_engine.file( path='/var/log/ost-engine-backup', state='directory', mode='0755' ) engine = engine_api.system_service() with test_utils.TestEvent(engine, [9024, 9025]): #backup started event, completed ansible_engine.shell(
'engine-backup '
'--mode=backup ' '--file=/var/log/ost-engine-backup/backup.tgz ' '--log=/var/log/ost-engine-backup/log.txt' ) ../basic-suite-master/test-scenarios/002_bootstrap_pytest.py:1246: _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ /usr/lib64/python2.7/contextlib.py:35: in __exit__ self.gen.throw(type, value, traceback) ../ost_utils/ost_utils/engine_utils.py:44: in wait_for_event lambda: ../ost_utils/ost_utils/assertions.py:97: in assert_true_within_long assert_equals_within_long(func, True, allowed_exceptions) ../ost_utils/ost_utils/assertions.py:82: in assert_equals_within_long func, value, LONG_TIMEOUT, allowed_exceptions=allowed_exceptions _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ func = <function <lambda> at 0x7f90b4864848>, value = True, timeout = 600 allowed_exceptions = [], initial_wait = 0 error_message = 'False != True after 600 seconds' def assert_equals_within( func, value, timeout, allowed_exceptions=None, initial_wait=10, error_message=None ): allowed_exceptions = allowed_exceptions or [] with _EggTimer(timeout) as timer: while not timer.elapsed(): try: res = func() if res == value: return except Exception as exc: if _instance_of_any(exc, allowed_exceptions): time.sleep(3) continue LOGGER.exception("Unhandled exception in %s", func) raise if initial_wait == 0: time.sleep(3) else: time.sleep(initial_wait) initial_wait = 0 try: if error_message is None: error_message = '%s != %s after %s seconds' % (res, value, timeout)
raise AssertionError(error_message)
E AssertionError: False != True after 600 seconds ../ost_utils/ost_utils/assertions.py:60: AssertionError
Project: https://jenkins.ovirt.org/job/ovirt-system-tests_compat-4.3-suite-master/ Build: https://jenkins.ovirt.org/job/ovirt-system-tests_compat-4.3-suite-master/572... Build Number: 572 Build Status: Still Failing Triggered By: Started by timer ------------------------------------- Changes Since Last Success: ------------------------------------- Changes for Build #563 [Milan Zamazal] basic: Disable smartcard temporarily Changes for Build #564 [Galit Rosenthal] Rm he basic iscsi4.3 he node ng 4.3 from check_patch Changes for Build #565 [Sandro Bonazzola] yum-repos: cbs.centos.org repos doesn't exist anymore Changes for Build #566 [Sandro Bonazzola] yum-repos: cbs.centos.org repos doesn't exist anymore [Gal Ben Haim] dockerfile-utils: Specify authfile [Gal Ben Haim] dockerfile-utils: Skopeo change args order Changes for Build #567 [Marcin Sobczyk] basic: Remove remaining lago deps from '002_bootstrap_pytest' Changes for Build #568 [Marcin Sobczyk] basic: Remove remaining lago deps from '002_bootstrap_pytest' Changes for Build #569 [Marcin Sobczyk] basic: Remove remaining lago deps from '002_bootstrap_pytest' Changes for Build #570 [Marcin Sobczyk] basic: Remove remaining lago deps from '002_bootstrap_pytest' Changes for Build #571 [Marcin Sobczyk] basic: Remove remaining lago deps from '002_bootstrap_pytest' Changes for Build #572 [Marcin Sobczyk] basic: Remove remaining lago deps from '002_bootstrap_pytest' [Gal Ben Haim] mock: Pass Docker conf env var ----------------- Failed Tests: ----------------- 1 tests failed. FAILED: basic-suite-master.test-scenarios.004_basic_sanity_pytest.test_ha_recovery Error Message: AssertionError: False != True after 600 seconds Stack Trace: prefix = <ovirtlago.prefix.OvirtPrefix object at 0x7fd9dfc97c10> @order_by(_TEST_LIST) def test_ha_recovery(prefix): engine = prefix.virt_env.engine_vm().get_api_v4().system_service() with test_utils.TestEvent(engine, [119, 9602, 506]): # VM_DOWN_ERROR event(119) # HA_VM_FAILED event event(9602) # VDS_INITIATED_RUN_VM event(506) vm_host = _vm_host(prefix, VM2_NAME) pid = vm_host.ssh(['pgrep', '-f', 'qemu.*guest=vm2']) vm_host.ssh(['kill', '-KILL', pid.out.decode('utf-8')]) vm_service = test_utils.get_vm_service(engine, VM2_NAME) testlib.assert_true_within_long(
lambda:
vm_service.get().status == types.VmStatus.UP ) ../basic-suite-master/test-scenarios/004_basic_sanity_pytest.py:819: _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ /usr/lib/python2.7/site-packages/ovirtlago/testlib.py:286: in assert_true_within_long assert_equals_within_long(func, True, allowed_exceptions) /usr/lib/python2.7/site-packages/ovirtlago/testlib.py:273: in assert_equals_within_long func, value, LONG_TIMEOUT, allowed_exceptions=allowed_exceptions _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ func = <function <lambda> at 0x7fd9e1f9b938>, value = True, timeout = 600 allowed_exceptions = [], initial_wait = 0 def assert_equals_within( func, value, timeout, allowed_exceptions=None, initial_wait=10 ): allowed_exceptions = allowed_exceptions or [] with utils.EggTimer(timeout) as timer: while not timer.elapsed(): try: res = func() if res == value: return except Exception as exc: if _instance_of_any(exc, allowed_exceptions): time.sleep(3) continue LOGGER.exception("Unhandled exception in %s", func) raise if initial_wait == 0: time.sleep(3) else: time.sleep(initial_wait) initial_wait = 0 try: raise AssertionError(
'%s != %s after %s seconds' % (res, value, timeout)
) E AssertionError: False != True after 600 seconds /usr/lib/python2.7/site-packages/ovirtlago/testlib.py:252: AssertionError
Project: https://jenkins.ovirt.org/job/ovirt-system-tests_compat-4.3-suite-master/ Build: https://jenkins.ovirt.org/job/ovirt-system-tests_compat-4.3-suite-master/573... Build Number: 573 Build Status: Still Failing Triggered By: Started by timer ------------------------------------- Changes Since Last Success: ------------------------------------- Changes for Build #564 [Galit Rosenthal] Rm he basic iscsi4.3 he node ng 4.3 from check_patch Changes for Build #565 [Sandro Bonazzola] yum-repos: cbs.centos.org repos doesn't exist anymore Changes for Build #566 [Sandro Bonazzola] yum-repos: cbs.centos.org repos doesn't exist anymore [Gal Ben Haim] dockerfile-utils: Specify authfile [Gal Ben Haim] dockerfile-utils: Skopeo change args order Changes for Build #567 [Marcin Sobczyk] basic: Remove remaining lago deps from '002_bootstrap_pytest' Changes for Build #568 [Marcin Sobczyk] basic: Remove remaining lago deps from '002_bootstrap_pytest' Changes for Build #569 [Marcin Sobczyk] basic: Remove remaining lago deps from '002_bootstrap_pytest' Changes for Build #570 [Marcin Sobczyk] basic: Remove remaining lago deps from '002_bootstrap_pytest' Changes for Build #571 [Marcin Sobczyk] basic: Remove remaining lago deps from '002_bootstrap_pytest' Changes for Build #572 [Marcin Sobczyk] basic: Remove remaining lago deps from '002_bootstrap_pytest' [Gal Ben Haim] mock: Pass Docker conf env var Changes for Build #573 [Eitan Raviv] network: ctx manager for new cluster assigned network ----------------- Failed Tests: ----------------- 1 tests failed. FAILED: basic-suite-master.test-scenarios.004_basic_sanity_pytest.test_ha_recovery Error Message: AssertionError: False != True after 600 seconds Stack Trace: prefix = <ovirtlago.prefix.OvirtPrefix object at 0x7ff9f9104c10> @order_by(_TEST_LIST) def test_ha_recovery(prefix): engine = prefix.virt_env.engine_vm().get_api_v4().system_service() with test_utils.TestEvent(engine, [119, 9602, 506]): # VM_DOWN_ERROR event(119) # HA_VM_FAILED event event(9602) # VDS_INITIATED_RUN_VM event(506) vm_host = _vm_host(prefix, VM2_NAME) pid = vm_host.ssh(['pgrep', '-f', 'qemu.*guest=vm2']) vm_host.ssh(['kill', '-KILL', pid.out.decode('utf-8')]) vm_service = test_utils.get_vm_service(engine, VM2_NAME) testlib.assert_true_within_long(
lambda:
vm_service.get().status == types.VmStatus.UP ) ../basic-suite-master/test-scenarios/004_basic_sanity_pytest.py:819: _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ /usr/lib/python2.7/site-packages/ovirtlago/testlib.py:286: in assert_true_within_long assert_equals_within_long(func, True, allowed_exceptions) /usr/lib/python2.7/site-packages/ovirtlago/testlib.py:273: in assert_equals_within_long func, value, LONG_TIMEOUT, allowed_exceptions=allowed_exceptions _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ func = <function <lambda> at 0x7ff9fb408938>, value = True, timeout = 600 allowed_exceptions = [], initial_wait = 0 def assert_equals_within( func, value, timeout, allowed_exceptions=None, initial_wait=10 ): allowed_exceptions = allowed_exceptions or [] with utils.EggTimer(timeout) as timer: while not timer.elapsed(): try: res = func() if res == value: return except Exception as exc: if _instance_of_any(exc, allowed_exceptions): time.sleep(3) continue LOGGER.exception("Unhandled exception in %s", func) raise if initial_wait == 0: time.sleep(3) else: time.sleep(initial_wait) initial_wait = 0 try: raise AssertionError(
'%s != %s after %s seconds' % (res, value, timeout)
) E AssertionError: False != True after 600 seconds /usr/lib/python2.7/site-packages/ovirtlago/testlib.py:252: AssertionError
Project: https://jenkins.ovirt.org/job/ovirt-system-tests_compat-4.3-suite-master/ Build: https://jenkins.ovirt.org/job/ovirt-system-tests_compat-4.3-suite-master/574... Build Number: 574 Build Status: Still Failing Triggered By: Started by timer ------------------------------------- Changes Since Last Success: ------------------------------------- Changes for Build #565 [Sandro Bonazzola] yum-repos: cbs.centos.org repos doesn't exist anymore Changes for Build #566 [Sandro Bonazzola] yum-repos: cbs.centos.org repos doesn't exist anymore [Gal Ben Haim] dockerfile-utils: Specify authfile [Gal Ben Haim] dockerfile-utils: Skopeo change args order Changes for Build #567 [Marcin Sobczyk] basic: Remove remaining lago deps from '002_bootstrap_pytest' Changes for Build #568 [Marcin Sobczyk] basic: Remove remaining lago deps from '002_bootstrap_pytest' Changes for Build #569 [Marcin Sobczyk] basic: Remove remaining lago deps from '002_bootstrap_pytest' Changes for Build #570 [Marcin Sobczyk] basic: Remove remaining lago deps from '002_bootstrap_pytest' Changes for Build #571 [Marcin Sobczyk] basic: Remove remaining lago deps from '002_bootstrap_pytest' Changes for Build #572 [Marcin Sobczyk] basic: Remove remaining lago deps from '002_bootstrap_pytest' [Gal Ben Haim] mock: Pass Docker conf env var Changes for Build #573 [Eitan Raviv] network: ctx manager for new cluster assigned network Changes for Build #574 [Marcin Sobczyk] ansible: Fix cleanup for multiple threads ----------------- Failed Tests: ----------------- 1 tests failed. FAILED: basic-suite-master.test-scenarios.004_basic_sanity_pytest.test_ha_recovery Error Message: AssertionError: False != True after 600 seconds Stack Trace: prefix = <ovirtlago.prefix.OvirtPrefix object at 0x7fd5d98df650> @order_by(_TEST_LIST) def test_ha_recovery(prefix): engine = prefix.virt_env.engine_vm().get_api_v4().system_service() with test_utils.TestEvent(engine, [119, 9602, 506]): # VM_DOWN_ERROR event(119) # HA_VM_FAILED event event(9602) # VDS_INITIATED_RUN_VM event(506) vm_host = _vm_host(prefix, VM2_NAME) pid = vm_host.ssh(['pgrep', '-f', 'qemu.*guest=vm2']) vm_host.ssh(['kill', '-KILL', pid.out.decode('utf-8')]) vm_service = test_utils.get_vm_service(engine, VM2_NAME) testlib.assert_true_within_long(
lambda:
vm_service.get().status == types.VmStatus.UP ) ../basic-suite-master/test-scenarios/004_basic_sanity_pytest.py:819: _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ /usr/lib/python2.7/site-packages/ovirtlago/testlib.py:286: in assert_true_within_long assert_equals_within_long(func, True, allowed_exceptions) /usr/lib/python2.7/site-packages/ovirtlago/testlib.py:273: in assert_equals_within_long func, value, LONG_TIMEOUT, allowed_exceptions=allowed_exceptions _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ func = <function <lambda> at 0x7fd5d8122758>, value = True, timeout = 600 allowed_exceptions = [], initial_wait = 0 def assert_equals_within( func, value, timeout, allowed_exceptions=None, initial_wait=10 ): allowed_exceptions = allowed_exceptions or [] with utils.EggTimer(timeout) as timer: while not timer.elapsed(): try: res = func() if res == value: return except Exception as exc: if _instance_of_any(exc, allowed_exceptions): time.sleep(3) continue LOGGER.exception("Unhandled exception in %s", func) raise if initial_wait == 0: time.sleep(3) else: time.sleep(initial_wait) initial_wait = 0 try: raise AssertionError(
'%s != %s after %s seconds' % (res, value, timeout)
) E AssertionError: False != True after 600 seconds /usr/lib/python2.7/site-packages/ovirtlago/testlib.py:252: AssertionError
Project: https://jenkins.ovirt.org/job/ovirt-system-tests_compat-4.3-suite-master/ Build: https://jenkins.ovirt.org/job/ovirt-system-tests_compat-4.3-suite-master/575... Build Number: 575 Build Status: Still Failing Triggered By: Started by timer ------------------------------------- Changes Since Last Success: ------------------------------------- Changes for Build #566 [Sandro Bonazzola] yum-repos: cbs.centos.org repos doesn't exist anymore [Gal Ben Haim] dockerfile-utils: Specify authfile [Gal Ben Haim] dockerfile-utils: Skopeo change args order Changes for Build #567 [Marcin Sobczyk] basic: Remove remaining lago deps from '002_bootstrap_pytest' Changes for Build #568 [Marcin Sobczyk] basic: Remove remaining lago deps from '002_bootstrap_pytest' Changes for Build #569 [Marcin Sobczyk] basic: Remove remaining lago deps from '002_bootstrap_pytest' Changes for Build #570 [Marcin Sobczyk] basic: Remove remaining lago deps from '002_bootstrap_pytest' Changes for Build #571 [Marcin Sobczyk] basic: Remove remaining lago deps from '002_bootstrap_pytest' Changes for Build #572 [Marcin Sobczyk] basic: Remove remaining lago deps from '002_bootstrap_pytest' [Gal Ben Haim] mock: Pass Docker conf env var Changes for Build #573 [Eitan Raviv] network: ctx manager for new cluster assigned network Changes for Build #574 [Marcin Sobczyk] ansible: Fix cleanup for multiple threads Changes for Build #575 [Marcin Sobczyk] ansible: Fix cleanup for multiple threads ----------------- Failed Tests: ----------------- 1 tests failed. FAILED: basic-suite-master.test-scenarios.004_basic_sanity_pytest.test_ha_recovery Error Message: AssertionError: False != True after 600 seconds Stack Trace: prefix = <ovirtlago.prefix.OvirtPrefix object at 0x7f4bff95e650> @order_by(_TEST_LIST) def test_ha_recovery(prefix): engine = prefix.virt_env.engine_vm().get_api_v4().system_service() with test_utils.TestEvent(engine, [119, 9602, 506]): # VM_DOWN_ERROR event(119) # HA_VM_FAILED event event(9602) # VDS_INITIATED_RUN_VM event(506) vm_host = _vm_host(prefix, VM2_NAME) pid = vm_host.ssh(['pgrep', '-f', 'qemu.*guest=vm2']) vm_host.ssh(['kill', '-KILL', pid.out.decode('utf-8')]) vm_service = test_utils.get_vm_service(engine, VM2_NAME) testlib.assert_true_within_long(
lambda:
vm_service.get().status == types.VmStatus.UP ) ../basic-suite-master/test-scenarios/004_basic_sanity_pytest.py:819: _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ /usr/lib/python2.7/site-packages/ovirtlago/testlib.py:286: in assert_true_within_long assert_equals_within_long(func, True, allowed_exceptions) /usr/lib/python2.7/site-packages/ovirtlago/testlib.py:273: in assert_equals_within_long func, value, LONG_TIMEOUT, allowed_exceptions=allowed_exceptions _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ func = <function <lambda> at 0x7f4bfc1205f0>, value = True, timeout = 600 allowed_exceptions = [], initial_wait = 0 def assert_equals_within( func, value, timeout, allowed_exceptions=None, initial_wait=10 ): allowed_exceptions = allowed_exceptions or [] with utils.EggTimer(timeout) as timer: while not timer.elapsed(): try: res = func() if res == value: return except Exception as exc: if _instance_of_any(exc, allowed_exceptions): time.sleep(3) continue LOGGER.exception("Unhandled exception in %s", func) raise if initial_wait == 0: time.sleep(3) else: time.sleep(initial_wait) initial_wait = 0 try: raise AssertionError(
'%s != %s after %s seconds' % (res, value, timeout)
) E AssertionError: False != True after 600 seconds /usr/lib/python2.7/site-packages/ovirtlago/testlib.py:252: AssertionError
Project: https://jenkins.ovirt.org/job/ovirt-system-tests_compat-4.3-suite-master/ Build: https://jenkins.ovirt.org/job/ovirt-system-tests_compat-4.3-suite-master/576... Build Number: 576 Build Status: Still Failing Triggered By: Started by timer ------------------------------------- Changes Since Last Success: ------------------------------------- Changes for Build #567 [Marcin Sobczyk] basic: Remove remaining lago deps from '002_bootstrap_pytest' Changes for Build #568 [Marcin Sobczyk] basic: Remove remaining lago deps from '002_bootstrap_pytest' Changes for Build #569 [Marcin Sobczyk] basic: Remove remaining lago deps from '002_bootstrap_pytest' Changes for Build #570 [Marcin Sobczyk] basic: Remove remaining lago deps from '002_bootstrap_pytest' Changes for Build #571 [Marcin Sobczyk] basic: Remove remaining lago deps from '002_bootstrap_pytest' Changes for Build #572 [Marcin Sobczyk] basic: Remove remaining lago deps from '002_bootstrap_pytest' [Gal Ben Haim] mock: Pass Docker conf env var Changes for Build #573 [Eitan Raviv] network: ctx manager for new cluster assigned network Changes for Build #574 [Marcin Sobczyk] ansible: Fix cleanup for multiple threads Changes for Build #575 [Marcin Sobczyk] ansible: Fix cleanup for multiple threads Changes for Build #576 [Marcin Sobczyk] ansible: Fix cleanup for multiple threads ----------------- Failed Tests: ----------------- 1 tests failed. FAILED: basic-suite-master.test-scenarios.004_basic_sanity_pytest.test_ha_recovery Error Message: AssertionError: False != True after 600 seconds Stack Trace: prefix = <ovirtlago.prefix.OvirtPrefix object at 0x7f4b1742a650> @order_by(_TEST_LIST) def test_ha_recovery(prefix): engine = prefix.virt_env.engine_vm().get_api_v4().system_service() with test_utils.TestEvent(engine, [119, 9602, 506]): # VM_DOWN_ERROR event(119) # HA_VM_FAILED event event(9602) # VDS_INITIATED_RUN_VM event(506) vm_host = _vm_host(prefix, VM2_NAME) pid = vm_host.ssh(['pgrep', '-f', 'qemu.*guest=vm2']) vm_host.ssh(['kill', '-KILL', pid.out.decode('utf-8')]) vm_service = test_utils.get_vm_service(engine, VM2_NAME) testlib.assert_true_within_long(
lambda:
vm_service.get().status == types.VmStatus.UP ) ../basic-suite-master/test-scenarios/004_basic_sanity_pytest.py:819: _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ /usr/lib/python2.7/site-packages/ovirtlago/testlib.py:286: in assert_true_within_long assert_equals_within_long(func, True, allowed_exceptions) /usr/lib/python2.7/site-packages/ovirtlago/testlib.py:273: in assert_equals_within_long func, value, LONG_TIMEOUT, allowed_exceptions=allowed_exceptions _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ func = <function <lambda> at 0x7f4b140fd5f0>, value = True, timeout = 600 allowed_exceptions = [], initial_wait = 0 def assert_equals_within( func, value, timeout, allowed_exceptions=None, initial_wait=10 ): allowed_exceptions = allowed_exceptions or [] with utils.EggTimer(timeout) as timer: while not timer.elapsed(): try: res = func() if res == value: return except Exception as exc: if _instance_of_any(exc, allowed_exceptions): time.sleep(3) continue LOGGER.exception("Unhandled exception in %s", func) raise if initial_wait == 0: time.sleep(3) else: time.sleep(initial_wait) initial_wait = 0 try: raise AssertionError(
'%s != %s after %s seconds' % (res, value, timeout)
) E AssertionError: False != True after 600 seconds /usr/lib/python2.7/site-packages/ovirtlago/testlib.py:252: AssertionError
Project: https://jenkins.ovirt.org/job/ovirt-system-tests_compat-4.3-suite-master/ Build: https://jenkins.ovirt.org/job/ovirt-system-tests_compat-4.3-suite-master/577... Build Number: 577 Build Status: Still Failing Triggered By: Started by timer ------------------------------------- Changes Since Last Success: ------------------------------------- Changes for Build #568 [Marcin Sobczyk] basic: Remove remaining lago deps from '002_bootstrap_pytest' Changes for Build #569 [Marcin Sobczyk] basic: Remove remaining lago deps from '002_bootstrap_pytest' Changes for Build #570 [Marcin Sobczyk] basic: Remove remaining lago deps from '002_bootstrap_pytest' Changes for Build #571 [Marcin Sobczyk] basic: Remove remaining lago deps from '002_bootstrap_pytest' Changes for Build #572 [Marcin Sobczyk] basic: Remove remaining lago deps from '002_bootstrap_pytest' [Gal Ben Haim] mock: Pass Docker conf env var Changes for Build #573 [Eitan Raviv] network: ctx manager for new cluster assigned network Changes for Build #574 [Marcin Sobczyk] ansible: Fix cleanup for multiple threads Changes for Build #575 [Marcin Sobczyk] ansible: Fix cleanup for multiple threads Changes for Build #576 [Marcin Sobczyk] ansible: Fix cleanup for multiple threads Changes for Build #577 [Marcin Sobczyk] ansible: Fix cleanup for multiple threads ----------------- Failed Tests: ----------------- 1 tests failed. FAILED: basic-suite-master.test-scenarios.004_basic_sanity_pytest.test_ha_recovery Error Message: AssertionError: False != True after 600 seconds Stack Trace: prefix = <ovirtlago.prefix.OvirtPrefix object at 0x7fd510a47550> @order_by(_TEST_LIST) def test_ha_recovery(prefix): engine = prefix.virt_env.engine_vm().get_api_v4().system_service() with test_utils.TestEvent(engine, [119, 9602, 506]): # VM_DOWN_ERROR event(119) # HA_VM_FAILED event event(9602) # VDS_INITIATED_RUN_VM event(506) vm_host = _vm_host(prefix, VM2_NAME) pid = vm_host.ssh(['pgrep', '-f', 'qemu.*guest=vm2']) vm_host.ssh(['kill', '-KILL', pid.out.decode('utf-8')]) vm_service = test_utils.get_vm_service(engine, VM2_NAME) testlib.assert_true_within_long(
lambda:
vm_service.get().status == types.VmStatus.UP ) ../basic-suite-master/test-scenarios/004_basic_sanity_pytest.py:819: _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ /usr/lib/python2.7/site-packages/ovirtlago/testlib.py:286: in assert_true_within_long assert_equals_within_long(func, True, allowed_exceptions) /usr/lib/python2.7/site-packages/ovirtlago/testlib.py:273: in assert_equals_within_long func, value, LONG_TIMEOUT, allowed_exceptions=allowed_exceptions _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ func = <function <lambda> at 0x7fd50d20a7d0>, value = True, timeout = 600 allowed_exceptions = [], initial_wait = 0 def assert_equals_within( func, value, timeout, allowed_exceptions=None, initial_wait=10 ): allowed_exceptions = allowed_exceptions or [] with utils.EggTimer(timeout) as timer: while not timer.elapsed(): try: res = func() if res == value: return except Exception as exc: if _instance_of_any(exc, allowed_exceptions): time.sleep(3) continue LOGGER.exception("Unhandled exception in %s", func) raise if initial_wait == 0: time.sleep(3) else: time.sleep(initial_wait) initial_wait = 0 try: raise AssertionError(
'%s != %s after %s seconds' % (res, value, timeout)
) E AssertionError: False != True after 600 seconds /usr/lib/python2.7/site-packages/ovirtlago/testlib.py:252: AssertionError
Project: https://jenkins.ovirt.org/job/ovirt-system-tests_compat-4.3-suite-master/ Build: https://jenkins.ovirt.org/job/ovirt-system-tests_compat-4.3-suite-master/578... Build Number: 578 Build Status: Still Failing Triggered By: Started by timer ------------------------------------- Changes Since Last Success: ------------------------------------- Changes for Build #569 [Marcin Sobczyk] basic: Remove remaining lago deps from '002_bootstrap_pytest' Changes for Build #570 [Marcin Sobczyk] basic: Remove remaining lago deps from '002_bootstrap_pytest' Changes for Build #571 [Marcin Sobczyk] basic: Remove remaining lago deps from '002_bootstrap_pytest' Changes for Build #572 [Marcin Sobczyk] basic: Remove remaining lago deps from '002_bootstrap_pytest' [Gal Ben Haim] mock: Pass Docker conf env var Changes for Build #573 [Eitan Raviv] network: ctx manager for new cluster assigned network Changes for Build #574 [Marcin Sobczyk] ansible: Fix cleanup for multiple threads Changes for Build #575 [Marcin Sobczyk] ansible: Fix cleanup for multiple threads Changes for Build #576 [Marcin Sobczyk] ansible: Fix cleanup for multiple threads Changes for Build #577 [Marcin Sobczyk] ansible: Fix cleanup for multiple threads Changes for Build #578 [Marcin Sobczyk] basic: Port '099_aaa-ldap' test module to ansible ----------------- Failed Tests: ----------------- 1 tests failed. FAILED: basic-suite-master.test-scenarios.002_bootstrap.test_verify_add_hosts Error Message: RuntimeError: 1 hosts failed installation: lago-compat-4-3-suite-master-host-0: install_failed Stack Trace: engine_api = <ovirtsdk4.Connection object at 0x7f635c456050> @order_by(_TEST_LIST) def test_verify_add_hosts(engine_api): hosts_service = engine_api.system_service().hosts_service() hosts_status = hosts_service.list(search='datacenter={}'.format(DC_NAME)) total_hosts = len(hosts_status) dump_hosts = _host_status_to_print(hosts_service, hosts_status) LOGGER.debug('Host status, verify_add_hosts:\n {}'.format(dump_hosts)) assertions.assert_true_within( lambda: _single_host_up(hosts_service, total_hosts),
timeout=constants.ADD_HOST_TIMEOUT
) ../basic-suite-master/test-scenarios/002_bootstrap.py:462: _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ ../ost_utils/ost_utils/assertions.py:87: in assert_true_within assert_equals_within(func, True, timeout, allowed_exceptions) ../ost_utils/ost_utils/assertions.py:41: in assert_equals_within res = func() ../basic-suite-master/test-scenarios/002_bootstrap.py:461: in <lambda> lambda: _single_host_up(hosts_service, total_hosts), ../basic-suite-master/test-scenarios/002_bootstrap.py:233: in _single_host_up _check_problematic_hosts(hosts_service) _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ hosts_service = <ovirtsdk4.services.HostsService object at 0x7f635c0f2710> def _check_problematic_hosts(hosts_service): problematic_hosts = hosts_service.list(search='datacenter={} AND status != installing and status != initializing and status != up'.format(DC_NAME)) if len(problematic_hosts): dump_hosts = '%s hosts failed installation:\n' % len(problematic_hosts) for host in problematic_hosts: host_service = hosts_service.host_service(host.id) dump_hosts += '%s: %s\n' % (host.name, host_service.get().status)
raise RuntimeError(dump_hosts)
E RuntimeError: 1 hosts failed installation: E lago-compat-4-3-suite-master-host-0: install_failed ../basic-suite-master/test-scenarios/002_bootstrap.py:242: RuntimeError
Project: https://jenkins.ovirt.org/job/ovirt-system-tests_compat-4.3-suite-master/ Build: https://jenkins.ovirt.org/job/ovirt-system-tests_compat-4.3-suite-master/579... Build Number: 579 Build Status: Still Failing Triggered By: Started by timer ------------------------------------- Changes Since Last Success: ------------------------------------- Changes for Build #570 [Marcin Sobczyk] basic: Remove remaining lago deps from '002_bootstrap_pytest' Changes for Build #571 [Marcin Sobczyk] basic: Remove remaining lago deps from '002_bootstrap_pytest' Changes for Build #572 [Marcin Sobczyk] basic: Remove remaining lago deps from '002_bootstrap_pytest' [Gal Ben Haim] mock: Pass Docker conf env var Changes for Build #573 [Eitan Raviv] network: ctx manager for new cluster assigned network Changes for Build #574 [Marcin Sobczyk] ansible: Fix cleanup for multiple threads Changes for Build #575 [Marcin Sobczyk] ansible: Fix cleanup for multiple threads Changes for Build #576 [Marcin Sobczyk] ansible: Fix cleanup for multiple threads Changes for Build #577 [Marcin Sobczyk] ansible: Fix cleanup for multiple threads Changes for Build #578 [Marcin Sobczyk] basic: Port '099_aaa-ldap' test module to ansible Changes for Build #579 [Marcin Sobczyk] basic: Port '099_aaa-ldap' test module to ansible ----------------- Failed Tests: ----------------- 1 tests failed. FAILED: basic-suite-master.test-scenarios.002_bootstrap.test_verify_add_hosts Error Message: RuntimeError: 1 hosts failed installation: lago-compat-4-3-suite-master-host-0: install_failed Stack Trace: engine_api = <ovirtsdk4.Connection object at 0x7f9efdba6050> @order_by(_TEST_LIST) def test_verify_add_hosts(engine_api): hosts_service = engine_api.system_service().hosts_service() hosts_status = hosts_service.list(search='datacenter={}'.format(DC_NAME)) total_hosts = len(hosts_status) dump_hosts = _host_status_to_print(hosts_service, hosts_status) LOGGER.debug('Host status, verify_add_hosts:\n {}'.format(dump_hosts)) assertions.assert_true_within( lambda: _single_host_up(hosts_service, total_hosts),
timeout=constants.ADD_HOST_TIMEOUT
) ../basic-suite-master/test-scenarios/002_bootstrap.py:462: _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ ../ost_utils/ost_utils/assertions.py:87: in assert_true_within assert_equals_within(func, True, timeout, allowed_exceptions) ../ost_utils/ost_utils/assertions.py:41: in assert_equals_within res = func() ../basic-suite-master/test-scenarios/002_bootstrap.py:461: in <lambda> lambda: _single_host_up(hosts_service, total_hosts), ../basic-suite-master/test-scenarios/002_bootstrap.py:233: in _single_host_up _check_problematic_hosts(hosts_service) _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ hosts_service = <ovirtsdk4.services.HostsService object at 0x7f9efd64e710> def _check_problematic_hosts(hosts_service): problematic_hosts = hosts_service.list(search='datacenter={} AND status != installing and status != initializing and status != up'.format(DC_NAME)) if len(problematic_hosts): dump_hosts = '%s hosts failed installation:\n' % len(problematic_hosts) for host in problematic_hosts: host_service = hosts_service.host_service(host.id) dump_hosts += '%s: %s\n' % (host.name, host_service.get().status)
raise RuntimeError(dump_hosts)
E RuntimeError: 1 hosts failed installation: E lago-compat-4-3-suite-master-host-0: install_failed ../basic-suite-master/test-scenarios/002_bootstrap.py:242: RuntimeError
Project: https://jenkins.ovirt.org/job/ovirt-system-tests_compat-4.3-suite-master/ Build: https://jenkins.ovirt.org/job/ovirt-system-tests_compat-4.3-suite-master/580... Build Number: 580 Build Status: Still Failing Triggered By: Started by timer ------------------------------------- Changes Since Last Success: ------------------------------------- Changes for Build #571 [Marcin Sobczyk] basic: Remove remaining lago deps from '002_bootstrap_pytest' Changes for Build #572 [Marcin Sobczyk] basic: Remove remaining lago deps from '002_bootstrap_pytest' [Gal Ben Haim] mock: Pass Docker conf env var Changes for Build #573 [Eitan Raviv] network: ctx manager for new cluster assigned network Changes for Build #574 [Marcin Sobczyk] ansible: Fix cleanup for multiple threads Changes for Build #575 [Marcin Sobczyk] ansible: Fix cleanup for multiple threads Changes for Build #576 [Marcin Sobczyk] ansible: Fix cleanup for multiple threads Changes for Build #577 [Marcin Sobczyk] ansible: Fix cleanup for multiple threads Changes for Build #578 [Marcin Sobczyk] basic: Port '099_aaa-ldap' test module to ansible Changes for Build #579 [Marcin Sobczyk] basic: Port '099_aaa-ldap' test module to ansible Changes for Build #580 [Marcin Sobczyk] basic: Port '099_aaa-ldap' test module to ansible ----------------- Failed Tests: ----------------- 1 tests failed. FAILED: basic-suite-master.test-scenarios.004_basic_sanity.test_ha_recovery Error Message: AssertionError: False != True after 600 seconds Stack Trace: prefix = <ovirtlago.prefix.OvirtPrefix object at 0x7fb728a93a90> @order_by(_TEST_LIST) def test_ha_recovery(prefix): engine = prefix.virt_env.engine_vm().get_api_v4().system_service() with test_utils.TestEvent(engine, [119, 9602, 506]): # VM_DOWN_ERROR event(119) # HA_VM_FAILED event event(9602) # VDS_INITIATED_RUN_VM event(506) vm_host = _vm_host(prefix, VM2_NAME) pid = vm_host.ssh(['pgrep', '-f', 'qemu.*guest=vm2']) vm_host.ssh(['kill', '-KILL', pid.out.decode('utf-8')]) vm_service = test_utils.get_vm_service(engine, VM2_NAME) testlib.assert_true_within_long(
lambda:
vm_service.get().status == types.VmStatus.UP ) ../basic-suite-master/test-scenarios/004_basic_sanity.py:819: _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ /usr/lib/python2.7/site-packages/ovirtlago/testlib.py:286: in assert_true_within_long assert_equals_within_long(func, True, allowed_exceptions) /usr/lib/python2.7/site-packages/ovirtlago/testlib.py:273: in assert_equals_within_long func, value, LONG_TIMEOUT, allowed_exceptions=allowed_exceptions _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ func = <function <lambda> at 0x7fb71da05938>, value = True, timeout = 600 allowed_exceptions = [], initial_wait = 0 def assert_equals_within( func, value, timeout, allowed_exceptions=None, initial_wait=10 ): allowed_exceptions = allowed_exceptions or [] with utils.EggTimer(timeout) as timer: while not timer.elapsed(): try: res = func() if res == value: return except Exception as exc: if _instance_of_any(exc, allowed_exceptions): time.sleep(3) continue LOGGER.exception("Unhandled exception in %s", func) raise if initial_wait == 0: time.sleep(3) else: time.sleep(initial_wait) initial_wait = 0 try: raise AssertionError(
'%s != %s after %s seconds' % (res, value, timeout)
) E AssertionError: False != True after 600 seconds /usr/lib/python2.7/site-packages/ovirtlago/testlib.py:252: AssertionError
Project: https://jenkins.ovirt.org/job/ovirt-system-tests_compat-4.3-suite-master/ Build: https://jenkins.ovirt.org/job/ovirt-system-tests_compat-4.3-suite-master/581... Build Number: 581 Build Status: Still Failing Triggered By: Started by timer ------------------------------------- Changes Since Last Success: ------------------------------------- Changes for Build #572 [Marcin Sobczyk] basic: Remove remaining lago deps from '002_bootstrap_pytest' [Gal Ben Haim] mock: Pass Docker conf env var Changes for Build #573 [Eitan Raviv] network: ctx manager for new cluster assigned network Changes for Build #574 [Marcin Sobczyk] ansible: Fix cleanup for multiple threads Changes for Build #575 [Marcin Sobczyk] ansible: Fix cleanup for multiple threads Changes for Build #576 [Marcin Sobczyk] ansible: Fix cleanup for multiple threads Changes for Build #577 [Marcin Sobczyk] ansible: Fix cleanup for multiple threads Changes for Build #578 [Marcin Sobczyk] basic: Port '099_aaa-ldap' test module to ansible Changes for Build #579 [Marcin Sobczyk] basic: Port '099_aaa-ldap' test module to ansible Changes for Build #580 [Marcin Sobczyk] basic: Port '099_aaa-ldap' test module to ansible Changes for Build #581 [Marcin Sobczyk] basic: Port '099_aaa-ldap' test module to ansible ----------------- Failed Tests: ----------------- 1 tests failed. FAILED: basic-suite-master.test-scenarios.004_basic_sanity.test_ha_recovery Error Message: AssertionError: False != True after 600 seconds Stack Trace: prefix = <ovirtlago.prefix.OvirtPrefix object at 0x7feb8caaba90> @order_by(_TEST_LIST) def test_ha_recovery(prefix): engine = prefix.virt_env.engine_vm().get_api_v4().system_service() with test_utils.TestEvent(engine, [119, 9602, 506]): # VM_DOWN_ERROR event(119) # HA_VM_FAILED event event(9602) # VDS_INITIATED_RUN_VM event(506) vm_host = _vm_host(prefix, VM2_NAME) pid = vm_host.ssh(['pgrep', '-f', 'qemu.*guest=vm2']) vm_host.ssh(['kill', '-KILL', pid.out.decode('utf-8')]) vm_service = test_utils.get_vm_service(engine, VM2_NAME) testlib.assert_true_within_long(
lambda:
vm_service.get().status == types.VmStatus.UP ) ../basic-suite-master/test-scenarios/004_basic_sanity.py:819: _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ /usr/lib/python2.7/site-packages/ovirtlago/testlib.py:286: in assert_true_within_long assert_equals_within_long(func, True, allowed_exceptions) /usr/lib/python2.7/site-packages/ovirtlago/testlib.py:273: in assert_equals_within_long func, value, LONG_TIMEOUT, allowed_exceptions=allowed_exceptions _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ func = <function <lambda> at 0x7feb82cc3488>, value = True, timeout = 600 allowed_exceptions = [], initial_wait = 0 def assert_equals_within( func, value, timeout, allowed_exceptions=None, initial_wait=10 ): allowed_exceptions = allowed_exceptions or [] with utils.EggTimer(timeout) as timer: while not timer.elapsed(): try: res = func() if res == value: return except Exception as exc: if _instance_of_any(exc, allowed_exceptions): time.sleep(3) continue LOGGER.exception("Unhandled exception in %s", func) raise if initial_wait == 0: time.sleep(3) else: time.sleep(initial_wait) initial_wait = 0 try: raise AssertionError(
'%s != %s after %s seconds' % (res, value, timeout)
) E AssertionError: False != True after 600 seconds /usr/lib/python2.7/site-packages/ovirtlago/testlib.py:252: AssertionError
Project: https://jenkins.ovirt.org/job/ovirt-system-tests_compat-4.3-suite-master/ Build: https://jenkins.ovirt.org/job/ovirt-system-tests_compat-4.3-suite-master/582... Build Number: 582 Build Status: Still Failing Triggered By: Started by timer ------------------------------------- Changes Since Last Success: ------------------------------------- Changes for Build #573 [Eitan Raviv] network: ctx manager for new cluster assigned network Changes for Build #574 [Marcin Sobczyk] ansible: Fix cleanup for multiple threads Changes for Build #575 [Marcin Sobczyk] ansible: Fix cleanup for multiple threads Changes for Build #576 [Marcin Sobczyk] ansible: Fix cleanup for multiple threads Changes for Build #577 [Marcin Sobczyk] ansible: Fix cleanup for multiple threads Changes for Build #578 [Marcin Sobczyk] basic: Port '099_aaa-ldap' test module to ansible Changes for Build #579 [Marcin Sobczyk] basic: Port '099_aaa-ldap' test module to ansible Changes for Build #580 [Marcin Sobczyk] basic: Port '099_aaa-ldap' test module to ansible Changes for Build #581 [Marcin Sobczyk] basic: Port '099_aaa-ldap' test module to ansible Changes for Build #582 [Marcin Sobczyk] basic: Port '099_aaa-ldap' test module to ansible ----------------- Failed Tests: ----------------- 1 tests failed. FAILED: basic-suite-master.test-scenarios.004_basic_sanity.test_ha_recovery Error Message: AssertionError: False != True after 600 seconds Stack Trace: prefix = <ovirtlago.prefix.OvirtPrefix object at 0x7fead697ea90> @order_by(_TEST_LIST) def test_ha_recovery(prefix): engine = prefix.virt_env.engine_vm().get_api_v4().system_service() with test_utils.TestEvent(engine, [119, 9602, 506]): # VM_DOWN_ERROR event(119) # HA_VM_FAILED event event(9602) # VDS_INITIATED_RUN_VM event(506) vm_host = _vm_host(prefix, VM2_NAME) pid = vm_host.ssh(['pgrep', '-f', 'qemu.*guest=vm2']) vm_host.ssh(['kill', '-KILL', pid.out.decode('utf-8')]) vm_service = test_utils.get_vm_service(engine, VM2_NAME) testlib.assert_true_within_long(
lambda:
vm_service.get().status == types.VmStatus.UP ) ../basic-suite-master/test-scenarios/004_basic_sanity.py:819: _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ /usr/lib/python2.7/site-packages/ovirtlago/testlib.py:286: in assert_true_within_long assert_equals_within_long(func, True, allowed_exceptions) /usr/lib/python2.7/site-packages/ovirtlago/testlib.py:273: in assert_equals_within_long func, value, LONG_TIMEOUT, allowed_exceptions=allowed_exceptions _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ func = <function <lambda> at 0x7feacc268cf8>, value = True, timeout = 600 allowed_exceptions = [], initial_wait = 0 def assert_equals_within( func, value, timeout, allowed_exceptions=None, initial_wait=10 ): allowed_exceptions = allowed_exceptions or [] with utils.EggTimer(timeout) as timer: while not timer.elapsed(): try: res = func() if res == value: return except Exception as exc: if _instance_of_any(exc, allowed_exceptions): time.sleep(3) continue LOGGER.exception("Unhandled exception in %s", func) raise if initial_wait == 0: time.sleep(3) else: time.sleep(initial_wait) initial_wait = 0 try: raise AssertionError(
'%s != %s after %s seconds' % (res, value, timeout)
) E AssertionError: False != True after 600 seconds /usr/lib/python2.7/site-packages/ovirtlago/testlib.py:252: AssertionError
Project: https://jenkins.ovirt.org/job/ovirt-system-tests_compat-4.3-suite-master/ Build: https://jenkins.ovirt.org/job/ovirt-system-tests_compat-4.3-suite-master/583... Build Number: 583 Build Status: Still Failing Triggered By: Started by timer ------------------------------------- Changes Since Last Success: ------------------------------------- Changes for Build #574 [Marcin Sobczyk] ansible: Fix cleanup for multiple threads Changes for Build #575 [Marcin Sobczyk] ansible: Fix cleanup for multiple threads Changes for Build #576 [Marcin Sobczyk] ansible: Fix cleanup for multiple threads Changes for Build #577 [Marcin Sobczyk] ansible: Fix cleanup for multiple threads Changes for Build #578 [Marcin Sobczyk] basic: Port '099_aaa-ldap' test module to ansible Changes for Build #579 [Marcin Sobczyk] basic: Port '099_aaa-ldap' test module to ansible Changes for Build #580 [Marcin Sobczyk] basic: Port '099_aaa-ldap' test module to ansible Changes for Build #581 [Marcin Sobczyk] basic: Port '099_aaa-ldap' test module to ansible Changes for Build #582 [Marcin Sobczyk] basic: Port '099_aaa-ldap' test module to ansible Changes for Build #583 [Marcin Sobczyk] basic: Port '099_aaa-ldap' test module to ansible [Gal Ben Haim] dockefile-utils: Unset env var in skopeo test ----------------- Failed Tests: ----------------- 1 tests failed. FAILED: basic-suite-master.test-scenarios.004_basic_sanity.test_ha_recovery Error Message: AssertionError: False != True after 600 seconds Stack Trace: prefix = <ovirtlago.prefix.OvirtPrefix object at 0x7f7ced72da90> @order_by(_TEST_LIST) def test_ha_recovery(prefix): engine = prefix.virt_env.engine_vm().get_api_v4().system_service() with test_utils.TestEvent(engine, [119, 9602, 506]): # VM_DOWN_ERROR event(119) # HA_VM_FAILED event event(9602) # VDS_INITIATED_RUN_VM event(506) vm_host = _vm_host(prefix, VM2_NAME) pid = vm_host.ssh(['pgrep', '-f', 'qemu.*guest=vm2']) vm_host.ssh(['kill', '-KILL', pid.out.decode('utf-8')]) vm_service = test_utils.get_vm_service(engine, VM2_NAME) testlib.assert_true_within_long(
lambda:
vm_service.get().status == types.VmStatus.UP ) ../basic-suite-master/test-scenarios/004_basic_sanity.py:819: _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ /usr/lib/python2.7/site-packages/ovirtlago/testlib.py:286: in assert_true_within_long assert_equals_within_long(func, True, allowed_exceptions) /usr/lib/python2.7/site-packages/ovirtlago/testlib.py:273: in assert_equals_within_long func, value, LONG_TIMEOUT, allowed_exceptions=allowed_exceptions _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ func = <function <lambda> at 0x7f7ce2f6d320>, value = True, timeout = 600 allowed_exceptions = [], initial_wait = 0 def assert_equals_within( func, value, timeout, allowed_exceptions=None, initial_wait=10 ): allowed_exceptions = allowed_exceptions or [] with utils.EggTimer(timeout) as timer: while not timer.elapsed(): try: res = func() if res == value: return except Exception as exc: if _instance_of_any(exc, allowed_exceptions): time.sleep(3) continue LOGGER.exception("Unhandled exception in %s", func) raise if initial_wait == 0: time.sleep(3) else: time.sleep(initial_wait) initial_wait = 0 try: raise AssertionError(
'%s != %s after %s seconds' % (res, value, timeout)
) E AssertionError: False != True after 600 seconds /usr/lib/python2.7/site-packages/ovirtlago/testlib.py:252: AssertionError
Project: https://jenkins.ovirt.org/job/ovirt-system-tests_compat-4.3-suite-master/ Build: https://jenkins.ovirt.org/job/ovirt-system-tests_compat-4.3-suite-master/584... Build Number: 584 Build Status: Still Failing Triggered By: Started by timer ------------------------------------- Changes Since Last Success: ------------------------------------- Changes for Build #575 [Marcin Sobczyk] ansible: Fix cleanup for multiple threads Changes for Build #576 [Marcin Sobczyk] ansible: Fix cleanup for multiple threads Changes for Build #577 [Marcin Sobczyk] ansible: Fix cleanup for multiple threads Changes for Build #578 [Marcin Sobczyk] basic: Port '099_aaa-ldap' test module to ansible Changes for Build #579 [Marcin Sobczyk] basic: Port '099_aaa-ldap' test module to ansible Changes for Build #580 [Marcin Sobczyk] basic: Port '099_aaa-ldap' test module to ansible Changes for Build #581 [Marcin Sobczyk] basic: Port '099_aaa-ldap' test module to ansible Changes for Build #582 [Marcin Sobczyk] basic: Port '099_aaa-ldap' test module to ansible Changes for Build #583 [Marcin Sobczyk] basic: Port '099_aaa-ldap' test module to ansible [Gal Ben Haim] dockefile-utils: Unset env var in skopeo test Changes for Build #584 [Marcin Sobczyk] basic: Port '099_aaa-ldap' test module to ansible ----------------- Failed Tests: ----------------- 1 tests failed. FAILED: basic-suite-master.test-scenarios.004_basic_sanity.test_ha_recovery Error Message: AssertionError: False != True after 600 seconds Stack Trace: prefix = <ovirtlago.prefix.OvirtPrefix object at 0x7f511ec5ca90> @order_by(_TEST_LIST) def test_ha_recovery(prefix): engine = prefix.virt_env.engine_vm().get_api_v4().system_service() with test_utils.TestEvent(engine, [119, 9602, 506]): # VM_DOWN_ERROR event(119) # HA_VM_FAILED event event(9602) # VDS_INITIATED_RUN_VM event(506) vm_host = _vm_host(prefix, VM2_NAME) pid = vm_host.ssh(['pgrep', '-f', 'qemu.*guest=vm2']) vm_host.ssh(['kill', '-KILL', pid.out.decode('utf-8')]) vm_service = test_utils.get_vm_service(engine, VM2_NAME) testlib.assert_true_within_long(
lambda:
vm_service.get().status == types.VmStatus.UP ) ../basic-suite-master/test-scenarios/004_basic_sanity.py:819: _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ /usr/lib/python2.7/site-packages/ovirtlago/testlib.py:286: in assert_true_within_long assert_equals_within_long(func, True, allowed_exceptions) /usr/lib/python2.7/site-packages/ovirtlago/testlib.py:273: in assert_equals_within_long func, value, LONG_TIMEOUT, allowed_exceptions=allowed_exceptions _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ func = <function <lambda> at 0x7f5114d50488>, value = True, timeout = 600 allowed_exceptions = [], initial_wait = 0 def assert_equals_within( func, value, timeout, allowed_exceptions=None, initial_wait=10 ): allowed_exceptions = allowed_exceptions or [] with utils.EggTimer(timeout) as timer: while not timer.elapsed(): try: res = func() if res == value: return except Exception as exc: if _instance_of_any(exc, allowed_exceptions): time.sleep(3) continue LOGGER.exception("Unhandled exception in %s", func) raise if initial_wait == 0: time.sleep(3) else: time.sleep(initial_wait) initial_wait = 0 try: raise AssertionError(
'%s != %s after %s seconds' % (res, value, timeout)
) E AssertionError: False != True after 600 seconds /usr/lib/python2.7/site-packages/ovirtlago/testlib.py:252: AssertionError
Project: https://jenkins.ovirt.org/job/ovirt-system-tests_compat-4.3-suite-master/ Build: https://jenkins.ovirt.org/job/ovirt-system-tests_compat-4.3-suite-master/585... Build Number: 585 Build Status: Still Failing Triggered By: Started by timer ------------------------------------- Changes Since Last Success: ------------------------------------- Changes for Build #576 [Marcin Sobczyk] ansible: Fix cleanup for multiple threads Changes for Build #577 [Marcin Sobczyk] ansible: Fix cleanup for multiple threads Changes for Build #578 [Marcin Sobczyk] basic: Port '099_aaa-ldap' test module to ansible Changes for Build #579 [Marcin Sobczyk] basic: Port '099_aaa-ldap' test module to ansible Changes for Build #580 [Marcin Sobczyk] basic: Port '099_aaa-ldap' test module to ansible Changes for Build #581 [Marcin Sobczyk] basic: Port '099_aaa-ldap' test module to ansible Changes for Build #582 [Marcin Sobczyk] basic: Port '099_aaa-ldap' test module to ansible Changes for Build #583 [Marcin Sobczyk] basic: Port '099_aaa-ldap' test module to ansible [Gal Ben Haim] dockefile-utils: Unset env var in skopeo test Changes for Build #584 [Marcin Sobczyk] basic: Port '099_aaa-ldap' test module to ansible Changes for Build #585 [Eitan Raviv] network: host change cluster - lib-erate + objectify ----------------- Failed Tests: ----------------- 1 tests failed. FAILED: basic-suite-master.test-scenarios.004_basic_sanity.test_ha_recovery Error Message: AssertionError: False != True after 600 seconds Stack Trace: prefix = <ovirtlago.prefix.OvirtPrefix object at 0x7f3635363a90> @order_by(_TEST_LIST) def test_ha_recovery(prefix): engine = prefix.virt_env.engine_vm().get_api_v4().system_service() with test_utils.TestEvent(engine, [119, 9602, 506]): # VM_DOWN_ERROR event(119) # HA_VM_FAILED event event(9602) # VDS_INITIATED_RUN_VM event(506) vm_host = _vm_host(prefix, VM2_NAME) pid = vm_host.ssh(['pgrep', '-f', 'qemu.*guest=vm2']) vm_host.ssh(['kill', '-KILL', pid.out.decode('utf-8')]) vm_service = test_utils.get_vm_service(engine, VM2_NAME) testlib.assert_true_within_long(
lambda:
vm_service.get().status == types.VmStatus.UP ) ../basic-suite-master/test-scenarios/004_basic_sanity.py:819: _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ /usr/lib/python2.7/site-packages/ovirtlago/testlib.py:286: in assert_true_within_long assert_equals_within_long(func, True, allowed_exceptions) /usr/lib/python2.7/site-packages/ovirtlago/testlib.py:273: in assert_equals_within_long func, value, LONG_TIMEOUT, allowed_exceptions=allowed_exceptions _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ func = <function <lambda> at 0x7f362b4b1938>, value = True, timeout = 600 allowed_exceptions = [], initial_wait = 0 def assert_equals_within( func, value, timeout, allowed_exceptions=None, initial_wait=10 ): allowed_exceptions = allowed_exceptions or [] with utils.EggTimer(timeout) as timer: while not timer.elapsed(): try: res = func() if res == value: return except Exception as exc: if _instance_of_any(exc, allowed_exceptions): time.sleep(3) continue LOGGER.exception("Unhandled exception in %s", func) raise if initial_wait == 0: time.sleep(3) else: time.sleep(initial_wait) initial_wait = 0 try: raise AssertionError(
'%s != %s after %s seconds' % (res, value, timeout)
) E AssertionError: False != True after 600 seconds /usr/lib/python2.7/site-packages/ovirtlago/testlib.py:252: AssertionError
Project: https://jenkins.ovirt.org/job/ovirt-system-tests_compat-4.3-suite-master/ Build: https://jenkins.ovirt.org/job/ovirt-system-tests_compat-4.3-suite-master/586... Build Number: 586 Build Status: Still Failing Triggered By: Started by timer ------------------------------------- Changes Since Last Success: ------------------------------------- Changes for Build #576 [Marcin Sobczyk] ansible: Fix cleanup for multiple threads Changes for Build #577 [Marcin Sobczyk] ansible: Fix cleanup for multiple threads Changes for Build #578 [Marcin Sobczyk] basic: Port '099_aaa-ldap' test module to ansible Changes for Build #579 [Marcin Sobczyk] basic: Port '099_aaa-ldap' test module to ansible Changes for Build #580 [Marcin Sobczyk] basic: Port '099_aaa-ldap' test module to ansible Changes for Build #581 [Marcin Sobczyk] basic: Port '099_aaa-ldap' test module to ansible Changes for Build #582 [Marcin Sobczyk] basic: Port '099_aaa-ldap' test module to ansible Changes for Build #583 [Marcin Sobczyk] basic: Port '099_aaa-ldap' test module to ansible [Gal Ben Haim] dockefile-utils: Unset env var in skopeo test Changes for Build #584 [Marcin Sobczyk] basic: Port '099_aaa-ldap' test module to ansible Changes for Build #585 [Eitan Raviv] network: host change cluster - lib-erate + objectify Changes for Build #586 [Yaniv Kaul] [WIP] SDKv4 tests. ----------------- Failed Tests: ----------------- No tests ran.
Project: https://jenkins.ovirt.org/job/ovirt-system-tests_compat-4.3-suite-master/ Build: https://jenkins.ovirt.org/job/ovirt-system-tests_compat-4.3-suite-master/587... Build Number: 587 Build Status: Still Failing Triggered By: Started by timer ------------------------------------- Changes Since Last Success: ------------------------------------- Changes for Build #577 [Marcin Sobczyk] ansible: Fix cleanup for multiple threads Changes for Build #578 [Marcin Sobczyk] basic: Port '099_aaa-ldap' test module to ansible Changes for Build #579 [Marcin Sobczyk] basic: Port '099_aaa-ldap' test module to ansible Changes for Build #580 [Marcin Sobczyk] basic: Port '099_aaa-ldap' test module to ansible Changes for Build #581 [Marcin Sobczyk] basic: Port '099_aaa-ldap' test module to ansible Changes for Build #582 [Marcin Sobczyk] basic: Port '099_aaa-ldap' test module to ansible Changes for Build #583 [Marcin Sobczyk] basic: Port '099_aaa-ldap' test module to ansible [Gal Ben Haim] dockefile-utils: Unset env var in skopeo test Changes for Build #584 [Marcin Sobczyk] basic: Port '099_aaa-ldap' test module to ansible Changes for Build #585 [Eitan Raviv] network: host change cluster - lib-erate + objectify Changes for Build #586 [Yaniv Kaul] [WIP] SDKv4 tests. Changes for Build #587 [Yaniv Kaul] [WIP] SDKv4 tests. [Ehud Yonasi] centos7-nodes: updated el7 mirrors links. ----------------- Failed Tests: ----------------- No tests ran.
Project: https://jenkins.ovirt.org/job/ovirt-system-tests_compat-4.3-suite-master/ Build: https://jenkins.ovirt.org/job/ovirt-system-tests_compat-4.3-suite-master/588... Build Number: 588 Build Status: Still Failing Triggered By: Started by timer ------------------------------------- Changes Since Last Success: ------------------------------------- Changes for Build #579 [Marcin Sobczyk] basic: Port '099_aaa-ldap' test module to ansible Changes for Build #580 [Marcin Sobczyk] basic: Port '099_aaa-ldap' test module to ansible Changes for Build #581 [Marcin Sobczyk] basic: Port '099_aaa-ldap' test module to ansible Changes for Build #582 [Marcin Sobczyk] basic: Port '099_aaa-ldap' test module to ansible Changes for Build #583 [Marcin Sobczyk] basic: Port '099_aaa-ldap' test module to ansible [Gal Ben Haim] dockefile-utils: Unset env var in skopeo test Changes for Build #584 [Marcin Sobczyk] basic: Port '099_aaa-ldap' test module to ansible Changes for Build #585 [Eitan Raviv] network: host change cluster - lib-erate + objectify Changes for Build #586 [Yaniv Kaul] [WIP] SDKv4 tests. Changes for Build #587 [Yaniv Kaul] [WIP] SDKv4 tests. [Ehud Yonasi] centos7-nodes: updated el7 mirrors links. Changes for Build #588 [Michal Skrivanek] pytest is needed for OST ----------------- Failed Tests: ----------------- 1 tests failed. FAILED: basic-suite-master.test-scenarios.004_basic_sanity.test_ha_recovery Error Message: AssertionError: False != True after 600 seconds Stack Trace: prefix = <ovirtlago.prefix.OvirtPrefix object at 0x7efdbc327a90> @order_by(_TEST_LIST) def test_ha_recovery(prefix): engine = prefix.virt_env.engine_vm().get_api_v4().system_service() with test_utils.TestEvent(engine, [119, 9602, 506]): # VM_DOWN_ERROR event(119) # HA_VM_FAILED event event(9602) # VDS_INITIATED_RUN_VM event(506) vm_host = _vm_host(prefix, VM2_NAME) pid = vm_host.ssh(['pgrep', '-f', 'qemu.*guest=vm2']) vm_host.ssh(['kill', '-KILL', pid.out.decode('utf-8')]) vm_service = test_utils.get_vm_service(engine, VM2_NAME) testlib.assert_true_within_long(
lambda:
vm_service.get().status == types.VmStatus.UP ) ../basic-suite-master/test-scenarios/004_basic_sanity.py:819: _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ /usr/lib/python2.7/site-packages/ovirtlago/testlib.py:286: in assert_true_within_long assert_equals_within_long(func, True, allowed_exceptions) /usr/lib/python2.7/site-packages/ovirtlago/testlib.py:273: in assert_equals_within_long func, value, LONG_TIMEOUT, allowed_exceptions=allowed_exceptions _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ func = <function <lambda> at 0x7efdb1c4d6e0>, value = True, timeout = 600 allowed_exceptions = [], initial_wait = 0 def assert_equals_within( func, value, timeout, allowed_exceptions=None, initial_wait=10 ): allowed_exceptions = allowed_exceptions or [] with utils.EggTimer(timeout) as timer: while not timer.elapsed(): try: res = func() if res == value: return except Exception as exc: if _instance_of_any(exc, allowed_exceptions): time.sleep(3) continue LOGGER.exception("Unhandled exception in %s", func) raise if initial_wait == 0: time.sleep(3) else: time.sleep(initial_wait) initial_wait = 0 try: raise AssertionError(
'%s != %s after %s seconds' % (res, value, timeout)
) E AssertionError: False != True after 600 seconds /usr/lib/python2.7/site-packages/ovirtlago/testlib.py:252: AssertionError
Project: https://jenkins.ovirt.org/job/ovirt-system-tests_compat-4.3-suite-master/ Build: https://jenkins.ovirt.org/job/ovirt-system-tests_compat-4.3-suite-master/589... Build Number: 589 Build Status: Still Failing Triggered By: Started by timer ------------------------------------- Changes Since Last Success: ------------------------------------- Changes for Build #580 [Marcin Sobczyk] basic: Port '099_aaa-ldap' test module to ansible Changes for Build #581 [Marcin Sobczyk] basic: Port '099_aaa-ldap' test module to ansible Changes for Build #582 [Marcin Sobczyk] basic: Port '099_aaa-ldap' test module to ansible Changes for Build #583 [Marcin Sobczyk] basic: Port '099_aaa-ldap' test module to ansible [Gal Ben Haim] dockefile-utils: Unset env var in skopeo test Changes for Build #584 [Marcin Sobczyk] basic: Port '099_aaa-ldap' test module to ansible Changes for Build #585 [Eitan Raviv] network: host change cluster - lib-erate + objectify Changes for Build #586 [Yaniv Kaul] [WIP] SDKv4 tests. Changes for Build #587 [Yaniv Kaul] [WIP] SDKv4 tests. [Ehud Yonasi] centos7-nodes: updated el7 mirrors links. Changes for Build #588 [Michal Skrivanek] pytest is needed for OST Changes for Build #589 [Michal Skrivanek] pytest is needed for OST ----------------- Failed Tests: ----------------- 1 tests failed. FAILED: basic-suite-master.test-scenarios.004_basic_sanity.test_ha_recovery Error Message: AssertionError: False != True after 600 seconds Stack Trace: prefix = <ovirtlago.prefix.OvirtPrefix object at 0x7f0628c21a90> @order_by(_TEST_LIST) def test_ha_recovery(prefix): engine = prefix.virt_env.engine_vm().get_api_v4().system_service() with test_utils.TestEvent(engine, [119, 9602, 506]): # VM_DOWN_ERROR event(119) # HA_VM_FAILED event event(9602) # VDS_INITIATED_RUN_VM event(506) vm_host = _vm_host(prefix, VM2_NAME) pid = vm_host.ssh(['pgrep', '-f', 'qemu.*guest=vm2']) vm_host.ssh(['kill', '-KILL', pid.out.decode('utf-8')]) vm_service = test_utils.get_vm_service(engine, VM2_NAME) testlib.assert_true_within_long(
lambda:
vm_service.get().status == types.VmStatus.UP ) ../basic-suite-master/test-scenarios/004_basic_sanity.py:819: _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ /usr/lib/python2.7/site-packages/ovirtlago/testlib.py:286: in assert_true_within_long assert_equals_within_long(func, True, allowed_exceptions) /usr/lib/python2.7/site-packages/ovirtlago/testlib.py:273: in assert_equals_within_long func, value, LONG_TIMEOUT, allowed_exceptions=allowed_exceptions _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ func = <function <lambda> at 0x7f061ef39de8>, value = True, timeout = 600 allowed_exceptions = [], initial_wait = 0 def assert_equals_within( func, value, timeout, allowed_exceptions=None, initial_wait=10 ): allowed_exceptions = allowed_exceptions or [] with utils.EggTimer(timeout) as timer: while not timer.elapsed(): try: res = func() if res == value: return except Exception as exc: if _instance_of_any(exc, allowed_exceptions): time.sleep(3) continue LOGGER.exception("Unhandled exception in %s", func) raise if initial_wait == 0: time.sleep(3) else: time.sleep(initial_wait) initial_wait = 0 try: raise AssertionError(
'%s != %s after %s seconds' % (res, value, timeout)
) E AssertionError: False != True after 600 seconds /usr/lib/python2.7/site-packages/ovirtlago/testlib.py:252: AssertionError
Project: https://jenkins.ovirt.org/job/ovirt-system-tests_compat-4.3-suite-master/ Build: https://jenkins.ovirt.org/job/ovirt-system-tests_compat-4.3-suite-master/590... Build Number: 590 Build Status: Still Failing Triggered By: Started by timer ------------------------------------- Changes Since Last Success: ------------------------------------- Changes for Build #581 [Marcin Sobczyk] basic: Port '099_aaa-ldap' test module to ansible Changes for Build #582 [Marcin Sobczyk] basic: Port '099_aaa-ldap' test module to ansible Changes for Build #583 [Marcin Sobczyk] basic: Port '099_aaa-ldap' test module to ansible [Gal Ben Haim] dockefile-utils: Unset env var in skopeo test Changes for Build #584 [Marcin Sobczyk] basic: Port '099_aaa-ldap' test module to ansible Changes for Build #585 [Eitan Raviv] network: host change cluster - lib-erate + objectify Changes for Build #586 [Yaniv Kaul] [WIP] SDKv4 tests. Changes for Build #587 [Yaniv Kaul] [WIP] SDKv4 tests. [Ehud Yonasi] centos7-nodes: updated el7 mirrors links. Changes for Build #588 [Michal Skrivanek] pytest is needed for OST Changes for Build #589 [Michal Skrivanek] pytest is needed for OST Changes for Build #590 [hbraha] network: bond active slave test ----------------- Failed Tests: ----------------- 1 tests failed. FAILED: basic-suite-master.test-scenarios.004_basic_sanity.test_ha_recovery Error Message: AssertionError: False != True after 600 seconds Stack Trace: prefix = <ovirtlago.prefix.OvirtPrefix object at 0x7f8a6af16a90> @order_by(_TEST_LIST) def test_ha_recovery(prefix): engine = prefix.virt_env.engine_vm().get_api_v4().system_service() with test_utils.TestEvent(engine, [119, 9602, 506]): # VM_DOWN_ERROR event(119) # HA_VM_FAILED event event(9602) # VDS_INITIATED_RUN_VM event(506) vm_host = _vm_host(prefix, VM2_NAME) pid = vm_host.ssh(['pgrep', '-f', 'qemu.*guest=vm2']) vm_host.ssh(['kill', '-KILL', pid.out.decode('utf-8')]) vm_service = test_utils.get_vm_service(engine, VM2_NAME) testlib.assert_true_within_long(
lambda:
vm_service.get().status == types.VmStatus.UP ) ../basic-suite-master/test-scenarios/004_basic_sanity.py:819: _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ /usr/lib/python2.7/site-packages/ovirtlago/testlib.py:286: in assert_true_within_long assert_equals_within_long(func, True, allowed_exceptions) /usr/lib/python2.7/site-packages/ovirtlago/testlib.py:273: in assert_equals_within_long func, value, LONG_TIMEOUT, allowed_exceptions=allowed_exceptions _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ func = <function <lambda> at 0x7f8a60886de8>, value = True, timeout = 600 allowed_exceptions = [], initial_wait = 0 def assert_equals_within( func, value, timeout, allowed_exceptions=None, initial_wait=10 ): allowed_exceptions = allowed_exceptions or [] with utils.EggTimer(timeout) as timer: while not timer.elapsed(): try: res = func() if res == value: return except Exception as exc: if _instance_of_any(exc, allowed_exceptions): time.sleep(3) continue LOGGER.exception("Unhandled exception in %s", func) raise if initial_wait == 0: time.sleep(3) else: time.sleep(initial_wait) initial_wait = 0 try: raise AssertionError(
'%s != %s after %s seconds' % (res, value, timeout)
) E AssertionError: False != True after 600 seconds /usr/lib/python2.7/site-packages/ovirtlago/testlib.py:252: AssertionError
Project: https://jenkins.ovirt.org/job/ovirt-system-tests_compat-4.3-suite-master/ Build: https://jenkins.ovirt.org/job/ovirt-system-tests_compat-4.3-suite-master/591... Build Number: 591 Build Status: Still Failing Triggered By: Started by timer ------------------------------------- Changes Since Last Success: ------------------------------------- Changes for Build #582 [Marcin Sobczyk] basic: Port '099_aaa-ldap' test module to ansible Changes for Build #583 [Marcin Sobczyk] basic: Port '099_aaa-ldap' test module to ansible [Gal Ben Haim] dockefile-utils: Unset env var in skopeo test Changes for Build #584 [Marcin Sobczyk] basic: Port '099_aaa-ldap' test module to ansible Changes for Build #585 [Eitan Raviv] network: host change cluster - lib-erate + objectify Changes for Build #586 [Yaniv Kaul] [WIP] SDKv4 tests. Changes for Build #587 [Yaniv Kaul] [WIP] SDKv4 tests. [Ehud Yonasi] centos7-nodes: updated el7 mirrors links. Changes for Build #588 [Michal Skrivanek] pytest is needed for OST Changes for Build #589 [Michal Skrivanek] pytest is needed for OST Changes for Build #590 [hbraha] network: bond active slave test Changes for Build #591 [hbraha] network: bond active slave test [Evgheni Dereveanchin] Change logic for module metadata detection ----------------- Failed Tests: ----------------- 1 tests failed. FAILED: basic-suite-master.test-scenarios.004_basic_sanity.test_ha_recovery Error Message: AssertionError: False != True after 600 seconds Stack Trace: prefix = <ovirtlago.prefix.OvirtPrefix object at 0x7f3f2c318a90> @order_by(_TEST_LIST) def test_ha_recovery(prefix): engine = prefix.virt_env.engine_vm().get_api_v4().system_service() with test_utils.TestEvent(engine, [119, 9602, 506]): # VM_DOWN_ERROR event(119) # HA_VM_FAILED event event(9602) # VDS_INITIATED_RUN_VM event(506) vm_host = _vm_host(prefix, VM2_NAME) pid = vm_host.ssh(['pgrep', '-f', 'qemu.*guest=vm2']) vm_host.ssh(['kill', '-KILL', pid.out.decode('utf-8')]) vm_service = test_utils.get_vm_service(engine, VM2_NAME) testlib.assert_true_within_long(
lambda:
vm_service.get().status == types.VmStatus.UP ) ../basic-suite-master/test-scenarios/004_basic_sanity.py:819: _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ /usr/lib/python2.7/site-packages/ovirtlago/testlib.py:286: in assert_true_within_long assert_equals_within_long(func, True, allowed_exceptions) /usr/lib/python2.7/site-packages/ovirtlago/testlib.py:273: in assert_equals_within_long func, value, LONG_TIMEOUT, allowed_exceptions=allowed_exceptions _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ func = <function <lambda> at 0x7f3f226bd1b8>, value = True, timeout = 600 allowed_exceptions = [], initial_wait = 0 def assert_equals_within( func, value, timeout, allowed_exceptions=None, initial_wait=10 ): allowed_exceptions = allowed_exceptions or [] with utils.EggTimer(timeout) as timer: while not timer.elapsed(): try: res = func() if res == value: return except Exception as exc: if _instance_of_any(exc, allowed_exceptions): time.sleep(3) continue LOGGER.exception("Unhandled exception in %s", func) raise if initial_wait == 0: time.sleep(3) else: time.sleep(initial_wait) initial_wait = 0 try: raise AssertionError(
'%s != %s after %s seconds' % (res, value, timeout)
) E AssertionError: False != True after 600 seconds /usr/lib/python2.7/site-packages/ovirtlago/testlib.py:252: AssertionError
Project: https://jenkins.ovirt.org/job/ovirt-system-tests_compat-4.3-suite-master/ Build: https://jenkins.ovirt.org/job/ovirt-system-tests_compat-4.3-suite-master/592... Build Number: 592 Build Status: Still Failing Triggered By: Started by timer ------------------------------------- Changes Since Last Success: ------------------------------------- Changes for Build #583 [Marcin Sobczyk] basic: Port '099_aaa-ldap' test module to ansible [Gal Ben Haim] dockefile-utils: Unset env var in skopeo test Changes for Build #584 [Marcin Sobczyk] basic: Port '099_aaa-ldap' test module to ansible Changes for Build #585 [Eitan Raviv] network: host change cluster - lib-erate + objectify Changes for Build #586 [Yaniv Kaul] [WIP] SDKv4 tests. Changes for Build #587 [Yaniv Kaul] [WIP] SDKv4 tests. [Ehud Yonasi] centos7-nodes: updated el7 mirrors links. Changes for Build #588 [Michal Skrivanek] pytest is needed for OST Changes for Build #589 [Michal Skrivanek] pytest is needed for OST Changes for Build #590 [hbraha] network: bond active slave test Changes for Build #591 [hbraha] network: bond active slave test [Evgheni Dereveanchin] Change logic for module metadata detection Changes for Build #592 [Marcin Sobczyk] basic: Remove '000_check_repo_closure' test module ----------------- Failed Tests: ----------------- 1 tests failed. FAILED: basic-suite-master.test-scenarios.004_basic_sanity.test_ha_recovery Error Message: AssertionError: False != True after 600 seconds Stack Trace: engine_api = <ovirtsdk4.Connection object at 0x7fdbb64f7e10> get_ansible_host_for_vm = <function ansible_host_for at 0x7fdbb5347cf8> @order_by(_TEST_LIST) def test_ha_recovery(engine_api, get_ansible_host_for_vm): engine = engine_api.system_service() with engine_utils.wait_for_event(engine, [119, 9602, 506]): # VM_DOWN_ERROR event(119) # HA_VM_FAILED event event(9602) # VDS_INITIATED_RUN_VM event(506) ansible_host = get_ansible_host_for_vm(VM2_NAME) pid = ansible_host.shell('pgrep -f qemu.*guest=vm2')['stdout'].strip() ansible_host.shell('kill -KILL {}'.format(pid)) vm_service = test_utils.get_vm_service(engine, VM2_NAME) assertions.assert_true_within_long(
lambda:
vm_service.get().status == types.VmStatus.UP ) ../basic-suite-master/test-scenarios/004_basic_sanity.py:825: _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ ../ost_utils/ost_utils/assertions.py:97: in assert_true_within_long assert_equals_within_long(func, True, allowed_exceptions) ../ost_utils/ost_utils/assertions.py:82: in assert_equals_within_long func, value, LONG_TIMEOUT, allowed_exceptions=allowed_exceptions _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ func = <function <lambda> at 0x7fdbb52c6c80>, value = True, timeout = 600 allowed_exceptions = [], initial_wait = 0 error_message = 'False != True after 600 seconds' def assert_equals_within( func, value, timeout, allowed_exceptions=None, initial_wait=10, error_message=None ): allowed_exceptions = allowed_exceptions or [] with _EggTimer(timeout) as timer: while not timer.elapsed(): try: res = func() if res == value: return except Exception as exc: if _instance_of_any(exc, allowed_exceptions): time.sleep(3) continue LOGGER.exception("Unhandled exception in %s", func) raise if initial_wait == 0: time.sleep(3) else: time.sleep(initial_wait) initial_wait = 0 try: if error_message is None: error_message = '%s != %s after %s seconds' % (res, value, timeout)
raise AssertionError(error_message)
E AssertionError: False != True after 600 seconds ../ost_utils/ost_utils/assertions.py:60: AssertionError
Project: https://jenkins.ovirt.org/job/ovirt-system-tests_compat-4.3-suite-master/ Build: https://jenkins.ovirt.org/job/ovirt-system-tests_compat-4.3-suite-master/593... Build Number: 593 Build Status: Still Failing Triggered By: Started by timer ------------------------------------- Changes Since Last Success: ------------------------------------- Changes for Build #584 [Marcin Sobczyk] basic: Port '099_aaa-ldap' test module to ansible Changes for Build #585 [Eitan Raviv] network: host change cluster - lib-erate + objectify Changes for Build #586 [Yaniv Kaul] [WIP] SDKv4 tests. Changes for Build #587 [Yaniv Kaul] [WIP] SDKv4 tests. [Ehud Yonasi] centos7-nodes: updated el7 mirrors links. Changes for Build #588 [Michal Skrivanek] pytest is needed for OST Changes for Build #589 [Michal Skrivanek] pytest is needed for OST Changes for Build #590 [hbraha] network: bond active slave test Changes for Build #591 [hbraha] network: bond active slave test [Evgheni Dereveanchin] Change logic for module metadata detection Changes for Build #592 [Marcin Sobczyk] basic: Remove '000_check_repo_closure' test module Changes for Build #593 [hbraha] network: fix BondingData constructor ----------------- Failed Tests: ----------------- 1 tests failed. FAILED: basic-suite-master.test-scenarios.004_basic_sanity.test_ha_recovery Error Message: AssertionError: False != True after 600 seconds Stack Trace: engine_api = <ovirtsdk4.Connection object at 0x7f74301f1e90> get_ansible_host_for_vm = <function ansible_host_for at 0x7f74313a5500> @order_by(_TEST_LIST) def test_ha_recovery(engine_api, get_ansible_host_for_vm): engine = engine_api.system_service() with engine_utils.wait_for_event(engine, [119, 9602, 506]): # VM_DOWN_ERROR event(119) # HA_VM_FAILED event event(9602) # VDS_INITIATED_RUN_VM event(506) ansible_host = get_ansible_host_for_vm(VM2_NAME) pid = ansible_host.shell('pgrep -f qemu.*guest=vm2')['stdout'].strip() ansible_host.shell('kill -KILL {}'.format(pid)) vm_service = test_utils.get_vm_service(engine, VM2_NAME) assertions.assert_true_within_long(
lambda:
vm_service.get().status == types.VmStatus.UP ) ../basic-suite-master/test-scenarios/004_basic_sanity.py:825: _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ ../ost_utils/ost_utils/assertions.py:97: in assert_true_within_long assert_equals_within_long(func, True, allowed_exceptions) ../ost_utils/ost_utils/assertions.py:82: in assert_equals_within_long func, value, LONG_TIMEOUT, allowed_exceptions=allowed_exceptions _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ func = <function <lambda> at 0x7f74313e3500>, value = True, timeout = 600 allowed_exceptions = [], initial_wait = 0 error_message = 'False != True after 600 seconds' def assert_equals_within( func, value, timeout, allowed_exceptions=None, initial_wait=10, error_message=None ): allowed_exceptions = allowed_exceptions or [] with _EggTimer(timeout) as timer: while not timer.elapsed(): try: res = func() if res == value: return except Exception as exc: if _instance_of_any(exc, allowed_exceptions): time.sleep(3) continue LOGGER.exception("Unhandled exception in %s", func) raise if initial_wait == 0: time.sleep(3) else: time.sleep(initial_wait) initial_wait = 0 try: if error_message is None: error_message = '%s != %s after %s seconds' % (res, value, timeout)
raise AssertionError(error_message)
E AssertionError: False != True after 600 seconds ../ost_utils/ost_utils/assertions.py:60: AssertionError
Project: https://jenkins.ovirt.org/job/ovirt-system-tests_compat-4.3-suite-master/ Build: https://jenkins.ovirt.org/job/ovirt-system-tests_compat-4.3-suite-master/594... Build Number: 594 Build Status: Still Failing Triggered By: Started by timer ------------------------------------- Changes Since Last Success: ------------------------------------- Changes for Build #585 [Eitan Raviv] network: host change cluster - lib-erate + objectify Changes for Build #586 [Yaniv Kaul] [WIP] SDKv4 tests. Changes for Build #587 [Yaniv Kaul] [WIP] SDKv4 tests. [Ehud Yonasi] centos7-nodes: updated el7 mirrors links. Changes for Build #588 [Michal Skrivanek] pytest is needed for OST Changes for Build #589 [Michal Skrivanek] pytest is needed for OST Changes for Build #590 [hbraha] network: bond active slave test Changes for Build #591 [hbraha] network: bond active slave test [Evgheni Dereveanchin] Change logic for module metadata detection Changes for Build #592 [Marcin Sobczyk] basic: Remove '000_check_repo_closure' test module Changes for Build #593 [hbraha] network: fix BondingData constructor Changes for Build #594 [Radoslaw Szwajkowski] UI test: Add PF4 selectors for Manage vGPU dialog ----------------- Failed Tests: ----------------- 1 tests failed. FAILED: basic-suite-master.test-scenarios.004_basic_sanity.test_ha_recovery Error Message: AssertionError: False != True after 600 seconds Stack Trace: engine_api = <ovirtsdk4.Connection object at 0x7f9b54d37dd0> get_ansible_host_for_vm = <function ansible_host_for at 0x7f9b54c8a9b0> @order_by(_TEST_LIST) def test_ha_recovery(engine_api, get_ansible_host_for_vm): engine = engine_api.system_service() with engine_utils.wait_for_event(engine, [119, 9602, 506]): # VM_DOWN_ERROR event(119) # HA_VM_FAILED event event(9602) # VDS_INITIATED_RUN_VM event(506) ansible_host = get_ansible_host_for_vm(VM2_NAME) pid = ansible_host.shell('pgrep -f qemu.*guest=vm2')['stdout'].strip() ansible_host.shell('kill -KILL {}'.format(pid)) vm_service = test_utils.get_vm_service(engine, VM2_NAME) assertions.assert_true_within_long(
lambda:
vm_service.get().status == types.VmStatus.UP ) ../basic-suite-master/test-scenarios/004_basic_sanity.py:825: _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ ../ost_utils/ost_utils/assertions.py:97: in assert_true_within_long assert_equals_within_long(func, True, allowed_exceptions) ../ost_utils/ost_utils/assertions.py:82: in assert_equals_within_long func, value, LONG_TIMEOUT, allowed_exceptions=allowed_exceptions _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ func = <function <lambda> at 0x7f9b543ed938>, value = True, timeout = 600 allowed_exceptions = [], initial_wait = 0 error_message = 'False != True after 600 seconds' def assert_equals_within( func, value, timeout, allowed_exceptions=None, initial_wait=10, error_message=None ): allowed_exceptions = allowed_exceptions or [] with _EggTimer(timeout) as timer: while not timer.elapsed(): try: res = func() if res == value: return except Exception as exc: if _instance_of_any(exc, allowed_exceptions): time.sleep(3) continue LOGGER.exception("Unhandled exception in %s", func) raise if initial_wait == 0: time.sleep(3) else: time.sleep(initial_wait) initial_wait = 0 try: if error_message is None: error_message = '%s != %s after %s seconds' % (res, value, timeout)
raise AssertionError(error_message)
E AssertionError: False != True after 600 seconds ../ost_utils/ost_utils/assertions.py:60: AssertionError
Project: https://jenkins.ovirt.org/job/ovirt-system-tests_compat-4.3-suite-master/ Build: https://jenkins.ovirt.org/job/ovirt-system-tests_compat-4.3-suite-master/595... Build Number: 595 Build Status: Still Failing Triggered By: Started by timer ------------------------------------- Changes Since Last Success: ------------------------------------- Changes for Build #586 [Yaniv Kaul] [WIP] SDKv4 tests. Changes for Build #587 [Yaniv Kaul] [WIP] SDKv4 tests. [Ehud Yonasi] centos7-nodes: updated el7 mirrors links. Changes for Build #588 [Michal Skrivanek] pytest is needed for OST Changes for Build #589 [Michal Skrivanek] pytest is needed for OST Changes for Build #590 [hbraha] network: bond active slave test Changes for Build #591 [hbraha] network: bond active slave test [Evgheni Dereveanchin] Change logic for module metadata detection Changes for Build #592 [Marcin Sobczyk] basic: Remove '000_check_repo_closure' test module Changes for Build #593 [hbraha] network: fix BondingData constructor Changes for Build #594 [Radoslaw Szwajkowski] UI test: Add PF4 selectors for Manage vGPU dialog Changes for Build #595 [Radoslaw Szwajkowski] UI test: Add PF4 selectors for Manage vGPU dialog ----------------- Failed Tests: ----------------- 1 tests failed. FAILED: basic-suite-master.test-scenarios.004_basic_sanity.test_ha_recovery Error Message: AssertionError: False != True after 600 seconds Stack Trace: engine_api = <ovirtsdk4.Connection object at 0x7fcf3ce4ced0> get_ansible_host_for_vm = <function ansible_host_for at 0x7fcf3dec4a28> @order_by(_TEST_LIST) def test_ha_recovery(engine_api, get_ansible_host_for_vm): engine = engine_api.system_service() with engine_utils.wait_for_event(engine, [119, 9602, 506]): # VM_DOWN_ERROR event(119) # HA_VM_FAILED event event(9602) # VDS_INITIATED_RUN_VM event(506) ansible_host = get_ansible_host_for_vm(VM2_NAME) pid = ansible_host.shell('pgrep -f qemu.*guest=vm2')['stdout'].strip() ansible_host.shell('kill -KILL {}'.format(pid)) vm_service = test_utils.get_vm_service(engine, VM2_NAME) assertions.assert_true_within_long(
lambda:
vm_service.get().status == types.VmStatus.UP ) ../basic-suite-master/test-scenarios/004_basic_sanity.py:825: _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ ../ost_utils/ost_utils/assertions.py:97: in assert_true_within_long assert_equals_within_long(func, True, allowed_exceptions) ../ost_utils/ost_utils/assertions.py:82: in assert_equals_within_long func, value, LONG_TIMEOUT, allowed_exceptions=allowed_exceptions _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ func = <function <lambda> at 0x7fcf3cd8ba28>, value = True, timeout = 600 allowed_exceptions = [], initial_wait = 0 error_message = 'False != True after 600 seconds' def assert_equals_within( func, value, timeout, allowed_exceptions=None, initial_wait=10, error_message=None ): allowed_exceptions = allowed_exceptions or [] with _EggTimer(timeout) as timer: while not timer.elapsed(): try: res = func() if res == value: return except Exception as exc: if _instance_of_any(exc, allowed_exceptions): time.sleep(3) continue LOGGER.exception("Unhandled exception in %s", func) raise if initial_wait == 0: time.sleep(3) else: time.sleep(initial_wait) initial_wait = 0 try: if error_message is None: error_message = '%s != %s after %s seconds' % (res, value, timeout)
raise AssertionError(error_message)
E AssertionError: False != True after 600 seconds ../ost_utils/ost_utils/assertions.py:60: AssertionError
Project: https://jenkins.ovirt.org/job/ovirt-system-tests_compat-4.3-suite-master/ Build: https://jenkins.ovirt.org/job/ovirt-system-tests_compat-4.3-suite-master/596... Build Number: 596 Build Status: Still Failing Triggered By: Started by timer ------------------------------------- Changes Since Last Success: ------------------------------------- Changes for Build #587 [Yaniv Kaul] [WIP] SDKv4 tests. [Ehud Yonasi] centos7-nodes: updated el7 mirrors links. Changes for Build #588 [Michal Skrivanek] pytest is needed for OST Changes for Build #589 [Michal Skrivanek] pytest is needed for OST Changes for Build #590 [hbraha] network: bond active slave test Changes for Build #591 [hbraha] network: bond active slave test [Evgheni Dereveanchin] Change logic for module metadata detection Changes for Build #592 [Marcin Sobczyk] basic: Remove '000_check_repo_closure' test module Changes for Build #593 [hbraha] network: fix BondingData constructor Changes for Build #594 [Radoslaw Szwajkowski] UI test: Add PF4 selectors for Manage vGPU dialog Changes for Build #595 [Radoslaw Szwajkowski] UI test: Add PF4 selectors for Manage vGPU dialog Changes for Build #596 [Radoslaw Szwajkowski] UI test: Add PF4 selectors for Manage vGPU dialog ----------------- Failed Tests: ----------------- 1 tests failed. FAILED: basic-suite-master.test-scenarios.004_basic_sanity.test_ha_recovery Error Message: AssertionError: False != True after 600 seconds Stack Trace: engine_api = <ovirtsdk4.Connection object at 0x7fc934539c90> get_ansible_host_for_vm = <function ansible_host_for at 0x7fc9345087d0> @order_by(_TEST_LIST) def test_ha_recovery(engine_api, get_ansible_host_for_vm): engine = engine_api.system_service() with engine_utils.wait_for_event(engine, [119, 9602, 506]): # VM_DOWN_ERROR event(119) # HA_VM_FAILED event event(9602) # VDS_INITIATED_RUN_VM event(506) ansible_host = get_ansible_host_for_vm(VM2_NAME) pid = ansible_host.shell('pgrep -f qemu.*guest=vm2')['stdout'].strip() ansible_host.shell('kill -KILL {}'.format(pid)) vm_service = test_utils.get_vm_service(engine, VM2_NAME) assertions.assert_true_within_long(
lambda:
vm_service.get().status == types.VmStatus.UP ) ../basic-suite-master/test-scenarios/004_basic_sanity.py:825: _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ ../ost_utils/ost_utils/assertions.py:97: in assert_true_within_long assert_equals_within_long(func, True, allowed_exceptions) ../ost_utils/ost_utils/assertions.py:82: in assert_equals_within_long func, value, LONG_TIMEOUT, allowed_exceptions=allowed_exceptions _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ func = <function <lambda> at 0x7fc934383668>, value = True, timeout = 600 allowed_exceptions = [], initial_wait = 0 error_message = 'False != True after 600 seconds' def assert_equals_within( func, value, timeout, allowed_exceptions=None, initial_wait=10, error_message=None ): allowed_exceptions = allowed_exceptions or [] with _EggTimer(timeout) as timer: while not timer.elapsed(): try: res = func() if res == value: return except Exception as exc: if _instance_of_any(exc, allowed_exceptions): time.sleep(3) continue LOGGER.exception("Unhandled exception in %s", func) raise if initial_wait == 0: time.sleep(3) else: time.sleep(initial_wait) initial_wait = 0 try: if error_message is None: error_message = '%s != %s after %s seconds' % (res, value, timeout)
raise AssertionError(error_message)
E AssertionError: False != True after 600 seconds ../ost_utils/ost_utils/assertions.py:60: AssertionError
Project: https://jenkins.ovirt.org/job/ovirt-system-tests_compat-4.3-suite-master/ Build: https://jenkins.ovirt.org/job/ovirt-system-tests_compat-4.3-suite-master/597... Build Number: 597 Build Status: Still Failing Triggered By: Started by timer ------------------------------------- Changes Since Last Success: ------------------------------------- Changes for Build #588 [Michal Skrivanek] pytest is needed for OST Changes for Build #589 [Michal Skrivanek] pytest is needed for OST Changes for Build #590 [hbraha] network: bond active slave test Changes for Build #591 [hbraha] network: bond active slave test [Evgheni Dereveanchin] Change logic for module metadata detection Changes for Build #592 [Marcin Sobczyk] basic: Remove '000_check_repo_closure' test module Changes for Build #593 [hbraha] network: fix BondingData constructor Changes for Build #594 [Radoslaw Szwajkowski] UI test: Add PF4 selectors for Manage vGPU dialog Changes for Build #595 [Radoslaw Szwajkowski] UI test: Add PF4 selectors for Manage vGPU dialog Changes for Build #596 [Radoslaw Szwajkowski] UI test: Add PF4 selectors for Manage vGPU dialog Changes for Build #597 [Radoslaw Szwajkowski] UI test: Add PF4 selectors for Manage vGPU dialog ----------------- Failed Tests: ----------------- 1 tests failed. FAILED: basic-suite-master.test-scenarios.004_basic_sanity.test_ha_recovery Error Message: AssertionError: False != True after 600 seconds Stack Trace: engine_api = <ovirtsdk4.Connection object at 0x7fbf75309e90> get_ansible_host_for_vm = <function ansible_host_for at 0x7fbf7645c1b8> @order_by(_TEST_LIST) def test_ha_recovery(engine_api, get_ansible_host_for_vm): engine = engine_api.system_service() with engine_utils.wait_for_event(engine, [119, 9602, 506]): # VM_DOWN_ERROR event(119) # HA_VM_FAILED event event(9602) # VDS_INITIATED_RUN_VM event(506) ansible_host = get_ansible_host_for_vm(VM2_NAME) pid = ansible_host.shell('pgrep -f qemu.*guest=vm2')['stdout'].strip() ansible_host.shell('kill -KILL {}'.format(pid)) vm_service = test_utils.get_vm_service(engine, VM2_NAME) assertions.assert_true_within_long(
lambda:
vm_service.get().status == types.VmStatus.UP ) ../basic-suite-master/test-scenarios/004_basic_sanity.py:825: _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ ../ost_utils/ost_utils/assertions.py:97: in assert_true_within_long assert_equals_within_long(func, True, allowed_exceptions) ../ost_utils/ost_utils/assertions.py:82: in assert_equals_within_long func, value, LONG_TIMEOUT, allowed_exceptions=allowed_exceptions _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ func = <function <lambda> at 0x7fbf7535b6e0>, value = True, timeout = 600 allowed_exceptions = [], initial_wait = 0 error_message = 'False != True after 600 seconds' def assert_equals_within( func, value, timeout, allowed_exceptions=None, initial_wait=10, error_message=None ): allowed_exceptions = allowed_exceptions or [] with _EggTimer(timeout) as timer: while not timer.elapsed(): try: res = func() if res == value: return except Exception as exc: if _instance_of_any(exc, allowed_exceptions): time.sleep(3) continue LOGGER.exception("Unhandled exception in %s", func) raise if initial_wait == 0: time.sleep(3) else: time.sleep(initial_wait) initial_wait = 0 try: if error_message is None: error_message = '%s != %s after %s seconds' % (res, value, timeout)
raise AssertionError(error_message)
E AssertionError: False != True after 600 seconds ../ost_utils/ost_utils/assertions.py:60: AssertionError
Project: https://jenkins.ovirt.org/job/ovirt-system-tests_compat-4.3-suite-master/ Build: https://jenkins.ovirt.org/job/ovirt-system-tests_compat-4.3-suite-master/598... Build Number: 598 Build Status: Still Failing Triggered By: Started by timer ------------------------------------- Changes Since Last Success: ------------------------------------- Changes for Build #589 [Michal Skrivanek] pytest is needed for OST Changes for Build #590 [hbraha] network: bond active slave test Changes for Build #591 [hbraha] network: bond active slave test [Evgheni Dereveanchin] Change logic for module metadata detection Changes for Build #592 [Marcin Sobczyk] basic: Remove '000_check_repo_closure' test module Changes for Build #593 [hbraha] network: fix BondingData constructor Changes for Build #594 [Radoslaw Szwajkowski] UI test: Add PF4 selectors for Manage vGPU dialog Changes for Build #595 [Radoslaw Szwajkowski] UI test: Add PF4 selectors for Manage vGPU dialog Changes for Build #596 [Radoslaw Szwajkowski] UI test: Add PF4 selectors for Manage vGPU dialog Changes for Build #597 [Radoslaw Szwajkowski] UI test: Add PF4 selectors for Manage vGPU dialog Changes for Build #598 [Andrej Cernek] network: add second copy_host_network test ----------------- Failed Tests: ----------------- 1 tests failed. FAILED: basic-suite-master.test-scenarios.004_basic_sanity.test_ha_recovery Error Message: AssertionError: False != True after 600 seconds Stack Trace: engine_api = <ovirtsdk4.Connection object at 0x7f97d8bb6f90> get_ansible_host_for_vm = <function ansible_host_for at 0x7f97d8135668> @order_by(_TEST_LIST) def test_ha_recovery(engine_api, get_ansible_host_for_vm): engine = engine_api.system_service() with engine_utils.wait_for_event(engine, [119, 9602, 506]): # VM_DOWN_ERROR event(119) # HA_VM_FAILED event event(9602) # VDS_INITIATED_RUN_VM event(506) ansible_host = get_ansible_host_for_vm(VM2_NAME) pid = ansible_host.shell('pgrep -f qemu.*guest=vm2')['stdout'].strip() ansible_host.shell('kill -KILL {}'.format(pid)) vm_service = test_utils.get_vm_service(engine, VM2_NAME) assertions.assert_true_within_long(
lambda:
vm_service.get().status == types.VmStatus.UP ) ../basic-suite-master/test-scenarios/004_basic_sanity.py:825: _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ ../ost_utils/ost_utils/assertions.py:97: in assert_true_within_long assert_equals_within_long(func, True, allowed_exceptions) ../ost_utils/ost_utils/assertions.py:82: in assert_equals_within_long func, value, LONG_TIMEOUT, allowed_exceptions=allowed_exceptions _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ func = <function <lambda> at 0x7f97d2d847d0>, value = True, timeout = 600 allowed_exceptions = [], initial_wait = 0 error_message = 'False != True after 600 seconds' def assert_equals_within( func, value, timeout, allowed_exceptions=None, initial_wait=10, error_message=None ): allowed_exceptions = allowed_exceptions or [] with _EggTimer(timeout) as timer: while not timer.elapsed(): try: res = func() if res == value: return except Exception as exc: if _instance_of_any(exc, allowed_exceptions): time.sleep(3) continue LOGGER.exception("Unhandled exception in %s", func) raise if initial_wait == 0: time.sleep(3) else: time.sleep(initial_wait) initial_wait = 0 try: if error_message is None: error_message = '%s != %s after %s seconds' % (res, value, timeout)
raise AssertionError(error_message)
E AssertionError: False != True after 600 seconds ../ost_utils/ost_utils/assertions.py:60: AssertionError
Project: https://jenkins.ovirt.org/job/ovirt-system-tests_compat-4.3-suite-master/ Build: https://jenkins.ovirt.org/job/ovirt-system-tests_compat-4.3-suite-master/599... Build Number: 599 Build Status: Still Failing Triggered By: Started by timer ------------------------------------- Changes Since Last Success: ------------------------------------- Changes for Build #590 [hbraha] network: bond active slave test Changes for Build #591 [hbraha] network: bond active slave test [Evgheni Dereveanchin] Change logic for module metadata detection Changes for Build #592 [Marcin Sobczyk] basic: Remove '000_check_repo_closure' test module Changes for Build #593 [hbraha] network: fix BondingData constructor Changes for Build #594 [Radoslaw Szwajkowski] UI test: Add PF4 selectors for Manage vGPU dialog Changes for Build #595 [Radoslaw Szwajkowski] UI test: Add PF4 selectors for Manage vGPU dialog Changes for Build #596 [Radoslaw Szwajkowski] UI test: Add PF4 selectors for Manage vGPU dialog Changes for Build #597 [Radoslaw Szwajkowski] UI test: Add PF4 selectors for Manage vGPU dialog Changes for Build #598 [Andrej Cernek] network: add second copy_host_network test Changes for Build #599 [Eitan Raviv] network: lib-erate create snapshot [Evgheni Dereveanchin] Modularity: replace empty var ----------------- Failed Tests: ----------------- 1 tests failed. FAILED: basic-suite-master.test-scenarios.004_basic_sanity.test_ha_recovery Error Message: AssertionError: False != True after 600 seconds Stack Trace: engine_api = <ovirtsdk4.Connection object at 0x7f3e49f34ad0> get_ansible_host_for_vm = <function ansible_host_for at 0x7f3e4b08cc80> @order_by(_TEST_LIST) def test_ha_recovery(engine_api, get_ansible_host_for_vm): engine = engine_api.system_service() with engine_utils.wait_for_event(engine, [119, 9602, 506]): # VM_DOWN_ERROR event(119) # HA_VM_FAILED event event(9602) # VDS_INITIATED_RUN_VM event(506) ansible_host = get_ansible_host_for_vm(VM2_NAME) pid = ansible_host.shell('pgrep -f qemu.*guest=vm2')['stdout'].strip() ansible_host.shell('kill -KILL {}'.format(pid)) vm_service = test_utils.get_vm_service(engine, VM2_NAME) assertions.assert_true_within_long(
lambda:
vm_service.get().status == types.VmStatus.UP ) ../basic-suite-master/test-scenarios/004_basic_sanity.py:825: _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ ../ost_utils/ost_utils/assertions.py:97: in assert_true_within_long assert_equals_within_long(func, True, allowed_exceptions) ../ost_utils/ost_utils/assertions.py:82: in assert_equals_within_long func, value, LONG_TIMEOUT, allowed_exceptions=allowed_exceptions _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ func = <function <lambda> at 0x7f3e4961ff50>, value = True, timeout = 600 allowed_exceptions = [], initial_wait = 0 error_message = 'False != True after 600 seconds' def assert_equals_within( func, value, timeout, allowed_exceptions=None, initial_wait=10, error_message=None ): allowed_exceptions = allowed_exceptions or [] with _EggTimer(timeout) as timer: while not timer.elapsed(): try: res = func() if res == value: return except Exception as exc: if _instance_of_any(exc, allowed_exceptions): time.sleep(3) continue LOGGER.exception("Unhandled exception in %s", func) raise if initial_wait == 0: time.sleep(3) else: time.sleep(initial_wait) initial_wait = 0 try: if error_message is None: error_message = '%s != %s after %s seconds' % (res, value, timeout)
raise AssertionError(error_message)
E AssertionError: False != True after 600 seconds ../ost_utils/ost_utils/assertions.py:60: AssertionError
Project: https://jenkins.ovirt.org/job/ovirt-system-tests_compat-4.3-suite-master/ Build: https://jenkins.ovirt.org/job/ovirt-system-tests_compat-4.3-suite-master/600... Build Number: 600 Build Status: Still Failing Triggered By: Started by timer ------------------------------------- Changes Since Last Success: ------------------------------------- Changes for Build #591 [hbraha] network: bond active slave test [Evgheni Dereveanchin] Change logic for module metadata detection Changes for Build #592 [Marcin Sobczyk] basic: Remove '000_check_repo_closure' test module Changes for Build #593 [hbraha] network: fix BondingData constructor Changes for Build #594 [Radoslaw Szwajkowski] UI test: Add PF4 selectors for Manage vGPU dialog Changes for Build #595 [Radoslaw Szwajkowski] UI test: Add PF4 selectors for Manage vGPU dialog Changes for Build #596 [Radoslaw Szwajkowski] UI test: Add PF4 selectors for Manage vGPU dialog Changes for Build #597 [Radoslaw Szwajkowski] UI test: Add PF4 selectors for Manage vGPU dialog Changes for Build #598 [Andrej Cernek] network: add second copy_host_network test Changes for Build #599 [Eitan Raviv] network: lib-erate create snapshot [Evgheni Dereveanchin] Modularity: replace empty var Changes for Build #600 [Andrej Cernek] network: use setters where possible ----------------- Failed Tests: ----------------- 1 tests failed. FAILED: basic-suite-master.test-scenarios.004_basic_sanity.test_ha_recovery Error Message: AssertionError: False != True after 600 seconds Stack Trace: engine_api = <ovirtsdk4.Connection object at 0x7fc2106bccd0> get_ansible_host_for_vm = <function ansible_host_for at 0x7fc2114e1500> @order_by(_TEST_LIST) def test_ha_recovery(engine_api, get_ansible_host_for_vm): engine = engine_api.system_service() with engine_utils.wait_for_event(engine, [119, 9602, 506]): # VM_DOWN_ERROR event(119) # HA_VM_FAILED event event(9602) # VDS_INITIATED_RUN_VM event(506) ansible_host = get_ansible_host_for_vm(VM2_NAME) pid = ansible_host.shell('pgrep -f qemu.*guest=vm2')['stdout'].strip() ansible_host.shell('kill -KILL {}'.format(pid)) vm_service = test_utils.get_vm_service(engine, VM2_NAME) assertions.assert_true_within_long(
lambda:
vm_service.get().status == types.VmStatus.UP ) ../basic-suite-master/test-scenarios/004_basic_sanity.py:825: _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ ../ost_utils/ost_utils/assertions.py:97: in assert_true_within_long assert_equals_within_long(func, True, allowed_exceptions) ../ost_utils/ost_utils/assertions.py:82: in assert_equals_within_long func, value, LONG_TIMEOUT, allowed_exceptions=allowed_exceptions _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ func = <function <lambda> at 0x7fc21055a938>, value = True, timeout = 600 allowed_exceptions = [], initial_wait = 0 error_message = 'False != True after 600 seconds' def assert_equals_within( func, value, timeout, allowed_exceptions=None, initial_wait=10, error_message=None ): allowed_exceptions = allowed_exceptions or [] with _EggTimer(timeout) as timer: while not timer.elapsed(): try: res = func() if res == value: return except Exception as exc: if _instance_of_any(exc, allowed_exceptions): time.sleep(3) continue LOGGER.exception("Unhandled exception in %s", func) raise if initial_wait == 0: time.sleep(3) else: time.sleep(initial_wait) initial_wait = 0 try: if error_message is None: error_message = '%s != %s after %s seconds' % (res, value, timeout)
raise AssertionError(error_message)
E AssertionError: False != True after 600 seconds ../ost_utils/ost_utils/assertions.py:60: AssertionError
Project: https://jenkins.ovirt.org/job/ovirt-system-tests_compat-4.3-suite-master/ Build: https://jenkins.ovirt.org/job/ovirt-system-tests_compat-4.3-suite-master/601... Build Number: 601 Build Status: Still Failing Triggered By: Started by timer ------------------------------------- Changes Since Last Success: ------------------------------------- Changes for Build #592 [Marcin Sobczyk] basic: Remove '000_check_repo_closure' test module Changes for Build #593 [hbraha] network: fix BondingData constructor Changes for Build #594 [Radoslaw Szwajkowski] UI test: Add PF4 selectors for Manage vGPU dialog Changes for Build #595 [Radoslaw Szwajkowski] UI test: Add PF4 selectors for Manage vGPU dialog Changes for Build #596 [Radoslaw Szwajkowski] UI test: Add PF4 selectors for Manage vGPU dialog Changes for Build #597 [Radoslaw Szwajkowski] UI test: Add PF4 selectors for Manage vGPU dialog Changes for Build #598 [Andrej Cernek] network: add second copy_host_network test Changes for Build #599 [Eitan Raviv] network: lib-erate create snapshot [Evgheni Dereveanchin] Modularity: replace empty var Changes for Build #600 [Andrej Cernek] network: use setters where possible Changes for Build #601 [Eitan Raviv] network: skip mac pool check used mac below 4.4 ----------------- Failed Tests: ----------------- 1 tests failed. FAILED: basic-suite-master.test-scenarios.004_basic_sanity.test_ha_recovery Error Message: AssertionError: False != True after 600 seconds Stack Trace: engine_api = <ovirtsdk4.Connection object at 0x7f5bcc364350> get_ansible_host_for_vm = <function ansible_host_for at 0x7f5bcc307e60> @order_by(_TEST_LIST) def test_ha_recovery(engine_api, get_ansible_host_for_vm): engine = engine_api.system_service() with engine_utils.wait_for_event(engine, [119, 9602, 506]): # VM_DOWN_ERROR event(119) # HA_VM_FAILED event event(9602) # VDS_INITIATED_RUN_VM event(506) ansible_host = get_ansible_host_for_vm(VM2_NAME) pid = ansible_host.shell('pgrep -f qemu.*guest=vm2')['stdout'].strip() ansible_host.shell('kill -KILL {}'.format(pid)) vm_service = test_utils.get_vm_service(engine, VM2_NAME) assertions.assert_true_within_long(
lambda:
vm_service.get().status == types.VmStatus.UP ) ../basic-suite-master/test-scenarios/004_basic_sanity.py:825: _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ ../ost_utils/ost_utils/assertions.py:97: in assert_true_within_long assert_equals_within_long(func, True, allowed_exceptions) ../ost_utils/ost_utils/assertions.py:82: in assert_equals_within_long func, value, LONG_TIMEOUT, allowed_exceptions=allowed_exceptions _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ func = <function <lambda> at 0x7f5bcc307c08>, value = True, timeout = 600 allowed_exceptions = [], initial_wait = 0 error_message = 'False != True after 600 seconds' def assert_equals_within( func, value, timeout, allowed_exceptions=None, initial_wait=10, error_message=None ): allowed_exceptions = allowed_exceptions or [] with _EggTimer(timeout) as timer: while not timer.elapsed(): try: res = func() if res == value: return except Exception as exc: if _instance_of_any(exc, allowed_exceptions): time.sleep(3) continue LOGGER.exception("Unhandled exception in %s", func) raise if initial_wait == 0: time.sleep(3) else: time.sleep(initial_wait) initial_wait = 0 try: if error_message is None: error_message = '%s != %s after %s seconds' % (res, value, timeout)
raise AssertionError(error_message)
E AssertionError: False != True after 600 seconds ../ost_utils/ost_utils/assertions.py:60: AssertionError
Project: https://jenkins.ovirt.org/job/ovirt-system-tests_compat-4.3-suite-master/ Build: https://jenkins.ovirt.org/job/ovirt-system-tests_compat-4.3-suite-master/602... Build Number: 602 Build Status: Still Failing Triggered By: Started by timer ------------------------------------- Changes Since Last Success: ------------------------------------- Changes for Build #593 [hbraha] network: fix BondingData constructor Changes for Build #594 [Radoslaw Szwajkowski] UI test: Add PF4 selectors for Manage vGPU dialog Changes for Build #595 [Radoslaw Szwajkowski] UI test: Add PF4 selectors for Manage vGPU dialog Changes for Build #596 [Radoslaw Szwajkowski] UI test: Add PF4 selectors for Manage vGPU dialog Changes for Build #597 [Radoslaw Szwajkowski] UI test: Add PF4 selectors for Manage vGPU dialog Changes for Build #598 [Andrej Cernek] network: add second copy_host_network test Changes for Build #599 [Eitan Raviv] network: lib-erate create snapshot [Evgheni Dereveanchin] Modularity: replace empty var Changes for Build #600 [Andrej Cernek] network: use setters where possible Changes for Build #601 [Eitan Raviv] network: skip mac pool check used mac below 4.4 Changes for Build #602 [Marcin Sobczyk] setup: Add setup playbook and script ----------------- Failed Tests: ----------------- 1 tests failed. FAILED: basic-suite-master.test-scenarios.004_basic_sanity.test_ha_recovery Error Message: AssertionError: False != True after 600 seconds Stack Trace: engine_api = <ovirtsdk4.Connection object at 0x7f80b3d81f90> get_ansible_host_for_vm = <function ansible_host_for at 0x7f80b8ab0230> @order_by(_TEST_LIST) def test_ha_recovery(engine_api, get_ansible_host_for_vm): engine = engine_api.system_service() with engine_utils.wait_for_event(engine, [119, 9602, 506]): # VM_DOWN_ERROR event(119) # HA_VM_FAILED event event(9602) # VDS_INITIATED_RUN_VM event(506) ansible_host = get_ansible_host_for_vm(VM2_NAME) pid = ansible_host.shell('pgrep -f qemu.*guest=vm2')['stdout'].strip() ansible_host.shell('kill -KILL {}'.format(pid)) vm_service = test_utils.get_vm_service(engine, VM2_NAME) assertions.assert_true_within_long(
lambda:
vm_service.get().status == types.VmStatus.UP ) ../basic-suite-master/test-scenarios/004_basic_sanity.py:825: _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ ../ost_utils/ost_utils/assertions.py:97: in assert_true_within_long assert_equals_within_long(func, True, allowed_exceptions) ../ost_utils/ost_utils/assertions.py:82: in assert_equals_within_long func, value, LONG_TIMEOUT, allowed_exceptions=allowed_exceptions _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ func = <function <lambda> at 0x7f80b2be2b90>, value = True, timeout = 600 allowed_exceptions = [], initial_wait = 0 error_message = 'False != True after 600 seconds' def assert_equals_within( func, value, timeout, allowed_exceptions=None, initial_wait=10, error_message=None ): allowed_exceptions = allowed_exceptions or [] with _EggTimer(timeout) as timer: while not timer.elapsed(): try: res = func() if res == value: return except Exception as exc: if _instance_of_any(exc, allowed_exceptions): time.sleep(3) continue LOGGER.exception("Unhandled exception in %s", func) raise if initial_wait == 0: time.sleep(3) else: time.sleep(initial_wait) initial_wait = 0 try: if error_message is None: error_message = '%s != %s after %s seconds' % (res, value, timeout)
raise AssertionError(error_message)
E AssertionError: False != True after 600 seconds ../ost_utils/ost_utils/assertions.py:60: AssertionError
Project: https://jenkins.ovirt.org/job/ovirt-system-tests_compat-4.3-suite-master/ Build: https://jenkins.ovirt.org/job/ovirt-system-tests_compat-4.3-suite-master/603... Build Number: 603 Build Status: Still Failing Triggered By: Started by timer ------------------------------------- Changes Since Last Success: ------------------------------------- Changes for Build #594 [Radoslaw Szwajkowski] UI test: Add PF4 selectors for Manage vGPU dialog Changes for Build #595 [Radoslaw Szwajkowski] UI test: Add PF4 selectors for Manage vGPU dialog Changes for Build #596 [Radoslaw Szwajkowski] UI test: Add PF4 selectors for Manage vGPU dialog Changes for Build #597 [Radoslaw Szwajkowski] UI test: Add PF4 selectors for Manage vGPU dialog Changes for Build #598 [Andrej Cernek] network: add second copy_host_network test Changes for Build #599 [Eitan Raviv] network: lib-erate create snapshot [Evgheni Dereveanchin] Modularity: replace empty var Changes for Build #600 [Andrej Cernek] network: use setters where possible Changes for Build #601 [Eitan Raviv] network: skip mac pool check used mac below 4.4 Changes for Build #602 [Marcin Sobczyk] setup: Add setup playbook and script Changes for Build #603 [Marcin Sobczyk] basic: Don't resolve to ip in 'assert_vm_is_alive' ----------------- Failed Tests: ----------------- 1 tests failed. FAILED: basic-suite-master.test-scenarios.004_basic_sanity.test_ha_recovery Error Message: AssertionError: False != True after 600 seconds Stack Trace: engine_api = <ovirtsdk4.Connection object at 0x7f51505df910> get_ansible_host_for_vm = <function ansible_host_for at 0x7f51517250c8> @order_by(_TEST_LIST) def test_ha_recovery(engine_api, get_ansible_host_for_vm): engine = engine_api.system_service() with engine_utils.wait_for_event(engine, [119, 9602, 506]): # VM_DOWN_ERROR event(119) # HA_VM_FAILED event event(9602) # VDS_INITIATED_RUN_VM event(506) ansible_host = get_ansible_host_for_vm(VM2_NAME) pid = ansible_host.shell('pgrep -f qemu.*guest=vm2')['stdout'].strip() ansible_host.shell('kill -KILL {}'.format(pid)) vm_service = test_utils.get_vm_service(engine, VM2_NAME) assertions.assert_true_within_long(
lambda:
vm_service.get().status == types.VmStatus.UP ) ../basic-suite-master/test-scenarios/004_basic_sanity.py:824: _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ ../ost_utils/ost_utils/assertions.py:97: in assert_true_within_long assert_equals_within_long(func, True, allowed_exceptions) ../ost_utils/ost_utils/assertions.py:82: in assert_equals_within_long func, value, LONG_TIMEOUT, allowed_exceptions=allowed_exceptions _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ func = <function <lambda> at 0x7f515040cde8>, value = True, timeout = 600 allowed_exceptions = [], initial_wait = 0 error_message = 'False != True after 600 seconds' def assert_equals_within( func, value, timeout, allowed_exceptions=None, initial_wait=10, error_message=None ): allowed_exceptions = allowed_exceptions or [] with _EggTimer(timeout) as timer: while not timer.elapsed(): try: res = func() if res == value: return except Exception as exc: if _instance_of_any(exc, allowed_exceptions): time.sleep(3) continue LOGGER.exception("Unhandled exception in %s", func) raise if initial_wait == 0: time.sleep(3) else: time.sleep(initial_wait) initial_wait = 0 try: if error_message is None: error_message = '%s != %s after %s seconds' % (res, value, timeout)
raise AssertionError(error_message)
E AssertionError: False != True after 600 seconds ../ost_utils/ost_utils/assertions.py:60: AssertionError
Project: https://jenkins.ovirt.org/job/ovirt-system-tests_compat-4.3-suite-master/ Build: https://jenkins.ovirt.org/job/ovirt-system-tests_compat-4.3-suite-master/604... Build Number: 604 Build Status: Still Failing Triggered By: Started by timer ------------------------------------- Changes Since Last Success: ------------------------------------- Changes for Build #595 [Radoslaw Szwajkowski] UI test: Add PF4 selectors for Manage vGPU dialog Changes for Build #596 [Radoslaw Szwajkowski] UI test: Add PF4 selectors for Manage vGPU dialog Changes for Build #597 [Radoslaw Szwajkowski] UI test: Add PF4 selectors for Manage vGPU dialog Changes for Build #598 [Andrej Cernek] network: add second copy_host_network test Changes for Build #599 [Eitan Raviv] network: lib-erate create snapshot [Evgheni Dereveanchin] Modularity: replace empty var Changes for Build #600 [Andrej Cernek] network: use setters where possible Changes for Build #601 [Eitan Raviv] network: skip mac pool check used mac below 4.4 Changes for Build #602 [Marcin Sobczyk] setup: Add setup playbook and script Changes for Build #603 [Marcin Sobczyk] basic: Don't resolve to ip in 'assert_vm_is_alive' Changes for Build #604 [Marcin Sobczyk] basic: Don't resolve to ip in 'assert_vm_is_alive' ----------------- Failed Tests: ----------------- 1 tests failed. FAILED: basic-suite-master.test-scenarios.004_basic_sanity.test_ha_recovery Error Message: AssertionError: False != True after 600 seconds Stack Trace: engine_api = <ovirtsdk4.Connection object at 0x7faf57013910> get_ansible_host_for_vm = <function ansible_host_for at 0x7faf57079b90> @order_by(_TEST_LIST) def test_ha_recovery(engine_api, get_ansible_host_for_vm): engine = engine_api.system_service() with engine_utils.wait_for_event(engine, [119, 9602, 506]): # VM_DOWN_ERROR event(119) # HA_VM_FAILED event event(9602) # VDS_INITIATED_RUN_VM event(506) ansible_host = get_ansible_host_for_vm(VM2_NAME) pid = ansible_host.shell('pgrep -f qemu.*guest=vm2')['stdout'].strip() ansible_host.shell('kill -KILL {}'.format(pid)) vm_service = test_utils.get_vm_service(engine, VM2_NAME) assertions.assert_true_within_long(
lambda:
vm_service.get().status == types.VmStatus.UP ) ../basic-suite-master/test-scenarios/004_basic_sanity.py:824: _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ ../ost_utils/ost_utils/assertions.py:97: in assert_true_within_long assert_equals_within_long(func, True, allowed_exceptions) ../ost_utils/ost_utils/assertions.py:82: in assert_equals_within_long func, value, LONG_TIMEOUT, allowed_exceptions=allowed_exceptions _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ func = <function <lambda> at 0x7faf5552e758>, value = True, timeout = 600 allowed_exceptions = [], initial_wait = 0 error_message = 'False != True after 600 seconds' def assert_equals_within( func, value, timeout, allowed_exceptions=None, initial_wait=10, error_message=None ): allowed_exceptions = allowed_exceptions or [] with _EggTimer(timeout) as timer: while not timer.elapsed(): try: res = func() if res == value: return except Exception as exc: if _instance_of_any(exc, allowed_exceptions): time.sleep(3) continue LOGGER.exception("Unhandled exception in %s", func) raise if initial_wait == 0: time.sleep(3) else: time.sleep(initial_wait) initial_wait = 0 try: if error_message is None: error_message = '%s != %s after %s seconds' % (res, value, timeout)
raise AssertionError(error_message)
E AssertionError: False != True after 600 seconds ../ost_utils/ost_utils/assertions.py:60: AssertionError
Project: https://jenkins.ovirt.org/job/ovirt-system-tests_compat-4.3-suite-master/ Build: https://jenkins.ovirt.org/job/ovirt-system-tests_compat-4.3-suite-master/605... Build Number: 605 Build Status: Still Failing Triggered By: Started by timer ------------------------------------- Changes Since Last Success: ------------------------------------- Changes for Build #596 [Radoslaw Szwajkowski] UI test: Add PF4 selectors for Manage vGPU dialog Changes for Build #597 [Radoslaw Szwajkowski] UI test: Add PF4 selectors for Manage vGPU dialog Changes for Build #598 [Andrej Cernek] network: add second copy_host_network test Changes for Build #599 [Eitan Raviv] network: lib-erate create snapshot [Evgheni Dereveanchin] Modularity: replace empty var Changes for Build #600 [Andrej Cernek] network: use setters where possible Changes for Build #601 [Eitan Raviv] network: skip mac pool check used mac below 4.4 Changes for Build #602 [Marcin Sobczyk] setup: Add setup playbook and script Changes for Build #603 [Marcin Sobczyk] basic: Don't resolve to ip in 'assert_vm_is_alive' Changes for Build #604 [Marcin Sobczyk] basic: Don't resolve to ip in 'assert_vm_is_alive' Changes for Build #605 [Marcin Sobczyk] basic: Improve 'assert_vm_is_alive' ----------------- Failed Tests: ----------------- 1 tests failed. FAILED: basic-suite-master.test-scenarios.004_basic_sanity.test_ha_recovery Error Message: AssertionError: False != True after 600 seconds Stack Trace: engine_api = <ovirtsdk4.Connection object at 0x7f04c57d27d0> get_ansible_host_for_vm = <function ansible_host_for at 0x7f04c591ded8> @order_by(_TEST_LIST) def test_ha_recovery(engine_api, get_ansible_host_for_vm): engine = engine_api.system_service() with engine_utils.wait_for_event(engine, [119, 9602, 506]): # VM_DOWN_ERROR event(119) # HA_VM_FAILED event event(9602) # VDS_INITIATED_RUN_VM event(506) ansible_host = get_ansible_host_for_vm(VM2_NAME) pid = ansible_host.shell('pgrep -f qemu.*guest=vm2')['stdout'].strip() ansible_host.shell('kill -KILL {}'.format(pid)) vm_service = test_utils.get_vm_service(engine, VM2_NAME) assertions.assert_true_within_long(
lambda:
vm_service.get().status == types.VmStatus.UP ) ../basic-suite-master/test-scenarios/004_basic_sanity.py:825: _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ ../ost_utils/ost_utils/assertions.py:97: in assert_true_within_long assert_equals_within_long(func, True, allowed_exceptions) ../ost_utils/ost_utils/assertions.py:82: in assert_equals_within_long func, value, LONG_TIMEOUT, allowed_exceptions=allowed_exceptions _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ func = <function <lambda> at 0x7f04c65499b0>, value = True, timeout = 600 allowed_exceptions = [], initial_wait = 0 error_message = 'False != True after 600 seconds' def assert_equals_within( func, value, timeout, allowed_exceptions=None, initial_wait=10, error_message=None ): allowed_exceptions = allowed_exceptions or [] with _EggTimer(timeout) as timer: while not timer.elapsed(): try: res = func() if res == value: return except Exception as exc: if _instance_of_any(exc, allowed_exceptions): time.sleep(3) continue LOGGER.exception("Unhandled exception in %s", func) raise if initial_wait == 0: time.sleep(3) else: time.sleep(initial_wait) initial_wait = 0 try: if error_message is None: error_message = '%s != %s after %s seconds' % (res, value, timeout)
raise AssertionError(error_message)
E AssertionError: False != True after 600 seconds ../ost_utils/ost_utils/assertions.py:60: AssertionError
Project: https://jenkins.ovirt.org/job/ovirt-system-tests_compat-4.3-suite-master/ Build: https://jenkins.ovirt.org/job/ovirt-system-tests_compat-4.3-suite-master/606... Build Number: 606 Build Status: Still Failing Triggered By: Started by timer ------------------------------------- Changes Since Last Success: ------------------------------------- Changes for Build #597 [Radoslaw Szwajkowski] UI test: Add PF4 selectors for Manage vGPU dialog Changes for Build #598 [Andrej Cernek] network: add second copy_host_network test Changes for Build #599 [Eitan Raviv] network: lib-erate create snapshot [Evgheni Dereveanchin] Modularity: replace empty var Changes for Build #600 [Andrej Cernek] network: use setters where possible Changes for Build #601 [Eitan Raviv] network: skip mac pool check used mac below 4.4 Changes for Build #602 [Marcin Sobczyk] setup: Add setup playbook and script Changes for Build #603 [Marcin Sobczyk] basic: Don't resolve to ip in 'assert_vm_is_alive' Changes for Build #604 [Marcin Sobczyk] basic: Don't resolve to ip in 'assert_vm_is_alive' Changes for Build #605 [Marcin Sobczyk] basic: Improve 'assert_vm_is_alive' Changes for Build #606 [hbraha] network: remove calls to lago env on fixture.host ----------------- Failed Tests: ----------------- 1 tests failed. FAILED: basic-suite-master.test-scenarios.004_basic_sanity.test_ha_recovery Error Message: AssertionError: False != True after 600 seconds Stack Trace: engine_api = <ovirtsdk4.Connection object at 0x7fe7ca5d88d0> get_ansible_host_for_vm = <function ansible_host_for at 0x7fe7ca5be8c0> @order_by(_TEST_LIST) def test_ha_recovery(engine_api, get_ansible_host_for_vm): engine = engine_api.system_service() with engine_utils.wait_for_event(engine, [119, 9602, 506]): # VM_DOWN_ERROR event(119) # HA_VM_FAILED event event(9602) # VDS_INITIATED_RUN_VM event(506) ansible_host = get_ansible_host_for_vm(VM2_NAME) pid = ansible_host.shell('pgrep -f qemu.*guest=vm2')['stdout'].strip() ansible_host.shell('kill -KILL {}'.format(pid)) vm_service = test_utils.get_vm_service(engine, VM2_NAME) assertions.assert_true_within_long(
lambda:
vm_service.get().status == types.VmStatus.UP ) ../basic-suite-master/test-scenarios/004_basic_sanity.py:825: _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ ../ost_utils/ost_utils/assertions.py:97: in assert_true_within_long assert_equals_within_long(func, True, allowed_exceptions) ../ost_utils/ost_utils/assertions.py:82: in assert_equals_within_long func, value, LONG_TIMEOUT, allowed_exceptions=allowed_exceptions _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ func = <function <lambda> at 0x7fe7ca5be9b0>, value = True, timeout = 600 allowed_exceptions = [], initial_wait = 0 error_message = 'False != True after 600 seconds' def assert_equals_within( func, value, timeout, allowed_exceptions=None, initial_wait=10, error_message=None ): allowed_exceptions = allowed_exceptions or [] with _EggTimer(timeout) as timer: while not timer.elapsed(): try: res = func() if res == value: return except Exception as exc: if _instance_of_any(exc, allowed_exceptions): time.sleep(3) continue LOGGER.exception("Unhandled exception in %s", func) raise if initial_wait == 0: time.sleep(3) else: time.sleep(initial_wait) initial_wait = 0 try: if error_message is None: error_message = '%s != %s after %s seconds' % (res, value, timeout)
raise AssertionError(error_message)
E AssertionError: False != True after 600 seconds ../ost_utils/ost_utils/assertions.py:60: AssertionError
Project: https://jenkins.ovirt.org/job/ovirt-system-tests_compat-4.3-suite-master/ Build: https://jenkins.ovirt.org/job/ovirt-system-tests_compat-4.3-suite-master/607... Build Number: 607 Build Status: Still Failing Triggered By: Started by timer ------------------------------------- Changes Since Last Success: ------------------------------------- Changes for Build #598 [Andrej Cernek] network: add second copy_host_network test Changes for Build #599 [Eitan Raviv] network: lib-erate create snapshot [Evgheni Dereveanchin] Modularity: replace empty var Changes for Build #600 [Andrej Cernek] network: use setters where possible Changes for Build #601 [Eitan Raviv] network: skip mac pool check used mac below 4.4 Changes for Build #602 [Marcin Sobczyk] setup: Add setup playbook and script Changes for Build #603 [Marcin Sobczyk] basic: Don't resolve to ip in 'assert_vm_is_alive' Changes for Build #604 [Marcin Sobczyk] basic: Don't resolve to ip in 'assert_vm_is_alive' Changes for Build #605 [Marcin Sobczyk] basic: Improve 'assert_vm_is_alive' Changes for Build #606 [hbraha] network: remove calls to lago env on fixture.host Changes for Build #607 [hbraha] network: remove calls to lago env on fixture.host ----------------- Failed Tests: ----------------- 1 tests failed. FAILED: basic-suite-master.test-scenarios.004_basic_sanity.test_ha_recovery Error Message: AssertionError: False != True after 600 seconds Stack Trace: engine_api = <ovirtsdk4.Connection object at 0x7f4768028fd0> get_ansible_host_for_vm = <function ansible_host_for at 0x7f4768a37a28> @order_by(_TEST_LIST) def test_ha_recovery(engine_api, get_ansible_host_for_vm): engine = engine_api.system_service() with engine_utils.wait_for_event(engine, [119, 9602, 506]): # VM_DOWN_ERROR event(119) # HA_VM_FAILED event event(9602) # VDS_INITIATED_RUN_VM event(506) ansible_host = get_ansible_host_for_vm(VM2_NAME) pid = ansible_host.shell('pgrep -f qemu.*guest=vm2')['stdout'].strip() ansible_host.shell('kill -KILL {}'.format(pid)) vm_service = test_utils.get_vm_service(engine, VM2_NAME) assertions.assert_true_within_long(
lambda:
vm_service.get().status == types.VmStatus.UP ) ../basic-suite-master/test-scenarios/004_basic_sanity.py:825: _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ ../ost_utils/ost_utils/assertions.py:97: in assert_true_within_long assert_equals_within_long(func, True, allowed_exceptions) ../ost_utils/ost_utils/assertions.py:82: in assert_equals_within_long func, value, LONG_TIMEOUT, allowed_exceptions=allowed_exceptions _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ func = <function <lambda> at 0x7f4769684938>, value = True, timeout = 600 allowed_exceptions = [], initial_wait = 0 error_message = 'False != True after 600 seconds' def assert_equals_within( func, value, timeout, allowed_exceptions=None, initial_wait=10, error_message=None ): allowed_exceptions = allowed_exceptions or [] with _EggTimer(timeout) as timer: while not timer.elapsed(): try: res = func() if res == value: return except Exception as exc: if _instance_of_any(exc, allowed_exceptions): time.sleep(3) continue LOGGER.exception("Unhandled exception in %s", func) raise if initial_wait == 0: time.sleep(3) else: time.sleep(initial_wait) initial_wait = 0 try: if error_message is None: error_message = '%s != %s after %s seconds' % (res, value, timeout)
raise AssertionError(error_message)
E AssertionError: False != True after 600 seconds ../ost_utils/ost_utils/assertions.py:60: AssertionError
Project: https://jenkins.ovirt.org/job/ovirt-system-tests_compat-4.3-suite-master/ Build: https://jenkins.ovirt.org/job/ovirt-system-tests_compat-4.3-suite-master/608... Build Number: 608 Build Status: Still Failing Triggered By: Started by timer ------------------------------------- Changes Since Last Success: ------------------------------------- Changes for Build #599 [Eitan Raviv] network: lib-erate create snapshot [Evgheni Dereveanchin] Modularity: replace empty var Changes for Build #600 [Andrej Cernek] network: use setters where possible Changes for Build #601 [Eitan Raviv] network: skip mac pool check used mac below 4.4 Changes for Build #602 [Marcin Sobczyk] setup: Add setup playbook and script Changes for Build #603 [Marcin Sobczyk] basic: Don't resolve to ip in 'assert_vm_is_alive' Changes for Build #604 [Marcin Sobczyk] basic: Don't resolve to ip in 'assert_vm_is_alive' Changes for Build #605 [Marcin Sobczyk] basic: Improve 'assert_vm_is_alive' Changes for Build #606 [hbraha] network: remove calls to lago env on fixture.host Changes for Build #607 [hbraha] network: remove calls to lago env on fixture.host Changes for Build #608 [Marcin Sobczyk] network-4.3: Fix missing 'ost_utils' module [Galit] Updated manual test custom report guide ----------------- Failed Tests: ----------------- 1 tests failed. FAILED: basic-suite-master.test-scenarios.004_basic_sanity.test_ha_recovery Error Message: AssertionError: False != True after 600 seconds Stack Trace: engine_api = <ovirtsdk4.Connection object at 0x7f87cc259ed0> get_ansible_host_for_vm = <function ansible_host_for at 0x7f87cd8d0320> @order_by(_TEST_LIST) def test_ha_recovery(engine_api, get_ansible_host_for_vm): engine = engine_api.system_service() with engine_utils.wait_for_event(engine, [119, 9602, 506]): # VM_DOWN_ERROR event(119) # HA_VM_FAILED event event(9602) # VDS_INITIATED_RUN_VM event(506) ansible_host = get_ansible_host_for_vm(VM2_NAME) pid = ansible_host.shell('pgrep -f qemu.*guest=vm2')['stdout'].strip() ansible_host.shell('kill -KILL {}'.format(pid)) vm_service = test_utils.get_vm_service(engine, VM2_NAME) assertions.assert_true_within_long(
lambda:
vm_service.get().status == types.VmStatus.UP ) ../basic-suite-master/test-scenarios/004_basic_sanity.py:825: _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ ../ost_utils/ost_utils/assertions.py:97: in assert_true_within_long assert_equals_within_long(func, True, allowed_exceptions) ../ost_utils/ost_utils/assertions.py:82: in assert_equals_within_long func, value, LONG_TIMEOUT, allowed_exceptions=allowed_exceptions _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ func = <function <lambda> at 0x7f87cc0d5e60>, value = True, timeout = 600 allowed_exceptions = [], initial_wait = 0 error_message = 'False != True after 600 seconds' def assert_equals_within( func, value, timeout, allowed_exceptions=None, initial_wait=10, error_message=None ): allowed_exceptions = allowed_exceptions or [] with _EggTimer(timeout) as timer: while not timer.elapsed(): try: res = func() if res == value: return except Exception as exc: if _instance_of_any(exc, allowed_exceptions): time.sleep(3) continue LOGGER.exception("Unhandled exception in %s", func) raise if initial_wait == 0: time.sleep(3) else: time.sleep(initial_wait) initial_wait = 0 try: if error_message is None: error_message = '%s != %s after %s seconds' % (res, value, timeout)
raise AssertionError(error_message)
E AssertionError: False != True after 600 seconds ../ost_utils/ost_utils/assertions.py:60: AssertionError
Project: https://jenkins.ovirt.org/job/ovirt-system-tests_compat-4.3-suite-master/ Build: https://jenkins.ovirt.org/job/ovirt-system-tests_compat-4.3-suite-master/609... Build Number: 609 Build Status: Still Failing Triggered By: Started by timer ------------------------------------- Changes Since Last Success: ------------------------------------- Changes for Build #600 [Andrej Cernek] network: use setters where possible Changes for Build #601 [Eitan Raviv] network: skip mac pool check used mac below 4.4 Changes for Build #602 [Marcin Sobczyk] setup: Add setup playbook and script Changes for Build #603 [Marcin Sobczyk] basic: Don't resolve to ip in 'assert_vm_is_alive' Changes for Build #604 [Marcin Sobczyk] basic: Don't resolve to ip in 'assert_vm_is_alive' Changes for Build #605 [Marcin Sobczyk] basic: Improve 'assert_vm_is_alive' Changes for Build #606 [hbraha] network: remove calls to lago env on fixture.host Changes for Build #607 [hbraha] network: remove calls to lago env on fixture.host Changes for Build #608 [Marcin Sobczyk] network-4.3: Fix missing 'ost_utils' module [Galit] Updated manual test custom report guide Changes for Build #609 [parthdhanjal] Updating gluster-inventory file ----------------- Failed Tests: ----------------- 1 tests failed. FAILED: basic-suite-master.test-scenarios.004_basic_sanity.test_ha_recovery Error Message: AssertionError: False != True after 600 seconds Stack Trace: engine_api = <ovirtsdk4.Connection object at 0x7fd69212b510> get_ansible_host_for_vm = <function ansible_host_for at 0x7fd6932b12a8> @order_by(_TEST_LIST) def test_ha_recovery(engine_api, get_ansible_host_for_vm): engine = engine_api.system_service() with engine_utils.wait_for_event(engine, [119, 9602, 506]): # VM_DOWN_ERROR event(119) # HA_VM_FAILED event event(9602) # VDS_INITIATED_RUN_VM event(506) ansible_host = get_ansible_host_for_vm(VM2_NAME) pid = ansible_host.shell('pgrep -f qemu.*guest=vm2')['stdout'].strip() ansible_host.shell('kill -KILL {}'.format(pid)) vm_service = test_utils.get_vm_service(engine, VM2_NAME) assertions.assert_true_within_long(
lambda:
vm_service.get().status == types.VmStatus.UP ) ../basic-suite-master/test-scenarios/004_basic_sanity.py:825: _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ ../ost_utils/ost_utils/assertions.py:97: in assert_true_within_long assert_equals_within_long(func, True, allowed_exceptions) ../ost_utils/ost_utils/assertions.py:82: in assert_equals_within_long func, value, LONG_TIMEOUT, allowed_exceptions=allowed_exceptions _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ func = <function <lambda> at 0x7fd69177ade8>, value = True, timeout = 600 allowed_exceptions = [], initial_wait = 0 error_message = 'False != True after 600 seconds' def assert_equals_within( func, value, timeout, allowed_exceptions=None, initial_wait=10, error_message=None ): allowed_exceptions = allowed_exceptions or [] with _EggTimer(timeout) as timer: while not timer.elapsed(): try: res = func() if res == value: return except Exception as exc: if _instance_of_any(exc, allowed_exceptions): time.sleep(3) continue LOGGER.exception("Unhandled exception in %s", func) raise if initial_wait == 0: time.sleep(3) else: time.sleep(initial_wait) initial_wait = 0 try: if error_message is None: error_message = '%s != %s after %s seconds' % (res, value, timeout)
raise AssertionError(error_message)
E AssertionError: False != True after 600 seconds ../ost_utils/ost_utils/assertions.py:60: AssertionError
Project: https://jenkins.ovirt.org/job/ovirt-system-tests_compat-4.3-suite-master/ Build: https://jenkins.ovirt.org/job/ovirt-system-tests_compat-4.3-suite-master/610... Build Number: 610 Build Status: Still Failing Triggered By: Started by timer ------------------------------------- Changes Since Last Success: ------------------------------------- Changes for Build #601 [Eitan Raviv] network: skip mac pool check used mac below 4.4 Changes for Build #602 [Marcin Sobczyk] setup: Add setup playbook and script Changes for Build #603 [Marcin Sobczyk] basic: Don't resolve to ip in 'assert_vm_is_alive' Changes for Build #604 [Marcin Sobczyk] basic: Don't resolve to ip in 'assert_vm_is_alive' Changes for Build #605 [Marcin Sobczyk] basic: Improve 'assert_vm_is_alive' Changes for Build #606 [hbraha] network: remove calls to lago env on fixture.host Changes for Build #607 [hbraha] network: remove calls to lago env on fixture.host Changes for Build #608 [Marcin Sobczyk] network-4.3: Fix missing 'ost_utils' module [Galit] Updated manual test custom report guide Changes for Build #609 [parthdhanjal] Updating gluster-inventory file Changes for Build #610 [parthdhanjal] Updating gluster-inventory file ----------------- Failed Tests: ----------------- 1 tests failed. FAILED: basic-suite-master.test-scenarios.004_basic_sanity.test_ha_recovery Error Message: AssertionError: False != True after 600 seconds Stack Trace: engine_api = <ovirtsdk4.Connection object at 0x7f822ae79850> get_ansible_host_for_vm = <function ansible_host_for at 0x7f822bc9e230> @order_by(_TEST_LIST) def test_ha_recovery(engine_api, get_ansible_host_for_vm): engine = engine_api.system_service() with engine_utils.wait_for_event(engine, [119, 9602, 506]): # VM_DOWN_ERROR event(119) # HA_VM_FAILED event event(9602) # VDS_INITIATED_RUN_VM event(506) ansible_host = get_ansible_host_for_vm(VM2_NAME) pid = ansible_host.shell('pgrep -f qemu.*guest=vm2')['stdout'].strip() ansible_host.shell('kill -KILL {}'.format(pid)) vm_service = test_utils.get_vm_service(engine, VM2_NAME) assertions.assert_true_within_long(
lambda:
vm_service.get().status == types.VmStatus.UP ) ../basic-suite-master/test-scenarios/004_basic_sanity.py:825: _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ ../ost_utils/ost_utils/assertions.py:97: in assert_true_within_long assert_equals_within_long(func, True, allowed_exceptions) ../ost_utils/ost_utils/assertions.py:82: in assert_equals_within_long func, value, LONG_TIMEOUT, allowed_exceptions=allowed_exceptions _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ func = <function <lambda> at 0x7f822af56c08>, value = True, timeout = 600 allowed_exceptions = [], initial_wait = 0 error_message = 'False != True after 600 seconds' def assert_equals_within( func, value, timeout, allowed_exceptions=None, initial_wait=10, error_message=None ): allowed_exceptions = allowed_exceptions or [] with _EggTimer(timeout) as timer: while not timer.elapsed(): try: res = func() if res == value: return except Exception as exc: if _instance_of_any(exc, allowed_exceptions): time.sleep(3) continue LOGGER.exception("Unhandled exception in %s", func) raise if initial_wait == 0: time.sleep(3) else: time.sleep(initial_wait) initial_wait = 0 try: if error_message is None: error_message = '%s != %s after %s seconds' % (res, value, timeout)
raise AssertionError(error_message)
E AssertionError: False != True after 600 seconds ../ost_utils/ost_utils/assertions.py:60: AssertionError
Project: https://jenkins.ovirt.org/job/ovirt-system-tests_compat-4.3-suite-master/ Build: https://jenkins.ovirt.org/job/ovirt-system-tests_compat-4.3-suite-master/611... Build Number: 611 Build Status: Still Failing Triggered By: Started by timer ------------------------------------- Changes Since Last Success: ------------------------------------- Changes for Build #602 [Marcin Sobczyk] setup: Add setup playbook and script Changes for Build #603 [Marcin Sobczyk] basic: Don't resolve to ip in 'assert_vm_is_alive' Changes for Build #604 [Marcin Sobczyk] basic: Don't resolve to ip in 'assert_vm_is_alive' Changes for Build #605 [Marcin Sobczyk] basic: Improve 'assert_vm_is_alive' Changes for Build #606 [hbraha] network: remove calls to lago env on fixture.host Changes for Build #607 [hbraha] network: remove calls to lago env on fixture.host Changes for Build #608 [Marcin Sobczyk] network-4.3: Fix missing 'ost_utils' module [Galit] Updated manual test custom report guide Changes for Build #609 [parthdhanjal] Updating gluster-inventory file Changes for Build #610 [parthdhanjal] Updating gluster-inventory file Changes for Build #611 [hbraha] network: remove xfail from active bond test ----------------- Failed Tests: ----------------- 1 tests failed. FAILED: basic-suite-master.test-scenarios.004_basic_sanity.test_ha_recovery Error Message: AssertionError: False != True after 600 seconds Stack Trace: engine_api = <ovirtsdk4.Connection object at 0x7fc9a278df10> get_ansible_host_for_vm = <function ansible_host_for at 0x7fc9a3840050> @order_by(_TEST_LIST) def test_ha_recovery(engine_api, get_ansible_host_for_vm): engine = engine_api.system_service() with engine_utils.wait_for_event(engine, [119, 9602, 506]): # VM_DOWN_ERROR event(119) # HA_VM_FAILED event event(9602) # VDS_INITIATED_RUN_VM event(506) ansible_host = get_ansible_host_for_vm(VM2_NAME) pid = ansible_host.shell('pgrep -f qemu.*guest=vm2')['stdout'].strip() ansible_host.shell('kill -KILL {}'.format(pid)) vm_service = test_utils.get_vm_service(engine, VM2_NAME) assertions.assert_true_within_long(
lambda:
vm_service.get().status == types.VmStatus.UP ) ../basic-suite-master/test-scenarios/004_basic_sanity.py:825: _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ ../ost_utils/ost_utils/assertions.py:97: in assert_true_within_long assert_equals_within_long(func, True, allowed_exceptions) ../ost_utils/ost_utils/assertions.py:82: in assert_equals_within_long func, value, LONG_TIMEOUT, allowed_exceptions=allowed_exceptions _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ func = <function <lambda> at 0x7fc9a8555d70>, value = True, timeout = 600 allowed_exceptions = [], initial_wait = 0 error_message = 'False != True after 600 seconds' def assert_equals_within( func, value, timeout, allowed_exceptions=None, initial_wait=10, error_message=None ): allowed_exceptions = allowed_exceptions or [] with _EggTimer(timeout) as timer: while not timer.elapsed(): try: res = func() if res == value: return except Exception as exc: if _instance_of_any(exc, allowed_exceptions): time.sleep(3) continue LOGGER.exception("Unhandled exception in %s", func) raise if initial_wait == 0: time.sleep(3) else: time.sleep(initial_wait) initial_wait = 0 try: if error_message is None: error_message = '%s != %s after %s seconds' % (res, value, timeout)
raise AssertionError(error_message)
E AssertionError: False != True after 600 seconds ../ost_utils/ost_utils/assertions.py:60: AssertionError
Project: https://jenkins.ovirt.org/job/ovirt-system-tests_compat-4.3-suite-master/ Build: https://jenkins.ovirt.org/job/ovirt-system-tests_compat-4.3-suite-master/612... Build Number: 612 Build Status: Still Failing Triggered By: Started by timer ------------------------------------- Changes Since Last Success: ------------------------------------- Changes for Build #603 [Marcin Sobczyk] basic: Don't resolve to ip in 'assert_vm_is_alive' Changes for Build #604 [Marcin Sobczyk] basic: Don't resolve to ip in 'assert_vm_is_alive' Changes for Build #605 [Marcin Sobczyk] basic: Improve 'assert_vm_is_alive' Changes for Build #606 [hbraha] network: remove calls to lago env on fixture.host Changes for Build #607 [hbraha] network: remove calls to lago env on fixture.host Changes for Build #608 [Marcin Sobczyk] network-4.3: Fix missing 'ost_utils' module [Galit] Updated manual test custom report guide Changes for Build #609 [parthdhanjal] Updating gluster-inventory file Changes for Build #610 [parthdhanjal] Updating gluster-inventory file Changes for Build #611 [hbraha] network: remove xfail from active bond test Changes for Build #612 [hbraha] network: remove lago engine_ip call [Galit] Add dparth mail to hc-basic ----------------- Failed Tests: ----------------- 1 tests failed. FAILED: basic-suite-master.test-scenarios.004_basic_sanity.test_ha_recovery Error Message: AssertionError: False != True after 600 seconds Stack Trace: engine_api = <ovirtsdk4.Connection object at 0x7f832e341c10> get_ansible_host_for_vm = <function ansible_host_for at 0x7f833408d7d0> @order_by(_TEST_LIST) def test_ha_recovery(engine_api, get_ansible_host_for_vm): engine = engine_api.system_service() with engine_utils.wait_for_event(engine, [119, 9602, 506]): # VM_DOWN_ERROR event(119) # HA_VM_FAILED event event(9602) # VDS_INITIATED_RUN_VM event(506) ansible_host = get_ansible_host_for_vm(VM2_NAME) pid = ansible_host.shell('pgrep -f qemu.*guest=vm2')['stdout'].strip() ansible_host.shell('kill -KILL {}'.format(pid)) vm_service = test_utils.get_vm_service(engine, VM2_NAME) assertions.assert_true_within_long(
lambda:
vm_service.get().status == types.VmStatus.UP ) ../basic-suite-master/test-scenarios/004_basic_sanity.py:825: _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ ../ost_utils/ost_utils/assertions.py:97: in assert_true_within_long assert_equals_within_long(func, True, allowed_exceptions) ../ost_utils/ost_utils/assertions.py:82: in assert_equals_within_long func, value, LONG_TIMEOUT, allowed_exceptions=allowed_exceptions _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ func = <function <lambda> at 0x7f832e364848>, value = True, timeout = 600 allowed_exceptions = [], initial_wait = 0 error_message = 'False != True after 600 seconds' def assert_equals_within( func, value, timeout, allowed_exceptions=None, initial_wait=10, error_message=None ): allowed_exceptions = allowed_exceptions or [] with _EggTimer(timeout) as timer: while not timer.elapsed(): try: res = func() if res == value: return except Exception as exc: if _instance_of_any(exc, allowed_exceptions): time.sleep(3) continue LOGGER.exception("Unhandled exception in %s", func) raise if initial_wait == 0: time.sleep(3) else: time.sleep(initial_wait) initial_wait = 0 try: if error_message is None: error_message = '%s != %s after %s seconds' % (res, value, timeout)
raise AssertionError(error_message)
E AssertionError: False != True after 600 seconds ../ost_utils/ost_utils/assertions.py:60: AssertionError
Project: https://jenkins.ovirt.org/job/ovirt-system-tests_compat-4.3-suite-master/ Build: https://jenkins.ovirt.org/job/ovirt-system-tests_compat-4.3-suite-master/613... Build Number: 613 Build Status: Still Failing Triggered By: Started by timer ------------------------------------- Changes Since Last Success: ------------------------------------- Changes for Build #604 [Marcin Sobczyk] basic: Don't resolve to ip in 'assert_vm_is_alive' Changes for Build #605 [Marcin Sobczyk] basic: Improve 'assert_vm_is_alive' Changes for Build #606 [hbraha] network: remove calls to lago env on fixture.host Changes for Build #607 [hbraha] network: remove calls to lago env on fixture.host Changes for Build #608 [Marcin Sobczyk] network-4.3: Fix missing 'ost_utils' module [Galit] Updated manual test custom report guide Changes for Build #609 [parthdhanjal] Updating gluster-inventory file Changes for Build #610 [parthdhanjal] Updating gluster-inventory file Changes for Build #611 [hbraha] network: remove xfail from active bond test Changes for Build #612 [hbraha] network: remove lago engine_ip call [Galit] Add dparth mail to hc-basic Changes for Build #613 [Eyal Shenitzky] basic_sanity.py: remove unused time import ----------------- Failed Tests: ----------------- 1 tests failed. FAILED: basic-suite-master.test-scenarios.004_basic_sanity.test_ha_recovery Error Message: AssertionError: False != True after 600 seconds Stack Trace: engine_api = <ovirtsdk4.Connection object at 0x7fd586d7ed10> get_ansible_host_for_vm = <function ansible_host_for at 0x7fd58c0fc938> @order_by(_TEST_LIST) def test_ha_recovery(engine_api, get_ansible_host_for_vm): engine = engine_api.system_service() with engine_utils.wait_for_event(engine, [119, 9602, 506]): # VM_DOWN_ERROR event(119) # HA_VM_FAILED event event(9602) # VDS_INITIATED_RUN_VM event(506) ansible_host = get_ansible_host_for_vm(VM2_NAME) pid = ansible_host.shell('pgrep -f qemu.*guest=vm2')['stdout'].strip() ansible_host.shell('kill -KILL {}'.format(pid)) vm_service = test_utils.get_vm_service(engine, VM2_NAME) assertions.assert_true_within_long(
lambda:
vm_service.get().status == types.VmStatus.UP ) ../basic-suite-master/test-scenarios/004_basic_sanity.py:824: _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ ../ost_utils/ost_utils/assertions.py:97: in assert_true_within_long assert_equals_within_long(func, True, allowed_exceptions) ../ost_utils/ost_utils/assertions.py:82: in assert_equals_within_long func, value, LONG_TIMEOUT, allowed_exceptions=allowed_exceptions _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ func = <function <lambda> at 0x7fd58feef758>, value = True, timeout = 600 allowed_exceptions = [], initial_wait = 0 error_message = 'False != True after 600 seconds' def assert_equals_within( func, value, timeout, allowed_exceptions=None, initial_wait=10, error_message=None ): allowed_exceptions = allowed_exceptions or [] with _EggTimer(timeout) as timer: while not timer.elapsed(): try: res = func() if res == value: return except Exception as exc: if _instance_of_any(exc, allowed_exceptions): time.sleep(3) continue LOGGER.exception("Unhandled exception in %s", func) raise if initial_wait == 0: time.sleep(3) else: time.sleep(initial_wait) initial_wait = 0 try: if error_message is None: error_message = '%s != %s after %s seconds' % (res, value, timeout)
raise AssertionError(error_message)
E AssertionError: False != True after 600 seconds ../ost_utils/ost_utils/assertions.py:60: AssertionError
Project: https://jenkins.ovirt.org/job/ovirt-system-tests_compat-4.3-suite-master/ Build: https://jenkins.ovirt.org/job/ovirt-system-tests_compat-4.3-suite-master/614... Build Number: 614 Build Status: Still Failing Triggered By: Started by timer ------------------------------------- Changes Since Last Success: ------------------------------------- Changes for Build #605 [Marcin Sobczyk] basic: Improve 'assert_vm_is_alive' Changes for Build #606 [hbraha] network: remove calls to lago env on fixture.host Changes for Build #607 [hbraha] network: remove calls to lago env on fixture.host Changes for Build #608 [Marcin Sobczyk] network-4.3: Fix missing 'ost_utils' module [Galit] Updated manual test custom report guide Changes for Build #609 [parthdhanjal] Updating gluster-inventory file Changes for Build #610 [parthdhanjal] Updating gluster-inventory file Changes for Build #611 [hbraha] network: remove xfail from active bond test Changes for Build #612 [hbraha] network: remove lago engine_ip call [Galit] Add dparth mail to hc-basic Changes for Build #613 [Eyal Shenitzky] basic_sanity.py: remove unused time import Changes for Build #614 [Eitan Raviv] network: remove workaround for BZ 1752282 ----------------- Failed Tests: ----------------- 1 tests failed. FAILED: basic-suite-master.test-scenarios.004_basic_sanity.test_ha_recovery Error Message: AssertionError: False != True after 600 seconds Stack Trace: engine_api = <ovirtsdk4.Connection object at 0x7f6df0359ad0> get_ansible_host_for_vm = <function ansible_host_for at 0x7f6df0c381b8> @order_by(_TEST_LIST) def test_ha_recovery(engine_api, get_ansible_host_for_vm): engine = engine_api.system_service() with engine_utils.wait_for_event(engine, [119, 9602, 506]): # VM_DOWN_ERROR event(119) # HA_VM_FAILED event event(9602) # VDS_INITIATED_RUN_VM event(506) ansible_host = get_ansible_host_for_vm(VM2_NAME) pid = ansible_host.shell('pgrep -f qemu.*guest=vm2')['stdout'].strip() ansible_host.shell('kill -KILL {}'.format(pid)) vm_service = test_utils.get_vm_service(engine, VM2_NAME) assertions.assert_true_within_long(
lambda:
vm_service.get().status == types.VmStatus.UP ) ../basic-suite-master/test-scenarios/004_basic_sanity.py:824: _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ ../ost_utils/ost_utils/assertions.py:97: in assert_true_within_long assert_equals_within_long(func, True, allowed_exceptions) ../ost_utils/ost_utils/assertions.py:82: in assert_equals_within_long func, value, LONG_TIMEOUT, allowed_exceptions=allowed_exceptions _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ func = <function <lambda> at 0x7f6df5dc4410>, value = True, timeout = 600 allowed_exceptions = [], initial_wait = 0 error_message = 'False != True after 600 seconds' def assert_equals_within( func, value, timeout, allowed_exceptions=None, initial_wait=10, error_message=None ): allowed_exceptions = allowed_exceptions or [] with _EggTimer(timeout) as timer: while not timer.elapsed(): try: res = func() if res == value: return except Exception as exc: if _instance_of_any(exc, allowed_exceptions): time.sleep(3) continue LOGGER.exception("Unhandled exception in %s", func) raise if initial_wait == 0: time.sleep(3) else: time.sleep(initial_wait) initial_wait = 0 try: if error_message is None: error_message = '%s != %s after %s seconds' % (res, value, timeout)
raise AssertionError(error_message)
E AssertionError: False != True after 600 seconds ../ost_utils/ost_utils/assertions.py:60: AssertionError
Project: https://jenkins.ovirt.org/job/ovirt-system-tests_compat-4.3-suite-master/ Build: https://jenkins.ovirt.org/job/ovirt-system-tests_compat-4.3-suite-master/615... Build Number: 615 Build Status: Still Failing Triggered By: Started by timer ------------------------------------- Changes Since Last Success: ------------------------------------- Changes for Build #606 [hbraha] network: remove calls to lago env on fixture.host Changes for Build #607 [hbraha] network: remove calls to lago env on fixture.host Changes for Build #608 [Marcin Sobczyk] network-4.3: Fix missing 'ost_utils' module [Galit] Updated manual test custom report guide Changes for Build #609 [parthdhanjal] Updating gluster-inventory file Changes for Build #610 [parthdhanjal] Updating gluster-inventory file Changes for Build #611 [hbraha] network: remove xfail from active bond test Changes for Build #612 [hbraha] network: remove lago engine_ip call [Galit] Add dparth mail to hc-basic Changes for Build #613 [Eyal Shenitzky] basic_sanity.py: remove unused time import Changes for Build #614 [Eitan Raviv] network: remove workaround for BZ 1752282 Changes for Build #615 [Eitan Raviv] network: remove workaround for BZ 1752282 ----------------- Failed Tests: ----------------- 1 tests failed. FAILED: basic-suite-master.test-scenarios.004_basic_sanity.test_ha_recovery Error Message: AssertionError: False != True after 600 seconds Stack Trace: engine_api = <ovirtsdk4.Connection object at 0x7f41a3152890> get_ansible_host_for_vm = <function ansible_host_for at 0x7f41a3088ed8> @order_by(_TEST_LIST) def test_ha_recovery(engine_api, get_ansible_host_for_vm): engine = engine_api.system_service() with engine_utils.wait_for_event(engine, [119, 9602, 506]): # VM_DOWN_ERROR event(119) # HA_VM_FAILED event event(9602) # VDS_INITIATED_RUN_VM event(506) ansible_host = get_ansible_host_for_vm(VM2_NAME) pid = ansible_host.shell('pgrep -f qemu.*guest=vm2')['stdout'].strip() ansible_host.shell('kill -KILL {}'.format(pid)) vm_service = test_utils.get_vm_service(engine, VM2_NAME) assertions.assert_true_within_long(
lambda:
vm_service.get().status == types.VmStatus.UP ) ../basic-suite-master/test-scenarios/004_basic_sanity.py:824: _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ ../ost_utils/ost_utils/assertions.py:97: in assert_true_within_long assert_equals_within_long(func, True, allowed_exceptions) ../ost_utils/ost_utils/assertions.py:82: in assert_equals_within_long func, value, LONG_TIMEOUT, allowed_exceptions=allowed_exceptions _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ func = <function <lambda> at 0x7f41a3131398>, value = True, timeout = 600 allowed_exceptions = [], initial_wait = 0 error_message = 'False != True after 600 seconds' def assert_equals_within( func, value, timeout, allowed_exceptions=None, initial_wait=10, error_message=None ): allowed_exceptions = allowed_exceptions or [] with _EggTimer(timeout) as timer: while not timer.elapsed(): try: res = func() if res == value: return except Exception as exc: if _instance_of_any(exc, allowed_exceptions): time.sleep(3) continue LOGGER.exception("Unhandled exception in %s", func) raise if initial_wait == 0: time.sleep(3) else: time.sleep(initial_wait) initial_wait = 0 try: if error_message is None: error_message = '%s != %s after %s seconds' % (res, value, timeout)
raise AssertionError(error_message)
E AssertionError: False != True after 600 seconds ../ost_utils/ost_utils/assertions.py:60: AssertionError
Project: https://jenkins.ovirt.org/job/ovirt-system-tests_compat-4.3-suite-master/ Build: https://jenkins.ovirt.org/job/ovirt-system-tests_compat-4.3-suite-master/616... Build Number: 616 Build Status: Still Failing Triggered By: Started by timer ------------------------------------- Changes Since Last Success: ------------------------------------- Changes for Build #607 [hbraha] network: remove calls to lago env on fixture.host Changes for Build #608 [Marcin Sobczyk] network-4.3: Fix missing 'ost_utils' module [Galit] Updated manual test custom report guide Changes for Build #609 [parthdhanjal] Updating gluster-inventory file Changes for Build #610 [parthdhanjal] Updating gluster-inventory file Changes for Build #611 [hbraha] network: remove xfail from active bond test Changes for Build #612 [hbraha] network: remove lago engine_ip call [Galit] Add dparth mail to hc-basic Changes for Build #613 [Eyal Shenitzky] basic_sanity.py: remove unused time import Changes for Build #614 [Eitan Raviv] network: remove workaround for BZ 1752282 Changes for Build #615 [Eitan Raviv] network: remove workaround for BZ 1752282 Changes for Build #616 [Marcin Sobczyk] basic: Report start times and runnings time of tests ----------------- Failed Tests: ----------------- 1 tests failed. FAILED: basic-suite-master.test-scenarios.004_basic_sanity.test_ha_recovery Error Message: AssertionError: False != True after 600 seconds Stack Trace: engine_api = <ovirtsdk4.Connection object at 0x7fdefe172990> get_ansible_host_for_vm = <function ansible_host_for at 0x7fdefe17b1b8> @order_by(_TEST_LIST) def test_ha_recovery(engine_api, get_ansible_host_for_vm): engine = engine_api.system_service() with engine_utils.wait_for_event(engine, [119, 9602, 506]): # VM_DOWN_ERROR event(119) # HA_VM_FAILED event event(9602) # VDS_INITIATED_RUN_VM event(506) ansible_host = get_ansible_host_for_vm(VM2_NAME) pid = ansible_host.shell('pgrep -f qemu.*guest=vm2')['stdout'].strip() ansible_host.shell('kill -KILL {}'.format(pid)) vm_service = test_utils.get_vm_service(engine, VM2_NAME) assertions.assert_true_within_long(
lambda:
vm_service.get().status == types.VmStatus.UP ) ../basic-suite-master/test-scenarios/004_basic_sanity.py:824: _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ ../ost_utils/ost_utils/assertions.py:97: in assert_true_within_long assert_equals_within_long(func, True, allowed_exceptions) ../ost_utils/ost_utils/assertions.py:82: in assert_equals_within_long func, value, LONG_TIMEOUT, allowed_exceptions=allowed_exceptions _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ func = <function <lambda> at 0x7fdefe234d70>, value = True, timeout = 600 allowed_exceptions = [], initial_wait = 0 error_message = 'False != True after 600 seconds' def assert_equals_within( func, value, timeout, allowed_exceptions=None, initial_wait=10, error_message=None ): allowed_exceptions = allowed_exceptions or [] with _EggTimer(timeout) as timer: while not timer.elapsed(): try: res = func() if res == value: return except Exception as exc: if _instance_of_any(exc, allowed_exceptions): time.sleep(3) continue LOGGER.exception("Unhandled exception in %s", func) raise if initial_wait == 0: time.sleep(3) else: time.sleep(initial_wait) initial_wait = 0 try: if error_message is None: error_message = '%s != %s after %s seconds' % (res, value, timeout)
raise AssertionError(error_message)
E AssertionError: False != True after 600 seconds ../ost_utils/ost_utils/assertions.py:60: AssertionError
Project: https://jenkins.ovirt.org/job/ovirt-system-tests_compat-4.3-suite-master/ Build: https://jenkins.ovirt.org/job/ovirt-system-tests_compat-4.3-suite-master/617... Build Number: 617 Build Status: Still Failing Triggered By: Started by timer ------------------------------------- Changes Since Last Success: ------------------------------------- Changes for Build #608 [Marcin Sobczyk] network-4.3: Fix missing 'ost_utils' module [Galit] Updated manual test custom report guide Changes for Build #609 [parthdhanjal] Updating gluster-inventory file Changes for Build #610 [parthdhanjal] Updating gluster-inventory file Changes for Build #611 [hbraha] network: remove xfail from active bond test Changes for Build #612 [hbraha] network: remove lago engine_ip call [Galit] Add dparth mail to hc-basic Changes for Build #613 [Eyal Shenitzky] basic_sanity.py: remove unused time import Changes for Build #614 [Eitan Raviv] network: remove workaround for BZ 1752282 Changes for Build #615 [Eitan Raviv] network: remove workaround for BZ 1752282 Changes for Build #616 [Marcin Sobczyk] basic: Report start times and runnings time of tests Changes for Build #617 [Marcin Sobczyk] basic: Report start times and runnings time of tests ----------------- Failed Tests: ----------------- 1 tests failed. FAILED: basic-suite-master.test-scenarios.004_basic_sanity.test_run_vms Error Message: Error: Fault reason is "Operation Failed". Fault detail is "[Network error during communication with the Host.]". HTTP response code is 400. Stack Trace: assert_vm_is_alive = <function is_alive at 0x7f86b85f8c08> engine_api = <ovirtsdk4.Connection object at 0x7f86b85f4750> management_gw_ip = '192.168.201.1' @order_by(_TEST_LIST) def test_run_vms(assert_vm_is_alive, engine_api, management_gw_ip): engine = engine_api.system_service() vm_params = types.Vm( initialization=types.Initialization( user_name=VM_USER_NAME, root_password=VM_PASSWORD ) ) vm_params.initialization.host_name = BACKUP_VM_NAME backup_vm_service = test_utils.get_vm_service(engine, BACKUP_VM_NAME) backup_vm_service.start(use_cloud_init=True, vm=vm_params) vm_params.initialization.host_name = VM2_NAME vm2_service = test_utils.get_vm_service(engine, VM2_NAME) vm2_service.start(use_cloud_init=True, vm=vm_params) # CirrOS cloud-init is different, networking doesn't work since it doesn't support the format oVirt is using vm_params.initialization.host_name = VM0_NAME # hostname seems to work, the others not vm_params.initialization.dns_search = 'lago.local' vm_params.initialization.domain = 'lago.local' vm_params.initialization.dns_servers = management_gw_ip vm0_service = test_utils.get_vm_service(engine, VM0_NAME)
vm0_service.start(use_cloud_init=True, vm=vm_params)
../basic-suite-master/test-scenarios/004_basic_sanity.py:754: _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ /usr/lib64/python2.7/site-packages/ovirtsdk4/services.py:32520: in start return self._internal_action(action, 'start', None, headers, query, wait) /usr/lib64/python2.7/site-packages/ovirtsdk4/service.py:299: in _internal_action return future.wait() if wait else future /usr/lib64/python2.7/site-packages/ovirtsdk4/service.py:55: in wait return self._code(response) /usr/lib64/python2.7/site-packages/ovirtsdk4/service.py:296: in callback self._check_fault(response) /usr/lib64/python2.7/site-packages/ovirtsdk4/service.py:134: in _check_fault self._raise_error(response, body.fault) _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ response = <ovirtsdk4.http.Response object at 0x7f86b8539b50> detail = <ovirtsdk4.types.Fault object at 0x7f86b8539bd0> @staticmethod def _raise_error(response, detail=None): """ Creates and raises an error containing the details of the given HTTP response and fault. This method is intended for internal use by other components of the SDK. Refrain from using it directly, as backwards compatibility isn't guaranteed. """ fault = detail if isinstance(detail, types.Fault) else None msg = '' if fault: if fault.reason: if msg: msg += ' ' msg = msg + 'Fault reason is "%s".' % fault.reason if fault.detail: if msg: msg += ' ' msg = msg + 'Fault detail is "%s".' % fault.detail if response: if response.code: if msg: msg += ' ' msg = msg + 'HTTP response code is %s.' % response.code if response.message: if msg: msg += ' ' msg = msg + 'HTTP response message is "%s".' % response.message if isinstance(detail, six.string_types): if msg: msg += ' ' msg = msg + detail + '.' class_ = Error if response is not None: if response.code in [401, 403]: class_ = AuthError elif response.code == 404: class_ = NotFoundError error = class_(msg) error.code = response.code if response else None error.fault = fault
raise error
E Error: Fault reason is "Operation Failed". Fault detail is "[Network error during communication with the Host.]". HTTP response code is 400. /usr/lib64/python2.7/site-packages/ovirtsdk4/service.py:118: Error
Project: https://jenkins.ovirt.org/job/ovirt-system-tests_compat-4.3-suite-master/ Build: https://jenkins.ovirt.org/job/ovirt-system-tests_compat-4.3-suite-master/618... Build Number: 618 Build Status: Still Failing Triggered By: Started by timer ------------------------------------- Changes Since Last Success: ------------------------------------- Changes for Build #609 [parthdhanjal] Updating gluster-inventory file Changes for Build #610 [parthdhanjal] Updating gluster-inventory file Changes for Build #611 [hbraha] network: remove xfail from active bond test Changes for Build #612 [hbraha] network: remove lago engine_ip call [Galit] Add dparth mail to hc-basic Changes for Build #613 [Eyal Shenitzky] basic_sanity.py: remove unused time import Changes for Build #614 [Eitan Raviv] network: remove workaround for BZ 1752282 Changes for Build #615 [Eitan Raviv] network: remove workaround for BZ 1752282 Changes for Build #616 [Marcin Sobczyk] basic: Report start times and runnings time of tests Changes for Build #617 [Marcin Sobczyk] basic: Report start times and runnings time of tests Changes for Build #618 [Marcin Sobczyk] basic: Report start times and runnings time of tests ----------------- Failed Tests: ----------------- 1 tests failed. FAILED: basic-suite-master.test-scenarios.004_basic_sanity.test_ha_recovery Error Message: AssertionError: False != True after 600 seconds Stack Trace: engine_api = <ovirtsdk4.Connection object at 0x7fe3cce33f10> get_ansible_host_for_vm = <function ansible_host_for at 0x7fe3cce2d5f0> @order_by(_TEST_LIST) def test_ha_recovery(engine_api, get_ansible_host_for_vm): engine = engine_api.system_service() with engine_utils.wait_for_event(engine, [119, 9602, 506]): # VM_DOWN_ERROR event(119) # HA_VM_FAILED event event(9602) # VDS_INITIATED_RUN_VM event(506) ansible_host = get_ansible_host_for_vm(VM2_NAME) pid = ansible_host.shell('pgrep -f qemu.*guest=vm2')['stdout'].strip() ansible_host.shell('kill -KILL {}'.format(pid)) vm_service = test_utils.get_vm_service(engine, VM2_NAME) assertions.assert_true_within_long(
lambda:
vm_service.get().status == types.VmStatus.UP ) ../basic-suite-master/test-scenarios/004_basic_sanity.py:824: _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ ../ost_utils/ost_utils/assertions.py:97: in assert_true_within_long assert_equals_within_long(func, True, allowed_exceptions) ../ost_utils/ost_utils/assertions.py:82: in assert_equals_within_long func, value, LONG_TIMEOUT, allowed_exceptions=allowed_exceptions _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ func = <function <lambda> at 0x7fe3ce0b11b8>, value = True, timeout = 600 allowed_exceptions = [], initial_wait = 0 error_message = 'False != True after 600 seconds' def assert_equals_within( func, value, timeout, allowed_exceptions=None, initial_wait=10, error_message=None ): allowed_exceptions = allowed_exceptions or [] with _EggTimer(timeout) as timer: while not timer.elapsed(): try: res = func() if res == value: return except Exception as exc: if _instance_of_any(exc, allowed_exceptions): time.sleep(3) continue LOGGER.exception("Unhandled exception in %s", func) raise if initial_wait == 0: time.sleep(3) else: time.sleep(initial_wait) initial_wait = 0 try: if error_message is None: error_message = '%s != %s after %s seconds' % (res, value, timeout)
raise AssertionError(error_message)
E AssertionError: False != True after 600 seconds ../ost_utils/ost_utils/assertions.py:60: AssertionError
Project: https://jenkins.ovirt.org/job/ovirt-system-tests_compat-4.3-suite-master/ Build: https://jenkins.ovirt.org/job/ovirt-system-tests_compat-4.3-suite-master/619... Build Number: 619 Build Status: Still Failing Triggered By: Started by timer ------------------------------------- Changes Since Last Success: ------------------------------------- Changes for Build #610 [parthdhanjal] Updating gluster-inventory file Changes for Build #611 [hbraha] network: remove xfail from active bond test Changes for Build #612 [hbraha] network: remove lago engine_ip call [Galit] Add dparth mail to hc-basic Changes for Build #613 [Eyal Shenitzky] basic_sanity.py: remove unused time import Changes for Build #614 [Eitan Raviv] network: remove workaround for BZ 1752282 Changes for Build #615 [Eitan Raviv] network: remove workaround for BZ 1752282 Changes for Build #616 [Marcin Sobczyk] basic: Report start times and runnings time of tests Changes for Build #617 [Marcin Sobczyk] basic: Report start times and runnings time of tests Changes for Build #618 [Marcin Sobczyk] basic: Report start times and runnings time of tests Changes for Build #619 [Marcin Sobczyk] basic: Report start times and runnings time of tests ----------------- Failed Tests: ----------------- 1 tests failed. FAILED: basic-suite-master.test-scenarios.004_basic_sanity.test_ha_recovery Error Message: AssertionError: False != True after 600 seconds Stack Trace: engine_api = <ovirtsdk4.Connection object at 0x7ff6e48f5790> get_ansible_host_for_vm = <function ansible_host_for at 0x7ff6e4931488> @order_by(_TEST_LIST) def test_ha_recovery(engine_api, get_ansible_host_for_vm): engine = engine_api.system_service() with engine_utils.wait_for_event(engine, [119, 9602, 506]): # VM_DOWN_ERROR event(119) # HA_VM_FAILED event event(9602) # VDS_INITIATED_RUN_VM event(506) ansible_host = get_ansible_host_for_vm(VM2_NAME) pid = ansible_host.shell('pgrep -f qemu.*guest=vm2')['stdout'].strip() ansible_host.shell('kill -KILL {}'.format(pid)) vm_service = test_utils.get_vm_service(engine, VM2_NAME) assertions.assert_true_within_long(
lambda:
vm_service.get().status == types.VmStatus.UP ) ../basic-suite-master/test-scenarios/004_basic_sanity.py:824: _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ ../ost_utils/ost_utils/assertions.py:97: in assert_true_within_long assert_equals_within_long(func, True, allowed_exceptions) ../ost_utils/ost_utils/assertions.py:82: in assert_equals_within_long func, value, LONG_TIMEOUT, allowed_exceptions=allowed_exceptions _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ func = <function <lambda> at 0x7ff6e5a2b1b8>, value = True, timeout = 600 allowed_exceptions = [], initial_wait = 0 error_message = 'False != True after 600 seconds' def assert_equals_within( func, value, timeout, allowed_exceptions=None, initial_wait=10, error_message=None ): allowed_exceptions = allowed_exceptions or [] with _EggTimer(timeout) as timer: while not timer.elapsed(): try: res = func() if res == value: return except Exception as exc: if _instance_of_any(exc, allowed_exceptions): time.sleep(3) continue LOGGER.exception("Unhandled exception in %s", func) raise if initial_wait == 0: time.sleep(3) else: time.sleep(initial_wait) initial_wait = 0 try: if error_message is None: error_message = '%s != %s after %s seconds' % (res, value, timeout)
raise AssertionError(error_message)
E AssertionError: False != True after 600 seconds ../ost_utils/ost_utils/assertions.py:60: AssertionError
Project: https://jenkins.ovirt.org/job/ovirt-system-tests_compat-4.3-suite-master/ Build: https://jenkins.ovirt.org/job/ovirt-system-tests_compat-4.3-suite-master/620... Build Number: 620 Build Status: Still Failing Triggered By: Started by timer ------------------------------------- Changes Since Last Success: ------------------------------------- Changes for Build #611 [hbraha] network: remove xfail from active bond test Changes for Build #612 [hbraha] network: remove lago engine_ip call [Galit] Add dparth mail to hc-basic Changes for Build #613 [Eyal Shenitzky] basic_sanity.py: remove unused time import Changes for Build #614 [Eitan Raviv] network: remove workaround for BZ 1752282 Changes for Build #615 [Eitan Raviv] network: remove workaround for BZ 1752282 Changes for Build #616 [Marcin Sobczyk] basic: Report start times and runnings time of tests Changes for Build #617 [Marcin Sobczyk] basic: Report start times and runnings time of tests Changes for Build #618 [Marcin Sobczyk] basic: Report start times and runnings time of tests Changes for Build #619 [Marcin Sobczyk] basic: Report start times and runnings time of tests Changes for Build #620 [Nir Soffer] nfs: Use async mount ----------------- Failed Tests: ----------------- 1 tests failed. FAILED: basic-suite-master.test-scenarios.004_basic_sanity.test_ha_recovery Error Message: AssertionError: False != True after 600 seconds Stack Trace: engine_api = <ovirtsdk4.Connection object at 0x7ff97d55dfd0> get_ansible_host_for_vm = <function ansible_host_for at 0x7ff97d5c2848> @order_by(_TEST_LIST) def test_ha_recovery(engine_api, get_ansible_host_for_vm): engine = engine_api.system_service() with engine_utils.wait_for_event(engine, [119, 9602, 506]): # VM_DOWN_ERROR event(119) # HA_VM_FAILED event event(9602) # VDS_INITIATED_RUN_VM event(506) ansible_host = get_ansible_host_for_vm(VM2_NAME) pid = ansible_host.shell('pgrep -f qemu.*guest=vm2')['stdout'].strip() ansible_host.shell('kill -KILL {}'.format(pid)) vm_service = test_utils.get_vm_service(engine, VM2_NAME) assertions.assert_true_within_long(
lambda:
vm_service.get().status == types.VmStatus.UP ) ../basic-suite-master/test-scenarios/004_basic_sanity.py:824: _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ ../ost_utils/ost_utils/assertions.py:98: in assert_true_within_long assert_equals_within_long(func, True, allowed_exceptions) ../ost_utils/ost_utils/assertions.py:83: in assert_equals_within_long func, value, LONG_TIMEOUT, allowed_exceptions=allowed_exceptions _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ func = <function <lambda> at 0x7ff97d5c2c80>, value = True, timeout = 600 allowed_exceptions = [], initial_wait = 0 error_message = 'False != True after 600 seconds' def assert_equals_within( func, value, timeout, allowed_exceptions=None, initial_wait=10, error_message=None ): allowed_exceptions = allowed_exceptions or [] res = '<no-result-obtained>' with _EggTimer(timeout) as timer: while not timer.elapsed(): try: res = func() if res == value: return except Exception as exc: if _instance_of_any(exc, allowed_exceptions): time.sleep(3) continue LOGGER.exception("Unhandled exception in %s", func) raise if initial_wait == 0: time.sleep(3) else: time.sleep(initial_wait) initial_wait = 0 try: if error_message is None: error_message = '%s != %s after %s seconds' % (res, value, timeout)
raise AssertionError(error_message)
E AssertionError: False != True after 600 seconds ../ost_utils/ost_utils/assertions.py:61: AssertionError
Project: https://jenkins.ovirt.org/job/ovirt-system-tests_compat-4.3-suite-master/ Build: https://jenkins.ovirt.org/job/ovirt-system-tests_compat-4.3-suite-master/621... Build Number: 621 Build Status: Still Failing Triggered By: Started by timer ------------------------------------- Changes Since Last Success: ------------------------------------- Changes for Build #612 [hbraha] network: remove lago engine_ip call [Galit] Add dparth mail to hc-basic Changes for Build #613 [Eyal Shenitzky] basic_sanity.py: remove unused time import Changes for Build #614 [Eitan Raviv] network: remove workaround for BZ 1752282 Changes for Build #615 [Eitan Raviv] network: remove workaround for BZ 1752282 Changes for Build #616 [Marcin Sobczyk] basic: Report start times and runnings time of tests Changes for Build #617 [Marcin Sobczyk] basic: Report start times and runnings time of tests Changes for Build #618 [Marcin Sobczyk] basic: Report start times and runnings time of tests Changes for Build #619 [Marcin Sobczyk] basic: Report start times and runnings time of tests Changes for Build #620 [Nir Soffer] nfs: Use async mount Changes for Build #621 [Shani Leviim] basic_sanity: add test for creating a floating disk ----------------- Failed Tests: ----------------- 1 tests failed. FAILED: basic-suite-master.test-scenarios.004_basic_sanity.test_run_vms Error Message: Error: Fault reason is "Operation Failed". Fault detail is "[Network error during communication with the Host.]". HTTP response code is 400. Stack Trace: assert_vm_is_alive = <function is_alive at 0x7f8e08196398> engine_api = <ovirtsdk4.Connection object at 0x7f8e08b68fd0> management_gw_ip = '192.168.201.1' @order_by(_TEST_LIST) def test_run_vms(assert_vm_is_alive, engine_api, management_gw_ip): engine = engine_api.system_service() vm_params = types.Vm( initialization=types.Initialization( user_name=VM_USER_NAME, root_password=VM_PASSWORD ) ) vm_params.initialization.host_name = BACKUP_VM_NAME backup_vm_service = test_utils.get_vm_service(engine, BACKUP_VM_NAME) backup_vm_service.start(use_cloud_init=True, vm=vm_params) vm_params.initialization.host_name = VM2_NAME vm2_service = test_utils.get_vm_service(engine, VM2_NAME) vm2_service.start(use_cloud_init=True, vm=vm_params) # CirrOS cloud-init is different, networking doesn't work since it doesn't support the format oVirt is using vm_params.initialization.host_name = VM0_NAME # hostname seems to work, the others not vm_params.initialization.dns_search = 'lago.local' vm_params.initialization.domain = 'lago.local' vm_params.initialization.dns_servers = management_gw_ip vm0_service = test_utils.get_vm_service(engine, VM0_NAME)
vm0_service.start(use_cloud_init=True, vm=vm_params)
../basic-suite-master/test-scenarios/004_basic_sanity.py:781: _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ /usr/lib64/python2.7/site-packages/ovirtsdk4/services.py:32520: in start return self._internal_action(action, 'start', None, headers, query, wait) /usr/lib64/python2.7/site-packages/ovirtsdk4/service.py:299: in _internal_action return future.wait() if wait else future /usr/lib64/python2.7/site-packages/ovirtsdk4/service.py:55: in wait return self._code(response) /usr/lib64/python2.7/site-packages/ovirtsdk4/service.py:296: in callback self._check_fault(response) /usr/lib64/python2.7/site-packages/ovirtsdk4/service.py:134: in _check_fault self._raise_error(response, body.fault) _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ response = <ovirtsdk4.http.Response object at 0x7f8e08b66d10> detail = <ovirtsdk4.types.Fault object at 0x7f8e08b66d90> @staticmethod def _raise_error(response, detail=None): """ Creates and raises an error containing the details of the given HTTP response and fault. This method is intended for internal use by other components of the SDK. Refrain from using it directly, as backwards compatibility isn't guaranteed. """ fault = detail if isinstance(detail, types.Fault) else None msg = '' if fault: if fault.reason: if msg: msg += ' ' msg = msg + 'Fault reason is "%s".' % fault.reason if fault.detail: if msg: msg += ' ' msg = msg + 'Fault detail is "%s".' % fault.detail if response: if response.code: if msg: msg += ' ' msg = msg + 'HTTP response code is %s.' % response.code if response.message: if msg: msg += ' ' msg = msg + 'HTTP response message is "%s".' % response.message if isinstance(detail, six.string_types): if msg: msg += ' ' msg = msg + detail + '.' class_ = Error if response is not None: if response.code in [401, 403]: class_ = AuthError elif response.code == 404: class_ = NotFoundError error = class_(msg) error.code = response.code if response else None error.fault = fault
raise error
E Error: Fault reason is "Operation Failed". Fault detail is "[Network error during communication with the Host.]". HTTP response code is 400. /usr/lib64/python2.7/site-packages/ovirtsdk4/service.py:118: Error
Project: https://jenkins.ovirt.org/job/ovirt-system-tests_compat-4.3-suite-master/ Build: https://jenkins.ovirt.org/job/ovirt-system-tests_compat-4.3-suite-master/622... Build Number: 622 Build Status: Still Failing Triggered By: Started by timer ------------------------------------- Changes Since Last Success: ------------------------------------- Changes for Build #613 [Eyal Shenitzky] basic_sanity.py: remove unused time import Changes for Build #614 [Eitan Raviv] network: remove workaround for BZ 1752282 Changes for Build #615 [Eitan Raviv] network: remove workaround for BZ 1752282 Changes for Build #616 [Marcin Sobczyk] basic: Report start times and runnings time of tests Changes for Build #617 [Marcin Sobczyk] basic: Report start times and runnings time of tests Changes for Build #618 [Marcin Sobczyk] basic: Report start times and runnings time of tests Changes for Build #619 [Marcin Sobczyk] basic: Report start times and runnings time of tests Changes for Build #620 [Nir Soffer] nfs: Use async mount Changes for Build #621 [Shani Leviim] basic_sanity: add test for creating a floating disk Changes for Build #622 [jenkins CI] Auto create ovirt-4.3 ----------------- Failed Tests: ----------------- 1 tests failed. FAILED: basic-suite-master.test-scenarios.004_basic_sanity.test_ha_recovery Error Message: AssertionError: False != True after 600 seconds Stack Trace: engine_api = <ovirtsdk4.Connection object at 0x7f8494584b90> get_ansible_host_for_vm = <function ansible_host_for at 0x7f8494f832a8> @order_by(_TEST_LIST) def test_ha_recovery(engine_api, get_ansible_host_for_vm): engine = engine_api.system_service() with engine_utils.wait_for_event(engine, [119, 9602, 506]): # VM_DOWN_ERROR event(119) # HA_VM_FAILED event event(9602) # VDS_INITIATED_RUN_VM event(506) ansible_host = get_ansible_host_for_vm(VM2_NAME) pid = ansible_host.shell('pgrep -f qemu.*guest=vm2')['stdout'].strip() ansible_host.shell('kill -KILL {}'.format(pid)) vm_service = test_utils.get_vm_service(engine, VM2_NAME) assertions.assert_true_within_long(
lambda:
vm_service.get().status == types.VmStatus.UP ) ../basic-suite-master/test-scenarios/004_basic_sanity.py:858: _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ ../ost_utils/ost_utils/assertions.py:98: in assert_true_within_long assert_equals_within_long(func, True, allowed_exceptions) ../ost_utils/ost_utils/assertions.py:83: in assert_equals_within_long func, value, LONG_TIMEOUT, allowed_exceptions=allowed_exceptions _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ func = <function <lambda> at 0x7f8495ccdaa0>, value = True, timeout = 600 allowed_exceptions = [], initial_wait = 0 error_message = 'False != True after 600 seconds' def assert_equals_within( func, value, timeout, allowed_exceptions=None, initial_wait=10, error_message=None ): allowed_exceptions = allowed_exceptions or [] res = '<no-result-obtained>' with _EggTimer(timeout) as timer: while not timer.elapsed(): try: res = func() if res == value: return except Exception as exc: if _instance_of_any(exc, allowed_exceptions): time.sleep(3) continue LOGGER.exception("Unhandled exception in %s", func) raise if initial_wait == 0: time.sleep(3) else: time.sleep(initial_wait) initial_wait = 0 try: if error_message is None: error_message = '%s != %s after %s seconds' % (res, value, timeout)
raise AssertionError(error_message)
E AssertionError: False != True after 600 seconds ../ost_utils/ost_utils/assertions.py:61: AssertionError
Project: https://jenkins.ovirt.org/job/ovirt-system-tests_compat-4.3-suite-master/ Build: https://jenkins.ovirt.org/job/ovirt-system-tests_compat-4.3-suite-master/623... Build Number: 623 Build Status: Still Failing Triggered By: Started by timer ------------------------------------- Changes Since Last Success: ------------------------------------- Changes for Build #614 [Eitan Raviv] network: remove workaround for BZ 1752282 Changes for Build #615 [Eitan Raviv] network: remove workaround for BZ 1752282 Changes for Build #616 [Marcin Sobczyk] basic: Report start times and runnings time of tests Changes for Build #617 [Marcin Sobczyk] basic: Report start times and runnings time of tests Changes for Build #618 [Marcin Sobczyk] basic: Report start times and runnings time of tests Changes for Build #619 [Marcin Sobczyk] basic: Report start times and runnings time of tests Changes for Build #620 [Nir Soffer] nfs: Use async mount Changes for Build #621 [Shani Leviim] basic_sanity: add test for creating a floating disk Changes for Build #622 [jenkins CI] Auto create ovirt-4.3 Changes for Build #623 [Ales Musil] basic: Move CirrOS image name to fixtures ----------------- Failed Tests: ----------------- 1 tests failed. FAILED: basic-suite-master.test-scenarios.004_basic_sanity.test_ha_recovery Error Message: AssertionError: False != True after 600 seconds Stack Trace: engine_api = <ovirtsdk4.Connection object at 0x7f2a89346a90> get_ansible_host_for_vm = <function ansible_host_for at 0x7f2a8a4d0c80> @order_by(_TEST_LIST) def test_ha_recovery(engine_api, get_ansible_host_for_vm): engine = engine_api.system_service() with engine_utils.wait_for_event(engine, [119, 9602, 506]): # VM_DOWN_ERROR event(119) # HA_VM_FAILED event event(9602) # VDS_INITIATED_RUN_VM event(506) ansible_host = get_ansible_host_for_vm(VM2_NAME) pid = ansible_host.shell('pgrep -f qemu.*guest=vm2')['stdout'].strip() ansible_host.shell('kill -KILL {}'.format(pid)) vm_service = test_utils.get_vm_service(engine, VM2_NAME) assertions.assert_true_within_long(
lambda:
vm_service.get().status == types.VmStatus.UP ) ../basic-suite-master/test-scenarios/004_basic_sanity.py:864: _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ ../ost_utils/ost_utils/assertions.py:98: in assert_true_within_long assert_equals_within_long(func, True, allowed_exceptions) ../ost_utils/ost_utils/assertions.py:83: in assert_equals_within_long func, value, LONG_TIMEOUT, allowed_exceptions=allowed_exceptions _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ func = <function <lambda> at 0x7f2a893cacf8>, value = True, timeout = 600 allowed_exceptions = [], initial_wait = 0 error_message = 'False != True after 600 seconds' def assert_equals_within( func, value, timeout, allowed_exceptions=None, initial_wait=10, error_message=None ): allowed_exceptions = allowed_exceptions or [] res = '<no-result-obtained>' with _EggTimer(timeout) as timer: while not timer.elapsed(): try: res = func() if res == value: return except Exception as exc: if _instance_of_any(exc, allowed_exceptions): time.sleep(3) continue LOGGER.exception("Unhandled exception in %s", func) raise if initial_wait == 0: time.sleep(3) else: time.sleep(initial_wait) initial_wait = 0 try: if error_message is None: error_message = '%s != %s after %s seconds' % (res, value, timeout)
raise AssertionError(error_message)
E AssertionError: False != True after 600 seconds ../ost_utils/ost_utils/assertions.py:61: AssertionError
Project: https://jenkins.ovirt.org/job/ovirt-system-tests_compat-4.3-suite-master/ Build: https://jenkins.ovirt.org/job/ovirt-system-tests_compat-4.3-suite-master/624... Build Number: 624 Build Status: Still Failing Triggered By: Started by timer ------------------------------------- Changes Since Last Success: ------------------------------------- Changes for Build #615 [Eitan Raviv] network: remove workaround for BZ 1752282 Changes for Build #616 [Marcin Sobczyk] basic: Report start times and runnings time of tests Changes for Build #617 [Marcin Sobczyk] basic: Report start times and runnings time of tests Changes for Build #618 [Marcin Sobczyk] basic: Report start times and runnings time of tests Changes for Build #619 [Marcin Sobczyk] basic: Report start times and runnings time of tests Changes for Build #620 [Nir Soffer] nfs: Use async mount Changes for Build #621 [Shani Leviim] basic_sanity: add test for creating a floating disk Changes for Build #622 [jenkins CI] Auto create ovirt-4.3 Changes for Build #623 [Ales Musil] basic: Move CirrOS image name to fixtures Changes for Build #624 [Ales Musil] basic: Move CirrOS image name to fixtures ----------------- Failed Tests: ----------------- 1 tests failed. FAILED: basic-suite-master.test-scenarios.004_basic_sanity.test_ha_recovery Error Message: AssertionError: False != True after 600 seconds Stack Trace: engine_api = <ovirtsdk4.Connection object at 0x7f3ad87f5750> get_ansible_host_for_vm = <function ansible_host_for at 0x7f3ad9116ed8> @order_by(_TEST_LIST) def test_ha_recovery(engine_api, get_ansible_host_for_vm): engine = engine_api.system_service() with engine_utils.wait_for_event(engine, [119, 9602, 506]): # VM_DOWN_ERROR event(119) # HA_VM_FAILED event event(9602) # VDS_INITIATED_RUN_VM event(506) ansible_host = get_ansible_host_for_vm(VM2_NAME) pid = ansible_host.shell('pgrep -f qemu.*guest=vm2')['stdout'].strip() ansible_host.shell('kill -KILL {}'.format(pid)) vm_service = test_utils.get_vm_service(engine, VM2_NAME) assertions.assert_true_within_long(
lambda:
vm_service.get().status == types.VmStatus.UP ) ../basic-suite-master/test-scenarios/004_basic_sanity.py:864: _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ ../ost_utils/ost_utils/assertions.py:98: in assert_true_within_long assert_equals_within_long(func, True, allowed_exceptions) ../ost_utils/ost_utils/assertions.py:83: in assert_equals_within_long func, value, LONG_TIMEOUT, allowed_exceptions=allowed_exceptions _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ func = <function <lambda> at 0x7f3ada15ade8>, value = True, timeout = 600 allowed_exceptions = [], initial_wait = 0 error_message = 'False != True after 600 seconds' def assert_equals_within( func, value, timeout, allowed_exceptions=None, initial_wait=10, error_message=None ): allowed_exceptions = allowed_exceptions or [] res = '<no-result-obtained>' with _EggTimer(timeout) as timer: while not timer.elapsed(): try: res = func() if res == value: return except Exception as exc: if _instance_of_any(exc, allowed_exceptions): time.sleep(3) continue LOGGER.exception("Unhandled exception in %s", func) raise if initial_wait == 0: time.sleep(3) else: time.sleep(initial_wait) initial_wait = 0 try: if error_message is None: error_message = '%s != %s after %s seconds' % (res, value, timeout)
raise AssertionError(error_message)
E AssertionError: False != True after 600 seconds ../ost_utils/ost_utils/assertions.py:61: AssertionError
Project: https://jenkins.ovirt.org/job/ovirt-system-tests_compat-4.3-suite-master/ Build: https://jenkins.ovirt.org/job/ovirt-system-tests_compat-4.3-suite-master/625... Build Number: 625 Build Status: Still Failing Triggered By: Started by timer ------------------------------------- Changes Since Last Success: ------------------------------------- Changes for Build #616 [Marcin Sobczyk] basic: Report start times and runnings time of tests Changes for Build #617 [Marcin Sobczyk] basic: Report start times and runnings time of tests Changes for Build #618 [Marcin Sobczyk] basic: Report start times and runnings time of tests Changes for Build #619 [Marcin Sobczyk] basic: Report start times and runnings time of tests Changes for Build #620 [Nir Soffer] nfs: Use async mount Changes for Build #621 [Shani Leviim] basic_sanity: add test for creating a floating disk Changes for Build #622 [jenkins CI] Auto create ovirt-4.3 Changes for Build #623 [Ales Musil] basic: Move CirrOS image name to fixtures Changes for Build #624 [Ales Musil] basic: Move CirrOS image name to fixtures Changes for Build #625 [Eitan Raviv] network: pytest skipif requires reason=STRING ----------------- Failed Tests: ----------------- 1 tests failed. FAILED: basic-suite-master.test-scenarios.004_basic_sanity.test_run_vms Error Message: Error: Fault reason is "Operation Failed". Fault detail is "[Network error during communication with the Host.]". HTTP response code is 400. Stack Trace: assert_vm_is_alive = <function is_alive at 0x7f7bd60fe6e0> engine_api = <ovirtsdk4.Connection object at 0x7f7bd60b8790> management_gw_ip = '192.168.201.1' @order_by(_TEST_LIST) def test_run_vms(assert_vm_is_alive, engine_api, management_gw_ip): engine = engine_api.system_service() vm_params = types.Vm( initialization=types.Initialization( user_name=VM_USER_NAME, root_password=VM_PASSWORD ) ) vm_params.initialization.host_name = BACKUP_VM_NAME backup_vm_service = test_utils.get_vm_service(engine, BACKUP_VM_NAME) backup_vm_service.start(use_cloud_init=True, vm=vm_params) vm_params.initialization.host_name = VM2_NAME vm2_service = test_utils.get_vm_service(engine, VM2_NAME) vm2_service.start(use_cloud_init=True, vm=vm_params) # CirrOS cloud-init is different, networking doesn't work since it doesn't support the format oVirt is using vm_params.initialization.host_name = VM0_NAME # hostname seems to work, the others not vm_params.initialization.dns_search = 'lago.local' vm_params.initialization.domain = 'lago.local' vm_params.initialization.dns_servers = management_gw_ip vm0_service = test_utils.get_vm_service(engine, VM0_NAME)
vm0_service.start(use_cloud_init=True, vm=vm_params)
../basic-suite-master/test-scenarios/004_basic_sanity.py:794: _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ /usr/lib64/python2.7/site-packages/ovirtsdk4/services.py:32520: in start return self._internal_action(action, 'start', None, headers, query, wait) /usr/lib64/python2.7/site-packages/ovirtsdk4/service.py:299: in _internal_action return future.wait() if wait else future /usr/lib64/python2.7/site-packages/ovirtsdk4/service.py:55: in wait return self._code(response) /usr/lib64/python2.7/site-packages/ovirtsdk4/service.py:296: in callback self._check_fault(response) /usr/lib64/python2.7/site-packages/ovirtsdk4/service.py:134: in _check_fault self._raise_error(response, body.fault) _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ response = <ovirtsdk4.http.Response object at 0x7f7bd60a0f10> detail = <ovirtsdk4.types.Fault object at 0x7f7bd60a0f90> @staticmethod def _raise_error(response, detail=None): """ Creates and raises an error containing the details of the given HTTP response and fault. This method is intended for internal use by other components of the SDK. Refrain from using it directly, as backwards compatibility isn't guaranteed. """ fault = detail if isinstance(detail, types.Fault) else None msg = '' if fault: if fault.reason: if msg: msg += ' ' msg = msg + 'Fault reason is "%s".' % fault.reason if fault.detail: if msg: msg += ' ' msg = msg + 'Fault detail is "%s".' % fault.detail if response: if response.code: if msg: msg += ' ' msg = msg + 'HTTP response code is %s.' % response.code if response.message: if msg: msg += ' ' msg = msg + 'HTTP response message is "%s".' % response.message if isinstance(detail, six.string_types): if msg: msg += ' ' msg = msg + detail + '.' class_ = Error if response is not None: if response.code in [401, 403]: class_ = AuthError elif response.code == 404: class_ = NotFoundError error = class_(msg) error.code = response.code if response else None error.fault = fault
raise error
E Error: Fault reason is "Operation Failed". Fault detail is "[Network error during communication with the Host.]". HTTP response code is 400. /usr/lib64/python2.7/site-packages/ovirtsdk4/service.py:118: Error
Project: https://jenkins.ovirt.org/job/ovirt-system-tests_compat-4.3-suite-master/ Build: https://jenkins.ovirt.org/job/ovirt-system-tests_compat-4.3-suite-master/626... Build Number: 626 Build Status: Still Failing Triggered By: Started by timer ------------------------------------- Changes Since Last Success: ------------------------------------- Changes for Build #617 [Marcin Sobczyk] basic: Report start times and runnings time of tests Changes for Build #618 [Marcin Sobczyk] basic: Report start times and runnings time of tests Changes for Build #619 [Marcin Sobczyk] basic: Report start times and runnings time of tests Changes for Build #620 [Nir Soffer] nfs: Use async mount Changes for Build #621 [Shani Leviim] basic_sanity: add test for creating a floating disk Changes for Build #622 [jenkins CI] Auto create ovirt-4.3 Changes for Build #623 [Ales Musil] basic: Move CirrOS image name to fixtures Changes for Build #624 [Ales Musil] basic: Move CirrOS image name to fixtures Changes for Build #625 [Eitan Raviv] network: pytest skipif requires reason=STRING Changes for Build #626 [Eitan Raviv] network: stabilize host & DC state on change cluster ----------------- Failed Tests: ----------------- 1 tests failed. FAILED: basic-suite-master.test-scenarios.004_basic_sanity.test_ha_recovery Error Message: AssertionError: False != True after 600 seconds Stack Trace: engine_api = <ovirtsdk4.Connection object at 0x7f83dcac4bd0> get_ansible_host_for_vm = <function ansible_host_for at 0x7f83ddaf47d0> @order_by(_TEST_LIST) def test_ha_recovery(engine_api, get_ansible_host_for_vm): engine = engine_api.system_service() with engine_utils.wait_for_event(engine, [119, 9602, 506]): # VM_DOWN_ERROR event(119) # HA_VM_FAILED event event(9602) # VDS_INITIATED_RUN_VM event(506) ansible_host = get_ansible_host_for_vm(VM2_NAME) pid = ansible_host.shell('pgrep -f qemu.*guest=vm2')['stdout'].strip() ansible_host.shell('kill -KILL {}'.format(pid)) vm_service = test_utils.get_vm_service(engine, VM2_NAME) assertions.assert_true_within_long(
lambda:
vm_service.get().status == types.VmStatus.UP ) ../basic-suite-master/test-scenarios/004_basic_sanity.py:864: _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ ../ost_utils/ost_utils/assertions.py:98: in assert_true_within_long assert_equals_within_long(func, True, allowed_exceptions) ../ost_utils/ost_utils/assertions.py:83: in assert_equals_within_long func, value, LONG_TIMEOUT, allowed_exceptions=allowed_exceptions _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ func = <function <lambda> at 0x7f83ddb34938>, value = True, timeout = 600 allowed_exceptions = [], initial_wait = 0 error_message = 'False != True after 600 seconds' def assert_equals_within( func, value, timeout, allowed_exceptions=None, initial_wait=10, error_message=None ): allowed_exceptions = allowed_exceptions or [] res = '<no-result-obtained>' with _EggTimer(timeout) as timer: while not timer.elapsed(): try: res = func() if res == value: return except Exception as exc: if _instance_of_any(exc, allowed_exceptions): time.sleep(3) continue LOGGER.exception("Unhandled exception in %s", func) raise if initial_wait == 0: time.sleep(3) else: time.sleep(initial_wait) initial_wait = 0 try: if error_message is None: error_message = '%s != %s after %s seconds' % (res, value, timeout)
raise AssertionError(error_message)
E AssertionError: False != True after 600 seconds ../ost_utils/ost_utils/assertions.py:61: AssertionError
Project: https://jenkins.ovirt.org/job/ovirt-system-tests_compat-4.3-suite-master/ Build: https://jenkins.ovirt.org/job/ovirt-system-tests_compat-4.3-suite-master/627... Build Number: 627 Build Status: Still Failing Triggered By: Started by timer ------------------------------------- Changes Since Last Success: ------------------------------------- Changes for Build #618 [Marcin Sobczyk] basic: Report start times and runnings time of tests Changes for Build #619 [Marcin Sobczyk] basic: Report start times and runnings time of tests Changes for Build #620 [Nir Soffer] nfs: Use async mount Changes for Build #621 [Shani Leviim] basic_sanity: add test for creating a floating disk Changes for Build #622 [jenkins CI] Auto create ovirt-4.3 Changes for Build #623 [Ales Musil] basic: Move CirrOS image name to fixtures Changes for Build #624 [Ales Musil] basic: Move CirrOS image name to fixtures Changes for Build #625 [Eitan Raviv] network: pytest skipif requires reason=STRING Changes for Build #626 [Eitan Raviv] network: stabilize host & DC state on change cluster Changes for Build #627 [Galit Rosenthal] basic: performance: decouple 'versioning.py' [Ehud Yonasi] remove upgrades suites and 4.3 from nightly. ----------------- Failed Tests: ----------------- 1 tests failed. FAILED: basic-suite-master.test-scenarios.004_basic_sanity.test_ha_recovery Error Message: AssertionError: False != True after 600 seconds Stack Trace: engine_api = <ovirtsdk4.Connection object at 0x7fd3b446c750> get_ansible_host_for_vm = <function ansible_host_for at 0x7fd3b4cf7410> @order_by(_TEST_LIST) def test_ha_recovery(engine_api, get_ansible_host_for_vm): engine = engine_api.system_service() with engine_utils.wait_for_event(engine, [119, 9602, 506]): # VM_DOWN_ERROR event(119) # HA_VM_FAILED event event(9602) # VDS_INITIATED_RUN_VM event(506) ansible_host = get_ansible_host_for_vm(VM2_NAME) pid = ansible_host.shell('pgrep -f qemu.*guest=vm2')['stdout'].strip() ansible_host.shell('kill -KILL {}'.format(pid)) vm_service = test_utils.get_vm_service(engine, VM2_NAME) assertions.assert_true_within_long(
lambda:
vm_service.get().status == types.VmStatus.UP ) ../basic-suite-master/test-scenarios/004_basic_sanity.py:864: _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ ../ost_utils/ost_utils/assertions.py:98: in assert_true_within_long assert_equals_within_long(func, True, allowed_exceptions) ../ost_utils/ost_utils/assertions.py:83: in assert_equals_within_long func, value, LONG_TIMEOUT, allowed_exceptions=allowed_exceptions _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ func = <function <lambda> at 0x7fd3b5ea3938>, value = True, timeout = 600 allowed_exceptions = [], initial_wait = 0 error_message = 'False != True after 600 seconds' def assert_equals_within( func, value, timeout, allowed_exceptions=None, initial_wait=10, error_message=None ): allowed_exceptions = allowed_exceptions or [] res = '<no-result-obtained>' with _EggTimer(timeout) as timer: while not timer.elapsed(): try: res = func() if res == value: return except Exception as exc: if _instance_of_any(exc, allowed_exceptions): time.sleep(3) continue LOGGER.exception("Unhandled exception in %s", func) raise if initial_wait == 0: time.sleep(3) else: time.sleep(initial_wait) initial_wait = 0 try: if error_message is None: error_message = '%s != %s after %s seconds' % (res, value, timeout)
raise AssertionError(error_message)
E AssertionError: False != True after 600 seconds ../ost_utils/ost_utils/assertions.py:61: AssertionError
Project: https://jenkins.ovirt.org/job/ovirt-system-tests_compat-4.3-suite-master/ Build: https://jenkins.ovirt.org/job/ovirt-system-tests_compat-4.3-suite-master/628... Build Number: 628 Build Status: Still Failing Triggered By: Started by timer ------------------------------------- Changes Since Last Success: ------------------------------------- Changes for Build #619 [Marcin Sobczyk] basic: Report start times and runnings time of tests Changes for Build #620 [Nir Soffer] nfs: Use async mount Changes for Build #621 [Shani Leviim] basic_sanity: add test for creating a floating disk Changes for Build #622 [jenkins CI] Auto create ovirt-4.3 Changes for Build #623 [Ales Musil] basic: Move CirrOS image name to fixtures Changes for Build #624 [Ales Musil] basic: Move CirrOS image name to fixtures Changes for Build #625 [Eitan Raviv] network: pytest skipif requires reason=STRING Changes for Build #626 [Eitan Raviv] network: stabilize host & DC state on change cluster Changes for Build #627 [Galit Rosenthal] basic: performance: decouple 'versioning.py' [Ehud Yonasi] remove upgrades suites and 4.3 from nightly. Changes for Build #628 [jenkins CI] Auto create ovirt-4.3 ----------------- Failed Tests: ----------------- 1 tests failed. FAILED: basic-suite-master.test-scenarios.test_004_basic_sanity.test_run_vms Error Message: Error: Fault reason is "Operation Failed". Fault detail is "[Network error during communication with the Host.]". HTTP response code is 400. Stack Trace: assert_vm_is_alive = <function is_alive at 0x7ff19254dc80> engine_api = <ovirtsdk4.Connection object at 0x7ff1935fbc50> management_gw_ip = '192.168.201.1' @order_by(_TEST_LIST) def test_run_vms(assert_vm_is_alive, engine_api, management_gw_ip): engine = engine_api.system_service() vm_params = types.Vm( initialization=types.Initialization( user_name=VM_USER_NAME, root_password=VM_PASSWORD ) ) vm_params.initialization.host_name = BACKUP_VM_NAME backup_vm_service = test_utils.get_vm_service(engine, BACKUP_VM_NAME) backup_vm_service.start(use_cloud_init=True, vm=vm_params) vm_params.initialization.host_name = VM2_NAME vm2_service = test_utils.get_vm_service(engine, VM2_NAME) vm2_service.start(use_cloud_init=True, vm=vm_params) # CirrOS cloud-init is different, networking doesn't work since it doesn't support the format oVirt is using vm_params.initialization.host_name = VM0_NAME # hostname seems to work, the others not vm_params.initialization.dns_search = 'lago.local' vm_params.initialization.domain = 'lago.local' vm_params.initialization.dns_servers = management_gw_ip vm0_service = test_utils.get_vm_service(engine, VM0_NAME)
vm0_service.start(use_cloud_init=True, vm=vm_params)
../basic-suite-master/test-scenarios/test_004_basic_sanity.py:794: _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ /usr/lib64/python2.7/site-packages/ovirtsdk4/services.py:32520: in start return self._internal_action(action, 'start', None, headers, query, wait) /usr/lib64/python2.7/site-packages/ovirtsdk4/service.py:299: in _internal_action return future.wait() if wait else future /usr/lib64/python2.7/site-packages/ovirtsdk4/service.py:55: in wait return self._code(response) /usr/lib64/python2.7/site-packages/ovirtsdk4/service.py:296: in callback self._check_fault(response) /usr/lib64/python2.7/site-packages/ovirtsdk4/service.py:134: in _check_fault self._raise_error(response, body.fault) _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ response = <ovirtsdk4.http.Response object at 0x7ff192584d10> detail = <ovirtsdk4.types.Fault object at 0x7ff192584d90> @staticmethod def _raise_error(response, detail=None): """ Creates and raises an error containing the details of the given HTTP response and fault. This method is intended for internal use by other components of the SDK. Refrain from using it directly, as backwards compatibility isn't guaranteed. """ fault = detail if isinstance(detail, types.Fault) else None msg = '' if fault: if fault.reason: if msg: msg += ' ' msg = msg + 'Fault reason is "%s".' % fault.reason if fault.detail: if msg: msg += ' ' msg = msg + 'Fault detail is "%s".' % fault.detail if response: if response.code: if msg: msg += ' ' msg = msg + 'HTTP response code is %s.' % response.code if response.message: if msg: msg += ' ' msg = msg + 'HTTP response message is "%s".' % response.message if isinstance(detail, six.string_types): if msg: msg += ' ' msg = msg + detail + '.' class_ = Error if response is not None: if response.code in [401, 403]: class_ = AuthError elif response.code == 404: class_ = NotFoundError error = class_(msg) error.code = response.code if response else None error.fault = fault
raise error
E Error: Fault reason is "Operation Failed". Fault detail is "[Network error during communication with the Host.]". HTTP response code is 400. /usr/lib64/python2.7/site-packages/ovirtsdk4/service.py:118: Error
Project: https://jenkins.ovirt.org/job/ovirt-system-tests_compat-4.3-suite-master/ Build: https://jenkins.ovirt.org/job/ovirt-system-tests_compat-4.3-suite-master/629... Build Number: 629 Build Status: Still Failing Triggered By: Started by timer ------------------------------------- Changes Since Last Success: ------------------------------------- Changes for Build #620 [Nir Soffer] nfs: Use async mount Changes for Build #621 [Shani Leviim] basic_sanity: add test for creating a floating disk Changes for Build #622 [jenkins CI] Auto create ovirt-4.3 Changes for Build #623 [Ales Musil] basic: Move CirrOS image name to fixtures Changes for Build #624 [Ales Musil] basic: Move CirrOS image name to fixtures Changes for Build #625 [Eitan Raviv] network: pytest skipif requires reason=STRING Changes for Build #626 [Eitan Raviv] network: stabilize host & DC state on change cluster Changes for Build #627 [Galit Rosenthal] basic: performance: decouple 'versioning.py' [Ehud Yonasi] remove upgrades suites and 4.3 from nightly. Changes for Build #628 [jenkins CI] Auto create ovirt-4.3 Changes for Build #629 [hbraha] network: remove lago engine.ssh usage ----------------- Failed Tests: ----------------- 1 tests failed. FAILED: basic-suite-master.test-scenarios.test_004_basic_sanity.test_ha_recovery Error Message: AssertionError: False != True after 600 seconds Stack Trace: engine_api = <ovirtsdk4.Connection object at 0x7f4d6e8c3c50> get_ansible_host_for_vm = <function ansible_host_for at 0x7f4d6f9c7488> @order_by(_TEST_LIST) def test_ha_recovery(engine_api, get_ansible_host_for_vm): engine = engine_api.system_service() with engine_utils.wait_for_event(engine, [119, 9602, 506]): # VM_DOWN_ERROR event(119) # HA_VM_FAILED event event(9602) # VDS_INITIATED_RUN_VM event(506) ansible_host = get_ansible_host_for_vm(VM2_NAME) pid = ansible_host.shell('pgrep -f qemu.*guest=vm2')['stdout'].strip() ansible_host.shell('kill -KILL {}'.format(pid)) vm_service = test_utils.get_vm_service(engine, VM2_NAME) assertions.assert_true_within_long(
lambda:
vm_service.get().status == types.VmStatus.UP ) ../basic-suite-master/test-scenarios/test_004_basic_sanity.py:864: _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ ../ost_utils/ost_utils/assertions.py:98: in assert_true_within_long assert_equals_within_long(func, True, allowed_exceptions) ../ost_utils/ost_utils/assertions.py:83: in assert_equals_within_long func, value, LONG_TIMEOUT, allowed_exceptions=allowed_exceptions _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ func = <function <lambda> at 0x7f4d6fa32b18>, value = True, timeout = 600 allowed_exceptions = [], initial_wait = 0 error_message = 'False != True after 600 seconds' def assert_equals_within( func, value, timeout, allowed_exceptions=None, initial_wait=10, error_message=None ): allowed_exceptions = allowed_exceptions or [] res = '<no-result-obtained>' with _EggTimer(timeout) as timer: while not timer.elapsed(): try: res = func() if res == value: return except Exception as exc: if _instance_of_any(exc, allowed_exceptions): time.sleep(3) continue LOGGER.exception("Unhandled exception in %s", func) raise if initial_wait == 0: time.sleep(3) else: time.sleep(initial_wait) initial_wait = 0 try: if error_message is None: error_message = '%s != %s after %s seconds' % (res, value, timeout)
raise AssertionError(error_message)
E AssertionError: False != True after 600 seconds ../ost_utils/ost_utils/assertions.py:61: AssertionError
Project: https://jenkins.ovirt.org/job/ovirt-system-tests_compat-4.3-suite-master/ Build: https://jenkins.ovirt.org/job/ovirt-system-tests_compat-4.3-suite-master/630... Build Number: 630 Build Status: Still Failing Triggered By: Started by timer ------------------------------------- Changes Since Last Success: ------------------------------------- Changes for Build #621 [Shani Leviim] basic_sanity: add test for creating a floating disk Changes for Build #622 [jenkins CI] Auto create ovirt-4.3 Changes for Build #623 [Ales Musil] basic: Move CirrOS image name to fixtures Changes for Build #624 [Ales Musil] basic: Move CirrOS image name to fixtures Changes for Build #625 [Eitan Raviv] network: pytest skipif requires reason=STRING Changes for Build #626 [Eitan Raviv] network: stabilize host & DC state on change cluster Changes for Build #627 [Galit Rosenthal] basic: performance: decouple 'versioning.py' [Ehud Yonasi] remove upgrades suites and 4.3 from nightly. Changes for Build #628 [jenkins CI] Auto create ovirt-4.3 Changes for Build #629 [hbraha] network: remove lago engine.ssh usage Changes for Build #630 [Michal Skrivanek] lagofy: better error handling when initialization fails ----------------- Failed Tests: ----------------- 1 tests failed. FAILED: basic-suite-master.test-scenarios.test_004_basic_sanity.test_ha_recovery Error Message: AssertionError: False != True after 600 seconds Stack Trace: engine_api = <ovirtsdk4.Connection object at 0x7fe048442190> get_ansible_host_for_vm = <function ansible_host_for at 0x7fe04845ee60> @order_by(_TEST_LIST) def test_ha_recovery(engine_api, get_ansible_host_for_vm): engine = engine_api.system_service() with engine_utils.wait_for_event(engine, [119, 9602, 506]): # VM_DOWN_ERROR event(119) # HA_VM_FAILED event event(9602) # VDS_INITIATED_RUN_VM event(506) ansible_host = get_ansible_host_for_vm(VM2_NAME) pid = ansible_host.shell('pgrep -f qemu.*guest=vm2')['stdout'].strip() ansible_host.shell('kill -KILL {}'.format(pid)) vm_service = test_utils.get_vm_service(engine, VM2_NAME) assertions.assert_true_within_long(
lambda:
vm_service.get().status == types.VmStatus.UP ) ../basic-suite-master/test-scenarios/test_004_basic_sanity.py:864: _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ ../ost_utils/ost_utils/assertions.py:98: in assert_true_within_long assert_equals_within_long(func, True, allowed_exceptions) ../ost_utils/ost_utils/assertions.py:83: in assert_equals_within_long func, value, LONG_TIMEOUT, allowed_exceptions=allowed_exceptions _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ func = <function <lambda> at 0x7fe04832c1b8>, value = True, timeout = 600 allowed_exceptions = [], initial_wait = 0 error_message = 'False != True after 600 seconds' def assert_equals_within( func, value, timeout, allowed_exceptions=None, initial_wait=10, error_message=None ): allowed_exceptions = allowed_exceptions or [] res = '<no-result-obtained>' with _EggTimer(timeout) as timer: while not timer.elapsed(): try: res = func() if res == value: return except Exception as exc: if _instance_of_any(exc, allowed_exceptions): time.sleep(3) continue LOGGER.exception("Unhandled exception in %s", func) raise if initial_wait == 0: time.sleep(3) else: time.sleep(initial_wait) initial_wait = 0 try: if error_message is None: error_message = '%s != %s after %s seconds' % (res, value, timeout)
raise AssertionError(error_message)
E AssertionError: False != True after 600 seconds ../ost_utils/ost_utils/assertions.py:61: AssertionError
Project: https://jenkins.ovirt.org/job/ovirt-system-tests_compat-4.3-suite-master/ Build: https://jenkins.ovirt.org/job/ovirt-system-tests_compat-4.3-suite-master/631... Build Number: 631 Build Status: Still Failing Triggered By: Started by timer ------------------------------------- Changes Since Last Success: ------------------------------------- Changes for Build #622 [jenkins CI] Auto create ovirt-4.3 Changes for Build #623 [Ales Musil] basic: Move CirrOS image name to fixtures Changes for Build #624 [Ales Musil] basic: Move CirrOS image name to fixtures Changes for Build #625 [Eitan Raviv] network: pytest skipif requires reason=STRING Changes for Build #626 [Eitan Raviv] network: stabilize host & DC state on change cluster Changes for Build #627 [Galit Rosenthal] basic: performance: decouple 'versioning.py' [Ehud Yonasi] remove upgrades suites and 4.3 from nightly. Changes for Build #628 [jenkins CI] Auto create ovirt-4.3 Changes for Build #629 [hbraha] network: remove lago engine.ssh usage Changes for Build #630 [Michal Skrivanek] lagofy: better error handling when initialization fails Changes for Build #631 [Michal Skrivanek] lagofy: better error handling when initialization fails ----------------- Failed Tests: ----------------- 1 tests failed. FAILED: basic-suite-master.test-scenarios.test_004_basic_sanity.test_run_vms Error Message: Error: Fault reason is "Operation Failed". Fault detail is "[Network error during communication with the Host.]". HTTP response code is 400. Stack Trace: assert_vm_is_alive = <function is_alive at 0x7fb96dc7d230> engine_api = <ovirtsdk4.Connection object at 0x7fb96ecf3c50> management_gw_ip = '192.168.201.1' @order_by(_TEST_LIST) def test_run_vms(assert_vm_is_alive, engine_api, management_gw_ip): engine = engine_api.system_service() vm_params = types.Vm( initialization=types.Initialization( user_name=VM_USER_NAME, root_password=VM_PASSWORD ) ) vm_params.initialization.host_name = BACKUP_VM_NAME backup_vm_service = test_utils.get_vm_service(engine, BACKUP_VM_NAME) backup_vm_service.start(use_cloud_init=True, vm=vm_params) vm_params.initialization.host_name = VM2_NAME vm2_service = test_utils.get_vm_service(engine, VM2_NAME) vm2_service.start(use_cloud_init=True, vm=vm_params) # CirrOS cloud-init is different, networking doesn't work since it doesn't support the format oVirt is using vm_params.initialization.host_name = VM0_NAME # hostname seems to work, the others not vm_params.initialization.dns_search = 'lago.local' vm_params.initialization.domain = 'lago.local' vm_params.initialization.dns_servers = management_gw_ip vm0_service = test_utils.get_vm_service(engine, VM0_NAME)
vm0_service.start(use_cloud_init=True, vm=vm_params)
../basic-suite-master/test-scenarios/test_004_basic_sanity.py:794: _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ /usr/lib64/python2.7/site-packages/ovirtsdk4/services.py:32520: in start return self._internal_action(action, 'start', None, headers, query, wait) /usr/lib64/python2.7/site-packages/ovirtsdk4/service.py:299: in _internal_action return future.wait() if wait else future /usr/lib64/python2.7/site-packages/ovirtsdk4/service.py:55: in wait return self._code(response) /usr/lib64/python2.7/site-packages/ovirtsdk4/service.py:296: in callback self._check_fault(response) /usr/lib64/python2.7/site-packages/ovirtsdk4/service.py:134: in _check_fault self._raise_error(response, body.fault) _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ response = <ovirtsdk4.http.Response object at 0x7fb96dc3cd10> detail = <ovirtsdk4.types.Fault object at 0x7fb96dc3cd90> @staticmethod def _raise_error(response, detail=None): """ Creates and raises an error containing the details of the given HTTP response and fault. This method is intended for internal use by other components of the SDK. Refrain from using it directly, as backwards compatibility isn't guaranteed. """ fault = detail if isinstance(detail, types.Fault) else None msg = '' if fault: if fault.reason: if msg: msg += ' ' msg = msg + 'Fault reason is "%s".' % fault.reason if fault.detail: if msg: msg += ' ' msg = msg + 'Fault detail is "%s".' % fault.detail if response: if response.code: if msg: msg += ' ' msg = msg + 'HTTP response code is %s.' % response.code if response.message: if msg: msg += ' ' msg = msg + 'HTTP response message is "%s".' % response.message if isinstance(detail, six.string_types): if msg: msg += ' ' msg = msg + detail + '.' class_ = Error if response is not None: if response.code in [401, 403]: class_ = AuthError elif response.code == 404: class_ = NotFoundError error = class_(msg) error.code = response.code if response else None error.fault = fault
raise error
E Error: Fault reason is "Operation Failed". Fault detail is "[Network error during communication with the Host.]". HTTP response code is 400. /usr/lib64/python2.7/site-packages/ovirtsdk4/service.py:118: Error
Project: https://jenkins.ovirt.org/job/ovirt-system-tests_compat-4.3-suite-master/ Build: https://jenkins.ovirt.org/job/ovirt-system-tests_compat-4.3-suite-master/632... Build Number: 632 Build Status: Still Failing Triggered By: Started by timer ------------------------------------- Changes Since Last Success: ------------------------------------- Changes for Build #623 [Ales Musil] basic: Move CirrOS image name to fixtures Changes for Build #624 [Ales Musil] basic: Move CirrOS image name to fixtures Changes for Build #625 [Eitan Raviv] network: pytest skipif requires reason=STRING Changes for Build #626 [Eitan Raviv] network: stabilize host & DC state on change cluster Changes for Build #627 [Galit Rosenthal] basic: performance: decouple 'versioning.py' [Ehud Yonasi] remove upgrades suites and 4.3 from nightly. Changes for Build #628 [jenkins CI] Auto create ovirt-4.3 Changes for Build #629 [hbraha] network: remove lago engine.ssh usage Changes for Build #630 [Michal Skrivanek] lagofy: better error handling when initialization fails Changes for Build #631 [Michal Skrivanek] lagofy: better error handling when initialization fails Changes for Build #632 [hbraha] network: remove engine.ssh from fixtures/storage ----------------- Failed Tests: ----------------- No tests ran.
Project: https://jenkins.ovirt.org/job/ovirt-system-tests_compat-4.3-suite-master/ Build: https://jenkins.ovirt.org/job/ovirt-system-tests_compat-4.3-suite-master/633... Build Number: 633 Build Status: Still Failing Triggered By: Started by timer ------------------------------------- Changes Since Last Success: ------------------------------------- Changes for Build #624 [Ales Musil] basic: Move CirrOS image name to fixtures Changes for Build #625 [Eitan Raviv] network: pytest skipif requires reason=STRING Changes for Build #626 [Eitan Raviv] network: stabilize host & DC state on change cluster Changes for Build #627 [Galit Rosenthal] basic: performance: decouple 'versioning.py' [Ehud Yonasi] remove upgrades suites and 4.3 from nightly. Changes for Build #628 [jenkins CI] Auto create ovirt-4.3 Changes for Build #629 [hbraha] network: remove lago engine.ssh usage Changes for Build #630 [Michal Skrivanek] lagofy: better error handling when initialization fails Changes for Build #631 [Michal Skrivanek] lagofy: better error handling when initialization fails Changes for Build #632 [hbraha] network: remove engine.ssh from fixtures/storage Changes for Build #633 [Shani Leviim] sd_reattach: add a test for registering a floating disk ----------------- Failed Tests: ----------------- No tests ran.
Project: https://jenkins.ovirt.org/job/ovirt-system-tests_compat-4.3-suite-master/ Build: https://jenkins.ovirt.org/job/ovirt-system-tests_compat-4.3-suite-master/634... Build Number: 634 Build Status: Still Failing Triggered By: Started by timer ------------------------------------- Changes Since Last Success: ------------------------------------- Changes for Build #624 [Ales Musil] basic: Move CirrOS image name to fixtures Changes for Build #625 [Eitan Raviv] network: pytest skipif requires reason=STRING Changes for Build #626 [Eitan Raviv] network: stabilize host & DC state on change cluster Changes for Build #627 [Galit Rosenthal] basic: performance: decouple 'versioning.py' [Ehud Yonasi] remove upgrades suites and 4.3 from nightly. Changes for Build #628 [jenkins CI] Auto create ovirt-4.3 Changes for Build #629 [hbraha] network: remove lago engine.ssh usage Changes for Build #630 [Michal Skrivanek] lagofy: better error handling when initialization fails Changes for Build #631 [Michal Skrivanek] lagofy: better error handling when initialization fails Changes for Build #632 [hbraha] network: remove engine.ssh from fixtures/storage Changes for Build #633 [Shani Leviim] sd_reattach: add a test for registering a floating disk Changes for Build #634 [Marcin Sobczyk] he: Add initial support for prebuilt OST images ----------------- Failed Tests: ----------------- No tests ran.
Project: https://jenkins.ovirt.org/job/ovirt-system-tests_compat-4.3-suite-master/ Build: https://jenkins.ovirt.org/job/ovirt-system-tests_compat-4.3-suite-master/635... Build Number: 635 Build Status: Still Failing Triggered By: Started by timer ------------------------------------- Changes Since Last Success: ------------------------------------- Changes for Build #625 [Eitan Raviv] network: pytest skipif requires reason=STRING Changes for Build #626 [Eitan Raviv] network: stabilize host & DC state on change cluster Changes for Build #627 [Galit Rosenthal] basic: performance: decouple 'versioning.py' [Ehud Yonasi] remove upgrades suites and 4.3 from nightly. Changes for Build #628 [jenkins CI] Auto create ovirt-4.3 Changes for Build #629 [hbraha] network: remove lago engine.ssh usage Changes for Build #630 [Michal Skrivanek] lagofy: better error handling when initialization fails Changes for Build #631 [Michal Skrivanek] lagofy: better error handling when initialization fails Changes for Build #632 [hbraha] network: remove engine.ssh from fixtures/storage Changes for Build #633 [Shani Leviim] sd_reattach: add a test for registering a floating disk Changes for Build #634 [Marcin Sobczyk] he: Add initial support for prebuilt OST images Changes for Build #635 [Eitan Raviv] network: remove lago-env arg [Gal Ben Haim] usrc: Add post update hook ----------------- Failed Tests: ----------------- No tests ran.
Project: https://jenkins.ovirt.org/job/ovirt-system-tests_compat-4.3-suite-master/ Build: https://jenkins.ovirt.org/job/ovirt-system-tests_compat-4.3-suite-master/636... Build Number: 636 Build Status: Still Failing Triggered By: Started by timer ------------------------------------- Changes Since Last Success: ------------------------------------- Changes for Build #626 [Eitan Raviv] network: stabilize host & DC state on change cluster Changes for Build #627 [Galit Rosenthal] basic: performance: decouple 'versioning.py' [Ehud Yonasi] remove upgrades suites and 4.3 from nightly. Changes for Build #628 [jenkins CI] Auto create ovirt-4.3 Changes for Build #629 [hbraha] network: remove lago engine.ssh usage Changes for Build #630 [Michal Skrivanek] lagofy: better error handling when initialization fails Changes for Build #631 [Michal Skrivanek] lagofy: better error handling when initialization fails Changes for Build #632 [hbraha] network: remove engine.ssh from fixtures/storage Changes for Build #633 [Shani Leviim] sd_reattach: add a test for registering a floating disk Changes for Build #634 [Marcin Sobczyk] he: Add initial support for prebuilt OST images Changes for Build #635 [Eitan Raviv] network: remove lago-env arg [Gal Ben Haim] usrc: Add post update hook Changes for Build #636 [Andrej Cernek] network: add remaining copy host network tests [Galit Rosenthal] Move hc nightly 4.3 to run from branch ovirt-engine-4.3 ----------------- Failed Tests: ----------------- No tests ran.
Project: https://jenkins.ovirt.org/job/ovirt-system-tests_compat-4.3-suite-master/ Build: https://jenkins.ovirt.org/job/ovirt-system-tests_compat-4.3-suite-master/637... Build Number: 637 Build Status: Still Failing Triggered By: Started by timer ------------------------------------- Changes Since Last Success: ------------------------------------- Changes for Build #627 [Galit Rosenthal] basic: performance: decouple 'versioning.py' [Ehud Yonasi] remove upgrades suites and 4.3 from nightly. Changes for Build #628 [jenkins CI] Auto create ovirt-4.3 Changes for Build #629 [hbraha] network: remove lago engine.ssh usage Changes for Build #630 [Michal Skrivanek] lagofy: better error handling when initialization fails Changes for Build #631 [Michal Skrivanek] lagofy: better error handling when initialization fails Changes for Build #632 [hbraha] network: remove engine.ssh from fixtures/storage Changes for Build #633 [Shani Leviim] sd_reattach: add a test for registering a floating disk Changes for Build #634 [Marcin Sobczyk] he: Add initial support for prebuilt OST images Changes for Build #635 [Eitan Raviv] network: remove lago-env arg [Gal Ben Haim] usrc: Add post update hook Changes for Build #636 [Andrej Cernek] network: add remaining copy host network tests [Galit Rosenthal] Move hc nightly 4.3 to run from branch ovirt-engine-4.3 Changes for Build #637 [Michal Skrivanek] setup_playbook: disable gpg check for epel repo ----------------- Failed Tests: ----------------- No tests ran.
Project: https://jenkins.ovirt.org/job/ovirt-system-tests_compat-4.3-suite-master/ Build: https://jenkins.ovirt.org/job/ovirt-system-tests_compat-4.3-suite-master/638... Build Number: 638 Build Status: Still Failing Triggered By: Started by timer ------------------------------------- Changes Since Last Success: ------------------------------------- Changes for Build #628 [jenkins CI] Auto create ovirt-4.3 Changes for Build #629 [hbraha] network: remove lago engine.ssh usage Changes for Build #630 [Michal Skrivanek] lagofy: better error handling when initialization fails Changes for Build #631 [Michal Skrivanek] lagofy: better error handling when initialization fails Changes for Build #632 [hbraha] network: remove engine.ssh from fixtures/storage Changes for Build #633 [Shani Leviim] sd_reattach: add a test for registering a floating disk Changes for Build #634 [Marcin Sobczyk] he: Add initial support for prebuilt OST images Changes for Build #635 [Eitan Raviv] network: remove lago-env arg [Gal Ben Haim] usrc: Add post update hook Changes for Build #636 [Andrej Cernek] network: add remaining copy host network tests [Galit Rosenthal] Move hc nightly 4.3 to run from branch ovirt-engine-4.3 Changes for Build #637 [Michal Skrivanek] setup_playbook: disable gpg check for epel repo Changes for Build #638 [Michal Skrivanek] setup_playbook: disable gpg check for epel repo ----------------- Failed Tests: ----------------- No tests ran.
Project: https://jenkins.ovirt.org/job/ovirt-system-tests_compat-4.3-suite-master/ Build: https://jenkins.ovirt.org/job/ovirt-system-tests_compat-4.3-suite-master/639... Build Number: 639 Build Status: Still Failing Triggered By: Started by timer ------------------------------------- Changes Since Last Success: ------------------------------------- Changes for Build #629 [hbraha] network: remove lago engine.ssh usage Changes for Build #630 [Michal Skrivanek] lagofy: better error handling when initialization fails Changes for Build #631 [Michal Skrivanek] lagofy: better error handling when initialization fails Changes for Build #632 [hbraha] network: remove engine.ssh from fixtures/storage Changes for Build #633 [Shani Leviim] sd_reattach: add a test for registering a floating disk Changes for Build #634 [Marcin Sobczyk] he: Add initial support for prebuilt OST images Changes for Build #635 [Eitan Raviv] network: remove lago-env arg [Gal Ben Haim] usrc: Add post update hook Changes for Build #636 [Andrej Cernek] network: add remaining copy host network tests [Galit Rosenthal] Move hc nightly 4.3 to run from branch ovirt-engine-4.3 Changes for Build #637 [Michal Skrivanek] setup_playbook: disable gpg check for epel repo Changes for Build #638 [Michal Skrivanek] setup_playbook: disable gpg check for epel repo Changes for Build #639 [Michal Skrivanek] setup_playbook: disable gpg check for epel repo ----------------- Failed Tests: ----------------- No tests ran.
Project: https://jenkins.ovirt.org/job/ovirt-system-tests_compat-4.3-suite-master/ Build: https://jenkins.ovirt.org/job/ovirt-system-tests_compat-4.3-suite-master/640... Build Number: 640 Build Status: Still Failing Triggered By: Started by timer ------------------------------------- Changes Since Last Success: ------------------------------------- Changes for Build #630 [Michal Skrivanek] lagofy: better error handling when initialization fails Changes for Build #631 [Michal Skrivanek] lagofy: better error handling when initialization fails Changes for Build #632 [hbraha] network: remove engine.ssh from fixtures/storage Changes for Build #633 [Shani Leviim] sd_reattach: add a test for registering a floating disk Changes for Build #634 [Marcin Sobczyk] he: Add initial support for prebuilt OST images Changes for Build #635 [Eitan Raviv] network: remove lago-env arg [Gal Ben Haim] usrc: Add post update hook Changes for Build #636 [Andrej Cernek] network: add remaining copy host network tests [Galit Rosenthal] Move hc nightly 4.3 to run from branch ovirt-engine-4.3 Changes for Build #637 [Michal Skrivanek] setup_playbook: disable gpg check for epel repo Changes for Build #638 [Michal Skrivanek] setup_playbook: disable gpg check for epel repo Changes for Build #639 [Michal Skrivanek] setup_playbook: disable gpg check for epel repo Changes for Build #640 [Andrej Cernek] he: Port '006_network_by_label' to pytest and ansible ----------------- Failed Tests: ----------------- No tests ran.
Project: https://jenkins.ovirt.org/job/ovirt-system-tests_compat-4.3-suite-master/ Build: https://jenkins.ovirt.org/job/ovirt-system-tests_compat-4.3-suite-master/641... Build Number: 641 Build Status: Still Failing Triggered By: Started by timer ------------------------------------- Changes Since Last Success: ------------------------------------- Changes for Build #631 [Michal Skrivanek] lagofy: better error handling when initialization fails Changes for Build #632 [hbraha] network: remove engine.ssh from fixtures/storage Changes for Build #633 [Shani Leviim] sd_reattach: add a test for registering a floating disk Changes for Build #634 [Marcin Sobczyk] he: Add initial support for prebuilt OST images Changes for Build #635 [Eitan Raviv] network: remove lago-env arg [Gal Ben Haim] usrc: Add post update hook Changes for Build #636 [Andrej Cernek] network: add remaining copy host network tests [Galit Rosenthal] Move hc nightly 4.3 to run from branch ovirt-engine-4.3 Changes for Build #637 [Michal Skrivanek] setup_playbook: disable gpg check for epel repo Changes for Build #638 [Michal Skrivanek] setup_playbook: disable gpg check for epel repo Changes for Build #639 [Michal Skrivanek] setup_playbook: disable gpg check for epel repo Changes for Build #640 [Andrej Cernek] he: Port '006_network_by_label' to pytest and ansible Changes for Build #641 [Eitan Raviv] network: generalize ssh password [Gal Ben Haim] Set the git user in the updater test ----------------- Failed Tests: ----------------- No tests ran.
Project: https://jenkins.ovirt.org/job/ovirt-system-tests_compat-4.3-suite-master/ Build: https://jenkins.ovirt.org/job/ovirt-system-tests_compat-4.3-suite-master/642... Build Number: 642 Build Status: Still Failing Triggered By: Started by timer ------------------------------------- Changes Since Last Success: ------------------------------------- Changes for Build #632 [hbraha] network: remove engine.ssh from fixtures/storage Changes for Build #633 [Shani Leviim] sd_reattach: add a test for registering a floating disk Changes for Build #634 [Marcin Sobczyk] he: Add initial support for prebuilt OST images Changes for Build #635 [Eitan Raviv] network: remove lago-env arg [Gal Ben Haim] usrc: Add post update hook Changes for Build #636 [Andrej Cernek] network: add remaining copy host network tests [Galit Rosenthal] Move hc nightly 4.3 to run from branch ovirt-engine-4.3 Changes for Build #637 [Michal Skrivanek] setup_playbook: disable gpg check for epel repo Changes for Build #638 [Michal Skrivanek] setup_playbook: disable gpg check for epel repo Changes for Build #639 [Michal Skrivanek] setup_playbook: disable gpg check for epel repo Changes for Build #640 [Andrej Cernek] he: Port '006_network_by_label' to pytest and ansible Changes for Build #641 [Eitan Raviv] network: generalize ssh password [Gal Ben Haim] Set the git user in the updater test Changes for Build #642 [Eitan Raviv] network: remove lago env dependency from fqdn fixture [Galit Rosenthal] Remove OST poll upstream sources ----------------- Failed Tests: ----------------- No tests ran.
Project: https://jenkins.ovirt.org/job/ovirt-system-tests_compat-4.3-suite-master/ Build: https://jenkins.ovirt.org/job/ovirt-system-tests_compat-4.3-suite-master/643... Build Number: 643 Build Status: Still Failing Triggered By: Started by timer ------------------------------------- Changes Since Last Success: ------------------------------------- Changes for Build #633 [Shani Leviim] sd_reattach: add a test for registering a floating disk Changes for Build #634 [Marcin Sobczyk] he: Add initial support for prebuilt OST images Changes for Build #635 [Eitan Raviv] network: remove lago-env arg [Gal Ben Haim] usrc: Add post update hook Changes for Build #636 [Andrej Cernek] network: add remaining copy host network tests [Galit Rosenthal] Move hc nightly 4.3 to run from branch ovirt-engine-4.3 Changes for Build #637 [Michal Skrivanek] setup_playbook: disable gpg check for epel repo Changes for Build #638 [Michal Skrivanek] setup_playbook: disable gpg check for epel repo Changes for Build #639 [Michal Skrivanek] setup_playbook: disable gpg check for epel repo Changes for Build #640 [Andrej Cernek] he: Port '006_network_by_label' to pytest and ansible Changes for Build #641 [Eitan Raviv] network: generalize ssh password [Gal Ben Haim] Set the git user in the updater test Changes for Build #642 [Eitan Raviv] network: remove lago env dependency from fqdn fixture [Galit Rosenthal] Remove OST poll upstream sources Changes for Build #643 [Liran Rotenberg] basic_sanity: check snapshot in clone test [Evgheni Dereveanchin] remove ovirt-release builds for fc30 [Mark Li] Add GitCommitModule to stdci_libs/git_utils [Ehud Yonasi] stdci.yaml: drop ost-containers from CI. ----------------- Failed Tests: ----------------- No tests ran.
Project: https://jenkins.ovirt.org/job/ovirt-system-tests_compat-4.3-suite-master/ Build: https://jenkins.ovirt.org/job/ovirt-system-tests_compat-4.3-suite-master/644... Build Number: 644 Build Status: Still Failing Triggered By: Started by timer ------------------------------------- Changes Since Last Success: ------------------------------------- Changes for Build #634 [Marcin Sobczyk] he: Add initial support for prebuilt OST images Changes for Build #635 [Eitan Raviv] network: remove lago-env arg [Gal Ben Haim] usrc: Add post update hook Changes for Build #636 [Andrej Cernek] network: add remaining copy host network tests [Galit Rosenthal] Move hc nightly 4.3 to run from branch ovirt-engine-4.3 Changes for Build #637 [Michal Skrivanek] setup_playbook: disable gpg check for epel repo Changes for Build #638 [Michal Skrivanek] setup_playbook: disable gpg check for epel repo Changes for Build #639 [Michal Skrivanek] setup_playbook: disable gpg check for epel repo Changes for Build #640 [Andrej Cernek] he: Port '006_network_by_label' to pytest and ansible Changes for Build #641 [Eitan Raviv] network: generalize ssh password [Gal Ben Haim] Set the git user in the updater test Changes for Build #642 [Eitan Raviv] network: remove lago env dependency from fqdn fixture [Galit Rosenthal] Remove OST poll upstream sources Changes for Build #643 [Liran Rotenberg] basic_sanity: check snapshot in clone test [Evgheni Dereveanchin] remove ovirt-release builds for fc30 [Mark Li] Add GitCommitModule to stdci_libs/git_utils [Ehud Yonasi] stdci.yaml: drop ost-containers from CI. Changes for Build #644 [Liran Rotenberg] basic_sanity: check snapshot in clone test ----------------- Failed Tests: ----------------- No tests ran.
Project: https://jenkins.ovirt.org/job/ovirt-system-tests_compat-4.3-suite-master/ Build: https://jenkins.ovirt.org/job/ovirt-system-tests_compat-4.3-suite-master/645... Build Number: 645 Build Status: Still Failing Triggered By: Started by timer ------------------------------------- Changes Since Last Success: ------------------------------------- Changes for Build #635 [Eitan Raviv] network: remove lago-env arg [Gal Ben Haim] usrc: Add post update hook Changes for Build #636 [Andrej Cernek] network: add remaining copy host network tests [Galit Rosenthal] Move hc nightly 4.3 to run from branch ovirt-engine-4.3 Changes for Build #637 [Michal Skrivanek] setup_playbook: disable gpg check for epel repo Changes for Build #638 [Michal Skrivanek] setup_playbook: disable gpg check for epel repo Changes for Build #639 [Michal Skrivanek] setup_playbook: disable gpg check for epel repo Changes for Build #640 [Andrej Cernek] he: Port '006_network_by_label' to pytest and ansible Changes for Build #641 [Eitan Raviv] network: generalize ssh password [Gal Ben Haim] Set the git user in the updater test Changes for Build #642 [Eitan Raviv] network: remove lago env dependency from fqdn fixture [Galit Rosenthal] Remove OST poll upstream sources Changes for Build #643 [Liran Rotenberg] basic_sanity: check snapshot in clone test [Evgheni Dereveanchin] remove ovirt-release builds for fc30 [Mark Li] Add GitCommitModule to stdci_libs/git_utils [Ehud Yonasi] stdci.yaml: drop ost-containers from CI. Changes for Build #644 [Liran Rotenberg] basic_sanity: check snapshot in clone test Changes for Build #645 [Liran Rotenberg] basic_sanity: check snapshot in clone test ----------------- Failed Tests: ----------------- No tests ran.
participants (1)
-
jenkins＠jenkins.phx.ovirt.org