[oVirt Jenkins] ovirt_master_hc-system-tests - Build # 65 - Failure!

------=_Part_18_795736980.1491968853625 Content-Type: text/plain; charset=UTF-8 Content-Transfer-Encoding: quoted-printable Project: http://jenkins.ovirt.org/job/ovirt_master_hc-system-tests/=20 Build: http://jenkins.ovirt.org/job/ovirt_master_hc-system-tests/65/ Build Number: 65 Build Status: Failure Triggered By: Started by timer ------------------------------------- Changes Since Last Success: ------------------------------------- Changes for Build #65 [Ond=C5=99ej Svoboda] 4.1 backport: Test creating, attaching and destroying= a bonding. ----------------- Failed Tests: ----------------- 2 tests failed. FAILED: 002_bootstrap.add_hosts Error Message: [ERROR]::oVirt API connection failure, (7, 'TCP connection reset by peer') -------------------- >> begin captured logging << -------------------- ovirtlago.testlib: ERROR: * Unhandled exception in <function _host_is_u= p at 0x2b9bde8> Traceback (most recent call last): File "/usr/lib/python2.7/site-packages/ovirtlago/testlib.py", line 217, i= n assert_equals_within res =3D func() File "/home/jenkins/workspace/ovirt_master_hc-system-tests/ovirt-system-t= ests/hc-basic-suite-master/test-scenarios/002_bootstrap.py", line 145, in _= host_is_up cur_state =3D api.hosts.get(host.name()).status.state File "/usr/lib/python2.7/site-packages/ovirtsdk/infrastructure/brokers.py= ", line 18338, in get headers=3D{"All-Content":all_content} File "/usr/lib/python2.7/site-packages/ovirtsdk/infrastructure/proxy.py",= line 46, in get return self.request(method=3D'GET', url=3Durl, headers=3Dheaders, cls= =3Dcls) File "/usr/lib/python2.7/site-packages/ovirtsdk/infrastructure/proxy.py",= line 122, in request persistent_auth=3Dself.__persistent_auth File "/usr/lib/python2.7/site-packages/ovirtsdk/infrastructure/connection= spool.py", line 81, in do_request raise errors.ConnectionError(error) ConnectionError: [ERROR]::oVirt API connection failure, (7, 'TCP connection= reset by peer') --------------------- >> end captured logging << --------------------- Stack Trace: File "/usr/lib64/python2.7/unittest/case.py", line 369, in run testMethod() File "/usr/lib/python2.7/site-packages/nose/case.py", line 197, in runTes= t self.test(*self.arg) File "/usr/lib/python2.7/site-packages/ovirtlago/testlib.py", line 129, i= n wrapped_test test() File "/usr/lib/python2.7/site-packages/ovirtlago/testlib.py", line 59, in= wrapper return func(get_test_prefix(), *args, **kwargs) File "/home/jenkins/workspace/ovirt_master_hc-system-tests/ovirt-system-t= ests/hc-basic-suite-master/test-scenarios/002_bootstrap.py", line 164, in a= dd_hosts testlib.assert_true_within(_host_is_up, timeout=3D15 * 60) File "/usr/lib/python2.7/site-packages/ovirtlago/testlib.py", line 256, i= n assert_true_within assert_equals_within(func, True, timeout, allowed_exceptions) File "/usr/lib/python2.7/site-packages/ovirtlago/testlib.py", line 217, i= n assert_equals_within res =3D func() File "/home/jenkins/workspace/ovirt_master_hc-system-tests/ovirt-system-t= ests/hc-basic-suite-master/test-scenarios/002_bootstrap.py", line 145, in _= host_is_up cur_state =3D api.hosts.get(host.name()).status.state File "/usr/lib/python2.7/site-packages/ovirtsdk/infrastructure/brokers.py= ", line 18338, in get headers=3D{"All-Content":all_content} File "/usr/lib/python2.7/site-packages/ovirtsdk/infrastructure/proxy.py",= line 46, in get return self.request(method=3D'GET', url=3Durl, headers=3Dheaders, cls= =3Dcls) File "/usr/lib/python2.7/site-packages/ovirtsdk/infrastructure/proxy.py",= line 122, in request persistent_auth=3Dself.__persistent_auth File "/usr/lib/python2.7/site-packages/ovirtsdk/infrastructure/connection= spool.py", line 81, in do_request raise errors.ConnectionError(error) [ERROR]::oVirt API connection failure, (7, 'TCP connection reset by peer') -------------------- >> begin captured logging << -------------------- ovirtlago.testlib: ERROR: * Unhandled exception in <function _host_is_u= p at 0x2b9bde8> Traceback (most recent call last): File "/usr/lib/python2.7/site-packages/ovirtlago/testlib.py", line 217, i= n assert_equals_within res =3D func() File "/home/jenkins/workspace/ovirt_master_hc-system-tests/ovirt-system-t= ests/hc-basic-suite-master/test-scenarios/002_bootstrap.py", line 145, in _= host_is_up cur_state =3D api.hosts.get(host.name()).status.state File "/usr/lib/python2.7/site-packages/ovirtsdk/infrastructure/brokers.py= ", line 18338, in get headers=3D{"All-Content":all_content} File "/usr/lib/python2.7/site-packages/ovirtsdk/infrastructure/proxy.py",= line 46, in get return self.request(method=3D'GET', url=3Durl, headers=3Dheaders, cls= =3Dcls) File "/usr/lib/python2.7/site-packages/ovirtsdk/infrastructure/proxy.py",= line 122, in request persistent_auth=3Dself.__persistent_auth File "/usr/lib/python2.7/site-packages/ovirtsdk/infrastructure/connection= spool.py", line 81, in do_request raise errors.ConnectionError(error) ConnectionError: [ERROR]::oVirt API connection failure, (7, 'TCP connection= reset by peer') --------------------- >> end captured logging << --------------------- FAILED: 002_bootstrap.add_hosts Error Message: Unable to extract paths from lago-hc-basic-suite-master-engine: unreachable= with SSH -------------------- >> begin captured logging << -------------------- ovirtlago.testlib: ERROR: * Unhandled exception in <function _host_is_u= p at 0x2b9bde8> Traceback (most recent call last): File "/usr/lib/python2.7/site-packages/ovirtlago/testlib.py", line 217, i= n assert_equals_within res =3D func() File "/home/jenkins/workspace/ovirt_master_hc-system-tests/ovirt-system-t= ests/hc-basic-suite-master/test-scenarios/002_bootstrap.py", line 145, in _= host_is_up cur_state =3D api.hosts.get(host.name()).status.state File "/usr/lib/python2.7/site-packages/ovirtsdk/infrastructure/brokers.py= ", line 18338, in get headers=3D{"All-Content":all_content} File "/usr/lib/python2.7/site-packages/ovirtsdk/infrastructure/proxy.py",= line 46, in get return self.request(method=3D'GET', url=3Durl, headers=3Dheaders, cls= =3Dcls) File "/usr/lib/python2.7/site-packages/ovirtsdk/infrastructure/proxy.py",= line 122, in request persistent_auth=3Dself.__persistent_auth File "/usr/lib/python2.7/site-packages/ovirtsdk/infrastructure/connection= spool.py", line 81, in do_request raise errors.ConnectionError(error) ConnectionError: [ERROR]::oVirt API connection failure, (7, 'TCP connection= reset by peer') lago.utils: DEBUG: Looking for a workdir lago.workdir: DEBUG: Checking if /home/jenkins/workspace/ovirt_master_hc-sy= stem-tests/ovirt-system-tests/deployment-hc-basic-suite-master is a workdir lago.prefix: INFO: * Collect artifacts: ?[0m?[0m lago.prefix: INFO: ?[0m?[0m lago.prefix: INFO: ?[0m?[0m lago.prefix: INFO: ?[0m?[0m lago.prefix: INFO: ?[0m?[0m lago.ssh: DEBUG: start task:f8bad1fc-60b9-49d1-9f31-672a3ee74270:Get ssh cl= ient for lago-hc-basic-suite-master-engine: lago.ssh: DEBUG: Still got 1 tries for lago-hc-basic-suite-master-engine lago.ssh: DEBUG: start task:e8d204a8-bfb1-4db6-a7ad-bbbd637db48a:Get ssh cl= ient for lago-hc-basic-suite-master-host1: lago.ssh: DEBUG: start task:be57934b-b829-4e74-b5df-82470742f701:Get ssh cl= ient for lago-hc-basic-suite-master-host0: lago.ssh: DEBUG: Still got 1 tries for lago-hc-basic-suite-master-host1 lago.ssh: DEBUG: Still got 1 tries for lago-hc-basic-suite-master-host0 lago.ssh: DEBUG: Socket error connecting to lago-hc-basic-suite-master-engi= ne: [Errno None] Unable to connect to port 22 on 192.168.201.99 lago.ssh: DEBUG: start task:c20b9072-ebcb-4c74-9704-a192f15cf282:Get ssh cl= ient for lago-hc-basic-suite-master-host2: lago.ssh: DEBUG: Still got 1 tries for lago-hc-basic-suite-master-host2 lago.ssh: DEBUG: end task:c20b9072-ebcb-4c74-9704-a192f15cf282:Get ssh clie= nt for lago-hc-basic-suite-master-host2: lago.ssh: DEBUG: end task:e8d204a8-bfb1-4db6-a7ad-bbbd637db48a:Get ssh clie= nt for lago-hc-basic-suite-master-host1: lago.ssh: DEBUG: end task:be57934b-b829-4e74-b5df-82470742f701:Get ssh clie= nt for lago-hc-basic-suite-master-host0: lago.plugins.vm: DEBUG: Extracting scp://lago-hc-basic-suite-master-host2:/= var/log to /home/jenkins/workspace/ovirt_master_hc-system-tests/ovirt-syste= m-tests/deployment-hc-basic-suite-master/default/test_logs/002_bootstrap.ad= d_hosts-20170412034659/lago-hc-basic-suite-master-host2/_var_log lago.plugins.vm: DEBUG: Extracting scp://lago-hc-basic-suite-master-host1:/= var/log to /home/jenkins/workspace/ovirt_master_hc-system-tests/ovirt-syste= m-tests/deployment-hc-basic-suite-master/default/test_logs/002_bootstrap.ad= d_hosts-20170412034659/lago-hc-basic-suite-master-host1/_var_log lago.ssh: DEBUG: start task:485f1b5b-19e3-4a5d-b6f5-ce410dfa93e7:Get ssh cl= ient for lago-hc-basic-suite-master-host2: lago.ssh: DEBUG: start task:7580203c-3c7f-463d-a9bb-d2e8240016ab:Get ssh cl= ient for lago-hc-basic-suite-master-host1: lago.ssh: DEBUG: Still got 100 tries for lago-hc-basic-suite-master-host2 lago.ssh: DEBUG: Still got 100 tries for lago-hc-basic-suite-master-host1 lago.plugins.vm: DEBUG: Extracting scp://lago-hc-basic-suite-master-host0:/= var/log to /home/jenkins/workspace/ovirt_master_hc-system-tests/ovirt-syste= m-tests/deployment-hc-basic-suite-master/default/test_logs/002_bootstrap.ad= d_hosts-20170412034659/lago-hc-basic-suite-master-host0/_var_log lago.ssh: DEBUG: start task:9d4f24eb-92ac-45f1-8d3b-e1474f8a77ed:Get ssh cl= ient for lago-hc-basic-suite-master-host0: lago.ssh: DEBUG: Still got 100 tries for lago-hc-basic-suite-master-host0 lago.ssh: DEBUG: end task:7580203c-3c7f-463d-a9bb-d2e8240016ab:Get ssh clie= nt for lago-hc-basic-suite-master-host1: lago.ssh: DEBUG: end task:485f1b5b-19e3-4a5d-b6f5-ce410dfa93e7:Get ssh clie= nt for lago-hc-basic-suite-master-host2: lago.ssh: DEBUG: end task:9d4f24eb-92ac-45f1-8d3b-e1474f8a77ed:Get ssh clie= nt for lago-hc-basic-suite-master-host0: lago.ssh: DEBUG: end task:f8bad1fc-60b9-49d1-9f31-672a3ee74270:Get ssh clie= nt for lago-hc-basic-suite-master-engine: root: ERROR: - [Thread-6] lago-hc-basic-suite-master-engine: ?[31mERR= OR?[0m (in 0:00:01) lago.utils: ERROR: Error while running thread Traceback (most recent call last): File "/usr/lib/python2.7/site-packages/lago/utils.py", line 57, in _ret_v= ia_queue queue.put({'return': func()}) File "/usr/lib/python2.7/site-packages/lago/prefix.py", line 1205, in _co= llect_artifacts vm.collect_artifacts(path, ignore_nopath) File "/usr/lib/python2.7/site-packages/lago/plugins/vm.py", line 562, in = collect_artifacts ignore_nopath=3Dignore_nopath File "/usr/lib/python2.7/site-packages/lago/plugins/vm.py", line 359, in = extract_paths return self.provider.extract_paths(paths, *args, **kwargs) File "/usr/lib/python2.7/site-packages/lago/plugins/vm.py", line 234, in = extract_paths format(self.vm.name()) ExtractPathError: Unable to extract paths from lago-hc-basic-suite-master-e= ngine: unreachable with SSH lago.prefix: INFO: ?[32mSuccess?[0m (in 0:00:02) lago.prefix: INFO: ?[32mSuccess?[0m (in 0:00:02) lago.prefix: INFO: ?[32mSuccess?[0m (in 0:00:04) root: ERROR: * Collect artifacts: ?[31mERROR?[0m (in 0:00:04) --------------------- >> end captured logging << --------------------- Stack Trace: File "/usr/lib/python2.7/site-packages/nose/case.py", line 133, in run self.runTest(result) File "/usr/lib/python2.7/site-packages/nose/case.py", line 151, in runTes= t test(result) File "/usr/lib64/python2.7/unittest/case.py", line 433, in __call__ return self.run(*args, **kwds) File "/usr/lib64/python2.7/unittest/case.py", line 393, in run result.addError(self, sys.exc_info()) File "/usr/lib/python2.7/site-packages/nose/proxy.py", line 131, in addEr= ror plugins.addError(self.test, err) File "/usr/lib/python2.7/site-packages/nose/plugins/manager.py", line 99,= in __call__ return self.call(*arg, **kw) File "/usr/lib/python2.7/site-packages/nose/plugins/manager.py", line 167= , in simple result =3D meth(*arg, **kw) File "/usr/lib/python2.7/site-packages/ovirtlago/testlib.py", line 150, i= n addError self._addFault(test, err) File "/usr/lib/python2.7/site-packages/ovirtlago/testlib.py", line 161, i= n _addFault no_skip=3DFalse File "/usr/lib/python2.7/site-packages/lago/plugins/cli.py", line 190, in= __call__ return self._do_run(*args, **kwargs) File "/usr/lib/python2.7/site-packages/lago/utils.py", line 495, in wrapp= er return func(*args, **kwargs) File "/usr/lib/python2.7/site-packages/lago/utils.py", line 506, in wrapp= er return func(*args, prefix=3Dprefix, **kwargs) File "/usr/lib/python2.7/site-packages/lago/cmd.py", line 714, in do_coll= ect prefix.collect_artifacts(output, ignore_nopath=3Dnot no_skip) File "/usr/lib/python2.7/site-packages/lago/log_utils.py", line 633, in w= rapper return func(*args, **kwargs) File "/usr/lib/python2.7/site-packages/lago/prefix.py", line 1209, in col= lect_artifacts self.virt_env.get_vms().values(), File "/usr/lib/python2.7/site-packages/lago/utils.py", line 99, in invoke= _in_parallel vt.join_all() File "/usr/lib/python2.7/site-packages/lago/utils.py", line 57, in _ret_v= ia_queue queue.put({'return': func()}) File "/usr/lib/python2.7/site-packages/lago/prefix.py", line 1205, in _co= llect_artifacts vm.collect_artifacts(path, ignore_nopath) File "/usr/lib/python2.7/site-packages/lago/plugins/vm.py", line 562, in = collect_artifacts ignore_nopath=3Dignore_nopath File "/usr/lib/python2.7/site-packages/lago/plugins/vm.py", line 359, in = extract_paths return self.provider.extract_paths(paths, *args, **kwargs) File "/usr/lib/python2.7/site-packages/lago/plugins/vm.py", line 234, in = extract_paths format(self.vm.name()) Unable to extract paths from lago-hc-basic-suite-master-engine: unreachable= with SSH -------------------- >> begin captured logging << -------------------- ovirtlago.testlib: ERROR: * Unhandled exception in <function _host_is_u= p at 0x2b9bde8> Traceback (most recent call last): File "/usr/lib/python2.7/site-packages/ovirtlago/testlib.py", line 217, i= n assert_equals_within res =3D func() File "/home/jenkins/workspace/ovirt_master_hc-system-tests/ovirt-system-t= ests/hc-basic-suite-master/test-scenarios/002_bootstrap.py", line 145, in _= host_is_up cur_state =3D api.hosts.get(host.name()).status.state File "/usr/lib/python2.7/site-packages/ovirtsdk/infrastructure/brokers.py= ", line 18338, in get headers=3D{"All-Content":all_content} File "/usr/lib/python2.7/site-packages/ovirtsdk/infrastructure/proxy.py",= line 46, in get return self.request(method=3D'GET', url=3Durl, headers=3Dheaders, cls= =3Dcls) File "/usr/lib/python2.7/site-packages/ovirtsdk/infrastructure/proxy.py",= line 122, in request persistent_auth=3Dself.__persistent_auth File "/usr/lib/python2.7/site-packages/ovirtsdk/infrastructure/connection= spool.py", line 81, in do_request raise errors.ConnectionError(error) ConnectionError: [ERROR]::oVirt API connection failure, (7, 'TCP connection= reset by peer') lago.utils: DEBUG: Looking for a workdir lago.workdir: DEBUG: Checking if /home/jenkins/workspace/ovirt_master_hc-sy= stem-tests/ovirt-system-tests/deployment-hc-basic-suite-master is a workdir lago.prefix: INFO: * Collect artifacts: ?[0m?[0m lago.prefix: INFO: ?[0m?[0m lago.prefix: INFO: ?[0m?[0m lago.prefix: INFO: ?[0m?[0m lago.prefix: INFO: ?[0m?[0m lago.ssh: DEBUG: start task:f8bad1fc-60b9-49d1-9f31-672a3ee74270:Get ssh cl= ient for lago-hc-basic-suite-master-engine: lago.ssh: DEBUG: Still got 1 tries for lago-hc-basic-suite-master-engine lago.ssh: DEBUG: start task:e8d204a8-bfb1-4db6-a7ad-bbbd637db48a:Get ssh cl= ient for lago-hc-basic-suite-master-host1: lago.ssh: DEBUG: start task:be57934b-b829-4e74-b5df-82470742f701:Get ssh cl= ient for lago-hc-basic-suite-master-host0: lago.ssh: DEBUG: Still got 1 tries for lago-hc-basic-suite-master-host1 lago.ssh: DEBUG: Still got 1 tries for lago-hc-basic-suite-master-host0 lago.ssh: DEBUG: Socket error connecting to lago-hc-basic-suite-master-engi= ne: [Errno None] Unable to connect to port 22 on 192.168.201.99 lago.ssh: DEBUG: start task:c20b9072-ebcb-4c74-9704-a192f15cf282:Get ssh cl= ient for lago-hc-basic-suite-master-host2: lago.ssh: DEBUG: Still got 1 tries for lago-hc-basic-suite-master-host2 lago.ssh: DEBUG: end task:c20b9072-ebcb-4c74-9704-a192f15cf282:Get ssh clie= nt for lago-hc-basic-suite-master-host2: lago.ssh: DEBUG: end task:e8d204a8-bfb1-4db6-a7ad-bbbd637db48a:Get ssh clie= nt for lago-hc-basic-suite-master-host1: lago.ssh: DEBUG: end task:be57934b-b829-4e74-b5df-82470742f701:Get ssh clie= nt for lago-hc-basic-suite-master-host0: lago.plugins.vm: DEBUG: Extracting scp://lago-hc-basic-suite-master-host2:/= var/log to /home/jenkins/workspace/ovirt_master_hc-system-tests/ovirt-syste= m-tests/deployment-hc-basic-suite-master/default/test_logs/002_bootstrap.ad= d_hosts-20170412034659/lago-hc-basic-suite-master-host2/_var_log lago.plugins.vm: DEBUG: Extracting scp://lago-hc-basic-suite-master-host1:/= var/log to /home/jenkins/workspace/ovirt_master_hc-system-tests/ovirt-syste= m-tests/deployment-hc-basic-suite-master/default/test_logs/002_bootstrap.ad= d_hosts-20170412034659/lago-hc-basic-suite-master-host1/_var_log lago.ssh: DEBUG: start task:485f1b5b-19e3-4a5d-b6f5-ce410dfa93e7:Get ssh cl= ient for lago-hc-basic-suite-master-host2: lago.ssh: DEBUG: start task:7580203c-3c7f-463d-a9bb-d2e8240016ab:Get ssh cl= ient for lago-hc-basic-suite-master-host1: lago.ssh: DEBUG: Still got 100 tries for lago-hc-basic-suite-master-host2 lago.ssh: DEBUG: Still got 100 tries for lago-hc-basic-suite-master-host1 lago.plugins.vm: DEBUG: Extracting scp://lago-hc-basic-suite-master-host0:/= var/log to /home/jenkins/workspace/ovirt_master_hc-system-tests/ovirt-syste= m-tests/deployment-hc-basic-suite-master/default/test_logs/002_bootstrap.ad= d_hosts-20170412034659/lago-hc-basic-suite-master-host0/_var_log lago.ssh: DEBUG: start task:9d4f24eb-92ac-45f1-8d3b-e1474f8a77ed:Get ssh cl= ient for lago-hc-basic-suite-master-host0: lago.ssh: DEBUG: Still got 100 tries for lago-hc-basic-suite-master-host0 lago.ssh: DEBUG: end task:7580203c-3c7f-463d-a9bb-d2e8240016ab:Get ssh clie= nt for lago-hc-basic-suite-master-host1: lago.ssh: DEBUG: end task:485f1b5b-19e3-4a5d-b6f5-ce410dfa93e7:Get ssh clie= nt for lago-hc-basic-suite-master-host2: lago.ssh: DEBUG: end task:9d4f24eb-92ac-45f1-8d3b-e1474f8a77ed:Get ssh clie= nt for lago-hc-basic-suite-master-host0: lago.ssh: DEBUG: end task:f8bad1fc-60b9-49d1-9f31-672a3ee74270:Get ssh clie= nt for lago-hc-basic-suite-master-engine: root: ERROR: - [Thread-6] lago-hc-basic-suite-master-engine: ?[31mERR= OR?[0m (in 0:00:01) lago.utils: ERROR: Error while running thread Traceback (most recent call last): File "/usr/lib/python2.7/site-packages/lago/utils.py", line 57, in _ret_v= ia_queue queue.put({'return': func()}) File "/usr/lib/python2.7/site-packages/lago/prefix.py", line 1205, in _co= llect_artifacts vm.collect_artifacts(path, ignore_nopath) File "/usr/lib/python2.7/site-packages/lago/plugins/vm.py", line 562, in = collect_artifacts ignore_nopath=3Dignore_nopath File "/usr/lib/python2.7/site-packages/lago/plugins/vm.py", line 359, in = extract_paths return self.provider.extract_paths(paths, *args, **kwargs) File "/usr/lib/python2.7/site-packages/lago/plugins/vm.py", line 234, in = extract_paths format(self.vm.name()) ExtractPathError: Unable to extract paths from lago-hc-basic-suite-master-e= ngine: unreachable with SSH lago.prefix: INFO: ?[32mSuccess?[0m (in 0:00:02) lago.prefix: INFO: ?[32mSuccess?[0m (in 0:00:02) lago.prefix: INFO: ?[32mSuccess?[0m (in 0:00:04) root: ERROR: * Collect artifacts: ?[31mERROR?[0m (in 0:00:04) --------------------- >> end captured logging << --------------------- ------=_Part_18_795736980.1491968853625--