- Infra - oVirt List Archives

[oVirt Jenkins] ovirt-system-tests_basic-suite-4.3_nightly - Build # 287 - Failure!
by jenkins＠jenkins.phx.ovirt.org 17 Aug '20

17 Aug '20

Project: https://jenkins.ovirt.org/job/ovirt-system-tests_basic-suite-4.3_nightly/ Build: https://jenkins.ovirt.org/job/ovirt-system-tests_basic-suite-4.3_nightly/28… Build Number: 287 Build Status: Failure Triggered By: Started by timer ------------------------------------- Changes Since Last Success: ------------------------------------- Changes for Build #287 [Michal Skrivanek] avoid ballooning crash in vm0 after memory hotplug ----------------- Failed Tests: ----------------- 1 tests failed. FAILED: 005_network_by_label.assign_labeled_network Error Message: False != True after 180 seconds Stack Trace: Traceback (most recent call last): File "/usr/lib64/python2.7/unittest/case.py", line 369, in run testMethod() File "/usr/lib/python2.7/site-packages/nose/case.py", line 197, in runTest self.test(*self.arg) File "/usr/lib/python2.7/site-packages/ovirtlago/testlib.py", line 142, in wrapped_test test() File "/usr/lib/python2.7/site-packages/ovirtlago/testlib.py", line 60, in wrapper return func(get_test_prefix(), *args, **kwargs) File "/usr/lib/python2.7/site-packages/ovirtlago/testlib.py", line 79, in wrapper prefix.virt_env.engine_vm().get_api(api_ver=4), *args, **kwargs File "/home/jenkins/agent/workspace/ovirt-system-tests_basic-suite-4.3_nightly/ovirt-system-tests/basic-suite-4.3/test-scenarios/005_network_by_label.py", line 149, in assign_labeled_network host_service, LABELED_NET_NAME)) File "/usr/lib/python2.7/site-packages/ovirtlago/testlib.py", line 282, in assert_true_within_short assert_equals_within_short(func, True, allowed_exceptions) File "/usr/lib/python2.7/site-packages/ovirtlago/testlib.py", line 266, in assert_equals_within_short func, value, SHORT_TIMEOUT, allowed_exceptions=allowed_exceptions File "/usr/lib/python2.7/site-packages/ovirtlago/testlib.py", line 252, in assert_equals_within '%s != %s after %s seconds' % (res, value, timeout) AssertionError: False != True after 180 seconds

1 1

oVirt infra daily report - unstable production jobs - 1285
by jenkins＠jenkins.phx.ovirt.org 17 Aug '20

17 Aug '20

Good morning! Attached is the HTML page with the jenkins status report. You can see it also here: - https://jenkins.ovirt.org/job/system_jenkins-report/1285//artifact/exported… Cheers, Jenkins

1 0

[oVirt Jenkins] cleanup-gated-ovirt-master - Build #245 - FAILURE!
by jenkins＠jenkins.phx.ovirt.org 16 Aug '20

16 Aug '20

Build: https://jenkins.ovirt.org/job/cleanup-gated-ovirt-master/245/ Build Name: #245 Build Status: FAILURE Gerrit change: null - title: null - project: null - branch: null - author: null <null>

1 0

oVirt infra daily report - unstable production jobs - 1284
by jenkins＠jenkins.phx.ovirt.org 16 Aug '20

16 Aug '20

Good morning! Attached is the HTML page with the jenkins status report. You can see it also here: - https://jenkins.ovirt.org/job/system_jenkins-report/1284//artifact/exported… Cheers, Jenkins

1 0

oVirt infra daily report - unstable production jobs - 1283
by jenkins＠jenkins.phx.ovirt.org 15 Aug '20

15 Aug '20

Good morning! Attached is the HTML page with the jenkins status report. You can see it also here: - https://jenkins.ovirt.org/job/system_jenkins-report/1283//artifact/exported… Cheers, Jenkins

1 0

[oVirt Jenkins] ovirt-system-tests_basic-suite-4.3_nightly - Build # 279 - Failure!
by jenkins＠jenkins.phx.ovirt.org 14 Aug '20

14 Aug '20

Project: https://jenkins.ovirt.org/job/ovirt-system-tests_basic-suite-4.3_nightly/ Build: https://jenkins.ovirt.org/job/ovirt-system-tests_basic-suite-4.3_nightly/27… Build Number: 279 Build Status: Failure Triggered By: Started by timer ------------------------------------- Changes Since Last Success: ------------------------------------- Changes for Build #279 [Michal Skrivanek] avoid ballooning crash in vm0 after memory hotplug ----------------- Failed Tests: ----------------- 1 tests failed. FAILED: 008_basic_ui_sanity.start_grid Error Message: could not get ip address of selenium hub. See previous messages for probable docker failure -------------------- >> begin captured stdout << --------------------- executing shell: docker ps CONTAINER ID IMAGE COMMAND CREATED STATUS PORTS NAMES executing shell: docker rm -f grid_node_chrome grid_node_firefox selenium-hub Error response from daemon: No such container: grid_node_chrome Error response from daemon: No such container: grid_node_firefox Error response from daemon: No such container: selenium-hub executing shell: docker kill grid_node_chrome grid_node_firefox selenium-hub Error response from daemon: Cannot kill container: grid_node_chrome: No such container: grid_node_chrome Error response from daemon: Cannot kill container: grid_node_firefox: No such container: grid_node_firefox Error response from daemon: Cannot kill container: selenium-hub: No such container: selenium-hub executing shell: docker ps CONTAINER ID IMAGE COMMAND CREATED STATUS PORTS NAMES executing shell: docker network ls NETWORK ID NAME DRIVER SCOPE 336b430c5db4 bridge bridge local bd7765e6670f host host local 245daf9e104c none null local executing shell: docker network rm grid Error response from daemon: network grid not found executing shell: docker network ls NETWORK ID NAME DRIVER SCOPE 336b430c5db4 bridge bridge local bd7765e6670f host host local 245daf9e104c none null local executing shell: docker network ls NETWORK ID NAME DRIVER SCOPE 336b430c5db4 bridge bridge local bd7765e6670f host host local 245daf9e104c none null local creating docker network for grid executing shell: docker network create grid 4aa37e0f55a57e7fa995d5786c78e60116d2ea099a390774afe8defbab8f4c23 executing shell: docker network ls NETWORK ID NAME DRIVER SCOPE 336b430c5db4 bridge bridge local 4aa37e0f55a5 grid bridge local bd7765e6670f host host local 245daf9e104c none null local starting hub executing shell: docker run -d -p 4444:4444 --net grid --name selenium-hub selenium/hub:3.9.1-actinium Unable to find image 'selenium/hub:3.9.1-actinium' locally 3.9.1-actinium: Pulling from selenium/hub 1be7f2b886e8: Pulling fs layer 6fbc4a21b806: Pulling fs layer c71a6f8e1378: Pulling fs layer 4be3072e5a37: Pulling fs layer 06c6d2f59700: Pulling fs layer edcd5e9f2f91: Pulling fs layer 4be3072e5a37: Waiting 0eeaf787f757: Pulling fs layer 06c6d2f59700: Waiting c949dee5af7e: Pulling fs layer df88a49b4162: Pulling fs layer edcd5e9f2f91: Waiting ce3c6f42fd24: Pulling fs layer 6c8e191c3abf: Pulling fs layer df88a49b4162: Waiting ce3c6f42fd24: Waiting c9aca8b50247: Pulling fs layer c9aca8b50247: Waiting 6c8e191c3abf: Waiting c949dee5af7e: Waiting 0eeaf787f757: Waiting /usr/bin/docker-current: error pulling image configuration: Get https://production.cloudflare.docker.com/registry-v2/docker/registry/v2/blo…: dial tcp: lookup production.cloudflare.docker.com on 172.19.11.6:53: read udp 10.129.5.217:34274->172.19.11.6:53: i/o timeout. See '/usr/bin/docker-current run --help'. getting ip of hub --------------------- >> end captured stdout << ---------------------- Stack Trace: File "/usr/lib64/python2.7/unittest/case.py", line 369, in run testMethod() File "/usr/lib/python2.7/site-packages/nose/case.py", line 197, in runTest self.test(*self.arg) File "/usr/lib/python2.7/site-packages/ovirtlago/testlib.py", line 142, in wrapped_test test() File "/home/jenkins/agent/workspace/ovirt-system-tests_basic-suite-4.3_nightly/ovirt-system-tests/basic-suite-4.3/test-scenarios/008_basic_ui_sanity.py", line 189, in start_grid raise RuntimeError("could not get ip address of selenium hub. See previous messages for probable docker failure") could not get ip address of selenium hub. See previous messages for probable docker failure -------------------- >> begin captured stdout << --------------------- executing shell: docker ps CONTAINER ID IMAGE COMMAND CREATED STATUS PORTS NAMES executing shell: docker rm -f grid_node_chrome grid_node_firefox selenium-hub Error response from daemon: No such container: grid_node_chrome Error response from daemon: No such container: grid_node_firefox Error response from daemon: No such container: selenium-hub executing shell: docker kill grid_node_chrome grid_node_firefox selenium-hub Error response from daemon: Cannot kill container: grid_node_chrome: No such container: grid_node_chrome Error response from daemon: Cannot kill container: grid_node_firefox: No such container: grid_node_firefox Error response from daemon: Cannot kill container: selenium-hub: No such container: selenium-hub executing shell: docker ps CONTAINER ID IMAGE COMMAND CREATED STATUS PORTS NAMES executing shell: docker network ls NETWORK ID NAME DRIVER SCOPE 336b430c5db4 bridge bridge local bd7765e6670f host host local 245daf9e104c none null local executing shell: docker network rm grid Error response from daemon: network grid not found executing shell: docker network ls NETWORK ID NAME DRIVER SCOPE 336b430c5db4 bridge bridge local bd7765e6670f host host local 245daf9e104c none null local executing shell: docker network ls NETWORK ID NAME DRIVER SCOPE 336b430c5db4 bridge bridge local bd7765e6670f host host local 245daf9e104c none null local creating docker network for grid executing shell: docker network create grid 4aa37e0f55a57e7fa995d5786c78e60116d2ea099a390774afe8defbab8f4c23 executing shell: docker network ls NETWORK ID NAME DRIVER SCOPE 336b430c5db4 bridge bridge local 4aa37e0f55a5 grid bridge local bd7765e6670f host host local 245daf9e104c none null local starting hub executing shell: docker run -d -p 4444:4444 --net grid --name selenium-hub selenium/hub:3.9.1-actinium Unable to find image 'selenium/hub:3.9.1-actinium' locally 3.9.1-actinium: Pulling from selenium/hub 1be7f2b886e8: Pulling fs layer 6fbc4a21b806: Pulling fs layer c71a6f8e1378: Pulling fs layer 4be3072e5a37: Pulling fs layer 06c6d2f59700: Pulling fs layer edcd5e9f2f91: Pulling fs layer 4be3072e5a37: Waiting 0eeaf787f757: Pulling fs layer 06c6d2f59700: Waiting c949dee5af7e: Pulling fs layer df88a49b4162: Pulling fs layer edcd5e9f2f91: Waiting ce3c6f42fd24: Pulling fs layer 6c8e191c3abf: Pulling fs layer df88a49b4162: Waiting ce3c6f42fd24: Waiting c9aca8b50247: Pulling fs layer c9aca8b50247: Waiting 6c8e191c3abf: Waiting c949dee5af7e: Waiting 0eeaf787f757: Waiting /usr/bin/docker-current: error pulling image configuration: Get https://production.cloudflare.docker.com/registry-v2/docker/registry/v2/blo…: dial tcp: lookup production.cloudflare.docker.com on 172.19.11.6:53: read udp 10.129.5.217:34274->172.19.11.6:53: i/o timeout. See '/usr/bin/docker-current run --help'. getting ip of hub --------------------- >> end captured stdout << ----------------------

1 1

oVirt infra daily report - unstable production jobs - 1282
by jenkins＠jenkins.phx.ovirt.org 14 Aug '20

14 Aug '20

Good morning! Attached is the HTML page with the jenkins status report. You can see it also here: - https://jenkins.ovirt.org/job/system_jenkins-report/1282//artifact/exported… Cheers, Jenkins

1 0

[oVirt Jenkins] cleanup-gated-ovirt-4.3 - Build #237 - FAILURE!
by jenkins＠jenkins.phx.ovirt.org 13 Aug '20

13 Aug '20

Build: https://jenkins.ovirt.org/job/cleanup-gated-ovirt-4.3/237/ Build Name: #237 Build Status: FAILURE Gerrit change: null - title: null - project: null - branch: null - author: null <null>

1 0

[oVirt Jenkins] ovirt-system-tests_basic-suite-master_nightly - Build # 274 - Failure!
by jenkins＠jenkins.phx.ovirt.org 13 Aug '20

13 Aug '20

Project: https://jenkins.ovirt.org/job/ovirt-system-tests_basic-suite-master_nightly/ Build: https://jenkins.ovirt.org/job/ovirt-system-tests_basic-suite-master_nightly… Build Number: 274 Build Status: Failure Triggered By: Started by timer ------------------------------------- Changes Since Last Success: ------------------------------------- Changes for Build #274 [Michal Skrivanek] smaller OST VMs ----------------- Failed Tests: ----------------- 1 tests failed. FAILED: basic-suite-master.test-scenarios.004_basic_sanity_pytest.test_suspend_resume_vm0 Error Message: LagoSSHTimeoutException: Timed out (in 117 s) trying to ssh to 192.168.201.223 Stack Trace: prefix = <ovirtlago.prefix.OvirtPrefix object at 0x7feb5b5eac10> @order_by(_TEST_LIST) def test_suspend_resume_vm0(prefix): vm_host = _vm_host(prefix, VM0_NAME) ret = vm_host.ssh(['tail', '-1', VDSM_LOG]) assert ret.code == EX_OK log_items = ret.out.decode('utf-8').split() global _log_time_before_suspend _log_time_before_suspend = log_items[0] + ' ' + log_items[1] # date + time > assert_vm0_is_alive(prefix) ../basic-suite-master/test-scenarios/004_basic_sanity_pytest.py:1210: _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ ../basic-suite-master/test-scenarios/004_basic_sanity_pytest.py:191: in assert_vm0_is_alive assert_vm_is_alive(prefix, VM0_NAME) ../basic-suite-master/test-scenarios/004_basic_sanity_pytest.py:199: in assert_vm_is_alive assert _vm_ssh(prefix, vm_hostname, ['true']).code == EX_OK ../basic-suite-master/test-scenarios/004_basic_sanity_pytest.py:186: in _vm_ssh tries=tries, /usr/lib/python2.7/site-packages/lago/ssh.py:46: in ssh password=password, _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ ip_addr = '192.168.201.223', ssh_key = None, host_name = '192.168.201.223' ssh_tries = 0, propagate_fail = True, username = 'cirros', password = 'gocubsgo' def get_ssh_client( ip_addr, ssh_key=None, host_name=None, ssh_tries=None, propagate_fail=True, username='root', password='123456', ): """ Get a connected SSH client Args: ip_addr(str): IP address of the endpoint ssh_key(str or list of str): Path to a file which contains the private key hotname(str): The hostname of the endpoint ssh_tries(int): The number of attempts to connect to the endpoint propagate_fail(bool): If set to true, this event will be in the log and fail the outer stage. Otherwise, it will be discarded. username(str): The username to authenticate with password(str): Used for password authentication or for private key decryption Raises: :exc:`~LagoSSHTimeoutException`: If the client failed to connect after "ssh_tries" """ host_name = host_name or ip_addr with LogTask( 'Get ssh client for %s' % host_name, level='debug', propagate_fail=propagate_fail, ): ssh_timeout = int(config.get('ssh_timeout')) if ssh_tries is None: ssh_tries = int(config.get('ssh_tries', 10)) start_time = time.time() client = paramiko.SSHClient() client.set_missing_host_key_policy(paramiko.AutoAddPolicy(), ) while ssh_tries > 0: try: client.connect( ip_addr, username=username, password=password, key_filename=ssh_key, timeout=ssh_timeout, ) break except (socket.error, socket.timeout) as err: LOGGER.debug( 'Socket error connecting to %s: %s', host_name, err, ) except paramiko.ssh_exception.SSHException as err: LOGGER.debug( 'SSH error connecting to %s: %s', host_name, err, ) except EOFError as err: LOGGER.debug('EOFError connecting to %s: %s', host_name, err) ssh_tries -= 1 LOGGER.debug( 'Still got %d tries for %s', ssh_tries, host_name, ) time.sleep(1) else: end_time = time.time() raise LagoSSHTimeoutException( 'Timed out (in %d s) trying to ssh to %s' % > (end_time - start_time, host_name) ) E LagoSSHTimeoutException: Timed out (in 117 s) trying to ssh to 192.168.201.223 /usr/lib/python2.7/site-packages/lago/ssh.py:388: LagoSSHTimeoutException

1 2

oVirt infra daily report - unstable production jobs - 1281
by jenkins＠jenkins.phx.ovirt.org 13 Aug '20

13 Aug '20

Good morning! Attached is the HTML page with the jenkins status report. You can see it also here: - https://jenkins.ovirt.org/job/system_jenkins-report/1281//artifact/exported… Cheers, Jenkins

1 0