- Infra - oVirt List Archives

[oVirt Jenkins] ovirt-system-tests_basic-suite-master_nightly - Build # 233 - Failure!
by jenkins＠jenkins.phx.ovirt.org 30 Jul '20

30 Jul '20

Project: https://jenkins.ovirt.org/job/ovirt-system-tests_basic-suite-master_nightly/ Build: https://jenkins.ovirt.org/job/ovirt-system-tests_basic-suite-master_nightly… Build Number: 233 Build Status: Failure Triggered By: Started by timer ------------------------------------- Changes Since Last Success: ------------------------------------- Changes for Build #233 [Michal Skrivanek] verify hotplug memory and test unplug [Tristan Cacqueray] zuul-jobs-gate: repparent to base-gate-job [Ehud Yonasi] stdci_runner: added the rhel labels selection. ----------------- Failed Tests: ----------------- 1 tests failed. FAILED: basic-suite-master.test-scenarios.002_bootstrap_pytest.test_verify_add_hosts Error Message: RuntimeError: 1 hosts failed installation: lago-basic-suite-master-host-1: install_failed Stack Trace: engine_api = <ovirtsdk4.Connection object at 0x7f28704d2310> @order_by(_TEST_LIST) def test_verify_add_hosts(engine_api): hosts_service = engine_api.system_service().hosts_service() hosts_status = hosts_service.list(search='datacenter={}'.format(DC_NAME)) total_hosts = len(hosts_status) dump_hosts = _host_status_to_print(hosts_service, hosts_status) LOGGER.debug('Host status, verify_add_hosts:\n {}'.format(dump_hosts)) testlib.assert_true_within( lambda: _single_host_up(hosts_service, total_hosts), > timeout=constants.ADD_HOST_TIMEOUT ) ../basic-suite-master/test-scenarios/002_bootstrap_pytest.py:434: _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ /usr/lib/python2.7/site-packages/ovirtlago/testlib.py:278: in assert_true_within assert_equals_within(func, True, timeout, allowed_exceptions) /usr/lib/python2.7/site-packages/ovirtlago/testlib.py:234: in assert_equals_within res = func() ../basic-suite-master/test-scenarios/002_bootstrap_pytest.py:433: in <lambda> lambda: _single_host_up(hosts_service, total_hosts), ../basic-suite-master/test-scenarios/002_bootstrap_pytest.py:230: in _single_host_up _check_problematic_hosts(hosts_service) _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ hosts_service = <ovirtsdk4.services.HostsService object at 0x7f286c6994d0> def _check_problematic_hosts(hosts_service): problematic_hosts = hosts_service.list(search='datacenter={} AND status != installing and status != initializing and status != up'.format(DC_NAME)) if len(problematic_hosts): dump_hosts = '%s hosts failed installation:\n' % len(problematic_hosts) for host in problematic_hosts: host_service = hosts_service.host_service(host.id) dump_hosts += '%s: %s\n' % (host.name, host_service.get().status) > raise RuntimeError(dump_hosts) E RuntimeError: 1 hosts failed installation: E lago-basic-suite-master-host-1: install_failed ../basic-suite-master/test-scenarios/002_bootstrap_pytest.py:239: RuntimeError

1 1

oVirt infra daily report - unstable production jobs - 1267
by jenkins＠jenkins.phx.ovirt.org 30 Jul '20

30 Jul '20

Good morning! Attached is the HTML page with the jenkins status report. You can see it also here: - https://jenkins.ovirt.org/job/system_jenkins-report/1267//artifact/exported… Cheers, Jenkins

1 0

[JIRA] (OVIRT-2991) [FIRING:1] InstanceUnreachable ibm-srv05.ovirt.org kubernetes-nodes-exporter (amd64 linux ibm-srv05.ovirt.org true external bare-metal-external ci)
by Alertmanager_Bot (oVirt JIRA) 29 Jul '20

29 Jul '20

Alertmanager_Bot created OVIRT-2991: --------------------------------------- Summary: [FIRING:1] InstanceUnreachable ibm-srv05.ovirt.org kubernetes-nodes-exporter (amd64 linux ibm-srv05.ovirt.org true external bare-metal-external ci) Key: OVIRT-2991 URL: https://ovirt-jira.atlassian.net/browse/OVIRT-2991 Project: oVirt - virtualization made easy Issue Type: Bug Reporter: Alertmanager_Bot Assignee: infra Labels: - alertname = InstanceUnreachable - beta_kubernetes_io_arch = amd64 - beta_kubernetes_io_os = linux - instance = ibm-srv05.ovirt.org - job = kubernetes-nodes-exporter - kubernetes_io_hostname = ibm-srv05.ovirt.org - node_role_kubernetes_io_compute = true - region = external - type = bare-metal-external - zone = ci Annotations: - description = ibm-srv05.ovirt.org of job kubernetes-nodes-exporter has been possibly down for more than 10 minutes. Source: http://prometheus-0:9090/graph?g0.expr=up%7Bjob%3D%22kubernetes-nodes-expor… -- This message was sent by Atlassian Jira (v1001.0.0-SNAPSHOT#100140)

1 0

[JIRA] (OVIRT-2988) [FIRING:1] InstanceUnreachable ibm-srv02.ovirt.org kubernetes-nodes-exporter (amd64 linux ibm-srv02.ovirt.org true external bare-metal-external ci)
by Alertmanager_Bot (oVirt JIRA) 29 Jul '20

29 Jul '20

Alertmanager_Bot created OVIRT-2988: --------------------------------------- Summary: [FIRING:1] InstanceUnreachable ibm-srv02.ovirt.org kubernetes-nodes-exporter (amd64 linux ibm-srv02.ovirt.org true external bare-metal-external ci) Key: OVIRT-2988 URL: https://ovirt-jira.atlassian.net/browse/OVIRT-2988 Project: oVirt - virtualization made easy Issue Type: Bug Reporter: Alertmanager_Bot Assignee: infra Labels: - alertname = InstanceUnreachable - beta_kubernetes_io_arch = amd64 - beta_kubernetes_io_os = linux - instance = ibm-srv02.ovirt.org - job = kubernetes-nodes-exporter - kubernetes_io_hostname = ibm-srv02.ovirt.org - node_role_kubernetes_io_compute = true - region = external - type = bare-metal-external - zone = ci Annotations: - description = ibm-srv02.ovirt.org of job kubernetes-nodes-exporter has been possibly down for more than 10 minutes. Source: http://prometheus-0:9090/graph?g0.expr=up%7Bjob%3D%22kubernetes-nodes-expor… -- This message was sent by Atlassian Jira (v1001.0.0-SNAPSHOT#100140)

1 0

[JIRA] (OVIRT-2987) [FIRING:1] InstanceUnreachable ibm-srv06.ovirt.org kubernetes-nodes-exporter (amd64 linux ibm-srv06.ovirt.org true external bare-metal-external ci)
by Alertmanager_Bot (oVirt JIRA) 29 Jul '20

29 Jul '20

Alertmanager_Bot created OVIRT-2987: --------------------------------------- Summary: [FIRING:1] InstanceUnreachable ibm-srv06.ovirt.org kubernetes-nodes-exporter (amd64 linux ibm-srv06.ovirt.org true external bare-metal-external ci) Key: OVIRT-2987 URL: https://ovirt-jira.atlassian.net/browse/OVIRT-2987 Project: oVirt - virtualization made easy Issue Type: Bug Reporter: Alertmanager_Bot Assignee: infra Labels: - alertname = InstanceUnreachable - beta_kubernetes_io_arch = amd64 - beta_kubernetes_io_os = linux - instance = ibm-srv06.ovirt.org - job = kubernetes-nodes-exporter - kubernetes_io_hostname = ibm-srv06.ovirt.org - node_role_kubernetes_io_compute = true - region = external - type = bare-metal-external - zone = ci Annotations: - description = ibm-srv06.ovirt.org of job kubernetes-nodes-exporter has been possibly down for more than 10 minutes. Source: http://prometheus-0:9090/graph?g0.expr=up%7Bjob%3D%22kubernetes-nodes-expor… -- This message was sent by Atlassian Jira (v1001.0.0-SNAPSHOT#100140)

1 0

[JIRA] (OVIRT-2989) [FIRING:1] InstanceUnreachable ibm-srv03.ovirt.org kubernetes-nodes-exporter (amd64 linux ibm-srv03.ovirt.org true external bare-metal-external ci)
by Alertmanager_Bot (oVirt JIRA) 29 Jul '20

29 Jul '20

Alertmanager_Bot created OVIRT-2989: --------------------------------------- Summary: [FIRING:1] InstanceUnreachable ibm-srv03.ovirt.org kubernetes-nodes-exporter (amd64 linux ibm-srv03.ovirt.org true external bare-metal-external ci) Key: OVIRT-2989 URL: https://ovirt-jira.atlassian.net/browse/OVIRT-2989 Project: oVirt - virtualization made easy Issue Type: Bug Reporter: Alertmanager_Bot Assignee: infra Labels: - alertname = InstanceUnreachable - beta_kubernetes_io_arch = amd64 - beta_kubernetes_io_os = linux - instance = ibm-srv03.ovirt.org - job = kubernetes-nodes-exporter - kubernetes_io_hostname = ibm-srv03.ovirt.org - node_role_kubernetes_io_compute = true - region = external - type = bare-metal-external - zone = ci Annotations: - description = ibm-srv03.ovirt.org of job kubernetes-nodes-exporter has been possibly down for more than 10 minutes. Source: http://prometheus-0:9090/graph?g0.expr=up%7Bjob%3D%22kubernetes-nodes-expor… -- This message was sent by Atlassian Jira (v1001.0.0-SNAPSHOT#100140)

1 0

[JIRA] (OVIRT-2990) [FIRING:1] InstanceUnreachable ibm-srv04.ovirt.org kubernetes-nodes-exporter (amd64 linux ibm-srv04.ovirt.org true external bare-metal-external ci)
by Alertmanager_Bot (oVirt JIRA) 29 Jul '20

29 Jul '20

Alertmanager_Bot created OVIRT-2990: --------------------------------------- Summary: [FIRING:1] InstanceUnreachable ibm-srv04.ovirt.org kubernetes-nodes-exporter (amd64 linux ibm-srv04.ovirt.org true external bare-metal-external ci) Key: OVIRT-2990 URL: https://ovirt-jira.atlassian.net/browse/OVIRT-2990 Project: oVirt - virtualization made easy Issue Type: Bug Reporter: Alertmanager_Bot Assignee: infra Labels: - alertname = InstanceUnreachable - beta_kubernetes_io_arch = amd64 - beta_kubernetes_io_os = linux - instance = ibm-srv04.ovirt.org - job = kubernetes-nodes-exporter - kubernetes_io_hostname = ibm-srv04.ovirt.org - node_role_kubernetes_io_compute = true - region = external - type = bare-metal-external - zone = ci Annotations: - description = ibm-srv04.ovirt.org of job kubernetes-nodes-exporter has been possibly down for more than 10 minutes. Source: http://prometheus-0:9090/graph?g0.expr=up%7Bjob%3D%22kubernetes-nodes-expor… -- This message was sent by Atlassian Jira (v1001.0.0-SNAPSHOT#100140)

1 0

[JIRA] (OVIRT-2986) [FIRING:1] InstanceUnreachable ibm-srv01.ovirt.org kubernetes-nodes-exporter (amd64 linux ibm-srv01.ovirt.org true external bare-metal-external ci)
by Alertmanager_Bot (oVirt JIRA) 29 Jul '20

29 Jul '20

Alertmanager_Bot created OVIRT-2986: --------------------------------------- Summary: [FIRING:1] InstanceUnreachable ibm-srv01.ovirt.org kubernetes-nodes-exporter (amd64 linux ibm-srv01.ovirt.org true external bare-metal-external ci) Key: OVIRT-2986 URL: https://ovirt-jira.atlassian.net/browse/OVIRT-2986 Project: oVirt - virtualization made easy Issue Type: Bug Reporter: Alertmanager_Bot Assignee: infra Labels: - alertname = InstanceUnreachable - beta_kubernetes_io_arch = amd64 - beta_kubernetes_io_os = linux - instance = ibm-srv01.ovirt.org - job = kubernetes-nodes-exporter - kubernetes_io_hostname = ibm-srv01.ovirt.org - node_role_kubernetes_io_compute = true - region = external - type = bare-metal-external - zone = ci Annotations: - description = ibm-srv01.ovirt.org of job kubernetes-nodes-exporter has been possibly down for more than 10 minutes. Source: http://prometheus-0:9090/graph?g0.expr=up%7Bjob%3D%22kubernetes-nodes-expor… -- This message was sent by Atlassian Jira (v1001.0.0-SNAPSHOT#100140)

1 0

[oVirt Jenkins] ovirt-system-tests_basic-suite-4.3_nightly - Build # 231 - Failure!
by jenkins＠jenkins.phx.ovirt.org 29 Jul '20

29 Jul '20

Project: https://jenkins.ovirt.org/job/ovirt-system-tests_basic-suite-4.3_nightly/ Build: https://jenkins.ovirt.org/job/ovirt-system-tests_basic-suite-4.3_nightly/23… Build Number: 231 Build Status: Failure Triggered By: Started by timer ------------------------------------- Changes Since Last Success: ------------------------------------- Changes for Build #231 [Nir Soffer] setup: Do not hide real issue by reserving ports ----------------- Failed Tests: ----------------- 1 tests failed. FAILED: 008_basic_ui_sanity.start_grid Error Message: could not get ip address of selenium hub. See previous messages for probable docker failure -------------------- >> begin captured stdout << --------------------- executing shell: docker ps CONTAINER ID IMAGE COMMAND CREATED STATUS PORTS NAMES executing shell: docker rm -f grid_node_chrome grid_node_firefox selenium-hub Error response from daemon: No such container: grid_node_chrome Error response from daemon: No such container: grid_node_firefox Error response from daemon: No such container: selenium-hub executing shell: docker kill grid_node_chrome grid_node_firefox selenium-hub Error response from daemon: Cannot kill container: grid_node_chrome: No such container: grid_node_chrome Error response from daemon: Cannot kill container: grid_node_firefox: No such container: grid_node_firefox Error response from daemon: Cannot kill container: selenium-hub: No such container: selenium-hub executing shell: docker ps CONTAINER ID IMAGE COMMAND CREATED STATUS PORTS NAMES executing shell: docker network ls NETWORK ID NAME DRIVER SCOPE 058a426a59b4 bridge bridge local bd7765e6670f host host local 245daf9e104c none null local executing shell: docker network rm grid Error response from daemon: network grid not found executing shell: docker network ls NETWORK ID NAME DRIVER SCOPE 058a426a59b4 bridge bridge local bd7765e6670f host host local 245daf9e104c none null local executing shell: docker network ls NETWORK ID NAME DRIVER SCOPE 058a426a59b4 bridge bridge local bd7765e6670f host host local 245daf9e104c none null local creating docker network for grid executing shell: docker network create grid 328da4cfdd441c835906d9674c744acf5ce32542b90053beed005cb2051505e1 executing shell: docker network ls NETWORK ID NAME DRIVER SCOPE 058a426a59b4 bridge bridge local 328da4cfdd44 grid bridge local bd7765e6670f host host local 245daf9e104c none null local starting hub executing shell: docker run -d -p 4444:4444 --net grid --name selenium-hub selenium/hub:3.9.1-actinium Unable to find image 'selenium/hub:3.9.1-actinium' locally 3.9.1-actinium: Pulling from selenium/hub 1be7f2b886e8: Pulling fs layer 6fbc4a21b806: Pulling fs layer c71a6f8e1378: Pulling fs layer 4be3072e5a37: Pulling fs layer 06c6d2f59700: Pulling fs layer edcd5e9f2f91: Pulling fs layer 0eeaf787f757: Pulling fs layer c949dee5af7e: Pulling fs layer df88a49b4162: Pulling fs layer ce3c6f42fd24: Pulling fs layer 6c8e191c3abf: Pulling fs layer c9aca8b50247: Pulling fs layer 0eeaf787f757: Waiting 4be3072e5a37: Waiting 06c6d2f59700: Waiting c949dee5af7e: Waiting c9aca8b50247: Waiting ce3c6f42fd24: Waiting 6c8e191c3abf: Waiting df88a49b4162: Waiting edcd5e9f2f91: Waiting /usr/bin/docker-current: error pulling image configuration: Get https://production.cloudflare.docker.com/registry-v2/docker/registry/v2/blo…: dial tcp: lookup production.cloudflare.docker.com on 172.19.11.6:53: read udp 10.129.5.89:52169->172.19.11.6:53: i/o timeout. See '/usr/bin/docker-current run --help'. getting ip of hub --------------------- >> end captured stdout << ---------------------- Stack Trace: File "/usr/lib64/python2.7/unittest/case.py", line 369, in run testMethod() File "/usr/lib/python2.7/site-packages/nose/case.py", line 197, in runTest self.test(*self.arg) File "/usr/lib/python2.7/site-packages/ovirtlago/testlib.py", line 142, in wrapped_test test() File "/home/jenkins/agent/workspace/ovirt-system-tests_basic-suite-4.3_nightly/ovirt-system-tests/basic-suite-4.3/test-scenarios/008_basic_ui_sanity.py", line 189, in start_grid raise RuntimeError("could not get ip address of selenium hub. See previous messages for probable docker failure") could not get ip address of selenium hub. See previous messages for probable docker failure -------------------- >> begin captured stdout << --------------------- executing shell: docker ps CONTAINER ID IMAGE COMMAND CREATED STATUS PORTS NAMES executing shell: docker rm -f grid_node_chrome grid_node_firefox selenium-hub Error response from daemon: No such container: grid_node_chrome Error response from daemon: No such container: grid_node_firefox Error response from daemon: No such container: selenium-hub executing shell: docker kill grid_node_chrome grid_node_firefox selenium-hub Error response from daemon: Cannot kill container: grid_node_chrome: No such container: grid_node_chrome Error response from daemon: Cannot kill container: grid_node_firefox: No such container: grid_node_firefox Error response from daemon: Cannot kill container: selenium-hub: No such container: selenium-hub executing shell: docker ps CONTAINER ID IMAGE COMMAND CREATED STATUS PORTS NAMES executing shell: docker network ls NETWORK ID NAME DRIVER SCOPE 058a426a59b4 bridge bridge local bd7765e6670f host host local 245daf9e104c none null local executing shell: docker network rm grid Error response from daemon: network grid not found executing shell: docker network ls NETWORK ID NAME DRIVER SCOPE 058a426a59b4 bridge bridge local bd7765e6670f host host local 245daf9e104c none null local executing shell: docker network ls NETWORK ID NAME DRIVER SCOPE 058a426a59b4 bridge bridge local bd7765e6670f host host local 245daf9e104c none null local creating docker network for grid executing shell: docker network create grid 328da4cfdd441c835906d9674c744acf5ce32542b90053beed005cb2051505e1 executing shell: docker network ls NETWORK ID NAME DRIVER SCOPE 058a426a59b4 bridge bridge local 328da4cfdd44 grid bridge local bd7765e6670f host host local 245daf9e104c none null local starting hub executing shell: docker run -d -p 4444:4444 --net grid --name selenium-hub selenium/hub:3.9.1-actinium Unable to find image 'selenium/hub:3.9.1-actinium' locally 3.9.1-actinium: Pulling from selenium/hub 1be7f2b886e8: Pulling fs layer 6fbc4a21b806: Pulling fs layer c71a6f8e1378: Pulling fs layer 4be3072e5a37: Pulling fs layer 06c6d2f59700: Pulling fs layer edcd5e9f2f91: Pulling fs layer 0eeaf787f757: Pulling fs layer c949dee5af7e: Pulling fs layer df88a49b4162: Pulling fs layer ce3c6f42fd24: Pulling fs layer 6c8e191c3abf: Pulling fs layer c9aca8b50247: Pulling fs layer 0eeaf787f757: Waiting 4be3072e5a37: Waiting 06c6d2f59700: Waiting c949dee5af7e: Waiting c9aca8b50247: Waiting ce3c6f42fd24: Waiting 6c8e191c3abf: Waiting df88a49b4162: Waiting edcd5e9f2f91: Waiting /usr/bin/docker-current: error pulling image configuration: Get https://production.cloudflare.docker.com/registry-v2/docker/registry/v2/blo…: dial tcp: lookup production.cloudflare.docker.com on 172.19.11.6:53: read udp 10.129.5.89:52169->172.19.11.6:53: i/o timeout. See '/usr/bin/docker-current run --help'. getting ip of hub --------------------- >> end captured stdout << ----------------------

1 1

oVirt infra daily report - unstable production jobs - 1266
by jenkins＠jenkins.phx.ovirt.org 29 Jul '20

29 Jul '20

Good morning! Attached is the HTML page with the jenkins status report. You can see it also here: - https://jenkins.ovirt.org/job/system_jenkins-report/1266//artifact/exported… Cheers, Jenkins

1 0