- Infra - oVirt List Archives

[oVirt Jenkins] ovirt-system-tests_basic-suite-master_nightly - Build # 233 - Failure!
by jenkins＠jenkins.phx.ovirt.org 30 Jul '20

30 Jul '20

Project: https://jenkins.ovirt.org/job/ovirt-system-tests_basic-suite-master_nightly/ Build: https://jenkins.ovirt.org/job/ovirt-system-tests_basic-suite-master_nightly… Build Number: 233 Build Status: Failure Triggered By: Started by timer ------------------------------------- Changes Since Last Success: ------------------------------------- Changes for Build #233 [Michal Skrivanek] verify hotplug memory and test unplug [Tristan Cacqueray] zuul-jobs-gate: repparent to base-gate-job [Ehud Yonasi] stdci_runner: added the rhel labels selection. ----------------- Failed Tests: ----------------- 1 tests failed. FAILED: basic-suite-master.test-scenarios.002_bootstrap_pytest.test_verify_add_hosts Error Message: RuntimeError: 1 hosts failed installation: lago-basic-suite-master-host-1: install_failed Stack Trace: engine_api = <ovirtsdk4.Connection object at 0x7f28704d2310> @order_by(_TEST_LIST) def test_verify_add_hosts(engine_api): hosts_service = engine_api.system_service().hosts_service() hosts_status = hosts_service.list(search='datacenter={}'.format(DC_NAME)) total_hosts = len(hosts_status) dump_hosts = _host_status_to_print(hosts_service, hosts_status) LOGGER.debug('Host status, verify_add_hosts:\n {}'.format(dump_hosts)) testlib.assert_true_within( lambda: _single_host_up(hosts_service, total_hosts), > timeout=constants.ADD_HOST_TIMEOUT ) ../basic-suite-master/test-scenarios/002_bootstrap_pytest.py:434: _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ /usr/lib/python2.7/site-packages/ovirtlago/testlib.py:278: in assert_true_within assert_equals_within(func, True, timeout, allowed_exceptions) /usr/lib/python2.7/site-packages/ovirtlago/testlib.py:234: in assert_equals_within res = func() ../basic-suite-master/test-scenarios/002_bootstrap_pytest.py:433: in <lambda> lambda: _single_host_up(hosts_service, total_hosts), ../basic-suite-master/test-scenarios/002_bootstrap_pytest.py:230: in _single_host_up _check_problematic_hosts(hosts_service) _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ hosts_service = <ovirtsdk4.services.HostsService object at 0x7f286c6994d0> def _check_problematic_hosts(hosts_service): problematic_hosts = hosts_service.list(search='datacenter={} AND status != installing and status != initializing and status != up'.format(DC_NAME)) if len(problematic_hosts): dump_hosts = '%s hosts failed installation:\n' % len(problematic_hosts) for host in problematic_hosts: host_service = hosts_service.host_service(host.id) dump_hosts += '%s: %s\n' % (host.name, host_service.get().status) > raise RuntimeError(dump_hosts) E RuntimeError: 1 hosts failed installation: E lago-basic-suite-master-host-1: install_failed ../basic-suite-master/test-scenarios/002_bootstrap_pytest.py:239: RuntimeError

1 1

oVirt infra daily report - unstable production jobs - 1267
by jenkins＠jenkins.phx.ovirt.org 30 Jul '20

30 Jul '20

Good morning! Attached is the HTML page with the jenkins status report. You can see it also here: - https://jenkins.ovirt.org/job/system_jenkins-report/1267//artifact/exported… Cheers, Jenkins

1 0

[JIRA] (OVIRT-2991) [FIRING:1] InstanceUnreachable ibm-srv05.ovirt.org kubernetes-nodes-exporter (amd64 linux ibm-srv05.ovirt.org true external bare-metal-external ci)
by Alertmanager_Bot (oVirt JIRA) 29 Jul '20

29 Jul '20

Alertmanager_Bot created OVIRT-2991: --------------------------------------- Summary: [FIRING:1] InstanceUnreachable ibm-srv05.ovirt.org kubernetes-nodes-exporter (amd64 linux ibm-srv05.ovirt.org true external bare-metal-external ci) Key: OVIRT-2991 URL: https://ovirt-jira.atlassian.net/browse/OVIRT-2991 Project: oVirt - virtualization made easy Issue Type: Bug Reporter: Alertmanager_Bot Assignee: infra Labels: - alertname = InstanceUnreachable - beta_kubernetes_io_arch = amd64 - beta_kubernetes_io_os = linux - instance = ibm-srv05.ovirt.org - job = kubernetes-nodes-exporter - kubernetes_io_hostname = ibm-srv05.ovirt.org - node_role_kubernetes_io_compute = true - region = external - type = bare-metal-external - zone = ci Annotations: - description = ibm-srv05.ovirt.org of job kubernetes-nodes-exporter has been possibly down for more than 10 minutes. Source: http://prometheus-0:9090/graph?g0.expr=up%7Bjob%3D%22kubernetes-nodes-expor… -- This message was sent by Atlassian Jira (v1001.0.0-SNAPSHOT#100140)

1 0

[JIRA] (OVIRT-2988) [FIRING:1] InstanceUnreachable ibm-srv02.ovirt.org kubernetes-nodes-exporter (amd64 linux ibm-srv02.ovirt.org true external bare-metal-external ci)
by Alertmanager_Bot (oVirt JIRA) 29 Jul '20

29 Jul '20

Alertmanager_Bot created OVIRT-2988: --------------------------------------- Summary: [FIRING:1] InstanceUnreachable ibm-srv02.ovirt.org kubernetes-nodes-exporter (amd64 linux ibm-srv02.ovirt.org true external bare-metal-external ci) Key: OVIRT-2988 URL: https://ovirt-jira.atlassian.net/browse/OVIRT-2988 Project: oVirt - virtualization made easy Issue Type: Bug Reporter: Alertmanager_Bot Assignee: infra Labels: - alertname = InstanceUnreachable - beta_kubernetes_io_arch = amd64 - beta_kubernetes_io_os = linux - instance = ibm-srv02.ovirt.org - job = kubernetes-nodes-exporter - kubernetes_io_hostname = ibm-srv02.ovirt.org - node_role_kubernetes_io_compute = true - region = external - type = bare-metal-external - zone = ci Annotations: - description = ibm-srv02.ovirt.org of job kubernetes-nodes-exporter has been possibly down for more than 10 minutes. Source: http://prometheus-0:9090/graph?g0.expr=up%7Bjob%3D%22kubernetes-nodes-expor… -- This message was sent by Atlassian Jira (v1001.0.0-SNAPSHOT#100140)

1 0

[JIRA] (OVIRT-2987) [FIRING:1] InstanceUnreachable ibm-srv06.ovirt.org kubernetes-nodes-exporter (amd64 linux ibm-srv06.ovirt.org true external bare-metal-external ci)
by Alertmanager_Bot (oVirt JIRA) 29 Jul '20

29 Jul '20

Alertmanager_Bot created OVIRT-2987: --------------------------------------- Summary: [FIRING:1] InstanceUnreachable ibm-srv06.ovirt.org kubernetes-nodes-exporter (amd64 linux ibm-srv06.ovirt.org true external bare-metal-external ci) Key: OVIRT-2987 URL: https://ovirt-jira.atlassian.net/browse/OVIRT-2987 Project: oVirt - virtualization made easy Issue Type: Bug Reporter: Alertmanager_Bot Assignee: infra Labels: - alertname = InstanceUnreachable - beta_kubernetes_io_arch = amd64 - beta_kubernetes_io_os = linux - instance = ibm-srv06.ovirt.org - job = kubernetes-nodes-exporter - kubernetes_io_hostname = ibm-srv06.ovirt.org - node_role_kubernetes_io_compute = true - region = external - type = bare-metal-external - zone = ci Annotations: - description = ibm-srv06.ovirt.org of job kubernetes-nodes-exporter has been possibly down for more than 10 minutes. Source: http://prometheus-0:9090/graph?g0.expr=up%7Bjob%3D%22kubernetes-nodes-expor… -- This message was sent by Atlassian Jira (v1001.0.0-SNAPSHOT#100140)

1 0

[JIRA] (OVIRT-2989) [FIRING:1] InstanceUnreachable ibm-srv03.ovirt.org kubernetes-nodes-exporter (amd64 linux ibm-srv03.ovirt.org true external bare-metal-external ci)
by Alertmanager_Bot (oVirt JIRA) 29 Jul '20

29 Jul '20

Alertmanager_Bot created OVIRT-2989: --------------------------------------- Summary: [FIRING:1] InstanceUnreachable ibm-srv03.ovirt.org kubernetes-nodes-exporter (amd64 linux ibm-srv03.ovirt.org true external bare-metal-external ci) Key: OVIRT-2989 URL: https://ovirt-jira.atlassian.net/browse/OVIRT-2989 Project: oVirt - virtualization made easy Issue Type: Bug Reporter: Alertmanager_Bot Assignee: infra Labels: - alertname = InstanceUnreachable - beta_kubernetes_io_arch = amd64 - beta_kubernetes_io_os = linux - instance = ibm-srv03.ovirt.org - job = kubernetes-nodes-exporter - kubernetes_io_hostname = ibm-srv03.ovirt.org - node_role_kubernetes_io_compute = true - region = external - type = bare-metal-external - zone = ci Annotations: - description = ibm-srv03.ovirt.org of job kubernetes-nodes-exporter has been possibly down for more than 10 minutes. Source: http://prometheus-0:9090/graph?g0.expr=up%7Bjob%3D%22kubernetes-nodes-expor… -- This message was sent by Atlassian Jira (v1001.0.0-SNAPSHOT#100140)

1 0

[JIRA] (OVIRT-2990) [FIRING:1] InstanceUnreachable ibm-srv04.ovirt.org kubernetes-nodes-exporter (amd64 linux ibm-srv04.ovirt.org true external bare-metal-external ci)
by Alertmanager_Bot (oVirt JIRA) 29 Jul '20

29 Jul '20

Alertmanager_Bot created OVIRT-2990: --------------------------------------- Summary: [FIRING:1] InstanceUnreachable ibm-srv04.ovirt.org kubernetes-nodes-exporter (amd64 linux ibm-srv04.ovirt.org true external bare-metal-external ci) Key: OVIRT-2990 URL: https://ovirt-jira.atlassian.net/browse/OVIRT-2990 Project: oVirt - virtualization made easy Issue Type: Bug Reporter: Alertmanager_Bot Assignee: infra Labels: - alertname = InstanceUnreachable - beta_kubernetes_io_arch = amd64 - beta_kubernetes_io_os = linux - instance = ibm-srv04.ovirt.org - job = kubernetes-nodes-exporter - kubernetes_io_hostname = ibm-srv04.ovirt.org - node_role_kubernetes_io_compute = true - region = external - type = bare-metal-external - zone = ci Annotations: - description = ibm-srv04.ovirt.org of job kubernetes-nodes-exporter has been possibly down for more than 10 minutes. Source: http://prometheus-0:9090/graph?g0.expr=up%7Bjob%3D%22kubernetes-nodes-expor… -- This message was sent by Atlassian Jira (v1001.0.0-SNAPSHOT#100140)

1 0

[JIRA] (OVIRT-2986) [FIRING:1] InstanceUnreachable ibm-srv01.ovirt.org kubernetes-nodes-exporter (amd64 linux ibm-srv01.ovirt.org true external bare-metal-external ci)
by Alertmanager_Bot (oVirt JIRA) 29 Jul '20

29 Jul '20

Alertmanager_Bot created OVIRT-2986: --------------------------------------- Summary: [FIRING:1] InstanceUnreachable ibm-srv01.ovirt.org kubernetes-nodes-exporter (amd64 linux ibm-srv01.ovirt.org true external bare-metal-external ci) Key: OVIRT-2986 URL: https://ovirt-jira.atlassian.net/browse/OVIRT-2986 Project: oVirt - virtualization made easy Issue Type: Bug Reporter: Alertmanager_Bot Assignee: infra Labels: - alertname = InstanceUnreachable - beta_kubernetes_io_arch = amd64 - beta_kubernetes_io_os = linux - instance = ibm-srv01.ovirt.org - job = kubernetes-nodes-exporter - kubernetes_io_hostname = ibm-srv01.ovirt.org - node_role_kubernetes_io_compute = true - region = external - type = bare-metal-external - zone = ci Annotations: - description = ibm-srv01.ovirt.org of job kubernetes-nodes-exporter has been possibly down for more than 10 minutes. Source: http://prometheus-0:9090/graph?g0.expr=up%7Bjob%3D%22kubernetes-nodes-expor… -- This message was sent by Atlassian Jira (v1001.0.0-SNAPSHOT#100140)

1 0

[oVirt Jenkins] ovirt-system-tests_basic-suite-4.3_nightly - Build # 231 - Failure!
by jenkins＠jenkins.phx.ovirt.org 29 Jul '20

29 Jul '20

Project: https://jenkins.ovirt.org/job/ovirt-system-tests_basic-suite-4.3_nightly/ Build: https://jenkins.ovirt.org/job/ovirt-system-tests_basic-suite-4.3_nightly/23… Build Number: 231 Build Status: Failure Triggered By: Started by timer ------------------------------------- Changes Since Last Success: ------------------------------------- Changes for Build #231 [Nir Soffer] setup: Do not hide real issue by reserving ports ----------------- Failed Tests: ----------------- 1 tests failed. FAILED: 008_basic_ui_sanity.start_grid Error Message: could not get ip address of selenium hub. See previous messages for probable docker failure -------------------- >> begin captured stdout << --------------------- executing shell: docker ps CONTAINER ID IMAGE COMMAND CREATED STATUS PORTS NAMES executing shell: docker rm -f grid_node_chrome grid_node_firefox selenium-hub Error response from daemon: No such container: grid_node_chrome Error response from daemon: No such container: grid_node_firefox Error response from daemon: No such container: selenium-hub executing shell: docker kill grid_node_chrome grid_node_firefox selenium-hub Error response from daemon: Cannot kill container: grid_node_chrome: No such container: grid_node_chrome Error response from daemon: Cannot kill container: grid_node_firefox: No such container: grid_node_firefox Error response from daemon: Cannot kill container: selenium-hub: No such container: selenium-hub executing shell: docker ps CONTAINER ID IMAGE COMMAND CREATED STATUS PORTS NAMES executing shell: docker network ls NETWORK ID NAME DRIVER SCOPE 058a426a59b4 bridge bridge local bd7765e6670f host host local 245daf9e104c none null local executing shell: docker network rm grid Error response from daemon: network grid not found executing shell: docker network ls NETWORK ID NAME DRIVER SCOPE 058a426a59b4 bridge bridge local bd7765e6670f host host local 245daf9e104c none null local executing shell: docker network ls NETWORK ID NAME DRIVER SCOPE 058a426a59b4 bridge bridge local bd7765e6670f host host local 245daf9e104c none null local creating docker network for grid executing shell: docker network create grid 328da4cfdd441c835906d9674c744acf5ce32542b90053beed005cb2051505e1 executing shell: docker network ls NETWORK ID NAME DRIVER SCOPE 058a426a59b4 bridge bridge local 328da4cfdd44 grid bridge local bd7765e6670f host host local 245daf9e104c none null local starting hub executing shell: docker run -d -p 4444:4444 --net grid --name selenium-hub selenium/hub:3.9.1-actinium Unable to find image 'selenium/hub:3.9.1-actinium' locally 3.9.1-actinium: Pulling from selenium/hub 1be7f2b886e8: Pulling fs layer 6fbc4a21b806: Pulling fs layer c71a6f8e1378: Pulling fs layer 4be3072e5a37: Pulling fs layer 06c6d2f59700: Pulling fs layer edcd5e9f2f91: Pulling fs layer 0eeaf787f757: Pulling fs layer c949dee5af7e: Pulling fs layer df88a49b4162: Pulling fs layer ce3c6f42fd24: Pulling fs layer 6c8e191c3abf: Pulling fs layer c9aca8b50247: Pulling fs layer 0eeaf787f757: Waiting 4be3072e5a37: Waiting 06c6d2f59700: Waiting c949dee5af7e: Waiting c9aca8b50247: Waiting ce3c6f42fd24: Waiting 6c8e191c3abf: Waiting df88a49b4162: Waiting edcd5e9f2f91: Waiting /usr/bin/docker-current: error pulling image configuration: Get https://production.cloudflare.docker.com/registry-v2/docker/registry/v2/blo…: dial tcp: lookup production.cloudflare.docker.com on 172.19.11.6:53: read udp 10.129.5.89:52169->172.19.11.6:53: i/o timeout. See '/usr/bin/docker-current run --help'. getting ip of hub --------------------- >> end captured stdout << ---------------------- Stack Trace: File "/usr/lib64/python2.7/unittest/case.py", line 369, in run testMethod() File "/usr/lib/python2.7/site-packages/nose/case.py", line 197, in runTest self.test(*self.arg) File "/usr/lib/python2.7/site-packages/ovirtlago/testlib.py", line 142, in wrapped_test test() File "/home/jenkins/agent/workspace/ovirt-system-tests_basic-suite-4.3_nightly/ovirt-system-tests/basic-suite-4.3/test-scenarios/008_basic_ui_sanity.py", line 189, in start_grid raise RuntimeError("could not get ip address of selenium hub. See previous messages for probable docker failure") could not get ip address of selenium hub. See previous messages for probable docker failure -------------------- >> begin captured stdout << --------------------- executing shell: docker ps CONTAINER ID IMAGE COMMAND CREATED STATUS PORTS NAMES executing shell: docker rm -f grid_node_chrome grid_node_firefox selenium-hub Error response from daemon: No such container: grid_node_chrome Error response from daemon: No such container: grid_node_firefox Error response from daemon: No such container: selenium-hub executing shell: docker kill grid_node_chrome grid_node_firefox selenium-hub Error response from daemon: Cannot kill container: grid_node_chrome: No such container: grid_node_chrome Error response from daemon: Cannot kill container: grid_node_firefox: No such container: grid_node_firefox Error response from daemon: Cannot kill container: selenium-hub: No such container: selenium-hub executing shell: docker ps CONTAINER ID IMAGE COMMAND CREATED STATUS PORTS NAMES executing shell: docker network ls NETWORK ID NAME DRIVER SCOPE 058a426a59b4 bridge bridge local bd7765e6670f host host local 245daf9e104c none null local executing shell: docker network rm grid Error response from daemon: network grid not found executing shell: docker network ls NETWORK ID NAME DRIVER SCOPE 058a426a59b4 bridge bridge local bd7765e6670f host host local 245daf9e104c none null local executing shell: docker network ls NETWORK ID NAME DRIVER SCOPE 058a426a59b4 bridge bridge local bd7765e6670f host host local 245daf9e104c none null local creating docker network for grid executing shell: docker network create grid 328da4cfdd441c835906d9674c744acf5ce32542b90053beed005cb2051505e1 executing shell: docker network ls NETWORK ID NAME DRIVER SCOPE 058a426a59b4 bridge bridge local 328da4cfdd44 grid bridge local bd7765e6670f host host local 245daf9e104c none null local starting hub executing shell: docker run -d -p 4444:4444 --net grid --name selenium-hub selenium/hub:3.9.1-actinium Unable to find image 'selenium/hub:3.9.1-actinium' locally 3.9.1-actinium: Pulling from selenium/hub 1be7f2b886e8: Pulling fs layer 6fbc4a21b806: Pulling fs layer c71a6f8e1378: Pulling fs layer 4be3072e5a37: Pulling fs layer 06c6d2f59700: Pulling fs layer edcd5e9f2f91: Pulling fs layer 0eeaf787f757: Pulling fs layer c949dee5af7e: Pulling fs layer df88a49b4162: Pulling fs layer ce3c6f42fd24: Pulling fs layer 6c8e191c3abf: Pulling fs layer c9aca8b50247: Pulling fs layer 0eeaf787f757: Waiting 4be3072e5a37: Waiting 06c6d2f59700: Waiting c949dee5af7e: Waiting c9aca8b50247: Waiting ce3c6f42fd24: Waiting 6c8e191c3abf: Waiting df88a49b4162: Waiting edcd5e9f2f91: Waiting /usr/bin/docker-current: error pulling image configuration: Get https://production.cloudflare.docker.com/registry-v2/docker/registry/v2/blo…: dial tcp: lookup production.cloudflare.docker.com on 172.19.11.6:53: read udp 10.129.5.89:52169->172.19.11.6:53: i/o timeout. See '/usr/bin/docker-current run --help'. getting ip of hub --------------------- >> end captured stdout << ----------------------

1 1

oVirt infra daily report - unstable production jobs - 1266
by jenkins＠jenkins.phx.ovirt.org 29 Jul '20

29 Jul '20

Good morning! Attached is the HTML page with the jenkins status report. You can see it also here: - https://jenkins.ovirt.org/job/system_jenkins-report/1266//artifact/exported… Cheers, Jenkins

1 0

[JIRA] (OVIRT-2985) [FIRING:1] MirrorIsOutOfSync web-ovirt-mirrorchecker.apps.ovirt.org:80 oVirt-mirrors-health http://www.gtlib.gatech.edu/pub/oVirt/pub
by Alertmanager_Bot (oVirt JIRA) 28 Jul '20

28 Jul '20

Alertmanager_Bot created OVIRT-2985: --------------------------------------- Summary: [FIRING:1] MirrorIsOutOfSync web-ovirt-mirrorchecker.apps.ovirt.org:80 oVirt-mirrors-health http://www.gtlib.gatech.edu/pub/oVirt/pub Key: OVIRT-2985 URL: https://ovirt-jira.atlassian.net/browse/OVIRT-2985 Project: oVirt - virtualization made easy Issue Type: Bug Reporter: Alertmanager_Bot Assignee: infra Labels: - alertname = MirrorIsOutOfSync - instance = web-ovirt-mirrorchecker.apps.ovirt.org:80 - job = oVirt-mirrors-health - url = http://www.gtlib.gatech.edu/pub/oVirt/pub Annotations: - description = http://www.gtlib.gatech.edu/pub/oVirt/pub (measured by oVirt-mirrors-health) has not been synced in the past two days. Source: http://prometheus-0:9090/graph?g0.expr=mirror_health_check+%3E+172800&g0.ta… -- This message was sent by Atlassian Jira (v1001.0.0-SNAPSHOT#100140)

1 0

[JIRA] (OVIRT-2980) [FIRING:1] TestAlert
by Shlomi Zidmi (oVirt JIRA) 28 Jul '20

28 Jul '20

[ https://ovirt-jira.atlassian.net/browse/OVIRT-2980?page=com.atlassian.jira.… ] Shlomi Zidmi updated OVIRT-2980: -------------------------------- Resolution: Fixed Status: Done (was: To Do) > [FIRING:1] TestAlert > --------------------- > > Key: OVIRT-2980 > URL: https://ovirt-jira.atlassian.net/browse/OVIRT-2980 > Project: oVirt - virtualization made easy > Issue Type: Bug > Reporter: Alertmanager_Bot > Assignee: infra > Labels: ALERT{alertname="TestAlert"} > > Labels: > - alertname = TestAlert > - url = simple_url.com > Annotations: > Source: -- This message was sent by Atlassian Jira (v1001.0.0-SNAPSHOT#100140)

1 0

[oVirt Jenkins] ovirt-system-tests_basic-suite-master_nightly - Build # 227 - Failure!
by jenkins＠jenkins.phx.ovirt.org 28 Jul '20

28 Jul '20

Project: https://jenkins.ovirt.org/job/ovirt-system-tests_basic-suite-master_nightly/ Build: https://jenkins.ovirt.org/job/ovirt-system-tests_basic-suite-master_nightly… Build Number: 227 Build Status: Failure Triggered By: Started by timer ------------------------------------- Changes Since Last Success: ------------------------------------- Changes for Build #227 [Nir Soffer] setup: Do not hide real issue by reserving ports [Sandro Bonazzola] ovirt-guest-agent: drop fc30 [Ehud Yonasi] mock_configs: added rhel configs. ----------------- Failed Tests: ----------------- 1 tests failed. FAILED: basic-suite-master.test-scenarios.002_bootstrap_pytest.test_verify_add_hosts Error Message: RuntimeError: 1 hosts failed installation: lago-basic-suite-master-host-1: install_failed Stack Trace: api_v4 = <ovirtsdk4.Connection object at 0x7fbb71fd58d0> @order_by(_TEST_LIST) def test_verify_add_hosts(api_v4): hosts_service = api_v4.system_service().hosts_service() hosts_status = hosts_service.list(search='datacenter={}'.format(DC_NAME)) total_hosts = len(hosts_status) dump_hosts = _host_status_to_print(hosts_service, hosts_status) LOGGER.debug('Host status, verify_add_hosts:\n {}'.format(dump_hosts)) testlib.assert_true_within( lambda: _single_host_up(hosts_service, total_hosts), > timeout=constants.ADD_HOST_TIMEOUT ) ../basic-suite-master/test-scenarios/002_bootstrap_pytest.py:433: _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ /usr/lib/python2.7/site-packages/ovirtlago/testlib.py:278: in assert_true_within assert_equals_within(func, True, timeout, allowed_exceptions) /usr/lib/python2.7/site-packages/ovirtlago/testlib.py:234: in assert_equals_within res = func() ../basic-suite-master/test-scenarios/002_bootstrap_pytest.py:432: in <lambda> lambda: _single_host_up(hosts_service, total_hosts), ../basic-suite-master/test-scenarios/002_bootstrap_pytest.py:229: in _single_host_up _check_problematic_hosts(hosts_service) _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ hosts_service = <ovirtsdk4.services.HostsService object at 0x7fbb72148350> def _check_problematic_hosts(hosts_service): problematic_hosts = hosts_service.list(search='datacenter={} AND status != installing and status != initializing and status != up'.format(DC_NAME)) if len(problematic_hosts): dump_hosts = '%s hosts failed installation:\n' % len(problematic_hosts) for host in problematic_hosts: host_service = hosts_service.host_service(host.id) dump_hosts += '%s: %s\n' % (host.name, host_service.get().status) > raise RuntimeError(dump_hosts) E RuntimeError: 1 hosts failed installation: E lago-basic-suite-master-host-1: install_failed ../basic-suite-master/test-scenarios/002_bootstrap_pytest.py:238: RuntimeError

1 1

oVirt infra daily report - unstable production jobs - 1265
by jenkins＠jenkins.phx.ovirt.org 28 Jul '20

28 Jul '20

Good morning! Attached is the HTML page with the jenkins status report. You can see it also here: - https://jenkins.ovirt.org/job/system_jenkins-report/1265//artifact/exported… Cheers, Jenkins

1 0

[JIRA] (OVIRT-2984) Replacing 'Ubuntu Server 18' image on ovirt-glance
by Arik Hadas (oVirt JIRA) 27 Jul '20

27 Jul '20

Arik Hadas created OVIRT-2984: --------------------------------- Summary: Replacing 'Ubuntu Server 18' image on ovirt-glance Key: OVIRT-2984 URL: https://ovirt-jira.atlassian.net/browse/OVIRT-2984 Project: oVirt - virtualization made easy Issue Type: Task Reporter: Arik Hadas Assignee: infra Please replace the Ubuntu Server 18 image on ovirt-glance (there seems to a problem with that image, see: https://bugzilla.redhat.com/show_bug.cgi?id=1858918) -- This message was sent by Atlassian Jira (v1001.0.0-SNAPSHOT#100140)

1 0

[JIRA] (OVIRT-2983) host mirrorlist on IPv6
by Evgheni Dereveanchin (oVirt JIRA) 27 Jul '20

27 Jul '20

Evgheni Dereveanchin created OVIRT-2983: ------------------------------------------- Summary: host mirrorlist on IPv6 Key: OVIRT-2983 URL: https://ovirt-jira.atlassian.net/browse/OVIRT-2983 Project: oVirt - virtualization made easy Issue Type: Improvement Reporter: Evgheni Dereveanchin Assignee: infra Currently, all oVirt users will get a similar URL to fetch the mirrorlist from ovirt-release: https://resources.ovirt.org/pub/yum-repo/mirrorlist-ovirt-4.4-el8 This server is ipv4-only so users on IPv6 can't get a list of mirrors which makes oVirt non-installable in such environments without specifying a mirror manually. Opening a ticket to move this service to a dedicated URL like mirrorlist.ovirt.org and host it on an IPv6-enabled service, maybe at OSAS. Once that is done we can change the URL shipped with ovirt-release to enable installation on IPv6-only setups. -- This message was sent by Atlassian Jira (v1001.0.0-SNAPSHOT#100140)

1 0

[oVirt Jenkins] ovirt-system-tests_basic-suite-master_nightly - Build # 225 - Failure!
by jenkins＠jenkins.phx.ovirt.org 27 Jul '20

27 Jul '20

Project: https://jenkins.ovirt.org/job/ovirt-system-tests_basic-suite-master_nightly/ Build: https://jenkins.ovirt.org/job/ovirt-system-tests_basic-suite-master_nightly… Build Number: 225 Build Status: Failure Triggered By: Started by timer ------------------------------------- Changes Since Last Success: ------------------------------------- Changes for Build #225 [Nir Soffer] setup: Do not hide real issue by reserving ports ----------------- Failed Tests: ----------------- No tests ran.

1 1

[oVirt Jenkins] cleanup-gated-dummy-master - Build #219 - FAILURE!
by jenkins＠jenkins.phx.ovirt.org 27 Jul '20

27 Jul '20

Build: https://jenkins.ovirt.org/job/cleanup-gated-dummy-master/219/ Build Name: #219 Build Status: FAILURE Gerrit change: null - title: null - project: null - branch: null - author: null <null>

1 0

[oVirt Jenkins] ovirt-system-tests_performance-suite-master - Build # 1530 - Failure!
by jenkins＠jenkins.phx.ovirt.org 27 Jul '20

27 Jul '20

Project: https://jenkins.ovirt.org/job/ovirt-system-tests_performance-suite-master/ Build: https://jenkins.ovirt.org/job/ovirt-system-tests_performance-suite-master/1… Build Number: 1530 Build Status: Failure Triggered By: Started by timer ------------------------------------- Changes Since Last Success: ------------------------------------- Changes for Build #1530 [Nir Soffer] setup: Do not hide real issue by reserving ports ----------------- Failed Tests: ----------------- 1 tests failed. FAILED: 040_add_hosts_vms.add_master_storage_domain Error Message: Fault reason is "Operation Failed". Fault detail is "[Cannot add storage server connection when Host status is not up]". HTTP response code is 409. -------------------- >> begin captured logging << -------------------- 040_add_hosts_vms: INFO: lago-performance-suite-master-host-0: installing lago-performance-suite-master-host-1: up ost_utils.general_utils: DEBUG: attempt 2/120, 1 seconds sleeping 040_add_hosts_vms: INFO: lago-performance-suite-master-host-0: installing lago-performance-suite-master-host-1: up --------------------- >> end captured logging << --------------------- Stack Trace: File "/usr/lib64/python2.7/unittest/case.py", line 369, in run testMethod() File "/usr/lib/python2.7/site-packages/nose/case.py", line 197, in runTest self.test(*self.arg) File "/usr/lib/python2.7/site-packages/ovirtlago/testlib.py", line 142, in wrapped_test test() File "/usr/lib/python2.7/site-packages/ovirtlago/testlib.py", line 60, in wrapper return func(get_test_prefix(), *args, **kwargs) File "/home/jenkins/agent/workspace/ovirt-system-tests_performance-suite-master/ovirt-system-tests/performance-suite-master/test-scenarios/040_add_hosts_vms.py", line 353, in add_master_storage_domain add_nfs_storage_domain(prefix) File "/home/jenkins/agent/workspace/ovirt-system-tests_performance-suite-master/ovirt-system-tests/performance-suite-master/test-scenarios/040_add_hosts_vms.py", line 357, in add_nfs_storage_domain add_generic_nfs_storage_domain_4(prefix, SD_NFS_NAME, SD_NFS_HOST_NAME, SD_NFS_PATH, nfs_version='v4_2') File "/home/jenkins/agent/workspace/ovirt-system-tests_performance-suite-master/ovirt-system-tests/performance-suite-master/test-scenarios/040_add_hosts_vms.py", line 407, in add_generic_nfs_storage_domain_4 _add_storage_domain_4(api, p) File "/home/jenkins/agent/workspace/ovirt-system-tests_performance-suite-master/ovirt-system-tests/performance-suite-master/test-scenarios/040_add_hosts_vms.py", line 327, in _add_storage_domain_4 sd = sds_service.add(p) File "/usr/lib64/python2.7/site-packages/ovirtsdk4/services.py", line 26097, in add return self._internal_add(storage_domain, headers, query, wait) File "/usr/lib64/python2.7/site-packages/ovirtsdk4/service.py", line 232, in _internal_add return future.wait() if wait else future File "/usr/lib64/python2.7/site-packages/ovirtsdk4/service.py", line 55, in wait return self._code(response) File "/usr/lib64/python2.7/site-packages/ovirtsdk4/service.py", line 229, in callback self._check_fault(response) File "/usr/lib64/python2.7/site-packages/ovirtsdk4/service.py", line 132, in _check_fault self._raise_error(response, body) File "/usr/lib64/python2.7/site-packages/ovirtsdk4/service.py", line 118, in _raise_error raise error 'Fault reason is "Operation Failed". Fault detail is "[Cannot add storage server connection when Host status is not up]". HTTP response code is 409.\n-------------------- >> begin captured logging << --------------------\n040_add_hosts_vms: INFO: lago-performance-suite-master-host-0: installing\nlago-performance-suite-master-host-1: up\n\nost_utils.general_utils: DEBUG: attempt 2/120, 1 seconds sleeping\n040_add_hosts_vms: INFO: lago-performance-suite-master-host-0: installing\nlago-performance-suite-master-host-1: up\n\n--------------------- >> end captured logging << ---------------------'

1 1

oVirt infra daily report - unstable production jobs - 1264
by jenkins＠jenkins.phx.ovirt.org 27 Jul '20

27 Jul '20

Good morning! Attached is the HTML page with the jenkins status report. You can see it also here: - https://jenkins.ovirt.org/job/system_jenkins-report/1264//artifact/exported… Cheers, Jenkins

1 0

[oVirt Jenkins] ovirt-system-tests_basic-suite-master_nightly - Build # 221 - Failure!
by jenkins＠jenkins.phx.ovirt.org 26 Jul '20

26 Jul '20

Project: https://jenkins.ovirt.org/job/ovirt-system-tests_basic-suite-master_nightly/ Build: https://jenkins.ovirt.org/job/ovirt-system-tests_basic-suite-master_nightly… Build Number: 221 Build Status: Failure Triggered By: Started by timer ------------------------------------- Changes Since Last Success: ------------------------------------- Changes for Build #221 [Nir Soffer] setup: Do not hide real issue by reserving ports ----------------- Failed Tests: ----------------- 1 tests failed. FAILED: basic-suite-master.test-scenarios.004_basic_sanity_pytest.test_live_storage_migration Error Message: AssertionError: False != True after 600 seconds Stack Trace: api_v4 = <ovirtsdk4.Connection object at 0x7fb0e92139d0> @order_by(_TEST_LIST) def test_live_storage_migration(api_v4): engine = api_v4.system_service() disk_service = test_utils.get_disk_service(engine, DISK0_NAME) correlation_id = 'live_storage_migration' disk_service.move( async=False, filter=False, storage_domain=types.StorageDomain( name=SD_ISCSI_NAME ), query={'correlation_id': correlation_id} ) testlib.assert_true_within_long(lambda: test_utils.all_jobs_finished(engine, correlation_id)) # Assert that the disk is on the correct storage domain, # its status is OK and the snapshot created for the migration # has been merged testlib.assert_true_within_long( lambda: api_v4.follow_link(disk_service.get().storage_domains[0]).name == SD_ISCSI_NAME ) vm0_snapshots_service = test_utils.get_vm_snapshots_service(engine, VM0_NAME) testlib.assert_true_within_long( > lambda: len(vm0_snapshots_service.list()) == 1 ) ../basic-suite-master/test-scenarios/004_basic_sanity_pytest.py:579: _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ /usr/lib/python2.7/site-packages/ovirtlago/testlib.py:286: in assert_true_within_long assert_equals_within_long(func, True, allowed_exceptions) /usr/lib/python2.7/site-packages/ovirtlago/testlib.py:273: in assert_equals_within_long func, value, LONG_TIMEOUT, allowed_exceptions=allowed_exceptions _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ func = <function <lambda> at 0x7fb0e5af2b90>, value = True, timeout = 600 allowed_exceptions = [], initial_wait = 0 def assert_equals_within( func, value, timeout, allowed_exceptions=None, initial_wait=10 ): allowed_exceptions = allowed_exceptions or [] with utils.EggTimer(timeout) as timer: while not timer.elapsed(): try: res = func() if res == value: return except Exception as exc: if _instance_of_any(exc, allowed_exceptions): time.sleep(3) continue LOGGER.exception("Unhandled exception in %s", func) raise if initial_wait == 0: time.sleep(3) else: time.sleep(initial_wait) initial_wait = 0 try: raise AssertionError( > '%s != %s after %s seconds' % (res, value, timeout) ) E AssertionError: False != True after 600 seconds /usr/lib/python2.7/site-packages/ovirtlago/testlib.py:252: AssertionError

1 1

[oVirt Jenkins] ovirt-system-tests_basic-suite-4.3_nightly - Build # 221 - Failure!
by jenkins＠jenkins.phx.ovirt.org 26 Jul '20

26 Jul '20

Project: https://jenkins.ovirt.org/job/ovirt-system-tests_basic-suite-4.3_nightly/ Build: https://jenkins.ovirt.org/job/ovirt-system-tests_basic-suite-4.3_nightly/22… Build Number: 221 Build Status: Failure Triggered By: Started by timer ------------------------------------- Changes Since Last Success: ------------------------------------- Changes for Build #221 [Nir Soffer] setup: Do not hide real issue by reserving ports ----------------- Failed Tests: ----------------- 1 tests failed. FAILED: 004_basic_sanity.disk_operations Error Message: False != True after 600 seconds -------------------- >> begin captured logging << -------------------- lago.utils: DEBUG: Error while running thread Thread-66 Traceback (most recent call last): File "/usr/lib/python2.7/site-packages/lago/utils.py", line 58, in _ret_via_queue queue.put({'return': func()}) File "/usr/lib/python2.7/site-packages/ovirtlago/testlib.py", line 60, in wrapper return func(get_test_prefix(), *args, **kwargs) File "/usr/lib/python2.7/site-packages/ovirtlago/testlib.py", line 79, in wrapper prefix.virt_env.engine_vm().get_api(api_ver=4), *args, **kwargs File "/home/jenkins/agent/workspace/ovirt-system-tests_basic-suite-4.3_nightly/ovirt-system-tests/basic-suite-4.3/test-scenarios/004_basic_sanity.py", line 508, in live_storage_migration testlib.assert_true_within_long(lambda: test_utils.all_jobs_finished(engine, correlation_id)) File "/usr/lib/python2.7/site-packages/ovirtlago/testlib.py", line 286, in assert_true_within_long assert_equals_within_long(func, True, allowed_exceptions) File "/usr/lib/python2.7/site-packages/ovirtlago/testlib.py", line 273, in assert_equals_within_long func, value, LONG_TIMEOUT, allowed_exceptions=allowed_exceptions File "/usr/lib/python2.7/site-packages/ovirtlago/testlib.py", line 252, in assert_equals_within '%s != %s after %s seconds' % (res, value, timeout) AssertionError: False != True after 600 seconds --------------------- >> end captured logging << --------------------- Stack Trace: File "/usr/lib64/python2.7/unittest/case.py", line 369, in run testMethod() File "/usr/lib/python2.7/site-packages/nose/case.py", line 197, in runTest self.test(*self.arg) File "/usr/lib/python2.7/site-packages/ovirtlago/testlib.py", line 142, in wrapped_test test() File "/usr/lib/python2.7/site-packages/ovirtlago/testlib.py", line 60, in wrapper return func(get_test_prefix(), *args, **kwargs) File "/usr/lib/python2.7/site-packages/ovirtlago/testlib.py", line 79, in wrapper prefix.virt_env.engine_vm().get_api(api_ver=4), *args, **kwargs File "/home/jenkins/agent/workspace/ovirt-system-tests_basic-suite-4.3_nightly/ovirt-system-tests/basic-suite-4.3/test-scenarios/004_basic_sanity.py", line 932, in disk_operations vt.join_all() File "/usr/lib/python2.7/site-packages/lago/utils.py", line 58, in _ret_via_queue queue.put({'return': func()}) File "/usr/lib/python2.7/site-packages/ovirtlago/testlib.py", line 60, in wrapper return func(get_test_prefix(), *args, **kwargs) File "/usr/lib/python2.7/site-packages/ovirtlago/testlib.py", line 79, in wrapper prefix.virt_env.engine_vm().get_api(api_ver=4), *args, **kwargs File "/home/jenkins/agent/workspace/ovirt-system-tests_basic-suite-4.3_nightly/ovirt-system-tests/basic-suite-4.3/test-scenarios/004_basic_sanity.py", line 508, in live_storage_migration testlib.assert_true_within_long(lambda: test_utils.all_jobs_finished(engine, correlation_id)) File "/usr/lib/python2.7/site-packages/ovirtlago/testlib.py", line 286, in assert_true_within_long assert_equals_within_long(func, True, allowed_exceptions) File "/usr/lib/python2.7/site-packages/ovirtlago/testlib.py", line 273, in assert_equals_within_long func, value, LONG_TIMEOUT, allowed_exceptions=allowed_exceptions File "/usr/lib/python2.7/site-packages/ovirtlago/testlib.py", line 252, in assert_equals_within '%s != %s after %s seconds' % (res, value, timeout) 'False != True after 600 seconds\n-------------------- >> begin captured logging << --------------------\nlago.utils: DEBUG: Error while running thread Thread-66\nTraceback (most recent call last):\n File "/usr/lib/python2.7/site-packages/lago/utils.py", line 58, in _ret_via_queue\n queue.put({\'return\': func()})\n File "/usr/lib/python2.7/site-packages/ovirtlago/testlib.py", line 60, in wrapper\n return func(get_test_prefix(), *args, **kwargs)\n File "/usr/lib/python2.7/site-packages/ovirtlago/testlib.py", line 79, in wrapper\n prefix.virt_env.engine_vm().get_api(api_ver=4), *args, **kwargs\n File "/home/jenkins/agent/workspace/ovirt-system-tests_basic-suite-4.3_nightly/ovirt-system-tests/basic-suite-4.3/test-scenarios/004_basic_sanity.py", line 508, in live_storage_migration\n testlib.assert_true_within_long(lambda: test_utils.all_jobs_finished(engine, correlation_id))\n File "/usr/lib/python2.7/site-packages/ovirtlago/testlib.py", line 286, in assert_true_within_long\n assert_equals_within_long(func, True, allowed_exceptions)\n File "/usr/lib/python2.7/site-packages/ovirtlago/testlib.py", line 273, in assert_equals_within_long\n func, value, LONG_TIMEOUT, allowed_exceptions=allowed_exceptions\n File "/usr/lib/python2.7/site-packages/ovirtlago/testlib.py", line 252, in assert_equals_within\n \'%s != %s after %s seconds\' % (res, value, timeout)\nAssertionError: False != True after 600 seconds\n--------------------- >> end captured logging << ---------------------'

1 1

oVirt infra daily report - unstable production jobs - 1263
by jenkins＠jenkins.phx.ovirt.org 26 Jul '20

26 Jul '20

Good morning! Attached is the HTML page with the jenkins status report. You can see it also here: - https://jenkins.ovirt.org/job/system_jenkins-report/1263//artifact/exported… Cheers, Jenkins

1 0

[oVirt Jenkins] ovirt-system-tests_basic-suite-master_nightly - Build # 219 - Failure!
by jenkins＠jenkins.phx.ovirt.org 25 Jul '20

25 Jul '20

Project: https://jenkins.ovirt.org/job/ovirt-system-tests_basic-suite-master_nightly/ Build: https://jenkins.ovirt.org/job/ovirt-system-tests_basic-suite-master_nightly… Build Number: 219 Build Status: Failure Triggered By: Started by timer ------------------------------------- Changes Since Last Success: ------------------------------------- Changes for Build #219 [Nir Soffer] setup: Do not hide real issue by reserving ports ----------------- Failed Tests: ----------------- 1 tests failed. FAILED: basic-suite-master.test-scenarios.002_bootstrap_pytest.test_verify_add_hosts Error Message: RuntimeError: 1 hosts failed installation: lago-basic-suite-master-host-1: install_failed Stack Trace: api_v4 = <ovirtsdk4.Connection object at 0x7fd026efc8d0> @order_by(_TEST_LIST) def test_verify_add_hosts(api_v4): hosts_service = api_v4.system_service().hosts_service() hosts_status = hosts_service.list(search='datacenter={}'.format(DC_NAME)) total_hosts = len(hosts_status) dump_hosts = _host_status_to_print(hosts_service, hosts_status) LOGGER.debug('Host status, verify_add_hosts:\n {}'.format(dump_hosts)) testlib.assert_true_within( lambda: _single_host_up(hosts_service, total_hosts), > timeout=constants.ADD_HOST_TIMEOUT ) ../basic-suite-master/test-scenarios/002_bootstrap_pytest.py:433: _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ /usr/lib/python2.7/site-packages/ovirtlago/testlib.py:278: in assert_true_within assert_equals_within(func, True, timeout, allowed_exceptions) /usr/lib/python2.7/site-packages/ovirtlago/testlib.py:234: in assert_equals_within res = func() ../basic-suite-master/test-scenarios/002_bootstrap_pytest.py:432: in <lambda> lambda: _single_host_up(hosts_service, total_hosts), ../basic-suite-master/test-scenarios/002_bootstrap_pytest.py:229: in _single_host_up _check_problematic_hosts(hosts_service) _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ hosts_service = <ovirtsdk4.services.HostsService object at 0x7fd02706f350> def _check_problematic_hosts(hosts_service): problematic_hosts = hosts_service.list(search='datacenter={} AND status != installing and status != initializing and status != up'.format(DC_NAME)) if len(problematic_hosts): dump_hosts = '%s hosts failed installation:\n' % len(problematic_hosts) for host in problematic_hosts: host_service = hosts_service.host_service(host.id) dump_hosts += '%s: %s\n' % (host.name, host_service.get().status) > raise RuntimeError(dump_hosts) E RuntimeError: 1 hosts failed installation: E lago-basic-suite-master-host-1: install_failed ../basic-suite-master/test-scenarios/002_bootstrap_pytest.py:238: RuntimeError

1 1

[oVirt Jenkins] ovirt-system-tests_basic-suite-4.3_nightly - Build # 217 - Failure!
by jenkins＠jenkins.phx.ovirt.org 25 Jul '20

25 Jul '20

Project: https://jenkins.ovirt.org/job/ovirt-system-tests_basic-suite-4.3_nightly/ Build: https://jenkins.ovirt.org/job/ovirt-system-tests_basic-suite-4.3_nightly/21… Build Number: 217 Build Status: Failure Triggered By: Started by timer ------------------------------------- Changes Since Last Success: ------------------------------------- Changes for Build #217 [Nir Soffer] setup: Do not hide real issue by reserving ports ----------------- Failed Tests: ----------------- 1 tests failed. FAILED: 008_basic_ui_sanity.initialize_chrome Error Message: Message: Error forwarding the new session cannot find : Capabilities {browserName: chrome, platform: LINUX, version: 64.0.3282.140} Stacktrace: at org.openqa.grid.web.servlet.handler.RequestHandler.process (RequestHandler.java:117) at org.openqa.grid.web.servlet.DriverServlet.process (DriverServlet.java:84) at org.openqa.grid.web.servlet.DriverServlet.doPost (DriverServlet.java:68) at javax.servlet.http.HttpServlet.service (HttpServlet.java:707) at javax.servlet.http.HttpServlet.service (HttpServlet.java:790) at org.seleniumhq.jetty9.servlet.ServletHolder.handle (ServletHolder.java:860) at org.seleniumhq.jetty9.servlet.ServletHandler.doHandle (ServletHandler.java:535) at org.seleniumhq.jetty9.server.handler.ScopedHandler.nextHandle (ScopedHandler.java:188) at org.seleniumhq.jetty9.server.session.SessionHandler.doHandle (SessionHandler.java:1595) at org.seleniumhq.jetty9.server.handler.ScopedHandler.nextHandle (ScopedHandler.java:188) at org.seleniumhq.jetty9.server.handler.ContextHandler.doHandle (ContextHandler.java:1253) at org.seleniumhq.jetty9.server.handler.ScopedHandler.nextScope (ScopedHandler.java:168) at org.seleniumhq.jetty9.servlet.ServletHandler.doScope (ServletHandler.java:473) at org.seleniumhq.jetty9.server.session.SessionHandler.doScope (SessionHandler.java:1564) at org.seleniumhq.jetty9.server.handler.ScopedHandler.nextScope (ScopedHandler.java:166) at org.seleniumhq.jetty9.server.handler.ContextHandler.doScope (ContextHandler.java:1155) at org.seleniumhq.jetty9.server.handler.ScopedHandler.handle (ScopedHandler.java:141) at org.seleniumhq.jetty9.server.handler.HandlerWrapper.handle (HandlerWrapper.java:132) at org.seleniumhq.jetty9.server.Server.handle (Server.java:530) at org.seleniumhq.jetty9.server.HttpChannel.handle (HttpChannel.java:347) at org.seleniumhq.jetty9.server.HttpConnection.onFillable (HttpConnection.java:256) at org.seleniumhq.jetty9.io.AbstractConnection$ReadCallback.succeeded (AbstractConnection.java:279) at org.seleniumhq.jetty9.io.FillInterest.fillable (FillInterest.java:102) at org.seleniumhq.jetty9.io.ChannelEndPoint$2.run (ChannelEndPoint.java:124) at org.seleniumhq.jetty9.util.thread.strategy.EatWhatYouKill.doProduce (EatWhatYouKill.java:247) at org.seleniumhq.jetty9.util.thread.strategy.EatWhatYouKill.produce (EatWhatYouKill.java:140) at org.seleniumhq.jetty9.util.thread.strategy.EatWhatYouKill.run (EatWhatYouKill.java:131) at org.seleniumhq.jetty9.util.thread.ReservedThreadExecutor$ReservedThread.run (ReservedThreadExecutor.java:382) at org.seleniumhq.jetty9.util.thread.QueuedThreadPool.runJob (QueuedThreadPool.java:708) at org.seleniumhq.jetty9.util.thread.QueuedThreadPool$2.run (QueuedThreadPool.java:626) at java.lang.Thread.run (Thread.java:748) -------------------- >> begin captured stdout << --------------------- _init_browser, connecting to hub at http://172.18.0.2:4444/wd/hub Message: Error forwarding the new session cannot find : Capabilities {browserName: chrome, platform: LINUX, version: 64.0.3282.140} Stacktrace: at org.openqa.grid.web.servlet.handler.RequestHandler.process (RequestHandler.java:117) at org.openqa.grid.web.servlet.DriverServlet.process (DriverServlet.java:84) at org.openqa.grid.web.servlet.DriverServlet.doPost (DriverServlet.java:68) at javax.servlet.http.HttpServlet.service (HttpServlet.java:707) at javax.servlet.http.HttpServlet.service (HttpServlet.java:790) at org.seleniumhq.jetty9.servlet.ServletHolder.handle (ServletHolder.java:860) at org.seleniumhq.jetty9.servlet.ServletHandler.doHandle (ServletHandler.java:535) at org.seleniumhq.jetty9.server.handler.ScopedHandler.nextHandle (ScopedHandler.java:188) at org.seleniumhq.jetty9.server.session.SessionHandler.doHandle (SessionHandler.java:1595) at org.seleniumhq.jetty9.server.handler.ScopedHandler.nextHandle (ScopedHandler.java:188) at org.seleniumhq.jetty9.server.handler.ContextHandler.doHandle (ContextHandler.java:1253) at org.seleniumhq.jetty9.server.handler.ScopedHandler.nextScope (ScopedHandler.java:168) at org.seleniumhq.jetty9.servlet.ServletHandler.doScope (ServletHandler.java:473) at org.seleniumhq.jetty9.server.session.SessionHandler.doScope (SessionHandler.java:1564) at org.seleniumhq.jetty9.server.handler.ScopedHandler.nextScope (ScopedHandler.java:166) at org.seleniumhq.jetty9.server.handler.ContextHandler.doScope (ContextHandler.java:1155) at org.seleniumhq.jetty9.server.handler.ScopedHandler.handle (ScopedHandler.java:141) at org.seleniumhq.jetty9.server.handler.HandlerWrapper.handle (HandlerWrapper.java:132) at org.seleniumhq.jetty9.server.Server.handle (Server.java:530) at org.seleniumhq.jetty9.server.HttpChannel.handle (HttpChannel.java:347) at org.seleniumhq.jetty9.server.HttpConnection.onFillable (HttpConnection.java:256) at org.seleniumhq.jetty9.io.AbstractConnection$ReadCallback.succeeded (AbstractConnection.java:279) at org.seleniumhq.jetty9.io.FillInterest.fillable (FillInterest.java:102) at org.seleniumhq.jetty9.io.ChannelEndPoint$2.run (ChannelEndPoint.java:124) at org.seleniumhq.jetty9.util.thread.strategy.EatWhatYouKill.doProduce (EatWhatYouKill.java:247) at org.seleniumhq.jetty9.util.thread.strategy.EatWhatYouKill.produce (EatWhatYouKill.java:140) at org.seleniumhq.jetty9.util.thread.strategy.EatWhatYouKill.run (EatWhatYouKill.java:131) at org.seleniumhq.jetty9.util.thread.ReservedThreadExecutor$ReservedThread.run (ReservedThreadExecutor.java:382) at org.seleniumhq.jetty9.util.thread.QueuedThreadPool.runJob (QueuedThreadPool.java:708) at org.seleniumhq.jetty9.util.thread.QueuedThreadPool$2.run (QueuedThreadPool.java:626) at java.lang.Thread.run (Thread.java:748) WebDriverException in _init_browser connecting to hub --------------------- >> end captured stdout << ---------------------- -------------------- >> begin captured logging << -------------------- selenium.webdriver.remote.remote_connection: DEBUG: POST http://172.18.0.2:4444/wd/hub/session {"capabilities": {"alwaysMatch": {"browserName": "chrome", "platformName": "linux", "browserVersion": "64.0.3282.140"}, "firstMatch": [{}]}, "desiredCapabilities": {"platform": "LINUX", "browserName": "chrome", "version": "64.0.3282.140"}} urllib3.connectionpool: DEBUG: Starting new HTTP connection (1): 172.18.0.2:4444 urllib3.connectionpool: DEBUG: http://172.18.0.2:4444 "POST /wd/hub/session HTTP/1.1" 500 7792 selenium.webdriver.remote.remote_connection: DEBUG: Finished Request --------------------- >> end captured logging << --------------------- Stack Trace: File "/usr/lib64/python2.7/unittest/case.py", line 369, in run testMethod() File "/usr/lib/python2.7/site-packages/nose/case.py", line 197, in runTest self.test(*self.arg) File "/usr/lib/python2.7/site-packages/ovirtlago/testlib.py", line 142, in wrapped_test test() File "/home/jenkins/agent/workspace/ovirt-system-tests_basic-suite-4.3_nightly/ovirt-system-tests/basic-suite-4.3/test-scenarios/008_basic_ui_sanity.py", line 223, in initialize_chrome _init_driver(_get_chrome_capabilities()) File "/home/jenkins/agent/workspace/ovirt-system-tests_basic-suite-4.3_nightly/ovirt-system-tests/basic-suite-4.3/test-scenarios/008_basic_ui_sanity.py", line 249, in _init_driver raise e 'Message: Error forwarding the new session cannot find : Capabilities {browserName: chrome, platform: LINUX, version: 64.0.3282.140}\nStacktrace:\n at org.openqa.grid.web.servlet.handler.RequestHandler.process (RequestHandler.java:117)\n at org.openqa.grid.web.servlet.DriverServlet.process (DriverServlet.java:84)\n at org.openqa.grid.web.servlet.DriverServlet.doPost (DriverServlet.java:68)\n at javax.servlet.http.HttpServlet.service (HttpServlet.java:707)\n at javax.servlet.http.HttpServlet.service (HttpServlet.java:790)\n at org.seleniumhq.jetty9.servlet.ServletHolder.handle (ServletHolder.java:860)\n at org.seleniumhq.jetty9.servlet.ServletHandler.doHandle (ServletHandler.java:535)\n at org.seleniumhq.jetty9.server.handler.ScopedHandler.nextHandle (ScopedHandler.java:188)\n at org.seleniumhq.jetty9.server.session.SessionHandler.doHandle (SessionHandler.java:1595)\n at org.seleniumhq.jetty9.server.handler.ScopedHandler.nextHandle (ScopedHandler.java:188)\n at org.seleniumhq.jetty9.server.handler.ContextHandler.doHandle (ContextHandler.java:1253)\n at org.seleniumhq.jetty9.server.handler.ScopedHandler.nextScope (ScopedHandler.java:168)\n at org.seleniumhq.jetty9.servlet.ServletHandler.doScope (ServletHandler.java:473)\n at org.seleniumhq.jetty9.server.session.SessionHandler.doScope (SessionHandler.java:1564)\n at org.seleniumhq.jetty9.server.handler.ScopedHandler.nextScope (ScopedHandler.java:166)\n at org.seleniumhq.jetty9.server.handler.ContextHandler.doScope (ContextHandler.java:1155)\n at org.seleniumhq.jetty9.server.handler.ScopedHandler.handle (ScopedHandler.java:141)\n at org.seleniumhq.jetty9.server.handler.HandlerWrapper.handle (HandlerWrapper.java:132)\n at org.seleniumhq.jetty9.server.Server.handle (Server.java:530)\n at org.seleniumhq.jetty9.server.HttpChannel.handle (HttpChannel.java:347)\n at org.seleniumhq.jetty9.server.HttpConnection.onFillable (HttpConnection.java:256)\n at org.seleniumhq.jetty9.io.AbstractConnection$ReadCallback.succeeded (AbstractConnection.java:279)\n at org.seleniumhq.jetty9.io.FillInterest.fillable (FillInterest.java:102)\n at org.seleniumhq.jetty9.io.ChannelEndPoint$2.run (ChannelEndPoint.java:124)\n at org.seleniumhq.jetty9.util.thread.strategy.EatWhatYouKill.doProduce (EatWhatYouKill.java:247)\n at org.seleniumhq.jetty9.util.thread.strategy.EatWhatYouKill.produce (EatWhatYouKill.java:140)\n at org.seleniumhq.jetty9.util.thread.strategy.EatWhatYouKill.run (EatWhatYouKill.java:131)\n at org.seleniumhq.jetty9.util.thread.ReservedThreadExecutor$ReservedThread.run (ReservedThreadExecutor.java:382)\n at org.seleniumhq.jetty9.util.thread.QueuedThreadPool.runJob (QueuedThreadPool.java:708)\n at org.seleniumhq.jetty9.util.thread.QueuedThreadPool$2.run (QueuedThreadPool.java:626)\n at java.lang.Thread.run (Thread.java:748)\n-------------------- >> begin captured stdout << ---------------------\n_init_browser, connecting to hub at http://172.18.0.2:4444/wd/hub\nMessage: Error forwarding the new session cannot find : Capabilities {browserName: chrome, platform: LINUX, version: 64.0.3282.140}\nStacktrace:\n at org.openqa.grid.web.servlet.handler.RequestHandler.process (RequestHandler.java:117)\n at org.openqa.grid.web.servlet.DriverServlet.process (DriverServlet.java:84)\n at org.openqa.grid.web.servlet.DriverServlet.doPost (DriverServlet.java:68)\n at javax.servlet.http.HttpServlet.service (HttpServlet.java:707)\n at javax.servlet.http.HttpServlet.service (HttpServlet.java:790)\n at org.seleniumhq.jetty9.servlet.ServletHolder.handle (ServletHolder.java:860)\n at org.seleniumhq.jetty9.servlet.ServletHandler.doHandle (ServletHandler.java:535)\n at org.seleniumhq.jetty9.server.handler.ScopedHandler.nextHandle (ScopedHandler.java:188)\n at org.seleniumhq.jetty9.server.session.SessionHandler.doHandle (SessionHandler.java:1595)\n at org.seleniumhq.jetty9.server.handler.ScopedHandler.nextHandle (ScopedHandler.java:188)\n at org.seleniumhq.jetty9.server.handler.ContextHandler.doHandle (ContextHandler.java:1253)\n at org.seleniumhq.jetty9.server.handler.ScopedHandler.nextScope (ScopedHandler.java:168)\n at org.seleniumhq.jetty9.servlet.ServletHandler.doScope (ServletHandler.java:473)\n at org.seleniumhq.jetty9.server.session.SessionHandler.doScope (SessionHandler.java:1564)\n at org.seleniumhq.jetty9.server.handler.ScopedHandler.nextScope (ScopedHandler.java:166)\n at org.seleniumhq.jetty9.server.handler.ContextHandler.doScope (ContextHandler.java:1155)\n at org.seleniumhq.jetty9.server.handler.ScopedHandler.handle (ScopedHandler.java:141)\n at org.seleniumhq.jetty9.server.handler.HandlerWrapper.handle (HandlerWrapper.java:132)\n at org.seleniumhq.jetty9.server.Server.handle (Server.java:530)\n at org.seleniumhq.jetty9.server.HttpChannel.handle (HttpChannel.java:347)\n at org.seleniumhq.jetty9.server.HttpConnection.onFillable (HttpConnection.java:256)\n at org.seleniumhq.jetty9.io.AbstractConnection$ReadCallback.succeeded (AbstractConnection.java:279)\n at org.seleniumhq.jetty9.io.FillInterest.fillable (FillInterest.java:102)\n at org.seleniumhq.jetty9.io.ChannelEndPoint$2.run (ChannelEndPoint.java:124)\n at org.seleniumhq.jetty9.util.thread.strategy.EatWhatYouKill.doProduce (EatWhatYouKill.java:247)\n at org.seleniumhq.jetty9.util.thread.strategy.EatWhatYouKill.produce (EatWhatYouKill.java:140)\n at org.seleniumhq.jetty9.util.thread.strategy.EatWhatYouKill.run (EatWhatYouKill.java:131)\n at org.seleniumhq.jetty9.util.thread.ReservedThreadExecutor$ReservedThread.run (ReservedThreadExecutor.java:382)\n at org.seleniumhq.jetty9.util.thread.QueuedThreadPool.runJob (QueuedThreadPool.java:708)\n at org.seleniumhq.jetty9.util.thread.QueuedThreadPool$2.run (QueuedThreadPool.java:626)\n at java.lang.Thread.run (Thread.java:748)\nWebDriverException in _init_browser connecting to hub\n\n--------------------- >> end captured stdout << ----------------------\n-------------------- >> begin captured logging << --------------------\nselenium.webdriver.remote.remote_connection: DEBUG: POST http://172.18.0.2:4444/wd/hub/session {"capabilities": {"alwaysMatch": {"browserName": "chrome", "platformName": "linux", "browserVersion": "64.0.3282.140"}, "firstMatch": [{}]}, "desiredCapabilities": {"platform": "LINUX", "browserName": "chrome", "version": "64.0.3282.140"}}\nurllib3.connectionpool: DEBUG: Starting new HTTP connection (1): 172.18.0.2:4444\nurllib3.connectionpool: DEBUG: http://172.18.0.2:4444 "POST /wd/hub/session HTTP/1.1" 500 7792\nselenium.webdriver.remote.remote_connection: DEBUG: Finished Request\n--------------------- >> end captured logging << ---------------------'

1 1

oVirt infra daily report - unstable production jobs - 1262
by jenkins＠jenkins.phx.ovirt.org 25 Jul '20

25 Jul '20

Good morning! Attached is the HTML page with the jenkins status report. You can see it also here: - https://jenkins.ovirt.org/job/system_jenkins-report/1262//artifact/exported… Cheers, Jenkins

1 0

oVirt infra daily report - unstable production jobs - 1261
by jenkins＠jenkins.phx.ovirt.org 24 Jul '20

24 Jul '20

Good morning! Attached is the HTML page with the jenkins status report. You can see it also here: - https://jenkins.ovirt.org/job/system_jenkins-report/1261//artifact/exported… Cheers, Jenkins

1 0

[JIRA] (OVIRT-2982) [FIRING:1] InstanceUnreachable shift-n12.phx.ovirt.org kubernetes-nodes-exporter (amd64 linux shift-n12.phx.ovirt.org true primary vm ci)
by Alertmanager_Bot (oVirt JIRA) 23 Jul '20

23 Jul '20

Alertmanager_Bot created OVIRT-2982: --------------------------------------- Summary: [FIRING:1] InstanceUnreachable shift-n12.phx.ovirt.org kubernetes-nodes-exporter (amd64 linux shift-n12.phx.ovirt.org true primary vm ci) Key: OVIRT-2982 URL: https://ovirt-jira.atlassian.net/browse/OVIRT-2982 Project: oVirt - virtualization made easy Issue Type: Bug Reporter: Alertmanager_Bot Assignee: infra Labels: - alertname = InstanceUnreachable - beta_kubernetes_io_arch = amd64 - beta_kubernetes_io_os = linux - instance = shift-n12.phx.ovirt.org - job = kubernetes-nodes-exporter - kubernetes_io_hostname = shift-n12.phx.ovirt.org - node_role_kubernetes_io_compute = true - region = primary - type = vm - zone = ci Annotations: - description = shift-n12.phx.ovirt.org of job kubernetes-nodes-exporter has been possibly down for more than 10 minutes. Source: http://prometheus-0:9090/graph?g0.expr=up%7Bjob%3D%22kubernetes-nodes-expor… -- This message was sent by Atlassian Jira (v1001.0.0-SNAPSHOT#100134)

1 0

[JIRA] (OVIRT-2981) [FIRING:1] MirrorIsOutOfSync web-ovirt-mirrorchecker.apps.ovirt.org:80 oVirt-mirrors-health http://www.gtlib.gatech.edu/pub/oVirt/pub
by Alertmanager_Bot (oVirt JIRA) 23 Jul '20

23 Jul '20

Alertmanager_Bot created OVIRT-2981: --------------------------------------- Summary: [FIRING:1] MirrorIsOutOfSync web-ovirt-mirrorchecker.apps.ovirt.org:80 oVirt-mirrors-health http://www.gtlib.gatech.edu/pub/oVirt/pub Key: OVIRT-2981 URL: https://ovirt-jira.atlassian.net/browse/OVIRT-2981 Project: oVirt - virtualization made easy Issue Type: Bug Reporter: Alertmanager_Bot Assignee: infra Labels: - alertname = MirrorIsOutOfSync - instance = web-ovirt-mirrorchecker.apps.ovirt.org:80 - job = oVirt-mirrors-health - url = http://www.gtlib.gatech.edu/pub/oVirt/pub Annotations: - description = http://www.gtlib.gatech.edu/pub/oVirt/pub (measured by oVirt-mirrors-health) has not been synced in the past two days. Source: http://prometheus-0:9090/graph?g0.expr=mirror_health_check+%3E+172800&g0.ta… -- This message was sent by Atlassian Jira (v1001.0.0-SNAPSHOT#100134)

1 0

[JIRA] (OVIRT-2980) [FIRING:1] TestAlert
by Alertmanager_Bot (oVirt JIRA) 23 Jul '20

23 Jul '20

Alertmanager_Bot created OVIRT-2980: --------------------------------------- Summary: [FIRING:1] TestAlert Key: OVIRT-2980 URL: https://ovirt-jira.atlassian.net/browse/OVIRT-2980 Project: oVirt - virtualization made easy Issue Type: Bug Reporter: Alertmanager_Bot Assignee: infra Labels: - alertname = TestAlert - url = simple_url.com Annotations: Source: -- This message was sent by Atlassian Jira (v1001.0.0-SNAPSHOT#100134)

1 0

[JIRA] (OVIRT-2977) [FIRING:1] TestAlert
by Shlomi Zidmi (oVirt JIRA) 23 Jul '20

23 Jul '20

[ https://ovirt-jira.atlassian.net/browse/OVIRT-2977?page=com.atlassian.jira.… ] Shlomi Zidmi closed OVIRT-2977. ------------------------------- Resolution: Fixed > [FIRING:1] TestAlert > --------------------- > > Key: OVIRT-2977 > URL: https://ovirt-jira.atlassian.net/browse/OVIRT-2977 > Project: oVirt - virtualization made easy > Issue Type: Bug > Reporter: Alertmanager_Bot > Assignee: infra > Labels: ALERT{alertname="TestAlert"} > > Labels: > - alertname = TestAlert > - label1 = value1 > Annotations: > Source: -- This message was sent by Atlassian Jira (v1001.0.0-SNAPSHOT#100134)

1 0

[JIRA] (OVIRT-2979) a new project on jira for jiralert test tickets
by Shlomi Zidmi (oVirt JIRA) 23 Jul '20

23 Jul '20

Shlomi Zidmi created OVIRT-2979: ----------------------------------- Summary: a new project on jira for jiralert test tickets Key: OVIRT-2979 URL: https://ovirt-jira.atlassian.net/browse/OVIRT-2979 Project: oVirt - virtualization made easy Issue Type: New Feature Reporter: Shlomi Zidmi Assignee: infra Priority: Low This ticket is for creating a dummy project on Jira that could be linked with Jiralert. Such a project would be helpful in testing different use cases. -- This message was sent by Atlassian Jira (v1001.0.0-SNAPSHOT#100134)

1 0

[JIRA] (OVIRT-2978) [FIRING:1] InstanceUnreachable (amd64 linux shift-n12.phx.ovirt.org kubernetes-nodes-exporter shift-n12.phx.ovirt.org true primary vm ci)
by Alertmanager_Bot (oVirt JIRA) 23 Jul '20

23 Jul '20

Alertmanager_Bot created OVIRT-2978: --------------------------------------- Summary: [FIRING:1] InstanceUnreachable (amd64 linux shift-n12.phx.ovirt.org kubernetes-nodes-exporter shift-n12.phx.ovirt.org true primary vm ci) Key: OVIRT-2978 URL: https://ovirt-jira.atlassian.net/browse/OVIRT-2978 Project: oVirt - virtualization made easy Issue Type: Bug Reporter: Alertmanager_Bot Assignee: infra Labels: - alertname = InstanceUnreachable - beta_kubernetes_io_arch = amd64 - beta_kubernetes_io_os = linux - instance = shift-n12.phx.ovirt.org - job = kubernetes-nodes-exporter - kubernetes_io_hostname = shift-n12.phx.ovirt.org - node_role_kubernetes_io_compute = true - region = primary - type = vm - zone = ci Annotations: - description = shift-n12.phx.ovirt.org of job kubernetes-nodes-exporter has been possibly down for more than 10 minutes. Source: http://prometheus-0:9090/graph?g0.expr=up%7Bjob%3D%22kubernetes-nodes-expor… -- This message was sent by Atlassian Jira (v1001.0.0-SNAPSHOT#100134)

1 0

[JIRA] (OVIRT-2977) [FIRING:1] TestAlert
by Shlomi Zidmi (oVirt JIRA) 23 Jul '20

23 Jul '20

[ https://ovirt-jira.atlassian.net/browse/OVIRT-2977?page=com.atlassian.jira.… ] Shlomi Zidmi commented on OVIRT-2977: ------------------------------------- This ticket is for examining how Jiralert acts when there is already an open ticket with the same alert, but different labels and values > [FIRING:1] TestAlert > --------------------- > > Key: OVIRT-2977 > URL: https://ovirt-jira.atlassian.net/browse/OVIRT-2977 > Project: oVirt - virtualization made easy > Issue Type: Bug > Reporter: Alertmanager_Bot > Assignee: infra > Labels: ALERT{alertname="TestAlert"} > > Labels: > - alertname = TestAlert > - label1 = value1 > Annotations: > Source: -- This message was sent by Atlassian Jira (v1001.0.0-SNAPSHOT#100134)

1 0

[JIRA] (OVIRT-2977) [FIRING:1] TestAlert
by Shlomi Zidmi (oVirt JIRA) 23 Jul '20

23 Jul '20

[ https://ovirt-jira.atlassian.net/browse/OVIRT-2977?page=com.atlassian.jira.… ] Shlomi Zidmi updated OVIRT-2977: -------------------------------- Status: In Progress (was: To Do) > [FIRING:1] TestAlert > --------------------- > > Key: OVIRT-2977 > URL: https://ovirt-jira.atlassian.net/browse/OVIRT-2977 > Project: oVirt - virtualization made easy > Issue Type: Bug > Reporter: Alertmanager_Bot > Assignee: infra > Labels: ALERT{alertname="TestAlert"} > > Labels: > - alertname = TestAlert > - label1 = value1 > Annotations: > Source: -- This message was sent by Atlassian Jira (v1001.0.0-SNAPSHOT#100134)

1 0

[JIRA] (OVIRT-2977) [FIRING:1] TestAlert
by Alertmanager_Bot (oVirt JIRA) 23 Jul '20

23 Jul '20

Alertmanager_Bot created OVIRT-2977: --------------------------------------- Summary: [FIRING:1] TestAlert Key: OVIRT-2977 URL: https://ovirt-jira.atlassian.net/browse/OVIRT-2977 Project: oVirt - virtualization made easy Issue Type: Bug Reporter: Alertmanager_Bot Assignee: infra Labels: - alertname = TestAlert - label1 = value1 Annotations: Source: -- This message was sent by Atlassian Jira (v1001.0.0-SNAPSHOT#100134)

1 0

[oVirt Jenkins] ovirt-system-tests_basic-suite-4.3_nightly - Build # 211 - Failure!
by jenkins＠jenkins.phx.ovirt.org 23 Jul '20

23 Jul '20

Project: https://jenkins.ovirt.org/job/ovirt-system-tests_basic-suite-4.3_nightly/ Build: https://jenkins.ovirt.org/job/ovirt-system-tests_basic-suite-4.3_nightly/21… Build Number: 211 Build Status: Failure Triggered By: Started by timer ------------------------------------- Changes Since Last Success: ------------------------------------- Changes for Build #211 [Marcin Sobczyk] lago: Skip sysprep when ssh key is available ----------------- Failed Tests: ----------------- 1 tests failed. FAILED: 004_basic_sanity.disk_operations Error Message: False != True after 600 seconds -------------------- >> begin captured logging << -------------------- lago.utils: DEBUG: Error while running thread Thread-70 Traceback (most recent call last): File "/usr/lib/python2.7/site-packages/lago/utils.py", line 58, in _ret_via_queue queue.put({'return': func()}) File "/usr/lib/python2.7/site-packages/ovirtlago/testlib.py", line 60, in wrapper return func(get_test_prefix(), *args, **kwargs) File "/usr/lib/python2.7/site-packages/ovirtlago/testlib.py", line 79, in wrapper prefix.virt_env.engine_vm().get_api(api_ver=4), *args, **kwargs File "/home/jenkins/agent/workspace/ovirt-system-tests_basic-suite-4.3_nightly/ovirt-system-tests/basic-suite-4.3/test-scenarios/004_basic_sanity.py", line 508, in live_storage_migration testlib.assert_true_within_long(lambda: test_utils.all_jobs_finished(engine, correlation_id)) File "/usr/lib/python2.7/site-packages/ovirtlago/testlib.py", line 286, in assert_true_within_long assert_equals_within_long(func, True, allowed_exceptions) File "/usr/lib/python2.7/site-packages/ovirtlago/testlib.py", line 273, in assert_equals_within_long func, value, LONG_TIMEOUT, allowed_exceptions=allowed_exceptions File "/usr/lib/python2.7/site-packages/ovirtlago/testlib.py", line 252, in assert_equals_within '%s != %s after %s seconds' % (res, value, timeout) AssertionError: False != True after 600 seconds --------------------- >> end captured logging << --------------------- Stack Trace: File "/usr/lib64/python2.7/unittest/case.py", line 369, in run testMethod() File "/usr/lib/python2.7/site-packages/nose/case.py", line 197, in runTest self.test(*self.arg) File "/usr/lib/python2.7/site-packages/ovirtlago/testlib.py", line 142, in wrapped_test test() File "/usr/lib/python2.7/site-packages/ovirtlago/testlib.py", line 60, in wrapper return func(get_test_prefix(), *args, **kwargs) File "/usr/lib/python2.7/site-packages/ovirtlago/testlib.py", line 79, in wrapper prefix.virt_env.engine_vm().get_api(api_ver=4), *args, **kwargs File "/home/jenkins/agent/workspace/ovirt-system-tests_basic-suite-4.3_nightly/ovirt-system-tests/basic-suite-4.3/test-scenarios/004_basic_sanity.py", line 932, in disk_operations vt.join_all() File "/usr/lib/python2.7/site-packages/lago/utils.py", line 58, in _ret_via_queue queue.put({'return': func()}) File "/usr/lib/python2.7/site-packages/ovirtlago/testlib.py", line 60, in wrapper return func(get_test_prefix(), *args, **kwargs) File "/usr/lib/python2.7/site-packages/ovirtlago/testlib.py", line 79, in wrapper prefix.virt_env.engine_vm().get_api(api_ver=4), *args, **kwargs File "/home/jenkins/agent/workspace/ovirt-system-tests_basic-suite-4.3_nightly/ovirt-system-tests/basic-suite-4.3/test-scenarios/004_basic_sanity.py", line 508, in live_storage_migration testlib.assert_true_within_long(lambda: test_utils.all_jobs_finished(engine, correlation_id)) File "/usr/lib/python2.7/site-packages/ovirtlago/testlib.py", line 286, in assert_true_within_long assert_equals_within_long(func, True, allowed_exceptions) File "/usr/lib/python2.7/site-packages/ovirtlago/testlib.py", line 273, in assert_equals_within_long func, value, LONG_TIMEOUT, allowed_exceptions=allowed_exceptions File "/usr/lib/python2.7/site-packages/ovirtlago/testlib.py", line 252, in assert_equals_within '%s != %s after %s seconds' % (res, value, timeout) 'False != True after 600 seconds\n-------------------- >> begin captured logging << --------------------\nlago.utils: DEBUG: Error while running thread Thread-70\nTraceback (most recent call last):\n File "/usr/lib/python2.7/site-packages/lago/utils.py", line 58, in _ret_via_queue\n queue.put({\'return\': func()})\n File "/usr/lib/python2.7/site-packages/ovirtlago/testlib.py", line 60, in wrapper\n return func(get_test_prefix(), *args, **kwargs)\n File "/usr/lib/python2.7/site-packages/ovirtlago/testlib.py", line 79, in wrapper\n prefix.virt_env.engine_vm().get_api(api_ver=4), *args, **kwargs\n File "/home/jenkins/agent/workspace/ovirt-system-tests_basic-suite-4.3_nightly/ovirt-system-tests/basic-suite-4.3/test-scenarios/004_basic_sanity.py", line 508, in live_storage_migration\n testlib.assert_true_within_long(lambda: test_utils.all_jobs_finished(engine, correlation_id))\n File "/usr/lib/python2.7/site-packages/ovirtlago/testlib.py", line 286, in assert_true_within_long\n assert_equals_within_long(func, True, allowed_exceptions)\n File "/usr/lib/python2.7/site-packages/ovirtlago/testlib.py", line 273, in assert_equals_within_long\n func, value, LONG_TIMEOUT, allowed_exceptions=allowed_exceptions\n File "/usr/lib/python2.7/site-packages/ovirtlago/testlib.py", line 252, in assert_equals_within\n \'%s != %s after %s seconds\' % (res, value, timeout)\nAssertionError: False != True after 600 seconds\n--------------------- >> end captured logging << ---------------------'

1 1

oVirt infra daily report - unstable production jobs - 1260
by jenkins＠jenkins.phx.ovirt.org 23 Jul '20

23 Jul '20

Good morning! Attached is the HTML page with the jenkins status report. You can see it also here: - https://jenkins.ovirt.org/job/system_jenkins-report/1260//artifact/exported… Cheers, Jenkins

1 0

[JIRA] (OVIRT-2976) [FIRING:1] MirrorIsOutOfSync (web-ovirt-mirrorchecker.apps.ovirt.org:80 oVirt-mirrors-health http://www.gtlib.gatech.edu/pub/oVirt/pub)
by Alertmanager_Bot (oVirt JIRA) 22 Jul '20

22 Jul '20

Alertmanager_Bot created OVIRT-2976: --------------------------------------- Summary: [FIRING:1] MirrorIsOutOfSync (web-ovirt-mirrorchecker.apps.ovirt.org:80 oVirt-mirrors-health http://www.gtlib.gatech.edu/pub/oVirt/pub) Key: OVIRT-2976 URL: https://ovirt-jira.atlassian.net/browse/OVIRT-2976 Project: oVirt - virtualization made easy Issue Type: Bug Reporter: Alertmanager_Bot Assignee: infra Labels: - alertname = MirrorIsOutOfSync - instance = web-ovirt-mirrorchecker.apps.ovirt.org:80 - job = oVirt-mirrors-health - url = http://www.gtlib.gatech.edu/pub/oVirt/pub Annotations: - description = http://www.gtlib.gatech.edu/pub/oVirt/pub (measured by oVirt-mirrors-health) has not been synced in the past two days. Source: http://prometheus-0:9090/graph?g0.expr=mirror_health_check+%3E+172800&g0.ta… -- This message was sent by Atlassian Jira (v1001.0.0-SNAPSHOT#100133)

1 0

[JIRA] (OVIRT-2268) Enable IPv6 for resources.ovirt.org
by Anton Marchukov (oVirt JIRA) 22 Jul '20

22 Jul '20

[ https://ovirt-jira.atlassian.net/browse/OVIRT-2268?page=com.atlassian.jira.… ] Anton Marchukov commented on OVIRT-2268: ---------------------------------------- Since we cannot provide IPv6 on [resources.ovirt.org|http://resources.ovirt.org] let’s redefine this ticket to make sure IPv6-enabled users will get mirrorlist wit h IPv6-enables mirrors only. The target is to make sure IPv6-only people can get oVirt. > Enable IPv6 for resources.ovirt.org > ----------------------------------- > > Key: OVIRT-2268 > URL: https://ovirt-jira.atlassian.net/browse/OVIRT-2268 > Project: oVirt - virtualization made easy > Issue Type: Improvement > Components: oVirt Infra > Reporter: Dominik Holler > Assignee: infra > > To allow access to resources.ovirt.org in IPv6 only setups, IPv6 should be enabled for resources.ovirt.org. > This would allow users in IPv6 only setups to install oVirt. -- This message was sent by Atlassian Jira (v1001.0.0-SNAPSHOT#100133)

1 0

[JIRA] (OVIRT-2975) upload CirrOS 0.5.1 to glance
by Evgheni Dereveanchin (oVirt JIRA) 22 Jul '20

22 Jul '20

Evgheni Dereveanchin created OVIRT-2975: ------------------------------------------- Summary: upload CirrOS 0.5.1 to glance Key: OVIRT-2975 URL: https://ovirt-jira.atlassian.net/browse/OVIRT-2975 Project: oVirt - virtualization made easy Issue Type: Bug Reporter: Evgheni Dereveanchin Assignee: infra http://download.cirros-cloud.net/0.5.1/cirros-0.5.1-x86_64-disk.img -- This message was sent by Atlassian Jira (v1001.0.0-SNAPSHOT#100133)

1 0

Re: [ovirt-users] invalid spf record for ovirt.org
by Yedidyah Bar David 22 Jul '20

22 Jul '20

On Wed, Jul 22, 2020 at 3:57 PM Jorick Astrego <jorick(a)netbulae.eu> wrote: > Hi, > > During routine maintenance on our mailserver I noticed the following in > the log: > > [22/Jul/2020 14:33:33] Error when parsing SPF TXT record for domain: > ovirt.org, envelope-from=users-bounces(a)ovirt.org, message: Invalid > character found near "" in "66.187.233.88" > > A check on MXtoolbox also gives an invalid systax error: > > v=spf1 a:mail.ovirt.org a:gerrit.ovirt.org 66.187.233.88 ~all > > Prefix Type Value PrefixDesc Description Error > > v spf1 > The SPF record version > + a mail.ovirt.org Pass Match if IP has a DNS 'A' record in given domain. > + a gerrit.ovirt.org Pass Match if IP has a DNS 'A' record in given > domain. > + 66.187.233.88 > Pass Unknown Unknown mechanisms are not allowed > ~ all > SoftFail Always matches. It goes at the end of your record. > > > Test Result > > <https://mxtoolbox.com/problem/spf/spf-syntax-check?page=prob_spf&action=spf…> SPF > Syntax Check Invalid syntax found > > Thanks for the report. Adding some more relevant people. Best regards, -- Didi

1 0

[oVirt Jenkins] ovirt-system-tests_basic-suite-master_nightly - Build # 208 - Failure!
by jenkins＠jenkins.phx.ovirt.org 22 Jul '20

22 Jul '20

Project: https://jenkins.ovirt.org/job/ovirt-system-tests_basic-suite-master_nightly/ Build: https://jenkins.ovirt.org/job/ovirt-system-tests_basic-suite-master_nightly… Build Number: 208 Build Status: Failure Triggered By: Started by timer ------------------------------------- Changes Since Last Success: ------------------------------------- Changes for Build #208 [Marcin Sobczyk] performance: Fix broken links for pgdg repos ----------------- Failed Tests: ----------------- 1 tests failed. FAILED: basic-suite-master.test-scenarios.004_basic_sanity_pytest.test_live_storage_migration Error Message: AssertionError: False != True after 600 seconds Stack Trace: api_v4 = <ovirtsdk4.Connection object at 0x7f86a012a050> @order_by(_TEST_LIST) def test_live_storage_migration(api_v4): engine = api_v4.system_service() disk_service = test_utils.get_disk_service(engine, DISK0_NAME) correlation_id = 'live_storage_migration' disk_service.move( async=False, filter=False, storage_domain=types.StorageDomain( name=SD_ISCSI_NAME ), query={'correlation_id': correlation_id} ) testlib.assert_true_within_long(lambda: test_utils.all_jobs_finished(engine, correlation_id)) # Assert that the disk is on the correct storage domain, # its status is OK and the snapshot created for the migration # has been merged testlib.assert_true_within_long( lambda: api_v4.follow_link(disk_service.get().storage_domains[0]).name == SD_ISCSI_NAME ) vm0_snapshots_service = test_utils.get_vm_snapshots_service(engine, VM0_NAME) testlib.assert_true_within_long( > lambda: len(vm0_snapshots_service.list()) == 1 ) ../basic-suite-master/test-scenarios/004_basic_sanity_pytest.py:576: _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ /usr/lib/python2.7/site-packages/ovirtlago/testlib.py:286: in assert_true_within_long assert_equals_within_long(func, True, allowed_exceptions) /usr/lib/python2.7/site-packages/ovirtlago/testlib.py:273: in assert_equals_within_long func, value, LONG_TIMEOUT, allowed_exceptions=allowed_exceptions _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ func = <function <lambda> at 0x7f869c96daa0>, value = True, timeout = 600 allowed_exceptions = [], initial_wait = 0 def assert_equals_within( func, value, timeout, allowed_exceptions=None, initial_wait=10 ): allowed_exceptions = allowed_exceptions or [] with utils.EggTimer(timeout) as timer: while not timer.elapsed(): try: res = func() if res == value: return except Exception as exc: if _instance_of_any(exc, allowed_exceptions): time.sleep(3) continue LOGGER.exception("Unhandled exception in %s", func) raise if initial_wait == 0: time.sleep(3) else: time.sleep(initial_wait) initial_wait = 0 try: raise AssertionError( > '%s != %s after %s seconds' % (res, value, timeout) ) E AssertionError: False != True after 600 seconds /usr/lib/python2.7/site-packages/ovirtlago/testlib.py:252: AssertionError

1 1

oVirt infra daily report - unstable production jobs - 1259
by jenkins＠jenkins.phx.ovirt.org 22 Jul '20

22 Jul '20

Good morning! Attached is the HTML page with the jenkins status report. You can see it also here: - https://jenkins.ovirt.org/job/system_jenkins-report/1259//artifact/exported… Cheers, Jenkins

1 0

Build failed in Jenkins: system-sync_mirrors-fedora-updates-fc30-x86_64 #1319
by jenkins＠jenkins.phx.ovirt.org 22 Jul '20

22 Jul '20

See <https://jenkins.ovirt.org/job/system-sync_mirrors-fedora-updates-fc30-x86_6…> Changes: ------------------------------------------ Started by timer Running as SYSTEM [EnvInject] - Loading node environment variables. Building remotely on mirrors.phx.ovirt.org (mirrors) in workspace <https://jenkins.ovirt.org/job/system-sync_mirrors-fedora-updates-fc30-x86_6…> No credentials specified > git rev-parse --is-inside-work-tree # timeout=10 Fetching changes from the remote Git repository > git config remote.origin.url http://gerrit.ovirt.org/jenkins.git # timeout=10 Cleaning workspace > git rev-parse --verify HEAD # timeout=10 Resetting working tree > git reset --hard # timeout=10 > git clean -fdx # timeout=10 Pruning obsolete local branches Fetching upstream changes from http://gerrit.ovirt.org/jenkins.git > git --version # timeout=10 > git fetch --tags --progress --prune http://gerrit.ovirt.org/jenkins.git +refs/heads/*:refs/remotes/origin/* # timeout=10 > git rev-parse origin/master^{commit} # timeout=10 Checking out Revision e74e3d234ca539ec7d6be740575b720d5577e5a9 (origin/master) > git config core.sparsecheckout # timeout=10 > git checkout -f e74e3d234ca539ec7d6be740575b720d5577e5a9 # timeout=10 Commit message: "Removing rgolan(a)redhat.com from email notifs" > git rev-list --no-walk e74e3d234ca539ec7d6be740575b720d5577e5a9 # timeout=10 [system-sync_mirrors-fedora-updates-fc30-x86_64] $ /bin/bash -xe /tmp/jenkins141163348633797447.sh + jenkins/scripts/mirror_mgr.sh resync_yum_mirror fedora-updates-fc30 x86_64 jenkins/data/mirrors-reposync.conf + MIRRORS_MP_BASE=/var/www/html/repos + MIRRORS_HTTP_BASE=http://mirrors.phx.ovirt.org/repos + MIRRORS_CACHE=/home/jenkins/mirrors_cache + YUM_PATH=/yum/ + MIRRORS_BASE_PREFIX=/var/www/html/repos/yum/ + REPOQUERY_HTTP_BASE=http://mirrors.phx.ovirt.org/repos/yum/ + MAX_LOCK_ATTEMPTS=120 + LOCK_WAIT_INTERVAL=5 + LOCK_BASE=/home/jenkins + OLD_MD_TO_KEEP=100 + MAX_AGE=14 + HTTP_SELINUX_TYPE=httpd_sys_content_t + HTTP_FILE_MODE=644 + main resync_yum_mirror fedora-updates-fc30 x86_64 jenkins/data/mirrors-reposync.conf + local command=resync_yum_mirror + command_args=("${@:2}") + local command_args + cmd_resync_yum_mirror fedora-updates-fc30 x86_64 jenkins/data/mirrors-reposync.conf + local repo_name=fedora-updates-fc30 + local repo_archs=x86_64 + local reposync_conf=jenkins/data/mirrors-reposync.conf + local sync_needed + mkdir -p /home/jenkins/mirrors_cache + verify_repo_fs fedora-updates-fc30 yum + local repo_name=fedora-updates-fc30 + local repo_type=yum + sudo install -o jenkins -d /var/www/html/repos/yum /var/www/html/repos/yum/fedora-updates-fc30 /var/www/html/repos/yum/fedora-updates-fc30/base + modify_reposync_conf fedora-updates-fc30 jenkins/data/mirrors-reposync.conf + local repo_name=fedora-updates-fc30 + local reposync_conf=jenkins/data/mirrors-reposync.conf + local tmp_file=reposync.tmp + awk -v RS= -v 'ORS=\n\n' /main/ jenkins/data/mirrors-reposync.conf + awk -v RS= -v 'ORS=\n\n' /fedora-updates-fc30/ jenkins/data/mirrors-reposync.conf + mv reposync.tmp jenkins/data/mirrors-reposync.conf + check_yum_sync_needed fedora-updates-fc30 x86_64 jenkins/data/mirrors-reposync.conf sync_needed + local repo_name=fedora-updates-fc30 + local repo_archs=x86_64 + local reposync_conf=jenkins/data/mirrors-reposync.conf + local p_sync_needed=sync_needed + local reposync_out + echo 'Checking if mirror needs a resync' Checking if mirror needs a resync + rm -rf /home/jenkins/mirrors_cache/fedora-updates-fc30 ++ IFS=, ++ echo x86_64 + for arch in '$(IFS=,; echo $repo_archs)' ++ run_reposync fedora-updates-fc30 x86_64 jenkins/data/mirrors-reposync.conf --urls --quiet ++ local repo_name=fedora-updates-fc30 ++ local repo_arch=x86_64 ++ local reposync_conf=jenkins/data/mirrors-reposync.conf ++ extra_args=("${@:4}") ++ local extra_args ++ reposync --config=jenkins/data/mirrors-reposync.conf --repoid=fedora-updates-fc30 --arch=x86_64 --cachedir=/home/jenkins/mirrors_cache --download_path=/var/www/html/repos/yum/fedora-updates-fc30/base --norepopath --urls --quiet Error setting up repositories: Cannot retrieve metalink for repository: fedora-updates-fc30. Please verify its path and try again + reposync_out= Build step 'Execute shell' marked build as failure

1 1

[JIRA] (OVIRT-2974) Tickets aren't created for fired alerts in Prometheus
by Shlomi Zidmi (oVirt JIRA) 21 Jul '20

21 Jul '20

Shlomi Zidmi created OVIRT-2974: ----------------------------------- Summary: Tickets aren't created for fired alerts in Prometheus Key: OVIRT-2974 URL: https://ovirt-jira.atlassian.net/browse/OVIRT-2974 Project: oVirt - virtualization made easy Issue Type: Bug Reporter: Shlomi Zidmi Assignee: infra There is a broken node_exporter on one of openshift nodes (shift-n12), however we haven't been notified about it with a ticket. For some reason, even though all application are running (Prometheus, Alertmanager, Jiralert) no ticket has been created for the broken exporter. Opening a ticket for investigating the issue -- This message was sent by Atlassian Jira (v1001.0.0-SNAPSHOT#100133)

1 0

[oVirt Jenkins] ovirt-system-tests_performance-suite-master - Build # 1521 - Still Failing!
by jenkins＠jenkins.phx.ovirt.org 21 Jul '20

21 Jul '20

Project: https://jenkins.ovirt.org/job/ovirt-system-tests_performance-suite-master/ Build: https://jenkins.ovirt.org/job/ovirt-system-tests_performance-suite-master/1… Build Number: 1521 Build Status: Still Failing Triggered By: Started by timer ------------------------------------- Changes Since Last Success: ------------------------------------- Changes for Build #1516 [Marcin Sobczyk] network: Run network suite master on el8 Changes for Build #1517 [Andrej Cernek] network, hostlib: refactor setup_networks method [Barak Korren] usrc: Support *.yml files Changes for Build #1518 [Galit] Remove basic-suite-4.2 suite Changes for Build #1519 [Marcin Sobczyk] basic: Verify glance template creation before tearing down the engine Changes for Build #1520 [Marcin Sobczyk] ost-images: Use OST images when available Changes for Build #1521 [Ehud Yonasi] fix el7 container label. [Roy Golan] Removing rgolan(a)redhat.com from email notifs ----------------- Failed Tests: ----------------- 1 tests failed. FAILED: 1000_analyze_postgresql.analyze_postgres Error Message: Failed to perform the postgres analysis. Exit code is 1 -------------------- >> begin captured logging << -------------------- lago.plugins.vm: INFO: * Copy /home/jenkins/agent/workspace/ovirt-system-tests_performance-suite-master/ovirt-system-tests/performance-suite-master/../common/test-scenarios-files/analyze_postgresql.sh to lago-performance-suite-master-engine:/root: ?[0m?[0m lago.ssh: DEBUG: start task:a9b6bf76-d6a4-4f6d-938d-59d14cc805d8:Get ssh client for lago-performance-suite-master-engine: lago.ssh: DEBUG: end task:a9b6bf76-d6a4-4f6d-938d-59d14cc805d8:Get ssh client for lago-performance-suite-master-engine: lago.plugins.vm: INFO: * Copy /home/jenkins/agent/workspace/ovirt-system-tests_performance-suite-master/ovirt-system-tests/performance-suite-master/../common/test-scenarios-files/analyze_postgresql.sh to lago-performance-suite-master-engine:/root: ?[32mSuccess?[0m (in 0:00:00) lago.ssh: DEBUG: start task:3cbcc980-918f-4be9-ac5e-e10f86305521:Get ssh client for lago-performance-suite-master-engine: lago.ssh: DEBUG: end task:3cbcc980-918f-4be9-ac5e-e10f86305521:Get ssh client for lago-performance-suite-master-engine: lago.ssh: DEBUG: Running aee507f2 on lago-performance-suite-master-engine: /root/analyze_postgresql.sh lago.ssh: DEBUG: Command aee507f2 on lago-performance-suite-master-engine returned with 1 lago.ssh: DEBUG: Command aee507f2 on lago-performance-suite-master-engine output: Last metadata expiration check: 0:11:06 ago on Thu 16 Jul 2020 11:03:48 PM EDT. [MIRROR] pgdg-redhat-repo-latest.noarch.rpm: Status code: 404 for https://download.postgresql.org/pub/repos/yum/12/redhat/rhel-8-x86_64/pgdg-… (IP: 72.32.157.246) [MIRROR] pgdg-redhat-repo-latest.noarch.rpm: Status code: 404 for https://download.postgresql.org/pub/repos/yum/12/redhat/rhel-8-x86_64/pgdg-… (IP: 72.32.157.246) [MIRROR] pgdg-redhat-repo-latest.noarch.rpm: Status code: 404 for https://download.postgresql.org/pub/repos/yum/12/redhat/rhel-8-x86_64/pgdg-… (IP: 72.32.157.246) [MIRROR] pgdg-redhat-repo-latest.noarch.rpm: Status code: 404 for https://download.postgresql.org/pub/repos/yum/12/redhat/rhel-8-x86_64/pgdg-… (IP: 72.32.157.246) [FAILED] pgdg-redhat-repo-latest.noarch.rpm: Status code: 404 for https://download.postgresql.org/pub/repos/yum/12/redhat/rhel-8-x86_64/pgdg-… (IP: 72.32.157.246) lago.ssh: DEBUG: Command aee507f2 on lago-performance-suite-master-engine errors: + pg_datadir=/var/lib/pgsql/data + pg_service=postgresql.service + pgdg= + scl_enable= ++ get_engine_pg_scl ++ engine_pg_scl_conf=(/etc/ovirt-engine/engine.conf.d/*scl-postgres*.conf) ++ local engine_pg_scl_conf ++ local res= ++ [[ -e /etc/ovirt-engine/engine.conf.d/*scl-postgres*.conf ]] ++ echo '' + engine_pg_scl= + [[ -n '' ]] ++ rpm -q postgresql + pgver=postgresql-12.1-2.module_el8.1.0+273+979c16e6.x86_64 + case "${pgver}" in + pgdg=https://download.postgresql.org/pub/repos/yum/12/redhat/rhel-8-x86_64/… + [[ -n https://download.postgresql.org/pub/repos/yum/12/redhat/rhel-8-x86_64/pgdg-… ]] + yum install -y https://download.postgresql.org/pub/repos/yum/12/redhat/rhel-8-x86_64/pgdg-… Status code: 404 for https://download.postgresql.org/pub/repos/yum/12/redhat/rhel-8-x86_64/pgdg-… (IP: 72.32.157.246) --------------------- >> end captured logging << --------------------- Stack Trace: File "/usr/lib64/python2.7/unittest/case.py", line 369, in run testMethod() File "/usr/lib/python2.7/site-packages/nose/case.py", line 197, in runTest self.test(*self.arg) File "/usr/lib/python2.7/site-packages/ovirtlago/testlib.py", line 142, in wrapped_test test() File "/usr/lib/python2.7/site-packages/ovirtlago/testlib.py", line 60, in wrapper return func(get_test_prefix(), *args, **kwargs) File "/home/jenkins/agent/workspace/ovirt-system-tests_performance-suite-master/ovirt-system-tests/performance-suite-master/test-scenarios/1000_analyze_postgresql.py", line 52, in analyze_postgres ' Exit code is %s' % result.code File "/usr/lib/python2.7/site-packages/nose/tools/trivial.py", line 29, in eq_ raise AssertionError(msg or "%r != %r" % (a, b)) 'Failed to perform the postgres analysis. Exit code is 1\n-------------------- >> begin captured logging << --------------------\nlago.plugins.vm: INFO: * Copy /home/jenkins/agent/workspace/ovirt-system-tests_performance-suite-master/ovirt-system-tests/performance-suite-master/../common/test-scenarios-files/analyze_postgresql.sh to lago-performance-suite-master-engine:/root: \x1b[0m\x1b[0m\nlago.ssh: DEBUG: start task:a9b6bf76-d6a4-4f6d-938d-59d14cc805d8:Get ssh client for lago-performance-suite-master-engine:\nlago.ssh: DEBUG: end task:a9b6bf76-d6a4-4f6d-938d-59d14cc805d8:Get ssh client for lago-performance-suite-master-engine:\nlago.plugins.vm: INFO: * Copy /home/jenkins/agent/workspace/ovirt-system-tests_performance-suite-master/ovirt-system-tests/performance-suite-master/../common/test-scenarios-files/analyze_postgresql.sh to lago-performance-suite-master-engine:/root: \x1b[32mSuccess\x1b[0m (in 0:00:00)\nlago.ssh: DEBUG: start task:3cbcc980-918f-4be9-ac5e-e10f86305521:Get ssh client for lago-performance-suite-master-engine:\nlago.ssh: DEBUG: end task:3cbcc980-918f-4be9-ac5e-e10f86305521:Get ssh client for lago-performance-suite-master-engine:\nlago.ssh: DEBUG: Running aee507f2 on lago-performance-suite-master-engine: /root/analyze_postgresql.sh\nlago.ssh: DEBUG: Command aee507f2 on lago-performance-suite-master-engine returned with 1\nlago.ssh: DEBUG: Command aee507f2 on lago-performance-suite-master-engine output:\n Last metadata expiration check: 0:11:06 ago on Thu 16 Jul 2020 11:03:48 PM EDT.\n[MIRROR] pgdg-redhat-repo-latest.noarch.rpm: Status code: 404 for https://download.postgresql.org/pub/repos/yum/12/redhat/rhel-8-x86_64/pgdg-… (IP: 72.32.157.246)\n[MIRROR] pgdg-redhat-repo-latest.noarch.rpm: Status code: 404 for https://download.postgresql.org/pub/repos/yum/12/redhat/rhel-8-x86_64/pgdg-… (IP: 72.32.157.246)\n[MIRROR] pgdg-redhat-repo-latest.noarch.rpm: Status code: 404 for https://download.postgresql.org/pub/repos/yum/12/redhat/rhel-8-x86_64/pgdg-… (IP: 72.32.157.246)\n[MIRROR] pgdg-redhat-repo-latest.noarch.rpm: Status code: 404 for https://download.postgresql.org/pub/repos/yum/12/redhat/rhel-8-x86_64/pgdg-… (IP: 72.32.157.246)\n[FAILED] pgdg-redhat-repo-latest.noarch.rpm: Status code: 404 for https://download.postgresql.org/pub/repos/yum/12/redhat/rhel-8-x86_64/pgdg-… (IP: 72.32.157.246)\n\nlago.ssh: DEBUG: Command aee507f2 on lago-performance-suite-master-engine errors:\n + pg_datadir=/var/lib/pgsql/data\n+ pg_service=postgresql.service\n+ pgdg=\n+ scl_enable=\n++ get_engine_pg_scl\n++ engine_pg_scl_conf=(/etc/ovirt-engine/engine.conf.d/*scl-postgres*.conf)\n++ local engine_pg_scl_conf\n++ local res=\n++ [[ -e /etc/ovirt-engine/engine.conf.d/*scl-postgres*.conf ]]\n++ echo \'\'\n+ engine_pg_scl=\n+ [[ -n \'\' ]]\n++ rpm -q postgresql\n+ pgver=postgresql-12.1-2.module_el8.1.0+273+979c16e6.x86_64\n+ case "${pgver}" in\n+ pgdg=https://download.postgresql.org/pub/repos/yum/12/redhat/rhel-8-x86_64/… [[ -n https://download.postgresql.org/pub/repos/yum/12/redhat/rhel-8-x86_64/pgdg-… ]]\n+ yum install -y https://download.postgresql.org/pub/repos/yum/12/redhat/rhel-8-x86_64/pgdg-… code: 404 for https://download.postgresql.org/pub/repos/yum/12/redhat/rhel-8-x86_64/pgdg-… (IP: 72.32.157.246)\n\n--------------------- >> end captured logging << ---------------------'

1 4

oVirt infra daily report - unstable production jobs - 1258
by jenkins＠jenkins.phx.ovirt.org 21 Jul '20

21 Jul '20

Good morning! Attached is the HTML page with the jenkins status report. You can see it also here: - https://jenkins.ovirt.org/job/system_jenkins-report/1258//artifact/exported… Cheers, Jenkins

1 0

[JIRA] (OVIRT-2972) resources.ovirt.org ran out of space for artifact publishing
by Evgheni Dereveanchin (oVirt JIRA) 20 Jul '20

20 Jul '20

[ https://ovirt-jira.atlassian.net/browse/OVIRT-2972?page=com.atlassian.jira.… ] Evgheni Dereveanchin commented on OVIRT-2972: --------------------------------------------- Suspect jobs that were reporting issues this weekend: [https://jenkins.ovirt.org/job/ovirt_master_publish-rpms_nightly/|https://je…] [https://jenkins.ovirt.org/job/ovirt_4.3_publish-rpms_nightly/|https://jenki…] [https://jenkins.ovirt.org/job/ovirt_4.2_publish-rpms_nightly/|https://jenki…] The errors are as follows: {noformat}13:15:32 2020-07-19 11:15:32,180::INFO::root::Saving /home/jenkins/ovirt-master-snapshot.tmp/rpm/fc29/x86_64/ovirt-engine-wildfly-18.0.0-1.fc29.x86_64.rpm 13:15:32 Traceback (most recent call last): 13:15:32 File "/bin/repoman", line 10, in <module> 13:15:32 sys.exit(main()) 13:15:32 File "/usr/lib/python2.7/site-packages/repoman/cmd.py", line 461, in main 13:15:32 exit_code = do_add(args, config, repo) 13:15:32 File "/usr/lib/python2.7/site-packages/repoman/cmd.py", line 361, in do_add 13:15:32 repo.save() 13:15:32 File "/usr/lib/python2.7/site-packages/repoman/common/repo.py", line 42, in _func 13:15:32 return func(self, *args, **kwargs) 13:15:32 File "/usr/lib/python2.7/site-packages/repoman/common/repo.py", line 201, in save 13:15:32 store.save() 13:15:32 File "/usr/lib/python2.7/site-packages/repoman/common/stores/RPM/__init__.py", line 263, in save 13:15:32 self._save(**args) 13:15:32 File "/usr/lib/python2.7/site-packages/repoman/common/stores/RPM/__init__.py", line 302, in _save 13:15:32 save_file(pkg.path, dst_path) 13:15:32 File "/usr/lib/python2.7/site-packages/repoman/common/utils.py", line 436, in save_file 13:15:32 copy(src_path, dst_path) 13:15:32 File "/usr/lib/python2.7/site-packages/repoman/common/utils.py", line 296, in copy 13:15:32 shutil.copy2(what, where) 13:15:32 File "/usr/lib64/python2.7/shutil.py", line 130, in copy2 13:15:32 copyfile(src, dst) 13:15:32 File "/usr/lib64/python2.7/shutil.py", line 84, in copyfile 13:15:32 copyfileobj(fsrc, fdst) 13:15:32 File "/usr/lib64/python2.7/shutil.py", line 52, in copyfileobj 13:15:32 fdst.write(buf) 13:15:32 IOError: [Errno 28] No space left on device 13:15:32 2020-07-19 11:15:32,917::INFO::repoman.common.repo::Cleaning up temporary dir /tmp/tmpRKo543 13:15:34 Waiting for scan_for_artifacts.sh cron job to finish (attempt 1 of 50) {noformat} looks like the job ran out of tmpfs space - did we publish too many files for some reason? Not entirely sure what tmp is used for in these jobs. > resources.ovirt.org ran out of space for artifact publishing > ------------------------------------------------------------ > > Key: OVIRT-2972 > URL: https://ovirt-jira.atlassian.net/browse/OVIRT-2972 > Project: oVirt - virtualization made easy > Issue Type: Bug > Reporter: Evgheni Dereveanchin > Assignee: infra > Attachments: image-20200720-083159.png, prom.png > > > This weekend Nagios sent disk space alerts for the /home/jenkins on resources.ovirt.org > AFAIR this is used as intermediate artifact storage for publishing and shouldn't fill up that much. Logging a ticket to investigate the root cause. > The partition is free again now so this is not blocking anything right now. -- This message was sent by Atlassian Jira (v1001.0.0-SNAPSHOT#100133)

1 0

[JIRA] (OVIRT-2972) resources.ovirt.org ran out of space for artifact publishing
by Evgheni Dereveanchin (oVirt JIRA) 20 Jul '20

20 Jul '20

[ https://ovirt-jira.atlassian.net/browse/OVIRT-2972?page=com.atlassian.jira.… ] Evgheni Dereveanchin commented on OVIRT-2972: --------------------------------------------- I did get emails from Nagios so we were alerted, just want to confirm the cause here as that’s something that doesn’t happen often: looks like some publishing job failed and did not clean up after itself which caused the next one to bring in even more files and fill up the partition. > resources.ovirt.org ran out of space for artifact publishing > ------------------------------------------------------------ > > Key: OVIRT-2972 > URL: https://ovirt-jira.atlassian.net/browse/OVIRT-2972 > Project: oVirt - virtualization made easy > Issue Type: Bug > Reporter: Evgheni Dereveanchin > Assignee: infra > Attachments: image-20200720-083159.png, prom.png > > > This weekend Nagios sent disk space alerts for the /home/jenkins on resources.ovirt.org > AFAIR this is used as intermediate artifact storage for publishing and shouldn't fill up that much. Logging a ticket to investigate the root cause. > The partition is free again now so this is not blocking anything right now. -- This message was sent by Atlassian Jira (v1001.0.0-SNAPSHOT#100133)

1 0