July 2020 - Infra - oVirt List Archives

[oVirt Jenkins] ovirt-system-tests_openshift-on-ovirt-suite-4.2 - Build # 682 - Still Failing!
by jenkins＠jenkins.phx.ovirt.org 06 Dec '20

06 Dec '20

Project: https://jenkins.ovirt.org/job/ovirt-system-tests_openshift-on-ovirt-suite-4… Build: https://jenkins.ovirt.org/job/ovirt-system-tests_openshift-on-ovirt-suite-4… Build Number: 682 Build Status: Still Failing Triggered By: Started by timer ------------------------------------- Changes Since Last Success: ------------------------------------- Changes for Build #676 [Marcin Sobczyk] network: Run network suite master on el8 Changes for Build #677 [Marcin Sobczyk] network: Run network suite master on el8 Changes for Build #678 [Andrej Cernek] network, hostlib: refactor setup_networks method [Barak Korren] usrc: Support *.yml files Changes for Build #679 [Galit] Remove basic-suite-4.2 suite Changes for Build #680 [Marcin Sobczyk] basic: Verify glance template creation before tearing down the engine Changes for Build #681 [Marcin Sobczyk] ost-images: Use OST images when available Changes for Build #682 [Ehud Yonasi] fix el7 container label. [Roy Golan] Removing rgolan(a)redhat.com from email notifs ----------------- Failed Tests: ----------------- No tests ran.

1 142

[oVirt Jenkins] ovirt-system-tests_compat-4.3-suite-master - Build # 394 - Failure!
by jenkins＠jenkins.phx.ovirt.org 06 Dec '20

06 Dec '20

Project: https://jenkins.ovirt.org/job/ovirt-system-tests_compat-4.3-suite-master/ Build: https://jenkins.ovirt.org/job/ovirt-system-tests_compat-4.3-suite-master/39… Build Number: 394 Build Status: Failure Triggered By: Started by timer ------------------------------------- Changes Since Last Success: ------------------------------------- Changes for Build #394 [Sandro Bonazzola] repos: fix Advanced Virtualization repos [Sandro Bonazzola] mirrors: drop gluster 5 mirroring ----------------- Failed Tests: ----------------- No tests ran.

1 251

[oVirt Jenkins] ovirt-system-tests_upgrade-from-release-suite-master_nightly - Build # 1 - Failure!
by jenkins＠jenkins.phx.ovirt.org 05 Nov '20

05 Nov '20

Project: https://jenkins.ovirt.org/job/ovirt-system-tests_upgrade-from-release-suite… Build: https://jenkins.ovirt.org/job/ovirt-system-tests_upgrade-from-release-suite… Build Number: 1 Build Status: Failure Triggered By: Started by timer ------------------------------------- Changes Since Last Success: ------------------------------------- ----------------- Failed Tests: ----------------- No tests ran.

1 483

[oVirt Jenkins] ovirt-system-tests_upgrade-from-prevrelease-suite-master_nightly - Build # 1 - Failure!
by jenkins＠jenkins.phx.ovirt.org 05 Nov '20

05 Nov '20

Project: https://jenkins.ovirt.org/job/ovirt-system-tests_upgrade-from-prevrelease-s… Build: https://jenkins.ovirt.org/job/ovirt-system-tests_upgrade-from-prevrelease-s… Build Number: 1 Build Status: Failure Triggered By: Started by timer ------------------------------------- Changes Since Last Success: ------------------------------------- ----------------- Failed Tests: ----------------- No tests ran.

1 483

[VDSM] Trouble merging patch, related to jenkins whitelist or new RHEL CI?
by Nir Soffer 03 Aug '20

03 Aug '20

I could not merge: https://gerrit.ovirt.org/c/109402/ Although it was approved and verified, and got +1 from Continuous Integration. There is a new "RHEL-Continuous-Integration", which does not run with this change, even when I trigger the tests manually with "ci test". I tried to add +1 for "RHEL-Continuous-Integration" but this is not possible, I see: RHEL-Continuous-Integration You don't have permission to edit this label. So finally I downloaded the patch and pushed it manually. I think this patch will fix the problem: https://gerrit.ovirt.org/#/c/110576/ But I need CI experts to review this. Since we have 50 projects that need this, this probably should be fixed elsewhere, and inherited by all projects. I know that Germano was not able to trigger tests because he was missing in the jenkins whitelist, but this was fixed last week. Germano, maybe just to check that everything works for you, you can post some trivial patch? Nir

4 9

oVirt infra daily report - unstable production jobs - 1269
by jenkins＠jenkins.phx.ovirt.org 01 Aug '20

01 Aug '20

Good morning! Attached is the HTML page with the jenkins status report. You can see it also here: - https://jenkins.ovirt.org/job/system_jenkins-report/1269//artifact/exported… Cheers, Jenkins

1 0

[oVirt Jenkins] ovirt-system-tests_basic-suite-4.3_nightly - Build # 233 - Failure!
by jenkins＠jenkins.phx.ovirt.org 31 Jul '20

31 Jul '20

Project: https://jenkins.ovirt.org/job/ovirt-system-tests_basic-suite-4.3_nightly/ Build: https://jenkins.ovirt.org/job/ovirt-system-tests_basic-suite-4.3_nightly/23… Build Number: 233 Build Status: Failure Triggered By: Started by timer ------------------------------------- Changes Since Last Success: ------------------------------------- Changes for Build #233 [Michal Skrivanek] verify hotplug memory and test unplug [Tristan Cacqueray] zuul-jobs-gate: repparent to base-gate-job [Ehud Yonasi] stdci_runner: added the rhel labels selection. ----------------- Failed Tests: ----------------- 1 tests failed. FAILED: 003_00_metrics_bootstrap.configure_vdsm_sos_plugin Error Message: Failed to copy files/directory /home/jenkins/agent/workspace/ovirt-system-tests_basic-suite-4.3_nightly/ovirt-system-tests/basic-suite-4.3/../common/test-scenarios-files/vdsm-sos-plugin [Errno 2] No such file or directory: '/home/jenkins/agent/workspace/ovirt-system-tests_basic-suite-4.3_nightly/ovirt-system-tests/basic-suite-4.3/../common/test-scenarios-files/vdsm-sos-plugin' -------------------- >> begin captured logging << -------------------- lago.plugins.vm: INFO: * Copy /home/jenkins/agent/workspace/ovirt-system-tests_basic-suite-4.3_nightly/ovirt-system-tests/basic-suite-4.3/../common/test-scenarios-files/vdsm-sos-plugin to lago-basic-suite-4-3-host-1:/tmp: ?[0m?[0m lago.ssh: DEBUG: start task:df0ccdbf-77a7-49e8-a3e3-46a2a949b9de:Get ssh client for lago-basic-suite-4-3-host-1: lago.ssh: DEBUG: end task:df0ccdbf-77a7-49e8-a3e3-46a2a949b9de:Get ssh client for lago-basic-suite-4-3-host-1: root: ERROR: * Copy /home/jenkins/agent/workspace/ovirt-system-tests_basic-suite-4.3_nightly/ovirt-system-tests/basic-suite-4.3/../common/test-scenarios-files/vdsm-sos-plugin to lago-basic-suite-4-3-host-1:/tmp: ?[31mERROR?[0m (in 0:00:01) lago.plugins.vm: DEBUG: File "/usr/lib/python2.7/site-packages/lago/plugins/vm.py", line 506, in copy_to raise LagoCopyFilesToVMError(local_path, str(err)) --------------------- >> end captured logging << --------------------- Stack Trace: File "/usr/lib64/python2.7/unittest/case.py", line 369, in run testMethod() File "/usr/lib/python2.7/site-packages/nose/case.py", line 197, in runTest self.test(*self.arg) File "/usr/lib/python2.7/site-packages/ovirtlago/testlib.py", line 142, in wrapped_test test() File "/usr/lib/python2.7/site-packages/ovirtlago/testlib.py", line 60, in wrapper return func(get_test_prefix(), *args, **kwargs) File "/home/jenkins/agent/workspace/ovirt-system-tests_basic-suite-4.3_nightly/ovirt-system-tests/basic-suite-4.3/test-scenarios/003_00_metrics_bootstrap.py", line 49, in configure_vdsm_sos_plugin '/tmp' File "/usr/lib/python2.7/site-packages/lago/plugins/vm.py", line 506, in copy_to raise LagoCopyFilesToVMError(local_path, str(err)) 'Failed to copy files/directory /home/jenkins/agent/workspace/ovirt-system-tests_basic-suite-4.3_nightly/ovirt-system-tests/basic-suite-4.3/../common/test-scenarios-files/vdsm-sos-plugin\n[Errno 2] No such file or directory: \'/home/jenkins/agent/workspace/ovirt-system-tests_basic-suite-4.3_nightly/ovirt-system-tests/basic-suite-4.3/../common/test-scenarios-files/vdsm-sos-plugin\'\n-------------------- >> begin captured logging << --------------------\nlago.plugins.vm: INFO: * Copy /home/jenkins/agent/workspace/ovirt-system-tests_basic-suite-4.3_nightly/ovirt-system-tests/basic-suite-4.3/../common/test-scenarios-files/vdsm-sos-plugin to lago-basic-suite-4-3-host-1:/tmp: \x1b[0m\x1b[0m\nlago.ssh: DEBUG: start task:df0ccdbf-77a7-49e8-a3e3-46a2a949b9de:Get ssh client for lago-basic-suite-4-3-host-1:\nlago.ssh: DEBUG: end task:df0ccdbf-77a7-49e8-a3e3-46a2a949b9de:Get ssh client for lago-basic-suite-4-3-host-1:\nroot: ERROR: * Copy /home/jenkins/agent/workspace/ovirt-system-tests_basic-suite-4.3_nightly/ovirt-system-tests/basic-suite-4.3/../common/test-scenarios-files/vdsm-sos-plugin to lago-basic-suite-4-3-host-1:/tmp: \x1b[31mERROR\x1b[0m (in 0:00:01)\nlago.plugins.vm: DEBUG: File "/usr/lib/python2.7/site-packages/lago/plugins/vm.py", line 506, in copy_to\n raise LagoCopyFilesToVMError(local_path, str(err))\n\n--------------------- >> end captured logging << ---------------------'

1 3

oVirt infra daily report - unstable production jobs - 1268
by jenkins＠jenkins.phx.ovirt.org 31 Jul '20

31 Jul '20

Good morning! Attached is the HTML page with the jenkins status report. You can see it also here: - https://jenkins.ovirt.org/job/system_jenkins-report/1268//artifact/exported… Cheers, Jenkins

1 0

[JIRA] (OVIRT-2992) [FIRING:1] TestAlert
by Alertmanager_Bot (oVirt JIRA) 30 Jul '20

30 Jul '20

Alertmanager_Bot created OVIRT-2992: --------------------------------------- Summary: [FIRING:1] TestAlert Key: OVIRT-2992 URL: https://ovirt-jira.atlassian.net/browse/OVIRT-2992 Project: oVirt - virtualization made easy Issue Type: Bug Reporter: Alertmanager_Bot Assignee: infra Labels: - alertname = TestAlert - label1 = value1 Annotations: Source: -- This message was sent by Atlassian Jira (v1001.0.0-SNAPSHOT#100140)

1 0

[CQ]: 110470,5 (ovirt-provider-ovn) failed "ovirt-4.3" system tests
by oVirt Jenkins 30 Jul '20

30 Jul '20

Change 110470,5 (ovirt-provider-ovn) is probably the reason behind recent system test failures in the "ovirt-4.3" change queue and needs to be fixed. This change had been removed from the testing queue. Artifacts build from this change will not be released until it is fixed. For further details about the change see: https://gerrit.ovirt.org/#/c/110470/5 For failed test results see: https://jenkins.ovirt.org/job/ovirt-4.3_change-queue-tester/3734/

1 0

[CQ]: 110470, 5 (ovirt-provider-ovn) failed "ovirt-master" system tests
by oVirt Jenkins 30 Jul '20

30 Jul '20

Change 110470,5 (ovirt-provider-ovn) is probably the reason behind recent system test failures in the "ovirt-master" change queue and needs to be fixed. This change had been removed from the testing queue. Artifacts build from this change will not be released until it is fixed. For further details about the change see: https://gerrit.ovirt.org/#/c/110470/5 For failed test results see: https://jenkins.ovirt.org/job/ovirt-master_change-queue-tester/25148/

1 0

[oVirt Jenkins] deploy-to-gated-ovirt-master - Build #432 - FAILURE!
by jenkins＠jenkins.phx.ovirt.org 30 Jul '20

30 Jul '20

Build: https://jenkins.ovirt.org/job/deploy-to-gated-ovirt-master/432/ Build Name: #432 Build Status: FAILURE Gerrit change: null - title: null - project: null - branch: null - author: null <null>

1 0

gerrit build whitelist
by Germano Veit Michel 30 Jul '20

30 Jul '20

Hi, Would it be possible to add me to the whitelist? I've got a build denied here: https://gerrit.ovirt.org/#/c/109402/ The account is germano(a)redhat.com Thanks, Germano

2 1

[oVirt Jenkins] cleanup-gated-ovirt-4.3 - Build #223 - FAILURE!
by jenkins＠jenkins.phx.ovirt.org 30 Jul '20

30 Jul '20

Build: https://jenkins.ovirt.org/job/cleanup-gated-ovirt-4.3/223/ Build Name: #223 Build Status: FAILURE Gerrit change: null - title: null - project: null - branch: null - author: null <null>

1 0

[oVirt Jenkins] ovirt-system-tests_basic-suite-master_nightly - Build # 233 - Failure!
by jenkins＠jenkins.phx.ovirt.org 30 Jul '20

30 Jul '20

Project: https://jenkins.ovirt.org/job/ovirt-system-tests_basic-suite-master_nightly/ Build: https://jenkins.ovirt.org/job/ovirt-system-tests_basic-suite-master_nightly… Build Number: 233 Build Status: Failure Triggered By: Started by timer ------------------------------------- Changes Since Last Success: ------------------------------------- Changes for Build #233 [Michal Skrivanek] verify hotplug memory and test unplug [Tristan Cacqueray] zuul-jobs-gate: repparent to base-gate-job [Ehud Yonasi] stdci_runner: added the rhel labels selection. ----------------- Failed Tests: ----------------- 1 tests failed. FAILED: basic-suite-master.test-scenarios.002_bootstrap_pytest.test_verify_add_hosts Error Message: RuntimeError: 1 hosts failed installation: lago-basic-suite-master-host-1: install_failed Stack Trace: engine_api = <ovirtsdk4.Connection object at 0x7f28704d2310> @order_by(_TEST_LIST) def test_verify_add_hosts(engine_api): hosts_service = engine_api.system_service().hosts_service() hosts_status = hosts_service.list(search='datacenter={}'.format(DC_NAME)) total_hosts = len(hosts_status) dump_hosts = _host_status_to_print(hosts_service, hosts_status) LOGGER.debug('Host status, verify_add_hosts:\n {}'.format(dump_hosts)) testlib.assert_true_within( lambda: _single_host_up(hosts_service, total_hosts), > timeout=constants.ADD_HOST_TIMEOUT ) ../basic-suite-master/test-scenarios/002_bootstrap_pytest.py:434: _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ /usr/lib/python2.7/site-packages/ovirtlago/testlib.py:278: in assert_true_within assert_equals_within(func, True, timeout, allowed_exceptions) /usr/lib/python2.7/site-packages/ovirtlago/testlib.py:234: in assert_equals_within res = func() ../basic-suite-master/test-scenarios/002_bootstrap_pytest.py:433: in <lambda> lambda: _single_host_up(hosts_service, total_hosts), ../basic-suite-master/test-scenarios/002_bootstrap_pytest.py:230: in _single_host_up _check_problematic_hosts(hosts_service) _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ hosts_service = <ovirtsdk4.services.HostsService object at 0x7f286c6994d0> def _check_problematic_hosts(hosts_service): problematic_hosts = hosts_service.list(search='datacenter={} AND status != installing and status != initializing and status != up'.format(DC_NAME)) if len(problematic_hosts): dump_hosts = '%s hosts failed installation:\n' % len(problematic_hosts) for host in problematic_hosts: host_service = hosts_service.host_service(host.id) dump_hosts += '%s: %s\n' % (host.name, host_service.get().status) > raise RuntimeError(dump_hosts) E RuntimeError: 1 hosts failed installation: E lago-basic-suite-master-host-1: install_failed ../basic-suite-master/test-scenarios/002_bootstrap_pytest.py:239: RuntimeError

1 1

oVirt infra daily report - unstable production jobs - 1267
by jenkins＠jenkins.phx.ovirt.org 30 Jul '20

30 Jul '20

Good morning! Attached is the HTML page with the jenkins status report. You can see it also here: - https://jenkins.ovirt.org/job/system_jenkins-report/1267//artifact/exported… Cheers, Jenkins

1 0

[JIRA] (OVIRT-2991) [FIRING:1] InstanceUnreachable ibm-srv05.ovirt.org kubernetes-nodes-exporter (amd64 linux ibm-srv05.ovirt.org true external bare-metal-external ci)
by Alertmanager_Bot (oVirt JIRA) 29 Jul '20

29 Jul '20

Alertmanager_Bot created OVIRT-2991: --------------------------------------- Summary: [FIRING:1] InstanceUnreachable ibm-srv05.ovirt.org kubernetes-nodes-exporter (amd64 linux ibm-srv05.ovirt.org true external bare-metal-external ci) Key: OVIRT-2991 URL: https://ovirt-jira.atlassian.net/browse/OVIRT-2991 Project: oVirt - virtualization made easy Issue Type: Bug Reporter: Alertmanager_Bot Assignee: infra Labels: - alertname = InstanceUnreachable - beta_kubernetes_io_arch = amd64 - beta_kubernetes_io_os = linux - instance = ibm-srv05.ovirt.org - job = kubernetes-nodes-exporter - kubernetes_io_hostname = ibm-srv05.ovirt.org - node_role_kubernetes_io_compute = true - region = external - type = bare-metal-external - zone = ci Annotations: - description = ibm-srv05.ovirt.org of job kubernetes-nodes-exporter has been possibly down for more than 10 minutes. Source: http://prometheus-0:9090/graph?g0.expr=up%7Bjob%3D%22kubernetes-nodes-expor… -- This message was sent by Atlassian Jira (v1001.0.0-SNAPSHOT#100140)

1 0

[JIRA] (OVIRT-2988) [FIRING:1] InstanceUnreachable ibm-srv02.ovirt.org kubernetes-nodes-exporter (amd64 linux ibm-srv02.ovirt.org true external bare-metal-external ci)
by Alertmanager_Bot (oVirt JIRA) 29 Jul '20

29 Jul '20

Alertmanager_Bot created OVIRT-2988: --------------------------------------- Summary: [FIRING:1] InstanceUnreachable ibm-srv02.ovirt.org kubernetes-nodes-exporter (amd64 linux ibm-srv02.ovirt.org true external bare-metal-external ci) Key: OVIRT-2988 URL: https://ovirt-jira.atlassian.net/browse/OVIRT-2988 Project: oVirt - virtualization made easy Issue Type: Bug Reporter: Alertmanager_Bot Assignee: infra Labels: - alertname = InstanceUnreachable - beta_kubernetes_io_arch = amd64 - beta_kubernetes_io_os = linux - instance = ibm-srv02.ovirt.org - job = kubernetes-nodes-exporter - kubernetes_io_hostname = ibm-srv02.ovirt.org - node_role_kubernetes_io_compute = true - region = external - type = bare-metal-external - zone = ci Annotations: - description = ibm-srv02.ovirt.org of job kubernetes-nodes-exporter has been possibly down for more than 10 minutes. Source: http://prometheus-0:9090/graph?g0.expr=up%7Bjob%3D%22kubernetes-nodes-expor… -- This message was sent by Atlassian Jira (v1001.0.0-SNAPSHOT#100140)

1 0

[JIRA] (OVIRT-2987) [FIRING:1] InstanceUnreachable ibm-srv06.ovirt.org kubernetes-nodes-exporter (amd64 linux ibm-srv06.ovirt.org true external bare-metal-external ci)
by Alertmanager_Bot (oVirt JIRA) 29 Jul '20

29 Jul '20

Alertmanager_Bot created OVIRT-2987: --------------------------------------- Summary: [FIRING:1] InstanceUnreachable ibm-srv06.ovirt.org kubernetes-nodes-exporter (amd64 linux ibm-srv06.ovirt.org true external bare-metal-external ci) Key: OVIRT-2987 URL: https://ovirt-jira.atlassian.net/browse/OVIRT-2987 Project: oVirt - virtualization made easy Issue Type: Bug Reporter: Alertmanager_Bot Assignee: infra Labels: - alertname = InstanceUnreachable - beta_kubernetes_io_arch = amd64 - beta_kubernetes_io_os = linux - instance = ibm-srv06.ovirt.org - job = kubernetes-nodes-exporter - kubernetes_io_hostname = ibm-srv06.ovirt.org - node_role_kubernetes_io_compute = true - region = external - type = bare-metal-external - zone = ci Annotations: - description = ibm-srv06.ovirt.org of job kubernetes-nodes-exporter has been possibly down for more than 10 minutes. Source: http://prometheus-0:9090/graph?g0.expr=up%7Bjob%3D%22kubernetes-nodes-expor… -- This message was sent by Atlassian Jira (v1001.0.0-SNAPSHOT#100140)

1 0

[JIRA] (OVIRT-2989) [FIRING:1] InstanceUnreachable ibm-srv03.ovirt.org kubernetes-nodes-exporter (amd64 linux ibm-srv03.ovirt.org true external bare-metal-external ci)
by Alertmanager_Bot (oVirt JIRA) 29 Jul '20

29 Jul '20

Alertmanager_Bot created OVIRT-2989: --------------------------------------- Summary: [FIRING:1] InstanceUnreachable ibm-srv03.ovirt.org kubernetes-nodes-exporter (amd64 linux ibm-srv03.ovirt.org true external bare-metal-external ci) Key: OVIRT-2989 URL: https://ovirt-jira.atlassian.net/browse/OVIRT-2989 Project: oVirt - virtualization made easy Issue Type: Bug Reporter: Alertmanager_Bot Assignee: infra Labels: - alertname = InstanceUnreachable - beta_kubernetes_io_arch = amd64 - beta_kubernetes_io_os = linux - instance = ibm-srv03.ovirt.org - job = kubernetes-nodes-exporter - kubernetes_io_hostname = ibm-srv03.ovirt.org - node_role_kubernetes_io_compute = true - region = external - type = bare-metal-external - zone = ci Annotations: - description = ibm-srv03.ovirt.org of job kubernetes-nodes-exporter has been possibly down for more than 10 minutes. Source: http://prometheus-0:9090/graph?g0.expr=up%7Bjob%3D%22kubernetes-nodes-expor… -- This message was sent by Atlassian Jira (v1001.0.0-SNAPSHOT#100140)

1 0

[JIRA] (OVIRT-2990) [FIRING:1] InstanceUnreachable ibm-srv04.ovirt.org kubernetes-nodes-exporter (amd64 linux ibm-srv04.ovirt.org true external bare-metal-external ci)
by Alertmanager_Bot (oVirt JIRA) 29 Jul '20

29 Jul '20

Alertmanager_Bot created OVIRT-2990: --------------------------------------- Summary: [FIRING:1] InstanceUnreachable ibm-srv04.ovirt.org kubernetes-nodes-exporter (amd64 linux ibm-srv04.ovirt.org true external bare-metal-external ci) Key: OVIRT-2990 URL: https://ovirt-jira.atlassian.net/browse/OVIRT-2990 Project: oVirt - virtualization made easy Issue Type: Bug Reporter: Alertmanager_Bot Assignee: infra Labels: - alertname = InstanceUnreachable - beta_kubernetes_io_arch = amd64 - beta_kubernetes_io_os = linux - instance = ibm-srv04.ovirt.org - job = kubernetes-nodes-exporter - kubernetes_io_hostname = ibm-srv04.ovirt.org - node_role_kubernetes_io_compute = true - region = external - type = bare-metal-external - zone = ci Annotations: - description = ibm-srv04.ovirt.org of job kubernetes-nodes-exporter has been possibly down for more than 10 minutes. Source: http://prometheus-0:9090/graph?g0.expr=up%7Bjob%3D%22kubernetes-nodes-expor… -- This message was sent by Atlassian Jira (v1001.0.0-SNAPSHOT#100140)

1 0

[JIRA] (OVIRT-2986) [FIRING:1] InstanceUnreachable ibm-srv01.ovirt.org kubernetes-nodes-exporter (amd64 linux ibm-srv01.ovirt.org true external bare-metal-external ci)
by Alertmanager_Bot (oVirt JIRA) 29 Jul '20

29 Jul '20

Alertmanager_Bot created OVIRT-2986: --------------------------------------- Summary: [FIRING:1] InstanceUnreachable ibm-srv01.ovirt.org kubernetes-nodes-exporter (amd64 linux ibm-srv01.ovirt.org true external bare-metal-external ci) Key: OVIRT-2986 URL: https://ovirt-jira.atlassian.net/browse/OVIRT-2986 Project: oVirt - virtualization made easy Issue Type: Bug Reporter: Alertmanager_Bot Assignee: infra Labels: - alertname = InstanceUnreachable - beta_kubernetes_io_arch = amd64 - beta_kubernetes_io_os = linux - instance = ibm-srv01.ovirt.org - job = kubernetes-nodes-exporter - kubernetes_io_hostname = ibm-srv01.ovirt.org - node_role_kubernetes_io_compute = true - region = external - type = bare-metal-external - zone = ci Annotations: - description = ibm-srv01.ovirt.org of job kubernetes-nodes-exporter has been possibly down for more than 10 minutes. Source: http://prometheus-0:9090/graph?g0.expr=up%7Bjob%3D%22kubernetes-nodes-expor… -- This message was sent by Atlassian Jira (v1001.0.0-SNAPSHOT#100140)

1 0

[oVirt Jenkins] ovirt-system-tests_basic-suite-4.3_nightly - Build # 231 - Failure!
by jenkins＠jenkins.phx.ovirt.org 29 Jul '20

29 Jul '20

Project: https://jenkins.ovirt.org/job/ovirt-system-tests_basic-suite-4.3_nightly/ Build: https://jenkins.ovirt.org/job/ovirt-system-tests_basic-suite-4.3_nightly/23… Build Number: 231 Build Status: Failure Triggered By: Started by timer ------------------------------------- Changes Since Last Success: ------------------------------------- Changes for Build #231 [Nir Soffer] setup: Do not hide real issue by reserving ports ----------------- Failed Tests: ----------------- 1 tests failed. FAILED: 008_basic_ui_sanity.start_grid Error Message: could not get ip address of selenium hub. See previous messages for probable docker failure -------------------- >> begin captured stdout << --------------------- executing shell: docker ps CONTAINER ID IMAGE COMMAND CREATED STATUS PORTS NAMES executing shell: docker rm -f grid_node_chrome grid_node_firefox selenium-hub Error response from daemon: No such container: grid_node_chrome Error response from daemon: No such container: grid_node_firefox Error response from daemon: No such container: selenium-hub executing shell: docker kill grid_node_chrome grid_node_firefox selenium-hub Error response from daemon: Cannot kill container: grid_node_chrome: No such container: grid_node_chrome Error response from daemon: Cannot kill container: grid_node_firefox: No such container: grid_node_firefox Error response from daemon: Cannot kill container: selenium-hub: No such container: selenium-hub executing shell: docker ps CONTAINER ID IMAGE COMMAND CREATED STATUS PORTS NAMES executing shell: docker network ls NETWORK ID NAME DRIVER SCOPE 058a426a59b4 bridge bridge local bd7765e6670f host host local 245daf9e104c none null local executing shell: docker network rm grid Error response from daemon: network grid not found executing shell: docker network ls NETWORK ID NAME DRIVER SCOPE 058a426a59b4 bridge bridge local bd7765e6670f host host local 245daf9e104c none null local executing shell: docker network ls NETWORK ID NAME DRIVER SCOPE 058a426a59b4 bridge bridge local bd7765e6670f host host local 245daf9e104c none null local creating docker network for grid executing shell: docker network create grid 328da4cfdd441c835906d9674c744acf5ce32542b90053beed005cb2051505e1 executing shell: docker network ls NETWORK ID NAME DRIVER SCOPE 058a426a59b4 bridge bridge local 328da4cfdd44 grid bridge local bd7765e6670f host host local 245daf9e104c none null local starting hub executing shell: docker run -d -p 4444:4444 --net grid --name selenium-hub selenium/hub:3.9.1-actinium Unable to find image 'selenium/hub:3.9.1-actinium' locally 3.9.1-actinium: Pulling from selenium/hub 1be7f2b886e8: Pulling fs layer 6fbc4a21b806: Pulling fs layer c71a6f8e1378: Pulling fs layer 4be3072e5a37: Pulling fs layer 06c6d2f59700: Pulling fs layer edcd5e9f2f91: Pulling fs layer 0eeaf787f757: Pulling fs layer c949dee5af7e: Pulling fs layer df88a49b4162: Pulling fs layer ce3c6f42fd24: Pulling fs layer 6c8e191c3abf: Pulling fs layer c9aca8b50247: Pulling fs layer 0eeaf787f757: Waiting 4be3072e5a37: Waiting 06c6d2f59700: Waiting c949dee5af7e: Waiting c9aca8b50247: Waiting ce3c6f42fd24: Waiting 6c8e191c3abf: Waiting df88a49b4162: Waiting edcd5e9f2f91: Waiting /usr/bin/docker-current: error pulling image configuration: Get https://production.cloudflare.docker.com/registry-v2/docker/registry/v2/blo…: dial tcp: lookup production.cloudflare.docker.com on 172.19.11.6:53: read udp 10.129.5.89:52169->172.19.11.6:53: i/o timeout. See '/usr/bin/docker-current run --help'. getting ip of hub --------------------- >> end captured stdout << ---------------------- Stack Trace: File "/usr/lib64/python2.7/unittest/case.py", line 369, in run testMethod() File "/usr/lib/python2.7/site-packages/nose/case.py", line 197, in runTest self.test(*self.arg) File "/usr/lib/python2.7/site-packages/ovirtlago/testlib.py", line 142, in wrapped_test test() File "/home/jenkins/agent/workspace/ovirt-system-tests_basic-suite-4.3_nightly/ovirt-system-tests/basic-suite-4.3/test-scenarios/008_basic_ui_sanity.py", line 189, in start_grid raise RuntimeError("could not get ip address of selenium hub. See previous messages for probable docker failure") could not get ip address of selenium hub. See previous messages for probable docker failure -------------------- >> begin captured stdout << --------------------- executing shell: docker ps CONTAINER ID IMAGE COMMAND CREATED STATUS PORTS NAMES executing shell: docker rm -f grid_node_chrome grid_node_firefox selenium-hub Error response from daemon: No such container: grid_node_chrome Error response from daemon: No such container: grid_node_firefox Error response from daemon: No such container: selenium-hub executing shell: docker kill grid_node_chrome grid_node_firefox selenium-hub Error response from daemon: Cannot kill container: grid_node_chrome: No such container: grid_node_chrome Error response from daemon: Cannot kill container: grid_node_firefox: No such container: grid_node_firefox Error response from daemon: Cannot kill container: selenium-hub: No such container: selenium-hub executing shell: docker ps CONTAINER ID IMAGE COMMAND CREATED STATUS PORTS NAMES executing shell: docker network ls NETWORK ID NAME DRIVER SCOPE 058a426a59b4 bridge bridge local bd7765e6670f host host local 245daf9e104c none null local executing shell: docker network rm grid Error response from daemon: network grid not found executing shell: docker network ls NETWORK ID NAME DRIVER SCOPE 058a426a59b4 bridge bridge local bd7765e6670f host host local 245daf9e104c none null local executing shell: docker network ls NETWORK ID NAME DRIVER SCOPE 058a426a59b4 bridge bridge local bd7765e6670f host host local 245daf9e104c none null local creating docker network for grid executing shell: docker network create grid 328da4cfdd441c835906d9674c744acf5ce32542b90053beed005cb2051505e1 executing shell: docker network ls NETWORK ID NAME DRIVER SCOPE 058a426a59b4 bridge bridge local 328da4cfdd44 grid bridge local bd7765e6670f host host local 245daf9e104c none null local starting hub executing shell: docker run -d -p 4444:4444 --net grid --name selenium-hub selenium/hub:3.9.1-actinium Unable to find image 'selenium/hub:3.9.1-actinium' locally 3.9.1-actinium: Pulling from selenium/hub 1be7f2b886e8: Pulling fs layer 6fbc4a21b806: Pulling fs layer c71a6f8e1378: Pulling fs layer 4be3072e5a37: Pulling fs layer 06c6d2f59700: Pulling fs layer edcd5e9f2f91: Pulling fs layer 0eeaf787f757: Pulling fs layer c949dee5af7e: Pulling fs layer df88a49b4162: Pulling fs layer ce3c6f42fd24: Pulling fs layer 6c8e191c3abf: Pulling fs layer c9aca8b50247: Pulling fs layer 0eeaf787f757: Waiting 4be3072e5a37: Waiting 06c6d2f59700: Waiting c949dee5af7e: Waiting c9aca8b50247: Waiting ce3c6f42fd24: Waiting 6c8e191c3abf: Waiting df88a49b4162: Waiting edcd5e9f2f91: Waiting /usr/bin/docker-current: error pulling image configuration: Get https://production.cloudflare.docker.com/registry-v2/docker/registry/v2/blo…: dial tcp: lookup production.cloudflare.docker.com on 172.19.11.6:53: read udp 10.129.5.89:52169->172.19.11.6:53: i/o timeout. See '/usr/bin/docker-current run --help'. getting ip of hub --------------------- >> end captured stdout << ----------------------

1 1

oVirt infra daily report - unstable production jobs - 1266
by jenkins＠jenkins.phx.ovirt.org 29 Jul '20

29 Jul '20

Good morning! Attached is the HTML page with the jenkins status report. You can see it also here: - https://jenkins.ovirt.org/job/system_jenkins-report/1266//artifact/exported… Cheers, Jenkins

1 0

[JIRA] (OVIRT-2985) [FIRING:1] MirrorIsOutOfSync web-ovirt-mirrorchecker.apps.ovirt.org:80 oVirt-mirrors-health http://www.gtlib.gatech.edu/pub/oVirt/pub
by Alertmanager_Bot (oVirt JIRA) 28 Jul '20

28 Jul '20

Alertmanager_Bot created OVIRT-2985: --------------------------------------- Summary: [FIRING:1] MirrorIsOutOfSync web-ovirt-mirrorchecker.apps.ovirt.org:80 oVirt-mirrors-health http://www.gtlib.gatech.edu/pub/oVirt/pub Key: OVIRT-2985 URL: https://ovirt-jira.atlassian.net/browse/OVIRT-2985 Project: oVirt - virtualization made easy Issue Type: Bug Reporter: Alertmanager_Bot Assignee: infra Labels: - alertname = MirrorIsOutOfSync - instance = web-ovirt-mirrorchecker.apps.ovirt.org:80 - job = oVirt-mirrors-health - url = http://www.gtlib.gatech.edu/pub/oVirt/pub Annotations: - description = http://www.gtlib.gatech.edu/pub/oVirt/pub (measured by oVirt-mirrors-health) has not been synced in the past two days. Source: http://prometheus-0:9090/graph?g0.expr=mirror_health_check+%3E+172800&g0.ta… -- This message was sent by Atlassian Jira (v1001.0.0-SNAPSHOT#100140)

1 0

[JIRA] (OVIRT-2980) [FIRING:1] TestAlert
by Shlomi Zidmi (oVirt JIRA) 28 Jul '20

28 Jul '20

[ https://ovirt-jira.atlassian.net/browse/OVIRT-2980?page=com.atlassian.jira.… ] Shlomi Zidmi updated OVIRT-2980: -------------------------------- Resolution: Fixed Status: Done (was: To Do) > [FIRING:1] TestAlert > --------------------- > > Key: OVIRT-2980 > URL: https://ovirt-jira.atlassian.net/browse/OVIRT-2980 > Project: oVirt - virtualization made easy > Issue Type: Bug > Reporter: Alertmanager_Bot > Assignee: infra > Labels: ALERT{alertname="TestAlert"} > > Labels: > - alertname = TestAlert > - url = simple_url.com > Annotations: > Source: -- This message was sent by Atlassian Jira (v1001.0.0-SNAPSHOT#100140)

1 0

[oVirt Jenkins] ovirt-system-tests_basic-suite-master_nightly - Build # 227 - Failure!
by jenkins＠jenkins.phx.ovirt.org 28 Jul '20

28 Jul '20

Project: https://jenkins.ovirt.org/job/ovirt-system-tests_basic-suite-master_nightly/ Build: https://jenkins.ovirt.org/job/ovirt-system-tests_basic-suite-master_nightly… Build Number: 227 Build Status: Failure Triggered By: Started by timer ------------------------------------- Changes Since Last Success: ------------------------------------- Changes for Build #227 [Nir Soffer] setup: Do not hide real issue by reserving ports [Sandro Bonazzola] ovirt-guest-agent: drop fc30 [Ehud Yonasi] mock_configs: added rhel configs. ----------------- Failed Tests: ----------------- 1 tests failed. FAILED: basic-suite-master.test-scenarios.002_bootstrap_pytest.test_verify_add_hosts Error Message: RuntimeError: 1 hosts failed installation: lago-basic-suite-master-host-1: install_failed Stack Trace: api_v4 = <ovirtsdk4.Connection object at 0x7fbb71fd58d0> @order_by(_TEST_LIST) def test_verify_add_hosts(api_v4): hosts_service = api_v4.system_service().hosts_service() hosts_status = hosts_service.list(search='datacenter={}'.format(DC_NAME)) total_hosts = len(hosts_status) dump_hosts = _host_status_to_print(hosts_service, hosts_status) LOGGER.debug('Host status, verify_add_hosts:\n {}'.format(dump_hosts)) testlib.assert_true_within( lambda: _single_host_up(hosts_service, total_hosts), > timeout=constants.ADD_HOST_TIMEOUT ) ../basic-suite-master/test-scenarios/002_bootstrap_pytest.py:433: _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ /usr/lib/python2.7/site-packages/ovirtlago/testlib.py:278: in assert_true_within assert_equals_within(func, True, timeout, allowed_exceptions) /usr/lib/python2.7/site-packages/ovirtlago/testlib.py:234: in assert_equals_within res = func() ../basic-suite-master/test-scenarios/002_bootstrap_pytest.py:432: in <lambda> lambda: _single_host_up(hosts_service, total_hosts), ../basic-suite-master/test-scenarios/002_bootstrap_pytest.py:229: in _single_host_up _check_problematic_hosts(hosts_service) _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ hosts_service = <ovirtsdk4.services.HostsService object at 0x7fbb72148350> def _check_problematic_hosts(hosts_service): problematic_hosts = hosts_service.list(search='datacenter={} AND status != installing and status != initializing and status != up'.format(DC_NAME)) if len(problematic_hosts): dump_hosts = '%s hosts failed installation:\n' % len(problematic_hosts) for host in problematic_hosts: host_service = hosts_service.host_service(host.id) dump_hosts += '%s: %s\n' % (host.name, host_service.get().status) > raise RuntimeError(dump_hosts) E RuntimeError: 1 hosts failed installation: E lago-basic-suite-master-host-1: install_failed ../basic-suite-master/test-scenarios/002_bootstrap_pytest.py:238: RuntimeError

1 1

oVirt infra daily report - unstable production jobs - 1265
by jenkins＠jenkins.phx.ovirt.org 28 Jul '20

28 Jul '20

Good morning! Attached is the HTML page with the jenkins status report. You can see it also here: - https://jenkins.ovirt.org/job/system_jenkins-report/1265//artifact/exported… Cheers, Jenkins

1 0

[JIRA] (OVIRT-2984) Replacing 'Ubuntu Server 18' image on ovirt-glance
by Arik Hadas (oVirt JIRA) 27 Jul '20

27 Jul '20

Arik Hadas created OVIRT-2984: --------------------------------- Summary: Replacing 'Ubuntu Server 18' image on ovirt-glance Key: OVIRT-2984 URL: https://ovirt-jira.atlassian.net/browse/OVIRT-2984 Project: oVirt - virtualization made easy Issue Type: Task Reporter: Arik Hadas Assignee: infra Please replace the Ubuntu Server 18 image on ovirt-glance (there seems to a problem with that image, see: https://bugzilla.redhat.com/show_bug.cgi?id=1858918) -- This message was sent by Atlassian Jira (v1001.0.0-SNAPSHOT#100140)

1 0

[JIRA] (OVIRT-2983) host mirrorlist on IPv6
by Evgheni Dereveanchin (oVirt JIRA) 27 Jul '20

27 Jul '20

Evgheni Dereveanchin created OVIRT-2983: ------------------------------------------- Summary: host mirrorlist on IPv6 Key: OVIRT-2983 URL: https://ovirt-jira.atlassian.net/browse/OVIRT-2983 Project: oVirt - virtualization made easy Issue Type: Improvement Reporter: Evgheni Dereveanchin Assignee: infra Currently, all oVirt users will get a similar URL to fetch the mirrorlist from ovirt-release: https://resources.ovirt.org/pub/yum-repo/mirrorlist-ovirt-4.4-el8 This server is ipv4-only so users on IPv6 can't get a list of mirrors which makes oVirt non-installable in such environments without specifying a mirror manually. Opening a ticket to move this service to a dedicated URL like mirrorlist.ovirt.org and host it on an IPv6-enabled service, maybe at OSAS. Once that is done we can change the URL shipped with ovirt-release to enable installation on IPv6-only setups. -- This message was sent by Atlassian Jira (v1001.0.0-SNAPSHOT#100140)

1 0

[oVirt Jenkins] ovirt-system-tests_basic-suite-master_nightly - Build # 225 - Failure!
by jenkins＠jenkins.phx.ovirt.org 27 Jul '20

27 Jul '20

Project: https://jenkins.ovirt.org/job/ovirt-system-tests_basic-suite-master_nightly/ Build: https://jenkins.ovirt.org/job/ovirt-system-tests_basic-suite-master_nightly… Build Number: 225 Build Status: Failure Triggered By: Started by timer ------------------------------------- Changes Since Last Success: ------------------------------------- Changes for Build #225 [Nir Soffer] setup: Do not hide real issue by reserving ports ----------------- Failed Tests: ----------------- No tests ran.

1 1

[oVirt Jenkins] cleanup-gated-dummy-master - Build #219 - FAILURE!
by jenkins＠jenkins.phx.ovirt.org 27 Jul '20

27 Jul '20

Build: https://jenkins.ovirt.org/job/cleanup-gated-dummy-master/219/ Build Name: #219 Build Status: FAILURE Gerrit change: null - title: null - project: null - branch: null - author: null <null>

1 0

[oVirt Jenkins] ovirt-system-tests_performance-suite-master - Build # 1530 - Failure!
by jenkins＠jenkins.phx.ovirt.org 27 Jul '20

27 Jul '20

Project: https://jenkins.ovirt.org/job/ovirt-system-tests_performance-suite-master/ Build: https://jenkins.ovirt.org/job/ovirt-system-tests_performance-suite-master/1… Build Number: 1530 Build Status: Failure Triggered By: Started by timer ------------------------------------- Changes Since Last Success: ------------------------------------- Changes for Build #1530 [Nir Soffer] setup: Do not hide real issue by reserving ports ----------------- Failed Tests: ----------------- 1 tests failed. FAILED: 040_add_hosts_vms.add_master_storage_domain Error Message: Fault reason is "Operation Failed". Fault detail is "[Cannot add storage server connection when Host status is not up]". HTTP response code is 409. -------------------- >> begin captured logging << -------------------- 040_add_hosts_vms: INFO: lago-performance-suite-master-host-0: installing lago-performance-suite-master-host-1: up ost_utils.general_utils: DEBUG: attempt 2/120, 1 seconds sleeping 040_add_hosts_vms: INFO: lago-performance-suite-master-host-0: installing lago-performance-suite-master-host-1: up --------------------- >> end captured logging << --------------------- Stack Trace: File "/usr/lib64/python2.7/unittest/case.py", line 369, in run testMethod() File "/usr/lib/python2.7/site-packages/nose/case.py", line 197, in runTest self.test(*self.arg) File "/usr/lib/python2.7/site-packages/ovirtlago/testlib.py", line 142, in wrapped_test test() File "/usr/lib/python2.7/site-packages/ovirtlago/testlib.py", line 60, in wrapper return func(get_test_prefix(), *args, **kwargs) File "/home/jenkins/agent/workspace/ovirt-system-tests_performance-suite-master/ovirt-system-tests/performance-suite-master/test-scenarios/040_add_hosts_vms.py", line 353, in add_master_storage_domain add_nfs_storage_domain(prefix) File "/home/jenkins/agent/workspace/ovirt-system-tests_performance-suite-master/ovirt-system-tests/performance-suite-master/test-scenarios/040_add_hosts_vms.py", line 357, in add_nfs_storage_domain add_generic_nfs_storage_domain_4(prefix, SD_NFS_NAME, SD_NFS_HOST_NAME, SD_NFS_PATH, nfs_version='v4_2') File "/home/jenkins/agent/workspace/ovirt-system-tests_performance-suite-master/ovirt-system-tests/performance-suite-master/test-scenarios/040_add_hosts_vms.py", line 407, in add_generic_nfs_storage_domain_4 _add_storage_domain_4(api, p) File "/home/jenkins/agent/workspace/ovirt-system-tests_performance-suite-master/ovirt-system-tests/performance-suite-master/test-scenarios/040_add_hosts_vms.py", line 327, in _add_storage_domain_4 sd = sds_service.add(p) File "/usr/lib64/python2.7/site-packages/ovirtsdk4/services.py", line 26097, in add return self._internal_add(storage_domain, headers, query, wait) File "/usr/lib64/python2.7/site-packages/ovirtsdk4/service.py", line 232, in _internal_add return future.wait() if wait else future File "/usr/lib64/python2.7/site-packages/ovirtsdk4/service.py", line 55, in wait return self._code(response) File "/usr/lib64/python2.7/site-packages/ovirtsdk4/service.py", line 229, in callback self._check_fault(response) File "/usr/lib64/python2.7/site-packages/ovirtsdk4/service.py", line 132, in _check_fault self._raise_error(response, body) File "/usr/lib64/python2.7/site-packages/ovirtsdk4/service.py", line 118, in _raise_error raise error 'Fault reason is "Operation Failed". Fault detail is "[Cannot add storage server connection when Host status is not up]". HTTP response code is 409.\n-------------------- >> begin captured logging << --------------------\n040_add_hosts_vms: INFO: lago-performance-suite-master-host-0: installing\nlago-performance-suite-master-host-1: up\n\nost_utils.general_utils: DEBUG: attempt 2/120, 1 seconds sleeping\n040_add_hosts_vms: INFO: lago-performance-suite-master-host-0: installing\nlago-performance-suite-master-host-1: up\n\n--------------------- >> end captured logging << ---------------------'

1 1

oVirt infra daily report - unstable production jobs - 1264
by jenkins＠jenkins.phx.ovirt.org 27 Jul '20

27 Jul '20

Good morning! Attached is the HTML page with the jenkins status report. You can see it also here: - https://jenkins.ovirt.org/job/system_jenkins-report/1264//artifact/exported… Cheers, Jenkins

1 0

[oVirt Jenkins] ovirt-system-tests_basic-suite-master_nightly - Build # 221 - Failure!
by jenkins＠jenkins.phx.ovirt.org 26 Jul '20

26 Jul '20

Project: https://jenkins.ovirt.org/job/ovirt-system-tests_basic-suite-master_nightly/ Build: https://jenkins.ovirt.org/job/ovirt-system-tests_basic-suite-master_nightly… Build Number: 221 Build Status: Failure Triggered By: Started by timer ------------------------------------- Changes Since Last Success: ------------------------------------- Changes for Build #221 [Nir Soffer] setup: Do not hide real issue by reserving ports ----------------- Failed Tests: ----------------- 1 tests failed. FAILED: basic-suite-master.test-scenarios.004_basic_sanity_pytest.test_live_storage_migration Error Message: AssertionError: False != True after 600 seconds Stack Trace: api_v4 = <ovirtsdk4.Connection object at 0x7fb0e92139d0> @order_by(_TEST_LIST) def test_live_storage_migration(api_v4): engine = api_v4.system_service() disk_service = test_utils.get_disk_service(engine, DISK0_NAME) correlation_id = 'live_storage_migration' disk_service.move( async=False, filter=False, storage_domain=types.StorageDomain( name=SD_ISCSI_NAME ), query={'correlation_id': correlation_id} ) testlib.assert_true_within_long(lambda: test_utils.all_jobs_finished(engine, correlation_id)) # Assert that the disk is on the correct storage domain, # its status is OK and the snapshot created for the migration # has been merged testlib.assert_true_within_long( lambda: api_v4.follow_link(disk_service.get().storage_domains[0]).name == SD_ISCSI_NAME ) vm0_snapshots_service = test_utils.get_vm_snapshots_service(engine, VM0_NAME) testlib.assert_true_within_long( > lambda: len(vm0_snapshots_service.list()) == 1 ) ../basic-suite-master/test-scenarios/004_basic_sanity_pytest.py:579: _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ /usr/lib/python2.7/site-packages/ovirtlago/testlib.py:286: in assert_true_within_long assert_equals_within_long(func, True, allowed_exceptions) /usr/lib/python2.7/site-packages/ovirtlago/testlib.py:273: in assert_equals_within_long func, value, LONG_TIMEOUT, allowed_exceptions=allowed_exceptions _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ func = <function <lambda> at 0x7fb0e5af2b90>, value = True, timeout = 600 allowed_exceptions = [], initial_wait = 0 def assert_equals_within( func, value, timeout, allowed_exceptions=None, initial_wait=10 ): allowed_exceptions = allowed_exceptions or [] with utils.EggTimer(timeout) as timer: while not timer.elapsed(): try: res = func() if res == value: return except Exception as exc: if _instance_of_any(exc, allowed_exceptions): time.sleep(3) continue LOGGER.exception("Unhandled exception in %s", func) raise if initial_wait == 0: time.sleep(3) else: time.sleep(initial_wait) initial_wait = 0 try: raise AssertionError( > '%s != %s after %s seconds' % (res, value, timeout) ) E AssertionError: False != True after 600 seconds /usr/lib/python2.7/site-packages/ovirtlago/testlib.py:252: AssertionError

1 1

[oVirt Jenkins] ovirt-system-tests_basic-suite-4.3_nightly - Build # 221 - Failure!
by jenkins＠jenkins.phx.ovirt.org 26 Jul '20

26 Jul '20

Project: https://jenkins.ovirt.org/job/ovirt-system-tests_basic-suite-4.3_nightly/ Build: https://jenkins.ovirt.org/job/ovirt-system-tests_basic-suite-4.3_nightly/22… Build Number: 221 Build Status: Failure Triggered By: Started by timer ------------------------------------- Changes Since Last Success: ------------------------------------- Changes for Build #221 [Nir Soffer] setup: Do not hide real issue by reserving ports ----------------- Failed Tests: ----------------- 1 tests failed. FAILED: 004_basic_sanity.disk_operations Error Message: False != True after 600 seconds -------------------- >> begin captured logging << -------------------- lago.utils: DEBUG: Error while running thread Thread-66 Traceback (most recent call last): File "/usr/lib/python2.7/site-packages/lago/utils.py", line 58, in _ret_via_queue queue.put({'return': func()}) File "/usr/lib/python2.7/site-packages/ovirtlago/testlib.py", line 60, in wrapper return func(get_test_prefix(), *args, **kwargs) File "/usr/lib/python2.7/site-packages/ovirtlago/testlib.py", line 79, in wrapper prefix.virt_env.engine_vm().get_api(api_ver=4), *args, **kwargs File "/home/jenkins/agent/workspace/ovirt-system-tests_basic-suite-4.3_nightly/ovirt-system-tests/basic-suite-4.3/test-scenarios/004_basic_sanity.py", line 508, in live_storage_migration testlib.assert_true_within_long(lambda: test_utils.all_jobs_finished(engine, correlation_id)) File "/usr/lib/python2.7/site-packages/ovirtlago/testlib.py", line 286, in assert_true_within_long assert_equals_within_long(func, True, allowed_exceptions) File "/usr/lib/python2.7/site-packages/ovirtlago/testlib.py", line 273, in assert_equals_within_long func, value, LONG_TIMEOUT, allowed_exceptions=allowed_exceptions File "/usr/lib/python2.7/site-packages/ovirtlago/testlib.py", line 252, in assert_equals_within '%s != %s after %s seconds' % (res, value, timeout) AssertionError: False != True after 600 seconds --------------------- >> end captured logging << --------------------- Stack Trace: File "/usr/lib64/python2.7/unittest/case.py", line 369, in run testMethod() File "/usr/lib/python2.7/site-packages/nose/case.py", line 197, in runTest self.test(*self.arg) File "/usr/lib/python2.7/site-packages/ovirtlago/testlib.py", line 142, in wrapped_test test() File "/usr/lib/python2.7/site-packages/ovirtlago/testlib.py", line 60, in wrapper return func(get_test_prefix(), *args, **kwargs) File "/usr/lib/python2.7/site-packages/ovirtlago/testlib.py", line 79, in wrapper prefix.virt_env.engine_vm().get_api(api_ver=4), *args, **kwargs File "/home/jenkins/agent/workspace/ovirt-system-tests_basic-suite-4.3_nightly/ovirt-system-tests/basic-suite-4.3/test-scenarios/004_basic_sanity.py", line 932, in disk_operations vt.join_all() File "/usr/lib/python2.7/site-packages/lago/utils.py", line 58, in _ret_via_queue queue.put({'return': func()}) File "/usr/lib/python2.7/site-packages/ovirtlago/testlib.py", line 60, in wrapper return func(get_test_prefix(), *args, **kwargs) File "/usr/lib/python2.7/site-packages/ovirtlago/testlib.py", line 79, in wrapper prefix.virt_env.engine_vm().get_api(api_ver=4), *args, **kwargs File "/home/jenkins/agent/workspace/ovirt-system-tests_basic-suite-4.3_nightly/ovirt-system-tests/basic-suite-4.3/test-scenarios/004_basic_sanity.py", line 508, in live_storage_migration testlib.assert_true_within_long(lambda: test_utils.all_jobs_finished(engine, correlation_id)) File "/usr/lib/python2.7/site-packages/ovirtlago/testlib.py", line 286, in assert_true_within_long assert_equals_within_long(func, True, allowed_exceptions) File "/usr/lib/python2.7/site-packages/ovirtlago/testlib.py", line 273, in assert_equals_within_long func, value, LONG_TIMEOUT, allowed_exceptions=allowed_exceptions File "/usr/lib/python2.7/site-packages/ovirtlago/testlib.py", line 252, in assert_equals_within '%s != %s after %s seconds' % (res, value, timeout) 'False != True after 600 seconds\n-------------------- >> begin captured logging << --------------------\nlago.utils: DEBUG: Error while running thread Thread-66\nTraceback (most recent call last):\n File "/usr/lib/python2.7/site-packages/lago/utils.py", line 58, in _ret_via_queue\n queue.put({\'return\': func()})\n File "/usr/lib/python2.7/site-packages/ovirtlago/testlib.py", line 60, in wrapper\n return func(get_test_prefix(), *args, **kwargs)\n File "/usr/lib/python2.7/site-packages/ovirtlago/testlib.py", line 79, in wrapper\n prefix.virt_env.engine_vm().get_api(api_ver=4), *args, **kwargs\n File "/home/jenkins/agent/workspace/ovirt-system-tests_basic-suite-4.3_nightly/ovirt-system-tests/basic-suite-4.3/test-scenarios/004_basic_sanity.py", line 508, in live_storage_migration\n testlib.assert_true_within_long(lambda: test_utils.all_jobs_finished(engine, correlation_id))\n File "/usr/lib/python2.7/site-packages/ovirtlago/testlib.py", line 286, in assert_true_within_long\n assert_equals_within_long(func, True, allowed_exceptions)\n File "/usr/lib/python2.7/site-packages/ovirtlago/testlib.py", line 273, in assert_equals_within_long\n func, value, LONG_TIMEOUT, allowed_exceptions=allowed_exceptions\n File "/usr/lib/python2.7/site-packages/ovirtlago/testlib.py", line 252, in assert_equals_within\n \'%s != %s after %s seconds\' % (res, value, timeout)\nAssertionError: False != True after 600 seconds\n--------------------- >> end captured logging << ---------------------'

1 1

oVirt infra daily report - unstable production jobs - 1263
by jenkins＠jenkins.phx.ovirt.org 26 Jul '20

26 Jul '20

Good morning! Attached is the HTML page with the jenkins status report. You can see it also here: - https://jenkins.ovirt.org/job/system_jenkins-report/1263//artifact/exported… Cheers, Jenkins

1 0

[oVirt Jenkins] ovirt-system-tests_basic-suite-master_nightly - Build # 219 - Failure!
by jenkins＠jenkins.phx.ovirt.org 25 Jul '20

25 Jul '20

Project: https://jenkins.ovirt.org/job/ovirt-system-tests_basic-suite-master_nightly/ Build: https://jenkins.ovirt.org/job/ovirt-system-tests_basic-suite-master_nightly… Build Number: 219 Build Status: Failure Triggered By: Started by timer ------------------------------------- Changes Since Last Success: ------------------------------------- Changes for Build #219 [Nir Soffer] setup: Do not hide real issue by reserving ports ----------------- Failed Tests: ----------------- 1 tests failed. FAILED: basic-suite-master.test-scenarios.002_bootstrap_pytest.test_verify_add_hosts Error Message: RuntimeError: 1 hosts failed installation: lago-basic-suite-master-host-1: install_failed Stack Trace: api_v4 = <ovirtsdk4.Connection object at 0x7fd026efc8d0> @order_by(_TEST_LIST) def test_verify_add_hosts(api_v4): hosts_service = api_v4.system_service().hosts_service() hosts_status = hosts_service.list(search='datacenter={}'.format(DC_NAME)) total_hosts = len(hosts_status) dump_hosts = _host_status_to_print(hosts_service, hosts_status) LOGGER.debug('Host status, verify_add_hosts:\n {}'.format(dump_hosts)) testlib.assert_true_within( lambda: _single_host_up(hosts_service, total_hosts), > timeout=constants.ADD_HOST_TIMEOUT ) ../basic-suite-master/test-scenarios/002_bootstrap_pytest.py:433: _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ /usr/lib/python2.7/site-packages/ovirtlago/testlib.py:278: in assert_true_within assert_equals_within(func, True, timeout, allowed_exceptions) /usr/lib/python2.7/site-packages/ovirtlago/testlib.py:234: in assert_equals_within res = func() ../basic-suite-master/test-scenarios/002_bootstrap_pytest.py:432: in <lambda> lambda: _single_host_up(hosts_service, total_hosts), ../basic-suite-master/test-scenarios/002_bootstrap_pytest.py:229: in _single_host_up _check_problematic_hosts(hosts_service) _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ hosts_service = <ovirtsdk4.services.HostsService object at 0x7fd02706f350> def _check_problematic_hosts(hosts_service): problematic_hosts = hosts_service.list(search='datacenter={} AND status != installing and status != initializing and status != up'.format(DC_NAME)) if len(problematic_hosts): dump_hosts = '%s hosts failed installation:\n' % len(problematic_hosts) for host in problematic_hosts: host_service = hosts_service.host_service(host.id) dump_hosts += '%s: %s\n' % (host.name, host_service.get().status) > raise RuntimeError(dump_hosts) E RuntimeError: 1 hosts failed installation: E lago-basic-suite-master-host-1: install_failed ../basic-suite-master/test-scenarios/002_bootstrap_pytest.py:238: RuntimeError

1 1

[oVirt Jenkins] ovirt-system-tests_basic-suite-4.3_nightly - Build # 217 - Failure!
by jenkins＠jenkins.phx.ovirt.org 25 Jul '20

25 Jul '20

Project: https://jenkins.ovirt.org/job/ovirt-system-tests_basic-suite-4.3_nightly/ Build: https://jenkins.ovirt.org/job/ovirt-system-tests_basic-suite-4.3_nightly/21… Build Number: 217 Build Status: Failure Triggered By: Started by timer ------------------------------------- Changes Since Last Success: ------------------------------------- Changes for Build #217 [Nir Soffer] setup: Do not hide real issue by reserving ports ----------------- Failed Tests: ----------------- 1 tests failed. FAILED: 008_basic_ui_sanity.initialize_chrome Error Message: Message: Error forwarding the new session cannot find : Capabilities {browserName: chrome, platform: LINUX, version: 64.0.3282.140} Stacktrace: at org.openqa.grid.web.servlet.handler.RequestHandler.process (RequestHandler.java:117) at org.openqa.grid.web.servlet.DriverServlet.process (DriverServlet.java:84) at org.openqa.grid.web.servlet.DriverServlet.doPost (DriverServlet.java:68) at javax.servlet.http.HttpServlet.service (HttpServlet.java:707) at javax.servlet.http.HttpServlet.service (HttpServlet.java:790) at org.seleniumhq.jetty9.servlet.ServletHolder.handle (ServletHolder.java:860) at org.seleniumhq.jetty9.servlet.ServletHandler.doHandle (ServletHandler.java:535) at org.seleniumhq.jetty9.server.handler.ScopedHandler.nextHandle (ScopedHandler.java:188) at org.seleniumhq.jetty9.server.session.SessionHandler.doHandle (SessionHandler.java:1595) at org.seleniumhq.jetty9.server.handler.ScopedHandler.nextHandle (ScopedHandler.java:188) at org.seleniumhq.jetty9.server.handler.ContextHandler.doHandle (ContextHandler.java:1253) at org.seleniumhq.jetty9.server.handler.ScopedHandler.nextScope (ScopedHandler.java:168) at org.seleniumhq.jetty9.servlet.ServletHandler.doScope (ServletHandler.java:473) at org.seleniumhq.jetty9.server.session.SessionHandler.doScope (SessionHandler.java:1564) at org.seleniumhq.jetty9.server.handler.ScopedHandler.nextScope (ScopedHandler.java:166) at org.seleniumhq.jetty9.server.handler.ContextHandler.doScope (ContextHandler.java:1155) at org.seleniumhq.jetty9.server.handler.ScopedHandler.handle (ScopedHandler.java:141) at org.seleniumhq.jetty9.server.handler.HandlerWrapper.handle (HandlerWrapper.java:132) at org.seleniumhq.jetty9.server.Server.handle (Server.java:530) at org.seleniumhq.jetty9.server.HttpChannel.handle (HttpChannel.java:347) at org.seleniumhq.jetty9.server.HttpConnection.onFillable (HttpConnection.java:256) at org.seleniumhq.jetty9.io.AbstractConnection$ReadCallback.succeeded (AbstractConnection.java:279) at org.seleniumhq.jetty9.io.FillInterest.fillable (FillInterest.java:102) at org.seleniumhq.jetty9.io.ChannelEndPoint$2.run (ChannelEndPoint.java:124) at org.seleniumhq.jetty9.util.thread.strategy.EatWhatYouKill.doProduce (EatWhatYouKill.java:247) at org.seleniumhq.jetty9.util.thread.strategy.EatWhatYouKill.produce (EatWhatYouKill.java:140) at org.seleniumhq.jetty9.util.thread.strategy.EatWhatYouKill.run (EatWhatYouKill.java:131) at org.seleniumhq.jetty9.util.thread.ReservedThreadExecutor$ReservedThread.run (ReservedThreadExecutor.java:382) at org.seleniumhq.jetty9.util.thread.QueuedThreadPool.runJob (QueuedThreadPool.java:708) at org.seleniumhq.jetty9.util.thread.QueuedThreadPool$2.run (QueuedThreadPool.java:626) at java.lang.Thread.run (Thread.java:748) -------------------- >> begin captured stdout << --------------------- _init_browser, connecting to hub at http://172.18.0.2:4444/wd/hub Message: Error forwarding the new session cannot find : Capabilities {browserName: chrome, platform: LINUX, version: 64.0.3282.140} Stacktrace: at org.openqa.grid.web.servlet.handler.RequestHandler.process (RequestHandler.java:117) at org.openqa.grid.web.servlet.DriverServlet.process (DriverServlet.java:84) at org.openqa.grid.web.servlet.DriverServlet.doPost (DriverServlet.java:68) at javax.servlet.http.HttpServlet.service (HttpServlet.java:707) at javax.servlet.http.HttpServlet.service (HttpServlet.java:790) at org.seleniumhq.jetty9.servlet.ServletHolder.handle (ServletHolder.java:860) at org.seleniumhq.jetty9.servlet.ServletHandler.doHandle (ServletHandler.java:535) at org.seleniumhq.jetty9.server.handler.ScopedHandler.nextHandle (ScopedHandler.java:188) at org.seleniumhq.jetty9.server.session.SessionHandler.doHandle (SessionHandler.java:1595) at org.seleniumhq.jetty9.server.handler.ScopedHandler.nextHandle (ScopedHandler.java:188) at org.seleniumhq.jetty9.server.handler.ContextHandler.doHandle (ContextHandler.java:1253) at org.seleniumhq.jetty9.server.handler.ScopedHandler.nextScope (ScopedHandler.java:168) at org.seleniumhq.jetty9.servlet.ServletHandler.doScope (ServletHandler.java:473) at org.seleniumhq.jetty9.server.session.SessionHandler.doScope (SessionHandler.java:1564) at org.seleniumhq.jetty9.server.handler.ScopedHandler.nextScope (ScopedHandler.java:166) at org.seleniumhq.jetty9.server.handler.ContextHandler.doScope (ContextHandler.java:1155) at org.seleniumhq.jetty9.server.handler.ScopedHandler.handle (ScopedHandler.java:141) at org.seleniumhq.jetty9.server.handler.HandlerWrapper.handle (HandlerWrapper.java:132) at org.seleniumhq.jetty9.server.Server.handle (Server.java:530) at org.seleniumhq.jetty9.server.HttpChannel.handle (HttpChannel.java:347) at org.seleniumhq.jetty9.server.HttpConnection.onFillable (HttpConnection.java:256) at org.seleniumhq.jetty9.io.AbstractConnection$ReadCallback.succeeded (AbstractConnection.java:279) at org.seleniumhq.jetty9.io.FillInterest.fillable (FillInterest.java:102) at org.seleniumhq.jetty9.io.ChannelEndPoint$2.run (ChannelEndPoint.java:124) at org.seleniumhq.jetty9.util.thread.strategy.EatWhatYouKill.doProduce (EatWhatYouKill.java:247) at org.seleniumhq.jetty9.util.thread.strategy.EatWhatYouKill.produce (EatWhatYouKill.java:140) at org.seleniumhq.jetty9.util.thread.strategy.EatWhatYouKill.run (EatWhatYouKill.java:131) at org.seleniumhq.jetty9.util.thread.ReservedThreadExecutor$ReservedThread.run (ReservedThreadExecutor.java:382) at org.seleniumhq.jetty9.util.thread.QueuedThreadPool.runJob (QueuedThreadPool.java:708) at org.seleniumhq.jetty9.util.thread.QueuedThreadPool$2.run (QueuedThreadPool.java:626) at java.lang.Thread.run (Thread.java:748) WebDriverException in _init_browser connecting to hub --------------------- >> end captured stdout << ---------------------- -------------------- >> begin captured logging << -------------------- selenium.webdriver.remote.remote_connection: DEBUG: POST http://172.18.0.2:4444/wd/hub/session {"capabilities": {"alwaysMatch": {"browserName": "chrome", "platformName": "linux", "browserVersion": "64.0.3282.140"}, "firstMatch": [{}]}, "desiredCapabilities": {"platform": "LINUX", "browserName": "chrome", "version": "64.0.3282.140"}} urllib3.connectionpool: DEBUG: Starting new HTTP connection (1): 172.18.0.2:4444 urllib3.connectionpool: DEBUG: http://172.18.0.2:4444 "POST /wd/hub/session HTTP/1.1" 500 7792 selenium.webdriver.remote.remote_connection: DEBUG: Finished Request --------------------- >> end captured logging << --------------------- Stack Trace: File "/usr/lib64/python2.7/unittest/case.py", line 369, in run testMethod() File "/usr/lib/python2.7/site-packages/nose/case.py", line 197, in runTest self.test(*self.arg) File "/usr/lib/python2.7/site-packages/ovirtlago/testlib.py", line 142, in wrapped_test test() File "/home/jenkins/agent/workspace/ovirt-system-tests_basic-suite-4.3_nightly/ovirt-system-tests/basic-suite-4.3/test-scenarios/008_basic_ui_sanity.py", line 223, in initialize_chrome _init_driver(_get_chrome_capabilities()) File "/home/jenkins/agent/workspace/ovirt-system-tests_basic-suite-4.3_nightly/ovirt-system-tests/basic-suite-4.3/test-scenarios/008_basic_ui_sanity.py", line 249, in _init_driver raise e 'Message: Error forwarding the new session cannot find : Capabilities {browserName: chrome, platform: LINUX, version: 64.0.3282.140}\nStacktrace:\n at org.openqa.grid.web.servlet.handler.RequestHandler.process (RequestHandler.java:117)\n at org.openqa.grid.web.servlet.DriverServlet.process (DriverServlet.java:84)\n at org.openqa.grid.web.servlet.DriverServlet.doPost (DriverServlet.java:68)\n at javax.servlet.http.HttpServlet.service (HttpServlet.java:707)\n at javax.servlet.http.HttpServlet.service (HttpServlet.java:790)\n at org.seleniumhq.jetty9.servlet.ServletHolder.handle (ServletHolder.java:860)\n at org.seleniumhq.jetty9.servlet.ServletHandler.doHandle (ServletHandler.java:535)\n at org.seleniumhq.jetty9.server.handler.ScopedHandler.nextHandle (ScopedHandler.java:188)\n at org.seleniumhq.jetty9.server.session.SessionHandler.doHandle (SessionHandler.java:1595)\n at org.seleniumhq.jetty9.server.handler.ScopedHandler.nextHandle (ScopedHandler.java:188)\n at org.seleniumhq.jetty9.server.handler.ContextHandler.doHandle (ContextHandler.java:1253)\n at org.seleniumhq.jetty9.server.handler.ScopedHandler.nextScope (ScopedHandler.java:168)\n at org.seleniumhq.jetty9.servlet.ServletHandler.doScope (ServletHandler.java:473)\n at org.seleniumhq.jetty9.server.session.SessionHandler.doScope (SessionHandler.java:1564)\n at org.seleniumhq.jetty9.server.handler.ScopedHandler.nextScope (ScopedHandler.java:166)\n at org.seleniumhq.jetty9.server.handler.ContextHandler.doScope (ContextHandler.java:1155)\n at org.seleniumhq.jetty9.server.handler.ScopedHandler.handle (ScopedHandler.java:141)\n at org.seleniumhq.jetty9.server.handler.HandlerWrapper.handle (HandlerWrapper.java:132)\n at org.seleniumhq.jetty9.server.Server.handle (Server.java:530)\n at org.seleniumhq.jetty9.server.HttpChannel.handle (HttpChannel.java:347)\n at org.seleniumhq.jetty9.server.HttpConnection.onFillable (HttpConnection.java:256)\n at org.seleniumhq.jetty9.io.AbstractConnection$ReadCallback.succeeded (AbstractConnection.java:279)\n at org.seleniumhq.jetty9.io.FillInterest.fillable (FillInterest.java:102)\n at org.seleniumhq.jetty9.io.ChannelEndPoint$2.run (ChannelEndPoint.java:124)\n at org.seleniumhq.jetty9.util.thread.strategy.EatWhatYouKill.doProduce (EatWhatYouKill.java:247)\n at org.seleniumhq.jetty9.util.thread.strategy.EatWhatYouKill.produce (EatWhatYouKill.java:140)\n at org.seleniumhq.jetty9.util.thread.strategy.EatWhatYouKill.run (EatWhatYouKill.java:131)\n at org.seleniumhq.jetty9.util.thread.ReservedThreadExecutor$ReservedThread.run (ReservedThreadExecutor.java:382)\n at org.seleniumhq.jetty9.util.thread.QueuedThreadPool.runJob (QueuedThreadPool.java:708)\n at org.seleniumhq.jetty9.util.thread.QueuedThreadPool$2.run (QueuedThreadPool.java:626)\n at java.lang.Thread.run (Thread.java:748)\n-------------------- >> begin captured stdout << ---------------------\n_init_browser, connecting to hub at http://172.18.0.2:4444/wd/hub\nMessage: Error forwarding the new session cannot find : Capabilities {browserName: chrome, platform: LINUX, version: 64.0.3282.140}\nStacktrace:\n at org.openqa.grid.web.servlet.handler.RequestHandler.process (RequestHandler.java:117)\n at org.openqa.grid.web.servlet.DriverServlet.process (DriverServlet.java:84)\n at org.openqa.grid.web.servlet.DriverServlet.doPost (DriverServlet.java:68)\n at javax.servlet.http.HttpServlet.service (HttpServlet.java:707)\n at javax.servlet.http.HttpServlet.service (HttpServlet.java:790)\n at org.seleniumhq.jetty9.servlet.ServletHolder.handle (ServletHolder.java:860)\n at org.seleniumhq.jetty9.servlet.ServletHandler.doHandle (ServletHandler.java:535)\n at org.seleniumhq.jetty9.server.handler.ScopedHandler.nextHandle (ScopedHandler.java:188)\n at org.seleniumhq.jetty9.server.session.SessionHandler.doHandle (SessionHandler.java:1595)\n at org.seleniumhq.jetty9.server.handler.ScopedHandler.nextHandle (ScopedHandler.java:188)\n at org.seleniumhq.jetty9.server.handler.ContextHandler.doHandle (ContextHandler.java:1253)\n at org.seleniumhq.jetty9.server.handler.ScopedHandler.nextScope (ScopedHandler.java:168)\n at org.seleniumhq.jetty9.servlet.ServletHandler.doScope (ServletHandler.java:473)\n at org.seleniumhq.jetty9.server.session.SessionHandler.doScope (SessionHandler.java:1564)\n at org.seleniumhq.jetty9.server.handler.ScopedHandler.nextScope (ScopedHandler.java:166)\n at org.seleniumhq.jetty9.server.handler.ContextHandler.doScope (ContextHandler.java:1155)\n at org.seleniumhq.jetty9.server.handler.ScopedHandler.handle (ScopedHandler.java:141)\n at org.seleniumhq.jetty9.server.handler.HandlerWrapper.handle (HandlerWrapper.java:132)\n at org.seleniumhq.jetty9.server.Server.handle (Server.java:530)\n at org.seleniumhq.jetty9.server.HttpChannel.handle (HttpChannel.java:347)\n at org.seleniumhq.jetty9.server.HttpConnection.onFillable (HttpConnection.java:256)\n at org.seleniumhq.jetty9.io.AbstractConnection$ReadCallback.succeeded (AbstractConnection.java:279)\n at org.seleniumhq.jetty9.io.FillInterest.fillable (FillInterest.java:102)\n at org.seleniumhq.jetty9.io.ChannelEndPoint$2.run (ChannelEndPoint.java:124)\n at org.seleniumhq.jetty9.util.thread.strategy.EatWhatYouKill.doProduce (EatWhatYouKill.java:247)\n at org.seleniumhq.jetty9.util.thread.strategy.EatWhatYouKill.produce (EatWhatYouKill.java:140)\n at org.seleniumhq.jetty9.util.thread.strategy.EatWhatYouKill.run (EatWhatYouKill.java:131)\n at org.seleniumhq.jetty9.util.thread.ReservedThreadExecutor$ReservedThread.run (ReservedThreadExecutor.java:382)\n at org.seleniumhq.jetty9.util.thread.QueuedThreadPool.runJob (QueuedThreadPool.java:708)\n at org.seleniumhq.jetty9.util.thread.QueuedThreadPool$2.run (QueuedThreadPool.java:626)\n at java.lang.Thread.run (Thread.java:748)\nWebDriverException in _init_browser connecting to hub\n\n--------------------- >> end captured stdout << ----------------------\n-------------------- >> begin captured logging << --------------------\nselenium.webdriver.remote.remote_connection: DEBUG: POST http://172.18.0.2:4444/wd/hub/session {"capabilities": {"alwaysMatch": {"browserName": "chrome", "platformName": "linux", "browserVersion": "64.0.3282.140"}, "firstMatch": [{}]}, "desiredCapabilities": {"platform": "LINUX", "browserName": "chrome", "version": "64.0.3282.140"}}\nurllib3.connectionpool: DEBUG: Starting new HTTP connection (1): 172.18.0.2:4444\nurllib3.connectionpool: DEBUG: http://172.18.0.2:4444 "POST /wd/hub/session HTTP/1.1" 500 7792\nselenium.webdriver.remote.remote_connection: DEBUG: Finished Request\n--------------------- >> end captured logging << ---------------------'

1 1

oVirt infra daily report - unstable production jobs - 1262
by jenkins＠jenkins.phx.ovirt.org 25 Jul '20

25 Jul '20

Good morning! Attached is the HTML page with the jenkins status report. You can see it also here: - https://jenkins.ovirt.org/job/system_jenkins-report/1262//artifact/exported… Cheers, Jenkins

1 0

oVirt infra daily report - unstable production jobs - 1261
by jenkins＠jenkins.phx.ovirt.org 24 Jul '20

24 Jul '20

Good morning! Attached is the HTML page with the jenkins status report. You can see it also here: - https://jenkins.ovirt.org/job/system_jenkins-report/1261//artifact/exported… Cheers, Jenkins

1 0

[JIRA] (OVIRT-2982) [FIRING:1] InstanceUnreachable shift-n12.phx.ovirt.org kubernetes-nodes-exporter (amd64 linux shift-n12.phx.ovirt.org true primary vm ci)
by Alertmanager_Bot (oVirt JIRA) 23 Jul '20

23 Jul '20

Alertmanager_Bot created OVIRT-2982: --------------------------------------- Summary: [FIRING:1] InstanceUnreachable shift-n12.phx.ovirt.org kubernetes-nodes-exporter (amd64 linux shift-n12.phx.ovirt.org true primary vm ci) Key: OVIRT-2982 URL: https://ovirt-jira.atlassian.net/browse/OVIRT-2982 Project: oVirt - virtualization made easy Issue Type: Bug Reporter: Alertmanager_Bot Assignee: infra Labels: - alertname = InstanceUnreachable - beta_kubernetes_io_arch = amd64 - beta_kubernetes_io_os = linux - instance = shift-n12.phx.ovirt.org - job = kubernetes-nodes-exporter - kubernetes_io_hostname = shift-n12.phx.ovirt.org - node_role_kubernetes_io_compute = true - region = primary - type = vm - zone = ci Annotations: - description = shift-n12.phx.ovirt.org of job kubernetes-nodes-exporter has been possibly down for more than 10 minutes. Source: http://prometheus-0:9090/graph?g0.expr=up%7Bjob%3D%22kubernetes-nodes-expor… -- This message was sent by Atlassian Jira (v1001.0.0-SNAPSHOT#100134)

1 0

[JIRA] (OVIRT-2981) [FIRING:1] MirrorIsOutOfSync web-ovirt-mirrorchecker.apps.ovirt.org:80 oVirt-mirrors-health http://www.gtlib.gatech.edu/pub/oVirt/pub
by Alertmanager_Bot (oVirt JIRA) 23 Jul '20

23 Jul '20

Alertmanager_Bot created OVIRT-2981: --------------------------------------- Summary: [FIRING:1] MirrorIsOutOfSync web-ovirt-mirrorchecker.apps.ovirt.org:80 oVirt-mirrors-health http://www.gtlib.gatech.edu/pub/oVirt/pub Key: OVIRT-2981 URL: https://ovirt-jira.atlassian.net/browse/OVIRT-2981 Project: oVirt - virtualization made easy Issue Type: Bug Reporter: Alertmanager_Bot Assignee: infra Labels: - alertname = MirrorIsOutOfSync - instance = web-ovirt-mirrorchecker.apps.ovirt.org:80 - job = oVirt-mirrors-health - url = http://www.gtlib.gatech.edu/pub/oVirt/pub Annotations: - description = http://www.gtlib.gatech.edu/pub/oVirt/pub (measured by oVirt-mirrors-health) has not been synced in the past two days. Source: http://prometheus-0:9090/graph?g0.expr=mirror_health_check+%3E+172800&g0.ta… -- This message was sent by Atlassian Jira (v1001.0.0-SNAPSHOT#100134)

1 0

[JIRA] (OVIRT-2980) [FIRING:1] TestAlert
by Alertmanager_Bot (oVirt JIRA) 23 Jul '20

23 Jul '20

Alertmanager_Bot created OVIRT-2980: --------------------------------------- Summary: [FIRING:1] TestAlert Key: OVIRT-2980 URL: https://ovirt-jira.atlassian.net/browse/OVIRT-2980 Project: oVirt - virtualization made easy Issue Type: Bug Reporter: Alertmanager_Bot Assignee: infra Labels: - alertname = TestAlert - url = simple_url.com Annotations: Source: -- This message was sent by Atlassian Jira (v1001.0.0-SNAPSHOT#100134)

1 0

[JIRA] (OVIRT-2977) [FIRING:1] TestAlert
by Shlomi Zidmi (oVirt JIRA) 23 Jul '20

23 Jul '20

[ https://ovirt-jira.atlassian.net/browse/OVIRT-2977?page=com.atlassian.jira.… ] Shlomi Zidmi closed OVIRT-2977. ------------------------------- Resolution: Fixed > [FIRING:1] TestAlert > --------------------- > > Key: OVIRT-2977 > URL: https://ovirt-jira.atlassian.net/browse/OVIRT-2977 > Project: oVirt - virtualization made easy > Issue Type: Bug > Reporter: Alertmanager_Bot > Assignee: infra > Labels: ALERT{alertname="TestAlert"} > > Labels: > - alertname = TestAlert > - label1 = value1 > Annotations: > Source: -- This message was sent by Atlassian Jira (v1001.0.0-SNAPSHOT#100134)

1 0

[JIRA] (OVIRT-2979) a new project on jira for jiralert test tickets
by Shlomi Zidmi (oVirt JIRA) 23 Jul '20

23 Jul '20

Shlomi Zidmi created OVIRT-2979: ----------------------------------- Summary: a new project on jira for jiralert test tickets Key: OVIRT-2979 URL: https://ovirt-jira.atlassian.net/browse/OVIRT-2979 Project: oVirt - virtualization made easy Issue Type: New Feature Reporter: Shlomi Zidmi Assignee: infra Priority: Low This ticket is for creating a dummy project on Jira that could be linked with Jiralert. Such a project would be helpful in testing different use cases. -- This message was sent by Atlassian Jira (v1001.0.0-SNAPSHOT#100134)

1 0

[JIRA] (OVIRT-2978) [FIRING:1] InstanceUnreachable (amd64 linux shift-n12.phx.ovirt.org kubernetes-nodes-exporter shift-n12.phx.ovirt.org true primary vm ci)
by Alertmanager_Bot (oVirt JIRA) 23 Jul '20

23 Jul '20

Alertmanager_Bot created OVIRT-2978: --------------------------------------- Summary: [FIRING:1] InstanceUnreachable (amd64 linux shift-n12.phx.ovirt.org kubernetes-nodes-exporter shift-n12.phx.ovirt.org true primary vm ci) Key: OVIRT-2978 URL: https://ovirt-jira.atlassian.net/browse/OVIRT-2978 Project: oVirt - virtualization made easy Issue Type: Bug Reporter: Alertmanager_Bot Assignee: infra Labels: - alertname = InstanceUnreachable - beta_kubernetes_io_arch = amd64 - beta_kubernetes_io_os = linux - instance = shift-n12.phx.ovirt.org - job = kubernetes-nodes-exporter - kubernetes_io_hostname = shift-n12.phx.ovirt.org - node_role_kubernetes_io_compute = true - region = primary - type = vm - zone = ci Annotations: - description = shift-n12.phx.ovirt.org of job kubernetes-nodes-exporter has been possibly down for more than 10 minutes. Source: http://prometheus-0:9090/graph?g0.expr=up%7Bjob%3D%22kubernetes-nodes-expor… -- This message was sent by Atlassian Jira (v1001.0.0-SNAPSHOT#100134)

1 0

[JIRA] (OVIRT-2977) [FIRING:1] TestAlert
by Shlomi Zidmi (oVirt JIRA) 23 Jul '20

23 Jul '20

[ https://ovirt-jira.atlassian.net/browse/OVIRT-2977?page=com.atlassian.jira.… ] Shlomi Zidmi commented on OVIRT-2977: ------------------------------------- This ticket is for examining how Jiralert acts when there is already an open ticket with the same alert, but different labels and values > [FIRING:1] TestAlert > --------------------- > > Key: OVIRT-2977 > URL: https://ovirt-jira.atlassian.net/browse/OVIRT-2977 > Project: oVirt - virtualization made easy > Issue Type: Bug > Reporter: Alertmanager_Bot > Assignee: infra > Labels: ALERT{alertname="TestAlert"} > > Labels: > - alertname = TestAlert > - label1 = value1 > Annotations: > Source: -- This message was sent by Atlassian Jira (v1001.0.0-SNAPSHOT#100134)

1 0

[JIRA] (OVIRT-2977) [FIRING:1] TestAlert
by Shlomi Zidmi (oVirt JIRA) 23 Jul '20

23 Jul '20

[ https://ovirt-jira.atlassian.net/browse/OVIRT-2977?page=com.atlassian.jira.… ] Shlomi Zidmi updated OVIRT-2977: -------------------------------- Status: In Progress (was: To Do) > [FIRING:1] TestAlert > --------------------- > > Key: OVIRT-2977 > URL: https://ovirt-jira.atlassian.net/browse/OVIRT-2977 > Project: oVirt - virtualization made easy > Issue Type: Bug > Reporter: Alertmanager_Bot > Assignee: infra > Labels: ALERT{alertname="TestAlert"} > > Labels: > - alertname = TestAlert > - label1 = value1 > Annotations: > Source: -- This message was sent by Atlassian Jira (v1001.0.0-SNAPSHOT#100134)

1 0

[JIRA] (OVIRT-2977) [FIRING:1] TestAlert
by Alertmanager_Bot (oVirt JIRA) 23 Jul '20

23 Jul '20

Alertmanager_Bot created OVIRT-2977: --------------------------------------- Summary: [FIRING:1] TestAlert Key: OVIRT-2977 URL: https://ovirt-jira.atlassian.net/browse/OVIRT-2977 Project: oVirt - virtualization made easy Issue Type: Bug Reporter: Alertmanager_Bot Assignee: infra Labels: - alertname = TestAlert - label1 = value1 Annotations: Source: -- This message was sent by Atlassian Jira (v1001.0.0-SNAPSHOT#100134)

1 0

[oVirt Jenkins] ovirt-system-tests_basic-suite-4.3_nightly - Build # 211 - Failure!
by jenkins＠jenkins.phx.ovirt.org 23 Jul '20

23 Jul '20

Project: https://jenkins.ovirt.org/job/ovirt-system-tests_basic-suite-4.3_nightly/ Build: https://jenkins.ovirt.org/job/ovirt-system-tests_basic-suite-4.3_nightly/21… Build Number: 211 Build Status: Failure Triggered By: Started by timer ------------------------------------- Changes Since Last Success: ------------------------------------- Changes for Build #211 [Marcin Sobczyk] lago: Skip sysprep when ssh key is available ----------------- Failed Tests: ----------------- 1 tests failed. FAILED: 004_basic_sanity.disk_operations Error Message: False != True after 600 seconds -------------------- >> begin captured logging << -------------------- lago.utils: DEBUG: Error while running thread Thread-70 Traceback (most recent call last): File "/usr/lib/python2.7/site-packages/lago/utils.py", line 58, in _ret_via_queue queue.put({'return': func()}) File "/usr/lib/python2.7/site-packages/ovirtlago/testlib.py", line 60, in wrapper return func(get_test_prefix(), *args, **kwargs) File "/usr/lib/python2.7/site-packages/ovirtlago/testlib.py", line 79, in wrapper prefix.virt_env.engine_vm().get_api(api_ver=4), *args, **kwargs File "/home/jenkins/agent/workspace/ovirt-system-tests_basic-suite-4.3_nightly/ovirt-system-tests/basic-suite-4.3/test-scenarios/004_basic_sanity.py", line 508, in live_storage_migration testlib.assert_true_within_long(lambda: test_utils.all_jobs_finished(engine, correlation_id)) File "/usr/lib/python2.7/site-packages/ovirtlago/testlib.py", line 286, in assert_true_within_long assert_equals_within_long(func, True, allowed_exceptions) File "/usr/lib/python2.7/site-packages/ovirtlago/testlib.py", line 273, in assert_equals_within_long func, value, LONG_TIMEOUT, allowed_exceptions=allowed_exceptions File "/usr/lib/python2.7/site-packages/ovirtlago/testlib.py", line 252, in assert_equals_within '%s != %s after %s seconds' % (res, value, timeout) AssertionError: False != True after 600 seconds --------------------- >> end captured logging << --------------------- Stack Trace: File "/usr/lib64/python2.7/unittest/case.py", line 369, in run testMethod() File "/usr/lib/python2.7/site-packages/nose/case.py", line 197, in runTest self.test(*self.arg) File "/usr/lib/python2.7/site-packages/ovirtlago/testlib.py", line 142, in wrapped_test test() File "/usr/lib/python2.7/site-packages/ovirtlago/testlib.py", line 60, in wrapper return func(get_test_prefix(), *args, **kwargs) File "/usr/lib/python2.7/site-packages/ovirtlago/testlib.py", line 79, in wrapper prefix.virt_env.engine_vm().get_api(api_ver=4), *args, **kwargs File "/home/jenkins/agent/workspace/ovirt-system-tests_basic-suite-4.3_nightly/ovirt-system-tests/basic-suite-4.3/test-scenarios/004_basic_sanity.py", line 932, in disk_operations vt.join_all() File "/usr/lib/python2.7/site-packages/lago/utils.py", line 58, in _ret_via_queue queue.put({'return': func()}) File "/usr/lib/python2.7/site-packages/ovirtlago/testlib.py", line 60, in wrapper return func(get_test_prefix(), *args, **kwargs) File "/usr/lib/python2.7/site-packages/ovirtlago/testlib.py", line 79, in wrapper prefix.virt_env.engine_vm().get_api(api_ver=4), *args, **kwargs File "/home/jenkins/agent/workspace/ovirt-system-tests_basic-suite-4.3_nightly/ovirt-system-tests/basic-suite-4.3/test-scenarios/004_basic_sanity.py", line 508, in live_storage_migration testlib.assert_true_within_long(lambda: test_utils.all_jobs_finished(engine, correlation_id)) File "/usr/lib/python2.7/site-packages/ovirtlago/testlib.py", line 286, in assert_true_within_long assert_equals_within_long(func, True, allowed_exceptions) File "/usr/lib/python2.7/site-packages/ovirtlago/testlib.py", line 273, in assert_equals_within_long func, value, LONG_TIMEOUT, allowed_exceptions=allowed_exceptions File "/usr/lib/python2.7/site-packages/ovirtlago/testlib.py", line 252, in assert_equals_within '%s != %s after %s seconds' % (res, value, timeout) 'False != True after 600 seconds\n-------------------- >> begin captured logging << --------------------\nlago.utils: DEBUG: Error while running thread Thread-70\nTraceback (most recent call last):\n File "/usr/lib/python2.7/site-packages/lago/utils.py", line 58, in _ret_via_queue\n queue.put({\'return\': func()})\n File "/usr/lib/python2.7/site-packages/ovirtlago/testlib.py", line 60, in wrapper\n return func(get_test_prefix(), *args, **kwargs)\n File "/usr/lib/python2.7/site-packages/ovirtlago/testlib.py", line 79, in wrapper\n prefix.virt_env.engine_vm().get_api(api_ver=4), *args, **kwargs\n File "/home/jenkins/agent/workspace/ovirt-system-tests_basic-suite-4.3_nightly/ovirt-system-tests/basic-suite-4.3/test-scenarios/004_basic_sanity.py", line 508, in live_storage_migration\n testlib.assert_true_within_long(lambda: test_utils.all_jobs_finished(engine, correlation_id))\n File "/usr/lib/python2.7/site-packages/ovirtlago/testlib.py", line 286, in assert_true_within_long\n assert_equals_within_long(func, True, allowed_exceptions)\n File "/usr/lib/python2.7/site-packages/ovirtlago/testlib.py", line 273, in assert_equals_within_long\n func, value, LONG_TIMEOUT, allowed_exceptions=allowed_exceptions\n File "/usr/lib/python2.7/site-packages/ovirtlago/testlib.py", line 252, in assert_equals_within\n \'%s != %s after %s seconds\' % (res, value, timeout)\nAssertionError: False != True after 600 seconds\n--------------------- >> end captured logging << ---------------------'

1 1

oVirt infra daily report - unstable production jobs - 1260
by jenkins＠jenkins.phx.ovirt.org 23 Jul '20

23 Jul '20

Good morning! Attached is the HTML page with the jenkins status report. You can see it also here: - https://jenkins.ovirt.org/job/system_jenkins-report/1260//artifact/exported… Cheers, Jenkins

1 0

[JIRA] (OVIRT-2976) [FIRING:1] MirrorIsOutOfSync (web-ovirt-mirrorchecker.apps.ovirt.org:80 oVirt-mirrors-health http://www.gtlib.gatech.edu/pub/oVirt/pub)
by Alertmanager_Bot (oVirt JIRA) 22 Jul '20

22 Jul '20

Alertmanager_Bot created OVIRT-2976: --------------------------------------- Summary: [FIRING:1] MirrorIsOutOfSync (web-ovirt-mirrorchecker.apps.ovirt.org:80 oVirt-mirrors-health http://www.gtlib.gatech.edu/pub/oVirt/pub) Key: OVIRT-2976 URL: https://ovirt-jira.atlassian.net/browse/OVIRT-2976 Project: oVirt - virtualization made easy Issue Type: Bug Reporter: Alertmanager_Bot Assignee: infra Labels: - alertname = MirrorIsOutOfSync - instance = web-ovirt-mirrorchecker.apps.ovirt.org:80 - job = oVirt-mirrors-health - url = http://www.gtlib.gatech.edu/pub/oVirt/pub Annotations: - description = http://www.gtlib.gatech.edu/pub/oVirt/pub (measured by oVirt-mirrors-health) has not been synced in the past two days. Source: http://prometheus-0:9090/graph?g0.expr=mirror_health_check+%3E+172800&g0.ta… -- This message was sent by Atlassian Jira (v1001.0.0-SNAPSHOT#100133)

1 0

[JIRA] (OVIRT-2268) Enable IPv6 for resources.ovirt.org
by Anton Marchukov (oVirt JIRA) 22 Jul '20

22 Jul '20

[ https://ovirt-jira.atlassian.net/browse/OVIRT-2268?page=com.atlassian.jira.… ] Anton Marchukov commented on OVIRT-2268: ---------------------------------------- Since we cannot provide IPv6 on [resources.ovirt.org|http://resources.ovirt.org] let’s redefine this ticket to make sure IPv6-enabled users will get mirrorlist wit h IPv6-enables mirrors only. The target is to make sure IPv6-only people can get oVirt. > Enable IPv6 for resources.ovirt.org > ----------------------------------- > > Key: OVIRT-2268 > URL: https://ovirt-jira.atlassian.net/browse/OVIRT-2268 > Project: oVirt - virtualization made easy > Issue Type: Improvement > Components: oVirt Infra > Reporter: Dominik Holler > Assignee: infra > > To allow access to resources.ovirt.org in IPv6 only setups, IPv6 should be enabled for resources.ovirt.org. > This would allow users in IPv6 only setups to install oVirt. -- This message was sent by Atlassian Jira (v1001.0.0-SNAPSHOT#100133)

1 0

[JIRA] (OVIRT-2975) upload CirrOS 0.5.1 to glance
by Evgheni Dereveanchin (oVirt JIRA) 22 Jul '20

22 Jul '20

Evgheni Dereveanchin created OVIRT-2975: ------------------------------------------- Summary: upload CirrOS 0.5.1 to glance Key: OVIRT-2975 URL: https://ovirt-jira.atlassian.net/browse/OVIRT-2975 Project: oVirt - virtualization made easy Issue Type: Bug Reporter: Evgheni Dereveanchin Assignee: infra http://download.cirros-cloud.net/0.5.1/cirros-0.5.1-x86_64-disk.img -- This message was sent by Atlassian Jira (v1001.0.0-SNAPSHOT#100133)

1 0

Re: [ovirt-users] invalid spf record for ovirt.org
by Yedidyah Bar David 22 Jul '20

22 Jul '20

On Wed, Jul 22, 2020 at 3:57 PM Jorick Astrego <jorick(a)netbulae.eu> wrote: > Hi, > > During routine maintenance on our mailserver I noticed the following in > the log: > > [22/Jul/2020 14:33:33] Error when parsing SPF TXT record for domain: > ovirt.org, envelope-from=users-bounces(a)ovirt.org, message: Invalid > character found near "" in "66.187.233.88" > > A check on MXtoolbox also gives an invalid systax error: > > v=spf1 a:mail.ovirt.org a:gerrit.ovirt.org 66.187.233.88 ~all > > Prefix Type Value PrefixDesc Description Error > > v spf1 > The SPF record version > + a mail.ovirt.org Pass Match if IP has a DNS 'A' record in given domain. > + a gerrit.ovirt.org Pass Match if IP has a DNS 'A' record in given > domain. > + 66.187.233.88 > Pass Unknown Unknown mechanisms are not allowed > ~ all > SoftFail Always matches. It goes at the end of your record. > > > Test Result > > <https://mxtoolbox.com/problem/spf/spf-syntax-check?page=prob_spf&action=spf…> SPF > Syntax Check Invalid syntax found > > Thanks for the report. Adding some more relevant people. Best regards, -- Didi

1 0

[oVirt Jenkins] ovirt-system-tests_basic-suite-master_nightly - Build # 208 - Failure!
by jenkins＠jenkins.phx.ovirt.org 22 Jul '20

22 Jul '20

Project: https://jenkins.ovirt.org/job/ovirt-system-tests_basic-suite-master_nightly/ Build: https://jenkins.ovirt.org/job/ovirt-system-tests_basic-suite-master_nightly… Build Number: 208 Build Status: Failure Triggered By: Started by timer ------------------------------------- Changes Since Last Success: ------------------------------------- Changes for Build #208 [Marcin Sobczyk] performance: Fix broken links for pgdg repos ----------------- Failed Tests: ----------------- 1 tests failed. FAILED: basic-suite-master.test-scenarios.004_basic_sanity_pytest.test_live_storage_migration Error Message: AssertionError: False != True after 600 seconds Stack Trace: api_v4 = <ovirtsdk4.Connection object at 0x7f86a012a050> @order_by(_TEST_LIST) def test_live_storage_migration(api_v4): engine = api_v4.system_service() disk_service = test_utils.get_disk_service(engine, DISK0_NAME) correlation_id = 'live_storage_migration' disk_service.move( async=False, filter=False, storage_domain=types.StorageDomain( name=SD_ISCSI_NAME ), query={'correlation_id': correlation_id} ) testlib.assert_true_within_long(lambda: test_utils.all_jobs_finished(engine, correlation_id)) # Assert that the disk is on the correct storage domain, # its status is OK and the snapshot created for the migration # has been merged testlib.assert_true_within_long( lambda: api_v4.follow_link(disk_service.get().storage_domains[0]).name == SD_ISCSI_NAME ) vm0_snapshots_service = test_utils.get_vm_snapshots_service(engine, VM0_NAME) testlib.assert_true_within_long( > lambda: len(vm0_snapshots_service.list()) == 1 ) ../basic-suite-master/test-scenarios/004_basic_sanity_pytest.py:576: _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ /usr/lib/python2.7/site-packages/ovirtlago/testlib.py:286: in assert_true_within_long assert_equals_within_long(func, True, allowed_exceptions) /usr/lib/python2.7/site-packages/ovirtlago/testlib.py:273: in assert_equals_within_long func, value, LONG_TIMEOUT, allowed_exceptions=allowed_exceptions _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ func = <function <lambda> at 0x7f869c96daa0>, value = True, timeout = 600 allowed_exceptions = [], initial_wait = 0 def assert_equals_within( func, value, timeout, allowed_exceptions=None, initial_wait=10 ): allowed_exceptions = allowed_exceptions or [] with utils.EggTimer(timeout) as timer: while not timer.elapsed(): try: res = func() if res == value: return except Exception as exc: if _instance_of_any(exc, allowed_exceptions): time.sleep(3) continue LOGGER.exception("Unhandled exception in %s", func) raise if initial_wait == 0: time.sleep(3) else: time.sleep(initial_wait) initial_wait = 0 try: raise AssertionError( > '%s != %s after %s seconds' % (res, value, timeout) ) E AssertionError: False != True after 600 seconds /usr/lib/python2.7/site-packages/ovirtlago/testlib.py:252: AssertionError

1 1

oVirt infra daily report - unstable production jobs - 1259
by jenkins＠jenkins.phx.ovirt.org 22 Jul '20

22 Jul '20

Good morning! Attached is the HTML page with the jenkins status report. You can see it also here: - https://jenkins.ovirt.org/job/system_jenkins-report/1259//artifact/exported… Cheers, Jenkins

1 0

Build failed in Jenkins: system-sync_mirrors-fedora-updates-fc30-x86_64 #1319
by jenkins＠jenkins.phx.ovirt.org 22 Jul '20

22 Jul '20

See <https://jenkins.ovirt.org/job/system-sync_mirrors-fedora-updates-fc30-x86_6…> Changes: ------------------------------------------ Started by timer Running as SYSTEM [EnvInject] - Loading node environment variables. Building remotely on mirrors.phx.ovirt.org (mirrors) in workspace <https://jenkins.ovirt.org/job/system-sync_mirrors-fedora-updates-fc30-x86_6…> No credentials specified > git rev-parse --is-inside-work-tree # timeout=10 Fetching changes from the remote Git repository > git config remote.origin.url http://gerrit.ovirt.org/jenkins.git # timeout=10 Cleaning workspace > git rev-parse --verify HEAD # timeout=10 Resetting working tree > git reset --hard # timeout=10 > git clean -fdx # timeout=10 Pruning obsolete local branches Fetching upstream changes from http://gerrit.ovirt.org/jenkins.git > git --version # timeout=10 > git fetch --tags --progress --prune http://gerrit.ovirt.org/jenkins.git +refs/heads/*:refs/remotes/origin/* # timeout=10 > git rev-parse origin/master^{commit} # timeout=10 Checking out Revision e74e3d234ca539ec7d6be740575b720d5577e5a9 (origin/master) > git config core.sparsecheckout # timeout=10 > git checkout -f e74e3d234ca539ec7d6be740575b720d5577e5a9 # timeout=10 Commit message: "Removing rgolan(a)redhat.com from email notifs" > git rev-list --no-walk e74e3d234ca539ec7d6be740575b720d5577e5a9 # timeout=10 [system-sync_mirrors-fedora-updates-fc30-x86_64] $ /bin/bash -xe /tmp/jenkins141163348633797447.sh + jenkins/scripts/mirror_mgr.sh resync_yum_mirror fedora-updates-fc30 x86_64 jenkins/data/mirrors-reposync.conf + MIRRORS_MP_BASE=/var/www/html/repos + MIRRORS_HTTP_BASE=http://mirrors.phx.ovirt.org/repos + MIRRORS_CACHE=/home/jenkins/mirrors_cache + YUM_PATH=/yum/ + MIRRORS_BASE_PREFIX=/var/www/html/repos/yum/ + REPOQUERY_HTTP_BASE=http://mirrors.phx.ovirt.org/repos/yum/ + MAX_LOCK_ATTEMPTS=120 + LOCK_WAIT_INTERVAL=5 + LOCK_BASE=/home/jenkins + OLD_MD_TO_KEEP=100 + MAX_AGE=14 + HTTP_SELINUX_TYPE=httpd_sys_content_t + HTTP_FILE_MODE=644 + main resync_yum_mirror fedora-updates-fc30 x86_64 jenkins/data/mirrors-reposync.conf + local command=resync_yum_mirror + command_args=("${@:2}") + local command_args + cmd_resync_yum_mirror fedora-updates-fc30 x86_64 jenkins/data/mirrors-reposync.conf + local repo_name=fedora-updates-fc30 + local repo_archs=x86_64 + local reposync_conf=jenkins/data/mirrors-reposync.conf + local sync_needed + mkdir -p /home/jenkins/mirrors_cache + verify_repo_fs fedora-updates-fc30 yum + local repo_name=fedora-updates-fc30 + local repo_type=yum + sudo install -o jenkins -d /var/www/html/repos/yum /var/www/html/repos/yum/fedora-updates-fc30 /var/www/html/repos/yum/fedora-updates-fc30/base + modify_reposync_conf fedora-updates-fc30 jenkins/data/mirrors-reposync.conf + local repo_name=fedora-updates-fc30 + local reposync_conf=jenkins/data/mirrors-reposync.conf + local tmp_file=reposync.tmp + awk -v RS= -v 'ORS=\n\n' /main/ jenkins/data/mirrors-reposync.conf + awk -v RS= -v 'ORS=\n\n' /fedora-updates-fc30/ jenkins/data/mirrors-reposync.conf + mv reposync.tmp jenkins/data/mirrors-reposync.conf + check_yum_sync_needed fedora-updates-fc30 x86_64 jenkins/data/mirrors-reposync.conf sync_needed + local repo_name=fedora-updates-fc30 + local repo_archs=x86_64 + local reposync_conf=jenkins/data/mirrors-reposync.conf + local p_sync_needed=sync_needed + local reposync_out + echo 'Checking if mirror needs a resync' Checking if mirror needs a resync + rm -rf /home/jenkins/mirrors_cache/fedora-updates-fc30 ++ IFS=, ++ echo x86_64 + for arch in '$(IFS=,; echo $repo_archs)' ++ run_reposync fedora-updates-fc30 x86_64 jenkins/data/mirrors-reposync.conf --urls --quiet ++ local repo_name=fedora-updates-fc30 ++ local repo_arch=x86_64 ++ local reposync_conf=jenkins/data/mirrors-reposync.conf ++ extra_args=("${@:4}") ++ local extra_args ++ reposync --config=jenkins/data/mirrors-reposync.conf --repoid=fedora-updates-fc30 --arch=x86_64 --cachedir=/home/jenkins/mirrors_cache --download_path=/var/www/html/repos/yum/fedora-updates-fc30/base --norepopath --urls --quiet Error setting up repositories: Cannot retrieve metalink for repository: fedora-updates-fc30. Please verify its path and try again + reposync_out= Build step 'Execute shell' marked build as failure

1 1

[JIRA] (OVIRT-2974) Tickets aren't created for fired alerts in Prometheus
by Shlomi Zidmi (oVirt JIRA) 21 Jul '20

21 Jul '20

Shlomi Zidmi created OVIRT-2974: ----------------------------------- Summary: Tickets aren't created for fired alerts in Prometheus Key: OVIRT-2974 URL: https://ovirt-jira.atlassian.net/browse/OVIRT-2974 Project: oVirt - virtualization made easy Issue Type: Bug Reporter: Shlomi Zidmi Assignee: infra There is a broken node_exporter on one of openshift nodes (shift-n12), however we haven't been notified about it with a ticket. For some reason, even though all application are running (Prometheus, Alertmanager, Jiralert) no ticket has been created for the broken exporter. Opening a ticket for investigating the issue -- This message was sent by Atlassian Jira (v1001.0.0-SNAPSHOT#100133)

1 0

[oVirt Jenkins] ovirt-system-tests_performance-suite-master - Build # 1521 - Still Failing!
by jenkins＠jenkins.phx.ovirt.org 21 Jul '20

21 Jul '20

Project: https://jenkins.ovirt.org/job/ovirt-system-tests_performance-suite-master/ Build: https://jenkins.ovirt.org/job/ovirt-system-tests_performance-suite-master/1… Build Number: 1521 Build Status: Still Failing Triggered By: Started by timer ------------------------------------- Changes Since Last Success: ------------------------------------- Changes for Build #1516 [Marcin Sobczyk] network: Run network suite master on el8 Changes for Build #1517 [Andrej Cernek] network, hostlib: refactor setup_networks method [Barak Korren] usrc: Support *.yml files Changes for Build #1518 [Galit] Remove basic-suite-4.2 suite Changes for Build #1519 [Marcin Sobczyk] basic: Verify glance template creation before tearing down the engine Changes for Build #1520 [Marcin Sobczyk] ost-images: Use OST images when available Changes for Build #1521 [Ehud Yonasi] fix el7 container label. [Roy Golan] Removing rgolan(a)redhat.com from email notifs ----------------- Failed Tests: ----------------- 1 tests failed. FAILED: 1000_analyze_postgresql.analyze_postgres Error Message: Failed to perform the postgres analysis. Exit code is 1 -------------------- >> begin captured logging << -------------------- lago.plugins.vm: INFO: * Copy /home/jenkins/agent/workspace/ovirt-system-tests_performance-suite-master/ovirt-system-tests/performance-suite-master/../common/test-scenarios-files/analyze_postgresql.sh to lago-performance-suite-master-engine:/root: ?[0m?[0m lago.ssh: DEBUG: start task:a9b6bf76-d6a4-4f6d-938d-59d14cc805d8:Get ssh client for lago-performance-suite-master-engine: lago.ssh: DEBUG: end task:a9b6bf76-d6a4-4f6d-938d-59d14cc805d8:Get ssh client for lago-performance-suite-master-engine: lago.plugins.vm: INFO: * Copy /home/jenkins/agent/workspace/ovirt-system-tests_performance-suite-master/ovirt-system-tests/performance-suite-master/../common/test-scenarios-files/analyze_postgresql.sh to lago-performance-suite-master-engine:/root: ?[32mSuccess?[0m (in 0:00:00) lago.ssh: DEBUG: start task:3cbcc980-918f-4be9-ac5e-e10f86305521:Get ssh client for lago-performance-suite-master-engine: lago.ssh: DEBUG: end task:3cbcc980-918f-4be9-ac5e-e10f86305521:Get ssh client for lago-performance-suite-master-engine: lago.ssh: DEBUG: Running aee507f2 on lago-performance-suite-master-engine: /root/analyze_postgresql.sh lago.ssh: DEBUG: Command aee507f2 on lago-performance-suite-master-engine returned with 1 lago.ssh: DEBUG: Command aee507f2 on lago-performance-suite-master-engine output: Last metadata expiration check: 0:11:06 ago on Thu 16 Jul 2020 11:03:48 PM EDT. [MIRROR] pgdg-redhat-repo-latest.noarch.rpm: Status code: 404 for https://download.postgresql.org/pub/repos/yum/12/redhat/rhel-8-x86_64/pgdg-… (IP: 72.32.157.246) [MIRROR] pgdg-redhat-repo-latest.noarch.rpm: Status code: 404 for https://download.postgresql.org/pub/repos/yum/12/redhat/rhel-8-x86_64/pgdg-… (IP: 72.32.157.246) [MIRROR] pgdg-redhat-repo-latest.noarch.rpm: Status code: 404 for https://download.postgresql.org/pub/repos/yum/12/redhat/rhel-8-x86_64/pgdg-… (IP: 72.32.157.246) [MIRROR] pgdg-redhat-repo-latest.noarch.rpm: Status code: 404 for https://download.postgresql.org/pub/repos/yum/12/redhat/rhel-8-x86_64/pgdg-… (IP: 72.32.157.246) [FAILED] pgdg-redhat-repo-latest.noarch.rpm: Status code: 404 for https://download.postgresql.org/pub/repos/yum/12/redhat/rhel-8-x86_64/pgdg-… (IP: 72.32.157.246) lago.ssh: DEBUG: Command aee507f2 on lago-performance-suite-master-engine errors: + pg_datadir=/var/lib/pgsql/data + pg_service=postgresql.service + pgdg= + scl_enable= ++ get_engine_pg_scl ++ engine_pg_scl_conf=(/etc/ovirt-engine/engine.conf.d/*scl-postgres*.conf) ++ local engine_pg_scl_conf ++ local res= ++ [[ -e /etc/ovirt-engine/engine.conf.d/*scl-postgres*.conf ]] ++ echo '' + engine_pg_scl= + [[ -n '' ]] ++ rpm -q postgresql + pgver=postgresql-12.1-2.module_el8.1.0+273+979c16e6.x86_64 + case "${pgver}" in + pgdg=https://download.postgresql.org/pub/repos/yum/12/redhat/rhel-8-x86_64/… + [[ -n https://download.postgresql.org/pub/repos/yum/12/redhat/rhel-8-x86_64/pgdg-… ]] + yum install -y https://download.postgresql.org/pub/repos/yum/12/redhat/rhel-8-x86_64/pgdg-… Status code: 404 for https://download.postgresql.org/pub/repos/yum/12/redhat/rhel-8-x86_64/pgdg-… (IP: 72.32.157.246) --------------------- >> end captured logging << --------------------- Stack Trace: File "/usr/lib64/python2.7/unittest/case.py", line 369, in run testMethod() File "/usr/lib/python2.7/site-packages/nose/case.py", line 197, in runTest self.test(*self.arg) File "/usr/lib/python2.7/site-packages/ovirtlago/testlib.py", line 142, in wrapped_test test() File "/usr/lib/python2.7/site-packages/ovirtlago/testlib.py", line 60, in wrapper return func(get_test_prefix(), *args, **kwargs) File "/home/jenkins/agent/workspace/ovirt-system-tests_performance-suite-master/ovirt-system-tests/performance-suite-master/test-scenarios/1000_analyze_postgresql.py", line 52, in analyze_postgres ' Exit code is %s' % result.code File "/usr/lib/python2.7/site-packages/nose/tools/trivial.py", line 29, in eq_ raise AssertionError(msg or "%r != %r" % (a, b)) 'Failed to perform the postgres analysis. Exit code is 1\n-------------------- >> begin captured logging << --------------------\nlago.plugins.vm: INFO: * Copy /home/jenkins/agent/workspace/ovirt-system-tests_performance-suite-master/ovirt-system-tests/performance-suite-master/../common/test-scenarios-files/analyze_postgresql.sh to lago-performance-suite-master-engine:/root: \x1b[0m\x1b[0m\nlago.ssh: DEBUG: start task:a9b6bf76-d6a4-4f6d-938d-59d14cc805d8:Get ssh client for lago-performance-suite-master-engine:\nlago.ssh: DEBUG: end task:a9b6bf76-d6a4-4f6d-938d-59d14cc805d8:Get ssh client for lago-performance-suite-master-engine:\nlago.plugins.vm: INFO: * Copy /home/jenkins/agent/workspace/ovirt-system-tests_performance-suite-master/ovirt-system-tests/performance-suite-master/../common/test-scenarios-files/analyze_postgresql.sh to lago-performance-suite-master-engine:/root: \x1b[32mSuccess\x1b[0m (in 0:00:00)\nlago.ssh: DEBUG: start task:3cbcc980-918f-4be9-ac5e-e10f86305521:Get ssh client for lago-performance-suite-master-engine:\nlago.ssh: DEBUG: end task:3cbcc980-918f-4be9-ac5e-e10f86305521:Get ssh client for lago-performance-suite-master-engine:\nlago.ssh: DEBUG: Running aee507f2 on lago-performance-suite-master-engine: /root/analyze_postgresql.sh\nlago.ssh: DEBUG: Command aee507f2 on lago-performance-suite-master-engine returned with 1\nlago.ssh: DEBUG: Command aee507f2 on lago-performance-suite-master-engine output:\n Last metadata expiration check: 0:11:06 ago on Thu 16 Jul 2020 11:03:48 PM EDT.\n[MIRROR] pgdg-redhat-repo-latest.noarch.rpm: Status code: 404 for https://download.postgresql.org/pub/repos/yum/12/redhat/rhel-8-x86_64/pgdg-… (IP: 72.32.157.246)\n[MIRROR] pgdg-redhat-repo-latest.noarch.rpm: Status code: 404 for https://download.postgresql.org/pub/repos/yum/12/redhat/rhel-8-x86_64/pgdg-… (IP: 72.32.157.246)\n[MIRROR] pgdg-redhat-repo-latest.noarch.rpm: Status code: 404 for https://download.postgresql.org/pub/repos/yum/12/redhat/rhel-8-x86_64/pgdg-… (IP: 72.32.157.246)\n[MIRROR] pgdg-redhat-repo-latest.noarch.rpm: Status code: 404 for https://download.postgresql.org/pub/repos/yum/12/redhat/rhel-8-x86_64/pgdg-… (IP: 72.32.157.246)\n[FAILED] pgdg-redhat-repo-latest.noarch.rpm: Status code: 404 for https://download.postgresql.org/pub/repos/yum/12/redhat/rhel-8-x86_64/pgdg-… (IP: 72.32.157.246)\n\nlago.ssh: DEBUG: Command aee507f2 on lago-performance-suite-master-engine errors:\n + pg_datadir=/var/lib/pgsql/data\n+ pg_service=postgresql.service\n+ pgdg=\n+ scl_enable=\n++ get_engine_pg_scl\n++ engine_pg_scl_conf=(/etc/ovirt-engine/engine.conf.d/*scl-postgres*.conf)\n++ local engine_pg_scl_conf\n++ local res=\n++ [[ -e /etc/ovirt-engine/engine.conf.d/*scl-postgres*.conf ]]\n++ echo \'\'\n+ engine_pg_scl=\n+ [[ -n \'\' ]]\n++ rpm -q postgresql\n+ pgver=postgresql-12.1-2.module_el8.1.0+273+979c16e6.x86_64\n+ case "${pgver}" in\n+ pgdg=https://download.postgresql.org/pub/repos/yum/12/redhat/rhel-8-x86_64/… [[ -n https://download.postgresql.org/pub/repos/yum/12/redhat/rhel-8-x86_64/pgdg-… ]]\n+ yum install -y https://download.postgresql.org/pub/repos/yum/12/redhat/rhel-8-x86_64/pgdg-… code: 404 for https://download.postgresql.org/pub/repos/yum/12/redhat/rhel-8-x86_64/pgdg-… (IP: 72.32.157.246)\n\n--------------------- >> end captured logging << ---------------------'

1 4

oVirt infra daily report - unstable production jobs - 1258
by jenkins＠jenkins.phx.ovirt.org 21 Jul '20

21 Jul '20

Good morning! Attached is the HTML page with the jenkins status report. You can see it also here: - https://jenkins.ovirt.org/job/system_jenkins-report/1258//artifact/exported… Cheers, Jenkins

1 0

[JIRA] (OVIRT-2972) resources.ovirt.org ran out of space for artifact publishing
by Evgheni Dereveanchin (oVirt JIRA) 20 Jul '20

20 Jul '20

[ https://ovirt-jira.atlassian.net/browse/OVIRT-2972?page=com.atlassian.jira.… ] Evgheni Dereveanchin commented on OVIRT-2972: --------------------------------------------- Suspect jobs that were reporting issues this weekend: [https://jenkins.ovirt.org/job/ovirt_master_publish-rpms_nightly/|https://je…] [https://jenkins.ovirt.org/job/ovirt_4.3_publish-rpms_nightly/|https://jenki…] [https://jenkins.ovirt.org/job/ovirt_4.2_publish-rpms_nightly/|https://jenki…] The errors are as follows: {noformat}13:15:32 2020-07-19 11:15:32,180::INFO::root::Saving /home/jenkins/ovirt-master-snapshot.tmp/rpm/fc29/x86_64/ovirt-engine-wildfly-18.0.0-1.fc29.x86_64.rpm 13:15:32 Traceback (most recent call last): 13:15:32 File "/bin/repoman", line 10, in <module> 13:15:32 sys.exit(main()) 13:15:32 File "/usr/lib/python2.7/site-packages/repoman/cmd.py", line 461, in main 13:15:32 exit_code = do_add(args, config, repo) 13:15:32 File "/usr/lib/python2.7/site-packages/repoman/cmd.py", line 361, in do_add 13:15:32 repo.save() 13:15:32 File "/usr/lib/python2.7/site-packages/repoman/common/repo.py", line 42, in _func 13:15:32 return func(self, *args, **kwargs) 13:15:32 File "/usr/lib/python2.7/site-packages/repoman/common/repo.py", line 201, in save 13:15:32 store.save() 13:15:32 File "/usr/lib/python2.7/site-packages/repoman/common/stores/RPM/__init__.py", line 263, in save 13:15:32 self._save(**args) 13:15:32 File "/usr/lib/python2.7/site-packages/repoman/common/stores/RPM/__init__.py", line 302, in _save 13:15:32 save_file(pkg.path, dst_path) 13:15:32 File "/usr/lib/python2.7/site-packages/repoman/common/utils.py", line 436, in save_file 13:15:32 copy(src_path, dst_path) 13:15:32 File "/usr/lib/python2.7/site-packages/repoman/common/utils.py", line 296, in copy 13:15:32 shutil.copy2(what, where) 13:15:32 File "/usr/lib64/python2.7/shutil.py", line 130, in copy2 13:15:32 copyfile(src, dst) 13:15:32 File "/usr/lib64/python2.7/shutil.py", line 84, in copyfile 13:15:32 copyfileobj(fsrc, fdst) 13:15:32 File "/usr/lib64/python2.7/shutil.py", line 52, in copyfileobj 13:15:32 fdst.write(buf) 13:15:32 IOError: [Errno 28] No space left on device 13:15:32 2020-07-19 11:15:32,917::INFO::repoman.common.repo::Cleaning up temporary dir /tmp/tmpRKo543 13:15:34 Waiting for scan_for_artifacts.sh cron job to finish (attempt 1 of 50) {noformat} looks like the job ran out of tmpfs space - did we publish too many files for some reason? Not entirely sure what tmp is used for in these jobs. > resources.ovirt.org ran out of space for artifact publishing > ------------------------------------------------------------ > > Key: OVIRT-2972 > URL: https://ovirt-jira.atlassian.net/browse/OVIRT-2972 > Project: oVirt - virtualization made easy > Issue Type: Bug > Reporter: Evgheni Dereveanchin > Assignee: infra > Attachments: image-20200720-083159.png, prom.png > > > This weekend Nagios sent disk space alerts for the /home/jenkins on resources.ovirt.org > AFAIR this is used as intermediate artifact storage for publishing and shouldn't fill up that much. Logging a ticket to investigate the root cause. > The partition is free again now so this is not blocking anything right now. -- This message was sent by Atlassian Jira (v1001.0.0-SNAPSHOT#100133)

1 0

[JIRA] (OVIRT-2972) resources.ovirt.org ran out of space for artifact publishing
by Evgheni Dereveanchin (oVirt JIRA) 20 Jul '20

20 Jul '20

[ https://ovirt-jira.atlassian.net/browse/OVIRT-2972?page=com.atlassian.jira.… ] Evgheni Dereveanchin commented on OVIRT-2972: --------------------------------------------- I did get emails from Nagios so we were alerted, just want to confirm the cause here as that’s something that doesn’t happen often: looks like some publishing job failed and did not clean up after itself which caused the next one to bring in even more files and fill up the partition. > resources.ovirt.org ran out of space for artifact publishing > ------------------------------------------------------------ > > Key: OVIRT-2972 > URL: https://ovirt-jira.atlassian.net/browse/OVIRT-2972 > Project: oVirt - virtualization made easy > Issue Type: Bug > Reporter: Evgheni Dereveanchin > Assignee: infra > Attachments: image-20200720-083159.png, prom.png > > > This weekend Nagios sent disk space alerts for the /home/jenkins on resources.ovirt.org > AFAIR this is used as intermediate artifact storage for publishing and shouldn't fill up that much. Logging a ticket to investigate the root cause. > The partition is free again now so this is not blocking anything right now. -- This message was sent by Atlassian Jira (v1001.0.0-SNAPSHOT#100133)

1 0

[JIRA] (OVIRT-2973) [FIRING:1] TestAlert
by Shlomi Zidmi (oVirt JIRA) 20 Jul '20

20 Jul '20

[ https://ovirt-jira.atlassian.net/browse/OVIRT-2973?page=com.atlassian.jira.… ] Shlomi Zidmi updated OVIRT-2973: -------------------------------- Resolution: Fixed Status: Done (was: To Do) Making sure jiralert is functioning properly > [FIRING:1] TestAlert > --------------------- > > Key: OVIRT-2973 > URL: https://ovirt-jira.atlassian.net/browse/OVIRT-2973 > Project: oVirt - virtualization made easy > Issue Type: Bug > Reporter: Alertmanager_Bot > Assignee: infra > Labels: ALERT{alertname="TestAlert"} > > Labels: > - alertname = TestAlert > - key = value > Annotations: > Source: -- This message was sent by Atlassian Jira (v1001.0.0-SNAPSHOT#100133)

1 0

[JIRA] (OVIRT-2973) [FIRING:1] TestAlert
by Alertmanager_Bot (oVirt JIRA) 20 Jul '20

20 Jul '20

Alertmanager_Bot created OVIRT-2973: --------------------------------------- Summary: [FIRING:1] TestAlert Key: OVIRT-2973 URL: https://ovirt-jira.atlassian.net/browse/OVIRT-2973 Project: oVirt - virtualization made easy Issue Type: Bug Reporter: Alertmanager_Bot Assignee: infra Labels: - alertname = TestAlert - key = value Annotations: Source: -- This message was sent by Atlassian Jira (v1001.0.0-SNAPSHOT#100133)

1 0

[JIRA] (OVIRT-2972) resources.ovirt.org ran out of space for artifact publishing
by Shlomi Zidmi (oVirt JIRA) 20 Jul '20

20 Jul '20

[ https://ovirt-jira.atlassian.net/browse/OVIRT-2972?page=com.atlassian.jira.… ] Shlomi Zidmi commented on OVIRT-2972: ------------------------------------- Actually we are monitoring [resources.ovirt.org|http://resources.ovirt.org] in Prometheus, and should have caught this on time !prom.png|width=83.33333333333334%! Looking at the alert rules defined, i see the following: {noformat}node_filesystem_free_bytes / node_filesystem_size_bytes{device!="tmpfs"} < 0.1{noformat} The blue line (/dev/sdb - mounted on /home/jenkins) in the picture is slightly above 0.1, and that’s why no alerts were fired. I’ll readjust the threshold so that Prometheus would be able to catch similar issues sooner next time. > resources.ovirt.org ran out of space for artifact publishing > ------------------------------------------------------------ > > Key: OVIRT-2972 > URL: https://ovirt-jira.atlassian.net/browse/OVIRT-2972 > Project: oVirt - virtualization made easy > Issue Type: Bug > Reporter: Evgheni Dereveanchin > Assignee: infra > Attachments: image-20200720-083159.png, prom.png > > > This weekend Nagios sent disk space alerts for the /home/jenkins on resources.ovirt.org > AFAIR this is used as intermediate artifact storage for publishing and shouldn't fill up that much. Logging a ticket to investigate the root cause. > The partition is free again now so this is not blocking anything right now. -- This message was sent by Atlassian Jira (v1001.0.0-SNAPSHOT#100133)

1 0

[JIRA] (OVIRT-2972) resources.ovirt.org ran out of space for artifact publishing
by Galit Rosenthal (oVirt JIRA) 20 Jul '20

20 Jul '20

[ https://ovirt-jira.atlassian.net/browse/OVIRT-2972?page=com.atlassian.jira.… ] Galit Rosenthal edited comment on OVIRT-2972 at 7/20/20 9:13 AM: ----------------------------------------------------------------- When I fixed the issues, the nagios already send a message that the disk space came back to normal. I just removed the *.ready directory that prevented from the publish to work was (Author: grosenth(a)redhat.com) When I fixed the issues, the jenkins already send a message that the disk space came back to normal. I just removed the *.ready directory that prevented from the publish to work > resources.ovirt.org ran out of space for artifact publishing > ------------------------------------------------------------ > > Key: OVIRT-2972 > URL: https://ovirt-jira.atlassian.net/browse/OVIRT-2972 > Project: oVirt - virtualization made easy > Issue Type: Bug > Reporter: Evgheni Dereveanchin > Assignee: infra > Attachments: image-20200720-083159.png > > > This weekend Nagios sent disk space alerts for the /home/jenkins on resources.ovirt.org > AFAIR this is used as intermediate artifact storage for publishing and shouldn't fill up that much. Logging a ticket to investigate the root cause. > The partition is free again now so this is not blocking anything right now. -- This message was sent by Atlassian Jira (v1001.0.0-SNAPSHOT#100133)

1 0

[JIRA] (OVIRT-2972) resources.ovirt.org ran out of space for artifact publishing
by Galit Rosenthal (oVirt JIRA) 20 Jul '20

20 Jul '20

[ https://ovirt-jira.atlassian.net/browse/OVIRT-2972?page=com.atlassian.jira.… ] Galit Rosenthal commented on OVIRT-2972: ---------------------------------------- When I fixed the issues, the jenkins already send a message that the disk space came back to normal. I just removed the *.ready directory that prevented from the publish to work > resources.ovirt.org ran out of space for artifact publishing > ------------------------------------------------------------ > > Key: OVIRT-2972 > URL: https://ovirt-jira.atlassian.net/browse/OVIRT-2972 > Project: oVirt - virtualization made easy > Issue Type: Bug > Reporter: Evgheni Dereveanchin > Assignee: infra > Attachments: image-20200720-083159.png > > > This weekend Nagios sent disk space alerts for the /home/jenkins on resources.ovirt.org > AFAIR this is used as intermediate artifact storage for publishing and shouldn't fill up that much. Logging a ticket to investigate the root cause. > The partition is free again now so this is not blocking anything right now. -- This message was sent by Atlassian Jira (v1001.0.0-SNAPSHOT#100133)

1 0

ovirt-engine-wildfly 19.1.0-2 publishing
by Yedidyah Bar David 20 Jul '20

20 Jul '20

Hi all, 1. Yesterday mperina pushed and merged a patch [1] to hopefully solve a rather-long-time breakage of, among others, he-basic-suite. I still do not see it in master-snapshot [2] - only 19.1.0-1. I do see it in tested [3]. Can someone please publish it to snapshot? 2. I tried to debug this myself a bit, but failed to find documentation for the publishing part. A few months ago it was said (not very clearly/officially, IIRC) that we do not anymore pass built-artifacts results through OST, because it's too fragile. Did we update docs? Where can I find a clear overview of the process from pressing Merge to having an rpm in master snapshot? Thanks and best regards, [1] https://gerrit.ovirt.org/110324 [2] https://resources.ovirt.org/pub/ovirt-master-snapshot/rpm/el8/ [3] https://resources.ovirt.org/repos/ovirt/tested/master/rpm/el8/ -- Didi

3 4

[JIRA] (OVIRT-2972) resources.ovirt.org ran out of space for artifact publishing
by Ehud Yonasi (oVirt JIRA) 20 Jul '20

20 Jul '20

[ https://ovirt-jira.atlassian.net/browse/OVIRT-2972?page=com.atlassian.jira.… ] Ehud Yonasi commented on OVIRT-2972: ------------------------------------ I read that on Sunday the repo was not accessible. I think [~accountid:5b25ec3f8b08c009c48b25c9] did cleanup there and might add more info on what used a lot of storage. > resources.ovirt.org ran out of space for artifact publishing > ------------------------------------------------------------ > > Key: OVIRT-2972 > URL: https://ovirt-jira.atlassian.net/browse/OVIRT-2972 > Project: oVirt - virtualization made easy > Issue Type: Bug > Reporter: Evgheni Dereveanchin > Assignee: infra > Attachments: image-20200720-083159.png > > > This weekend Nagios sent disk space alerts for the /home/jenkins on resources.ovirt.org > AFAIR this is used as intermediate artifact storage for publishing and shouldn't fill up that much. Logging a ticket to investigate the root cause. > The partition is free again now so this is not blocking anything right now. -- This message was sent by Atlassian Jira (v1001.0.0-SNAPSHOT#100133)

1 0

[JIRA] (OVIRT-2972) resources.ovirt.org ran out of space for artifact publishing
by Evgheni Dereveanchin (oVirt JIRA) 20 Jul '20

20 Jul '20

[ https://ovirt-jira.atlassian.net/browse/OVIRT-2972?page=com.atlassian.jira.… ] Evgheni Dereveanchin commented on OVIRT-2972: --------------------------------------------- !image-20200720-083159.png|width=597,height=215! Here’s the usage graph, looks like something happened on the night of 17th that caused files not to clean up, then repeated on the 18th and has been fixed during the 19th. [~accountid:5aa0f39f5a4d022884128a0f] [~accountid:5c977456c430371a3c67dbf6] did you get any reports on Sunday? May this be related to some nightly jobs? > resources.ovirt.org ran out of space for artifact publishing > ------------------------------------------------------------ > > Key: OVIRT-2972 > URL: https://ovirt-jira.atlassian.net/browse/OVIRT-2972 > Project: oVirt - virtualization made easy > Issue Type: Bug > Reporter: Evgheni Dereveanchin > Assignee: infra > Attachments: image-20200720-083159.png > > > This weekend Nagios sent disk space alerts for the /home/jenkins on resources.ovirt.org > AFAIR this is used as intermediate artifact storage for publishing and shouldn't fill up that much. Logging a ticket to investigate the root cause. > The partition is free again now so this is not blocking anything right now. -- This message was sent by Atlassian Jira (v1001.0.0-SNAPSHOT#100133)

1 0

[JIRA] (OVIRT-2972) resources.ovirt.org ran out of space for artifact publishing
by Evgheni Dereveanchin (oVirt JIRA) 20 Jul '20

20 Jul '20

Evgheni Dereveanchin created OVIRT-2972: ------------------------------------------- Summary: resources.ovirt.org ran out of space for artifact publishing Key: OVIRT-2972 URL: https://ovirt-jira.atlassian.net/browse/OVIRT-2972 Project: oVirt - virtualization made easy Issue Type: Bug Reporter: Evgheni Dereveanchin Assignee: infra This weekend Nagios sent disk space alerts for the /home/jenkins on resources.ovirt.org AFAIR this is used as intermediate artifact storage for publishing and shouldn't fill up that much. Logging a ticket to investigate the root cause. The partition is free again now so this is not blocking anything right now. -- This message was sent by Atlassian Jira (v1001.0.0-SNAPSHOT#100133)

1 0

[oVirt Jenkins] ovirt-system-tests_basic-suite-master_nightly - Build # 204 - Failure!
by jenkins＠jenkins.phx.ovirt.org 20 Jul '20

20 Jul '20

Project: https://jenkins.ovirt.org/job/ovirt-system-tests_basic-suite-master_nightly/ Build: https://jenkins.ovirt.org/job/ovirt-system-tests_basic-suite-master_nightly… Build Number: 204 Build Status: Failure Triggered By: Started by timer ------------------------------------- Changes Since Last Success: ------------------------------------- Changes for Build #204 [Marcin Sobczyk] ost_utils: Add ansible fixtures ----------------- Failed Tests: ----------------- No tests ran.

1 1

oVirt infra daily report - unstable production jobs - 1257
by jenkins＠jenkins.phx.ovirt.org 20 Jul '20

20 Jul '20

Good morning! Attached is the HTML page with the jenkins status report. You can see it also here: - https://jenkins.ovirt.org/job/system_jenkins-report/1257//artifact/exported… Cheers, Jenkins

1 0

Jenkins build became unstable: ovirt_4.2_publish-rpms_nightly #1047
by jenkins＠jenkins.phx.ovirt.org 19 Jul '20

19 Jul '20

See <https://jenkins.ovirt.org/job/ovirt_4.2_publish-rpms_nightly/1047/display/r…>

1 2

Jenkins build became unstable: ovirt_4.3_publish-rpms_nightly #561
by jenkins＠jenkins.phx.ovirt.org 19 Jul '20

19 Jul '20

See <https://jenkins.ovirt.org/job/ovirt_4.3_publish-rpms_nightly/561/display/re…>

1 2

Jenkins build became unstable: ovirt_master_publish-rpms_nightly #1803
by jenkins＠jenkins.phx.ovirt.org 19 Jul '20

19 Jul '20

See <https://jenkins.ovirt.org/job/ovirt_master_publish-rpms_nightly/1803/displa…>

1 4

[oVirt Jenkins] ovirt-system-tests_basic-suite-master_nightly - Build # 198 - Failure!
by jenkins＠jenkins.phx.ovirt.org 19 Jul '20

19 Jul '20

Project: https://jenkins.ovirt.org/job/ovirt-system-tests_basic-suite-master_nightly/ Build: https://jenkins.ovirt.org/job/ovirt-system-tests_basic-suite-master_nightly… Build Number: 198 Build Status: Failure Triggered By: Started by timer ------------------------------------- Changes Since Last Success: ------------------------------------- Changes for Build #198 [Marcin Sobczyk] ost_utils: Add ansible fixtures ----------------- Failed Tests: ----------------- 1 tests failed. FAILED: basic-suite-master.test-scenarios.002_bootstrap_pytest.test_verify_add_hosts Error Message: RuntimeError: 1 hosts failed installation: lago-basic-suite-master-host-1: install_failed Stack Trace: api_v4 = <ovirtsdk4.Connection object at 0x7fc206ada8d0> @order_by(_TEST_LIST) def test_verify_add_hosts(api_v4): hosts_service = api_v4.system_service().hosts_service() hosts_status = hosts_service.list(search='datacenter={}'.format(DC_NAME)) total_hosts = len(hosts_status) dump_hosts = _host_status_to_print(hosts_service, hosts_status) LOGGER.debug('Host status, verify_add_hosts:\n {}'.format(dump_hosts)) testlib.assert_true_within( lambda: _single_host_up(hosts_service, total_hosts), > timeout=constants.ADD_HOST_TIMEOUT ) ../basic-suite-master/test-scenarios/002_bootstrap_pytest.py:433: _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ /usr/lib/python2.7/site-packages/ovirtlago/testlib.py:278: in assert_true_within assert_equals_within(func, True, timeout, allowed_exceptions) /usr/lib/python2.7/site-packages/ovirtlago/testlib.py:234: in assert_equals_within res = func() ../basic-suite-master/test-scenarios/002_bootstrap_pytest.py:432: in <lambda> lambda: _single_host_up(hosts_service, total_hosts), ../basic-suite-master/test-scenarios/002_bootstrap_pytest.py:229: in _single_host_up _check_problematic_hosts(hosts_service) _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ hosts_service = <ovirtsdk4.services.HostsService object at 0x7fc206c4d350> def _check_problematic_hosts(hosts_service): problematic_hosts = hosts_service.list(search='datacenter={} AND status != installing and status != initializing and status != up'.format(DC_NAME)) if len(problematic_hosts): dump_hosts = '%s hosts failed installation:\n' % len(problematic_hosts) for host in problematic_hosts: host_service = hosts_service.host_service(host.id) dump_hosts += '%s: %s\n' % (host.name, host_service.get().status) > raise RuntimeError(dump_hosts) E RuntimeError: 1 hosts failed installation: E lago-basic-suite-master-host-1: install_failed ../basic-suite-master/test-scenarios/002_bootstrap_pytest.py:238: RuntimeError

1 2

oVirt infra daily report - unstable production jobs - 1256
by jenkins＠jenkins.phx.ovirt.org 19 Jul '20

19 Jul '20

Good morning! Attached is the HTML page with the jenkins status report. You can see it also here: - https://jenkins.ovirt.org/job/system_jenkins-report/1256//artifact/exported… Cheers, Jenkins

1 0

[oVirt Jenkins] ovirt-system-tests_basic-suite-master_nightly - Build # 196 - Failure!
by jenkins＠jenkins.phx.ovirt.org 18 Jul '20

18 Jul '20

Project: https://jenkins.ovirt.org/job/ovirt-system-tests_basic-suite-master_nightly/ Build: https://jenkins.ovirt.org/job/ovirt-system-tests_basic-suite-master_nightly… Build Number: 196 Build Status: Failure Triggered By: Started by timer ------------------------------------- Changes Since Last Success: ------------------------------------- Changes for Build #196 [Ehud Yonasi] fix el7 container label. ----------------- Failed Tests: ----------------- 1 tests failed. FAILED: basic-suite-master.test-scenarios.002_bootstrap_pytest.test_verify_add_hosts Error Message: RuntimeError: 1 hosts failed installation: lago-basic-suite-master-host-1: install_failed Stack Trace: api_v4 = <ovirtsdk4.Connection object at 0x7ffba7fe4e10> @order_by(_TEST_LIST) def test_verify_add_hosts(api_v4): hosts_service = api_v4.system_service().hosts_service() hosts_status = hosts_service.list(search='datacenter={}'.format(DC_NAME)) total_hosts = len(hosts_status) dump_hosts = _host_status_to_print(hosts_service, hosts_status) LOGGER.debug('Host status, verify_add_hosts:\n {}'.format(dump_hosts)) testlib.assert_true_within( lambda: _single_host_up(hosts_service, total_hosts), > timeout=constants.ADD_HOST_TIMEOUT ) ../basic-suite-master/test-scenarios/002_bootstrap_pytest.py:433: _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ /usr/lib/python2.7/site-packages/ovirtlago/testlib.py:278: in assert_true_within assert_equals_within(func, True, timeout, allowed_exceptions) /usr/lib/python2.7/site-packages/ovirtlago/testlib.py:234: in assert_equals_within res = func() ../basic-suite-master/test-scenarios/002_bootstrap_pytest.py:432: in <lambda> lambda: _single_host_up(hosts_service, total_hosts), ../basic-suite-master/test-scenarios/002_bootstrap_pytest.py:229: in _single_host_up _check_problematic_hosts(hosts_service) _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ hosts_service = <ovirtsdk4.services.HostsService object at 0x7ffba813f850> def _check_problematic_hosts(hosts_service): problematic_hosts = hosts_service.list(search='datacenter={} AND status != installing and status != initializing and status != up'.format(DC_NAME)) if len(problematic_hosts): dump_hosts = '%s hosts failed installation:\n' % len(problematic_hosts) for host in problematic_hosts: host_service = hosts_service.host_service(host.id) dump_hosts += '%s: %s\n' % (host.name, host_service.get().status) > raise RuntimeError(dump_hosts) E RuntimeError: 1 hosts failed installation: E lago-basic-suite-master-host-1: install_failed ../basic-suite-master/test-scenarios/002_bootstrap_pytest.py:238: RuntimeError

1 1

oVirt infra daily report - unstable production jobs - 1255
by jenkins＠jenkins.phx.ovirt.org 18 Jul '20

18 Jul '20

Good morning! Attached is the HTML page with the jenkins status report. You can see it also here: - https://jenkins.ovirt.org/job/system_jenkins-report/1255//artifact/exported… Cheers, Jenkins

1 0

[oVirt Jenkins] ovirt-system-tests_upgrade-from-prevrelease-suite-4.3_nightly - Build # 150 - Failure!
by jenkins＠jenkins.phx.ovirt.org 17 Jul '20

17 Jul '20

Project: https://jenkins.ovirt.org/job/ovirt-system-tests_upgrade-from-prevrelease-s… Build: https://jenkins.ovirt.org/job/ovirt-system-tests_upgrade-from-prevrelease-s… Build Number: 150 Build Status: Failure Triggered By: Started by timer ------------------------------------- Changes Since Last Success: ------------------------------------- Changes for Build #150 [Marcin Sobczyk] ost-images: Use OST images when available ----------------- Failed Tests: ----------------- 1 tests failed. FAILED: 002_bootstrap.verify_add_hosts Error Message: 1 hosts failed installation: lago-upgrade-from-prevrelease-suite-4-3-host-0: install_failed -------------------- >> begin captured logging << -------------------- 002_bootstrap: DEBUG: Host status, verify_add_hosts: lago-upgrade-from-prevrelease-suite-4-3-host-0: installing ovirtlago.testlib: ERROR: * Unhandled exception in <function <lambda> at 0x7f8c245a9578> Traceback (most recent call last): File "/usr/lib/python2.7/site-packages/ovirtlago/testlib.py", line 234, in assert_equals_within res = func() File "/home/jenkins/agent/workspace/ovirt-system-tests_upgrade-from-prevrelease-suite-4.3_nightly/ovirt-system-tests/upgrade-from-prevrelease-suite-4.3/test-scenarios-after-upgrade/002_bootstrap.py", line 227, in <lambda> lambda: _single_host_up(hosts_service, total_hosts), File "/home/jenkins/agent/workspace/ovirt-system-tests_upgrade-from-prevrelease-suite-4.3_nightly/ovirt-system-tests/upgrade-from-prevrelease-suite-4.3/test-scenarios-after-upgrade/002_bootstrap.py", line 98, in _single_host_up _check_problematic_hosts(hosts_service) File "/home/jenkins/agent/workspace/ovirt-system-tests_upgrade-from-prevrelease-suite-4.3_nightly/ovirt-system-tests/upgrade-from-prevrelease-suite-4.3/test-scenarios-after-upgrade/002_bootstrap.py", line 107, in _check_problematic_hosts raise RuntimeError(dump_hosts) RuntimeError: 1 hosts failed installation: lago-upgrade-from-prevrelease-suite-4-3-host-0: install_failed --------------------- >> end captured logging << --------------------- Stack Trace: File "/usr/lib64/python2.7/unittest/case.py", line 369, in run testMethod() File "/usr/lib/python2.7/site-packages/nose/case.py", line 197, in runTest self.test(*self.arg) File "/usr/lib/python2.7/site-packages/ovirtlago/testlib.py", line 142, in wrapped_test test() File "/usr/lib/python2.7/site-packages/ovirtlago/testlib.py", line 60, in wrapper return func(get_test_prefix(), *args, **kwargs) File "/usr/lib/python2.7/site-packages/ovirtlago/testlib.py", line 79, in wrapper prefix.virt_env.engine_vm().get_api(api_ver=4), *args, **kwargs File "/home/jenkins/agent/workspace/ovirt-system-tests_upgrade-from-prevrelease-suite-4.3_nightly/ovirt-system-tests/upgrade-from-prevrelease-suite-4.3/test-scenarios-after-upgrade/002_bootstrap.py", line 228, in verify_add_hosts timeout=constants.ADD_HOST_TIMEOUT File "/usr/lib/python2.7/site-packages/ovirtlago/testlib.py", line 278, in assert_true_within assert_equals_within(func, True, timeout, allowed_exceptions) File "/usr/lib/python2.7/site-packages/ovirtlago/testlib.py", line 234, in assert_equals_within res = func() File "/home/jenkins/agent/workspace/ovirt-system-tests_upgrade-from-prevrelease-suite-4.3_nightly/ovirt-system-tests/upgrade-from-prevrelease-suite-4.3/test-scenarios-after-upgrade/002_bootstrap.py", line 227, in <lambda> lambda: _single_host_up(hosts_service, total_hosts), File "/home/jenkins/agent/workspace/ovirt-system-tests_upgrade-from-prevrelease-suite-4.3_nightly/ovirt-system-tests/upgrade-from-prevrelease-suite-4.3/test-scenarios-after-upgrade/002_bootstrap.py", line 98, in _single_host_up _check_problematic_hosts(hosts_service) File "/home/jenkins/agent/workspace/ovirt-system-tests_upgrade-from-prevrelease-suite-4.3_nightly/ovirt-system-tests/upgrade-from-prevrelease-suite-4.3/test-scenarios-after-upgrade/002_bootstrap.py", line 107, in _check_problematic_hosts raise RuntimeError(dump_hosts) '1 hosts failed installation:\nlago-upgrade-from-prevrelease-suite-4-3-host-0: install_failed\n\n-------------------- >> begin captured logging << --------------------\n002_bootstrap: DEBUG: Host status, verify_add_hosts:\n lago-upgrade-from-prevrelease-suite-4-3-host-0: installing\n\novirtlago.testlib: ERROR: * Unhandled exception in <function <lambda> at 0x7f8c245a9578>\nTraceback (most recent call last):\n File "/usr/lib/python2.7/site-packages/ovirtlago/testlib.py", line 234, in assert_equals_within\n res = func()\n File "/home/jenkins/agent/workspace/ovirt-system-tests_upgrade-from-prevrelease-suite-4.3_nightly/ovirt-system-tests/upgrade-from-prevrelease-suite-4.3/test-scenarios-after-upgrade/002_bootstrap.py", line 227, in <lambda>\n lambda: _single_host_up(hosts_service, total_hosts),\n File "/home/jenkins/agent/workspace/ovirt-system-tests_upgrade-from-prevrelease-suite-4.3_nightly/ovirt-system-tests/upgrade-from-prevrelease-suite-4.3/test-scenarios-after-upgrade/002_bootstrap.py", line 98, in _single_host_up\n _check_problematic_hosts(hosts_service)\n File "/home/jenkins/agent/workspace/ovirt-system-tests_upgrade-from-prevrelease-suite-4.3_nightly/ovirt-system-tests/upgrade-from-prevrelease-suite-4.3/test-scenarios-after-upgrade/002_bootstrap.py", line 107, in _check_problematic_hosts\n raise RuntimeError(dump_hosts)\nRuntimeError: 1 hosts failed installation:\nlago-upgrade-from-prevrelease-suite-4-3-host-0: install_failed\n\n--------------------- >> end captured logging << ---------------------'

1 1

oVirt infra daily report - unstable production jobs - 1254
by jenkins＠jenkins.phx.ovirt.org 17 Jul '20

17 Jul '20

Good morning! Attached is the HTML page with the jenkins status report. You can see it also here: - https://jenkins.ovirt.org/job/system_jenkins-report/1254//artifact/exported… Cheers, Jenkins

1 0

Missing build-artifacts
by Yedidyah Bar David 16 Jul '20

16 Jul '20

Hi all, Any idea why [1] has no collected artifacts? I ran it to try to reproduce something similar to [2], using the same OST patch [3] - both as-is [1], for reproduction, and with a patched [4] engine [5], for verification. [5] passed, so I am happy with the verification. Former [1] failed, as expected, and at a reasonable point to be considered a reproduction (based on console output), but I can't be sure without some more logs. Going to mark the patch verified anyway, just wondering - mainly for the next time I'll want to use CI like that (which I do often). Thanks and best regards, [1] https://jenkins.ovirt.org/job/ovirt-system-tests_standard-check-patch/10297/ [2] https://bugzilla.redhat.com/show_bug.cgi?id=1847963 [3] https://gerrit.ovirt.org/110302 [4] https://gerrit.ovirt.org/110284 [5] https://jenkins.ovirt.org/view/oVirt%20system%20tests/job/ovirt-system-test… -- Didi

1 0

[CQ]: 109378, 19 (ovirt-system-tests) failed "ovirt-master" system tests
by oVirt Jenkins 16 Jul '20

16 Jul '20

Change 109378,19 (ovirt-system-tests) is probably the reason behind recent system test failures in the "ovirt-master" change queue and needs to be fixed. This change had been removed from the testing queue. Artifacts build from this change will not be released until it is fixed. For further details about the change see: https://gerrit.ovirt.org/#/c/109378/19 For failed test results see: https://jenkins.ovirt.org/job/ovirt-master_change-queue-tester/24897/

1 0

[CQ]: 109378,19 (ovirt-system-tests) failed "ovirt-4.3" system tests
by oVirt Jenkins 16 Jul '20

16 Jul '20

Change 109378,19 (ovirt-system-tests) is probably the reason behind recent system test failures in the "ovirt-4.3" change queue and needs to be fixed. This change had been removed from the testing queue. Artifacts build from this change will not be released until it is fixed. For further details about the change see: https://gerrit.ovirt.org/#/c/109378/19 For failed test results see: https://jenkins.ovirt.org/job/ovirt-4.3_change-queue-tester/3685/

1 0

Build failed in Jenkins: system-mk_mirrors_index-yum #91868
by jenkins＠jenkins.phx.ovirt.org 16 Jul '20

16 Jul '20

See <https://jenkins.ovirt.org/job/system-mk_mirrors_index-yum/91868/display/red…> Changes: ------------------------------------------ Started by upstream project "system-sync_mirrors-centos-extras-7.6-el7-x86_64" build number 801 originally caused by: Started by timer Running as SYSTEM [EnvInject] - Loading node environment variables. Building remotely on mirrors.phx.ovirt.org (mirrors) in workspace <https://jenkins.ovirt.org/job/system-mk_mirrors_index-yum/ws/> No credentials specified > git rev-parse --is-inside-work-tree # timeout=10 Fetching changes from the remote Git repository > git config remote.origin.url http://gerrit.ovirt.org/jenkins.git # timeout=10 Cleaning workspace > git rev-parse --verify HEAD # timeout=10 Resetting working tree > git reset --hard # timeout=10 > git clean -fdx # timeout=10 Pruning obsolete local branches Fetching upstream changes from http://gerrit.ovirt.org/jenkins.git > git --version # timeout=10 > git fetch --tags --progress --prune http://gerrit.ovirt.org/jenkins.git +refs/heads/*:refs/remotes/origin/* # timeout=10 ERROR: Error fetching remote repo 'origin' hudson.plugins.git.GitException: Failed to fetch from http://gerrit.ovirt.org/jenkins.git at hudson.plugins.git.GitSCM.fetchFrom(GitSCM.java:909) at hudson.plugins.git.GitSCM.retrieveChanges(GitSCM.java:1131) at hudson.plugins.git.GitSCM.checkout(GitSCM.java:1167) at hudson.scm.SCM.checkout(SCM.java:505) at hudson.model.AbstractProject.checkout(AbstractProject.java:1206) at hudson.model.AbstractBuild$AbstractBuildExecution.defaultCheckout(AbstractBuild.java:574) at jenkins.scm.SCMCheckoutStrategy.checkout(SCMCheckoutStrategy.java:86) at hudson.model.AbstractBuild$AbstractBuildExecution.run(AbstractBuild.java:499) at hudson.model.Run.execute(Run.java:1856) at hudson.model.FreeStyleBuild.run(FreeStyleBuild.java:43) at hudson.model.ResourceController.execute(ResourceController.java:97) at hudson.model.Executor.run(Executor.java:428) Caused by: hudson.plugins.git.GitException: Command "git fetch --tags --progress --prune http://gerrit.ovirt.org/jenkins.git +refs/heads/*:refs/remotes/origin/*" returned status code 128: stdout: stderr: fatal: unable to access 'http://gerrit.ovirt.org/jenkins.git/': Failed to connect to 2600:1f18:26e2:3333::3333: Network is unreachable at org.jenkinsci.plugins.gitclient.CliGitAPIImpl.launchCommandIn(CliGitAPIImpl.java:2430) at org.jenkinsci.plugins.gitclient.CliGitAPIImpl.launchCommandWithCredentials(CliGitAPIImpl.java:2044) at org.jenkinsci.plugins.gitclient.CliGitAPIImpl.access$500(CliGitAPIImpl.java:81) at org.jenkinsci.plugins.gitclient.CliGitAPIImpl$1.execute(CliGitAPIImpl.java:569) at org.jenkinsci.plugins.gitclient.RemoteGitImpl$CommandInvocationHandler$GitCommandMasterToSlaveCallable.call(RemoteGitImpl.java:161) at org.jenkinsci.plugins.gitclient.RemoteGitImpl$CommandInvocationHandler$GitCommandMasterToSlaveCallable.call(RemoteGitImpl.java:154) at hudson.remoting.UserRequest.perform(UserRequest.java:211) at hudson.remoting.UserRequest.perform(UserRequest.java:54) at hudson.remoting.Request$2.run(Request.java:369) at hudson.remoting.InterceptingExecutorService$1.call(InterceptingExecutorService.java:72) at java.util.concurrent.FutureTask.run(FutureTask.java:266) at java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1149) at java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:624) at java.lang.Thread.run(Thread.java:748) Suppressed: hudson.remoting.Channel$CallSiteStackTrace: Remote call to mirrors.phx.ovirt.org at hudson.remoting.Channel.attachCallSiteStackTrace(Channel.java:1788) at hudson.remoting.UserRequest$ExceptionResponse.retrieve(UserRequest.java:356) at hudson.remoting.Channel.call(Channel.java:998) at org.jenkinsci.plugins.gitclient.RemoteGitImpl$CommandInvocationHandler.execute(RemoteGitImpl.java:146) at sun.reflect.GeneratedMethodAccessor933.invoke(Unknown Source) at sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:43) at java.lang.reflect.Method.invoke(Method.java:498) at org.jenkinsci.plugins.gitclient.RemoteGitImpl$CommandInvocationHandler.invoke(RemoteGitImpl.java:132) at com.sun.proxy.$Proxy90.execute(Unknown Source) at hudson.plugins.git.GitSCM.fetchFrom(GitSCM.java:907) at hudson.plugins.git.GitSCM.retrieveChanges(GitSCM.java:1131) at hudson.plugins.git.GitSCM.checkout(GitSCM.java:1167) at hudson.scm.SCM.checkout(SCM.java:505) at hudson.model.AbstractProject.checkout(AbstractProject.java:1206) at hudson.model.AbstractBuild$AbstractBuildExecution.defaultCheckout(AbstractBuild.java:574) at jenkins.scm.SCMCheckoutStrategy.checkout(SCMCheckoutStrategy.java:86) at hudson.model.AbstractBuild$AbstractBuildExecution.run(AbstractBuild.java:499) at hudson.model.Run.execute(Run.java:1856) at hudson.model.FreeStyleBuild.run(FreeStyleBuild.java:43) at hudson.model.ResourceController.execute(ResourceController.java:97) at hudson.model.Executor.run(Executor.java:428) ERROR: Error fetching remote repo 'origin'

1 1

[oVirt Jenkins] cleanup-gated-dummy-master - Build #208 - FAILURE!
by jenkins＠jenkins.phx.ovirt.org 16 Jul '20

16 Jul '20

Build: https://jenkins.ovirt.org/job/cleanup-gated-dummy-master/208/ Build Name: #208 Build Status: FAILURE Gerrit change: null - title: null - project: null - branch: null - author: null <null>

1 0

oVirt infra daily report - unstable production jobs - 1253
by jenkins＠jenkins.phx.ovirt.org 16 Jul '20

16 Jul '20

Good morning! Attached is the HTML page with the jenkins status report. You can see it also here: - https://jenkins.ovirt.org/job/system_jenkins-report/1253//artifact/exported… Cheers, Jenkins

1 0

Permissions
by Aviv Litman 15 Jul '20

15 Jul '20

Hi, my name is aviv litman and i work with shirly on the ovirt-dwh project. I don't have permissions to abort commits. Thanks, Aviv

2 1

Unsubscribe request
by calinux81 . 15 Jul '20

15 Jul '20

Please remove my email address from this mailing list Thank You

2 1

[oVirt Jenkins] ovirt-system-tests_basic-suite-master_nightly - Build # 185 - Failure!
by jenkins＠jenkins.phx.ovirt.org 15 Jul '20

15 Jul '20

Project: https://jenkins.ovirt.org/job/ovirt-system-tests_basic-suite-master_nightly/ Build: https://jenkins.ovirt.org/job/ovirt-system-tests_basic-suite-master_nightly… Build Number: 185 Build Status: Failure Triggered By: Started by timer ------------------------------------- Changes Since Last Success: ------------------------------------- Changes for Build #185 [Galit] Remove basic-suite-4.2 suite ----------------- Failed Tests: ----------------- 1 tests failed. FAILED: basic-suite-master.test-scenarios.002_bootstrap_pytest.test_verify_add_hosts Error Message: RuntimeError: 1 hosts failed installation: lago-basic-suite-master-host-0: install_failed Stack Trace: api_v4 = <ovirtsdk4.Connection object at 0x7f8754690510> @order_by(_TEST_LIST) def test_verify_add_hosts(api_v4): hosts_service = api_v4.system_service().hosts_service() hosts_status = hosts_service.list(search='datacenter={}'.format(DC_NAME)) total_hosts = len(hosts_status) dump_hosts = _host_status_to_print(hosts_service, hosts_status) LOGGER.debug('Host status, verify_add_hosts:\n {}'.format(dump_hosts)) testlib.assert_true_within( lambda: _single_host_up(hosts_service, total_hosts), > timeout=constants.ADD_HOST_TIMEOUT ) ../basic-suite-master/test-scenarios/002_bootstrap_pytest.py:432: _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ /usr/lib/python2.7/site-packages/ovirtlago/testlib.py:278: in assert_true_within assert_equals_within(func, True, timeout, allowed_exceptions) /usr/lib/python2.7/site-packages/ovirtlago/testlib.py:234: in assert_equals_within res = func() ../basic-suite-master/test-scenarios/002_bootstrap_pytest.py:431: in <lambda> lambda: _single_host_up(hosts_service, total_hosts), ../basic-suite-master/test-scenarios/002_bootstrap_pytest.py:228: in _single_host_up _check_problematic_hosts(hosts_service) _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ hosts_service = <ovirtsdk4.services.HostsService object at 0x7f875490e910> def _check_problematic_hosts(hosts_service): problematic_hosts = hosts_service.list(search='datacenter={} AND status != installing and status != initializing and status != up'.format(DC_NAME)) if len(problematic_hosts): dump_hosts = '%s hosts failed installation:\n' % len(problematic_hosts) for host in problematic_hosts: host_service = hosts_service.host_service(host.id) dump_hosts += '%s: %s\n' % (host.name, host_service.get().status) > raise RuntimeError(dump_hosts) E RuntimeError: 1 hosts failed installation: E lago-basic-suite-master-host-0: install_failed ../basic-suite-master/test-scenarios/002_bootstrap_pytest.py:237: RuntimeError

1 4

oVirt infra daily report - unstable production jobs - 1252
by jenkins＠jenkins.phx.ovirt.org 15 Jul '20

15 Jul '20

Good morning! Attached is the HTML page with the jenkins status report. You can see it also here: - https://jenkins.ovirt.org/job/system_jenkins-report/1252//artifact/exported… Cheers, Jenkins

1 0

[oVirt Jenkins] ovirt-system-tests_basic-suite-4.3_nightly - Build # 185 - Failure!
by jenkins＠jenkins.phx.ovirt.org 14 Jul '20

14 Jul '20

Project: https://jenkins.ovirt.org/job/ovirt-system-tests_basic-suite-4.3_nightly/ Build: https://jenkins.ovirt.org/job/ovirt-system-tests_basic-suite-4.3_nightly/18… Build Number: 185 Build Status: Failure Triggered By: Started by timer ------------------------------------- Changes Since Last Success: ------------------------------------- Changes for Build #185 [Galit] Remove basic-suite-4.2 suite ----------------- Failed Tests: ----------------- 1 tests failed. FAILED: 005_network_by_label.assign_labeled_network Error Message: False != True after 180 seconds Stack Trace: Traceback (most recent call last): File "/usr/lib64/python2.7/unittest/case.py", line 369, in run testMethod() File "/usr/lib/python2.7/site-packages/nose/case.py", line 197, in runTest self.test(*self.arg) File "/usr/lib/python2.7/site-packages/ovirtlago/testlib.py", line 142, in wrapped_test test() File "/usr/lib/python2.7/site-packages/ovirtlago/testlib.py", line 60, in wrapper return func(get_test_prefix(), *args, **kwargs) File "/usr/lib/python2.7/site-packages/ovirtlago/testlib.py", line 79, in wrapper prefix.virt_env.engine_vm().get_api(api_ver=4), *args, **kwargs File "/home/jenkins/agent/workspace/ovirt-system-tests_basic-suite-4.3_nightly/ovirt-system-tests/basic-suite-4.3/test-scenarios/005_network_by_label.py", line 149, in assign_labeled_network host_service, LABELED_NET_NAME)) File "/usr/lib/python2.7/site-packages/ovirtlago/testlib.py", line 282, in assert_true_within_short assert_equals_within_short(func, True, allowed_exceptions) File "/usr/lib/python2.7/site-packages/ovirtlago/testlib.py", line 266, in assert_equals_within_short func, value, SHORT_TIMEOUT, allowed_exceptions=allowed_exceptions File "/usr/lib/python2.7/site-packages/ovirtlago/testlib.py", line 252, in assert_equals_within '%s != %s after %s seconds' % (res, value, timeout) AssertionError: False != True after 180 seconds

1 1

oVirt infra daily report - unstable production jobs - 1251
by jenkins＠jenkins.phx.ovirt.org 14 Jul '20

14 Jul '20

Good morning! Attached is the HTML page with the jenkins status report. You can see it also here: - https://jenkins.ovirt.org/job/system_jenkins-report/1251//artifact/exported… Cheers, Jenkins

1 0

[JIRA] (OVIRT-2268) Enable IPv6 for resources.ovirt.org
by Sandro Bonazzola (oVirt JIRA) 13 Jul '20

13 Jul '20

[ https://ovirt-jira.atlassian.net/browse/OVIRT-2268?page=com.atlassian.jira.… ] Sandro Bonazzola commented on OVIRT-2268: ----------------------------------------- Any update here? > Enable IPv6 for resources.ovirt.org > ----------------------------------- > > Key: OVIRT-2268 > URL: https://ovirt-jira.atlassian.net/browse/OVIRT-2268 > Project: oVirt - virtualization made easy > Issue Type: Improvement > Components: oVirt Infra > Reporter: Dominik Holler > Assignee: infra > > To allow access to resources.ovirt.org in IPv6 only setups, IPv6 should be enabled for resources.ovirt.org. > This would allow users in IPv6 only setups to install oVirt. -- This message was sent by Atlassian Jira (v1001.0.0-SNAPSHOT#100133)

1 0

[CQ]: 110151, 3 (ovirt-system-tests) failed "ovirt-4.3" system tests, but isn't the failure root cause
by oVirt Jenkins 13 Jul '20

13 Jul '20

A system test invoked by the "ovirt-4.3" change queue including change 110151,3 (ovirt-system-tests) failed. However, this change seems not to be the root cause for this failure. Change 110152,3 (ovirt-system-tests) that this change depends on or is based on, was detected as the cause of the testing failures. This change had been removed from the testing queue. Artifacts built from this change will not be released until either change 110152,3 (ovirt-system-tests) is fixed and this change is updated to refer to or rebased on the fixed version, or this change is modified to no longer depend on it. For further details about the change see: https://gerrit.ovirt.org/#/c/110151/3 For further details about the change that seems to be the root cause behind the testing failures see: https://gerrit.ovirt.org/#/c/110152/3 For failed test results see: https://jenkins.ovirt.org/job/ovirt-4.3_change-queue-tester/3673/

1 0

[oVirt Jenkins] deploy-to-gated-ovirt-master - Build #401 - FAILURE!
by jenkins＠jenkins.phx.ovirt.org 13 Jul '20

13 Jul '20

Build: https://jenkins.ovirt.org/job/deploy-to-gated-ovirt-master/401/ Build Name: #401 Build Status: FAILURE Gerrit change: null - title: null - project: null - branch: null - author: null <null>

1 1

[oVirt Jenkins] deploy-to-gated-ovirt-4.3 - Build #406 - FAILURE!
by jenkins＠jenkins.phx.ovirt.org 13 Jul '20

13 Jul '20

Build: https://jenkins.ovirt.org/job/deploy-to-gated-ovirt-4.3/406/ Build Name: #406 Build Status: FAILURE Gerrit change: null - title: null - project: null - branch: null - author: null <null>

1 1