October 2020 - Infra - oVirt List Archives

[oVirt Jenkins] ovirt-system-tests_basic-suite-4.3_nightly - Build # 468 - Failure!
by jenkins＠jenkins.phx.ovirt.org 16 Oct '20

16 Oct '20

Project: https://jenkins.ovirt.org/job/ovirt-system-tests_basic-suite-4.3_nightly/ Build: https://jenkins.ovirt.org/job/ovirt-system-tests_basic-suite-4.3_nightly/46… Build Number: 468 Build Status: Failure Triggered By: Started by timer ------------------------------------- Changes Since Last Success: ------------------------------------- Changes for Build #468 [Radoslaw Szwajkowski] UI test: Add PF4 selectors for Manage vGPU dialog ----------------- Failed Tests: ----------------- 1 tests failed. FAILED: 005_network_by_label.assign_labeled_network Error Message: False != True after 180 seconds Stack Trace: Traceback (most recent call last): File "/usr/lib64/python2.7/unittest/case.py", line 369, in run testMethod() File "/usr/lib/python2.7/site-packages/nose/case.py", line 197, in runTest self.test(*self.arg) File "/usr/lib/python2.7/site-packages/ovirtlago/testlib.py", line 142, in wrapped_test test() File "/usr/lib/python2.7/site-packages/ovirtlago/testlib.py", line 60, in wrapper return func(get_test_prefix(), *args, **kwargs) File "/usr/lib/python2.7/site-packages/ovirtlago/testlib.py", line 79, in wrapper prefix.virt_env.engine_vm().get_api(api_ver=4), *args, **kwargs File "/home/jenkins/agent/workspace/ovirt-system-tests_basic-suite-4.3_nightly/ovirt-system-tests/basic-suite-4.3/test-scenarios/005_network_by_label.py", line 149, in assign_labeled_network host_service, LABELED_NET_NAME)) File "/usr/lib/python2.7/site-packages/ovirtlago/testlib.py", line 282, in assert_true_within_short assert_equals_within_short(func, True, allowed_exceptions) File "/usr/lib/python2.7/site-packages/ovirtlago/testlib.py", line 266, in assert_equals_within_short func, value, SHORT_TIMEOUT, allowed_exceptions=allowed_exceptions File "/usr/lib/python2.7/site-packages/ovirtlago/testlib.py", line 252, in assert_equals_within '%s != %s after %s seconds' % (res, value, timeout) AssertionError: False != True after 180 seconds

1 1

[oVirt Jenkins] ovirt-system-tests_basic-suite-master_nightly - Build # 482 - Failure!
by jenkins＠jenkins.phx.ovirt.org 16 Oct '20

16 Oct '20

Project: https://jenkins.ovirt.org/job/ovirt-system-tests_basic-suite-master_nightly/ Build: https://jenkins.ovirt.org/job/ovirt-system-tests_basic-suite-master_nightly… Build Number: 482 Build Status: Failure Triggered By: Started by timer ------------------------------------- Changes Since Last Success: ------------------------------------- Changes for Build #482 [hbraha] network: bond active slave test [Evgheni Dereveanchin] Change logic for module metadata detection ----------------- Failed Tests: ----------------- 1 tests failed. FAILED: basic-suite-master.test-scenarios.100_basic_ui_sanity.test_virtual_machines[firefox] Error Message: TimeoutException: Message: Wait until Manage vGPU dialog is displayed failed Stack Trace: ovirt_driver = <test_utils.navigation.driver.Driver object at 0x7fca7f1e4290> setup_virtual_machines = None save_screenshot = <function save at 0x7fca7f1d6aa0> save_page_source = <function save at 0x7fca75055488> def test_virtual_machines(ovirt_driver, setup_virtual_machines, save_screenshot, save_page_source): try: webadmin_menu = WebAdminLeftMenu(ovirt_driver) vm_list_view = webadmin_menu.open_vm_list_view() vms = vm_list_view.get_vms() assert 'vm0' in vms assert vm_list_view.is_new_button_enabled() is True assert vm_list_view.is_edit_button_enabled() is False assert vm_list_view.is_shutdown_button_enabled() is False assert vm_list_view.is_export_button_enabled() is False assert vm_list_view.is_migrate_button_enabled() is False vm_list_view.select_vm('vm0') assert vm_list_view.is_new_button_enabled() is True assert vm_list_view.is_edit_button_enabled() is True assert vm_list_view.is_shutdown_button_enabled() is True assert vm_list_view.is_export_button_enabled() is False assert vm_list_view.is_migrate_button_enabled() is True vm_list_view.poweroff() assert vm_list_view.is_new_button_enabled() is True assert vm_list_view.is_edit_button_enabled() is True assert vm_list_view.is_shutdown_button_enabled() is False assert vm_list_view.is_export_button_enabled() is True assert vm_list_view.is_migrate_button_enabled() is False vm_detail_view = vm_list_view.open_detail_view('vm0') assert vm_detail_view.get_name() == 'vm0' assert vm_detail_view.get_status() == 'Down' vm_list_view.run_once() # Waiting for Powering Up instead of Up to speed up the test execution vm_detail_view.wait_for_statuses(['Powering Up', 'Up']) vm_status = vm_detail_view.get_status() assert vm_status == 'Powering Up' or vm_status == 'Up' save_screenshot('vms-after-run-once') vm_detail_host_devices_tab = vm_detail_view.open_host_devices_tab() > vm_vgpu_dialog = vm_detail_host_devices_tab.open_manage_vgpu_dialog() ../basic-suite-master/test-scenarios/100_basic_ui_sanity.py:300: _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ ../basic-suite-master/test_utils/page_objects/VmDetailView.py:51: in open_manage_vgpu_dialog vm_vgpu_dialog.wait_for_displayed() ../basic-suite-master/test_utils/page_objects/Displayable.py:14: in wait_for_displayed self.ovirt_driver.wait_until('Wait until ' + self.get_displayable_name() + ' is displayed failed', self.is_displayed) ../basic-suite-master/test_utils/navigation/driver.py:282: in wait_until self._wait_until(message, testlib.SHORT_TIMEOUT, condition_method, *args) ../basic-suite-master/test_utils/navigation/driver.py:288: in _wait_until WebDriverWait(self.driver, timeout).until(ConditionClass(condition_method, *args), message) _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ self = <selenium.webdriver.support.wait.WebDriverWait (session="2439723b-11fc-4659-bae4-5849fb651d8c")> method = <test_utils.navigation.driver.ConditionClass object at 0x7fca6f4d2f10> message = 'Wait until Manage vGPU dialog is displayed failed' def until(self, method, message=''): """Calls the method provided with the driver as an argument until the \ return value is not False.""" screen = None stacktrace = None end_time = time.time() + self._timeout while True: try: value = method(self._driver) if value: return value except self._ignored_exceptions as exc: screen = getattr(exc, 'screen', None) stacktrace = getattr(exc, 'stacktrace', None) time.sleep(self._poll) if time.time() > end_time: break > raise TimeoutException(message, screen, stacktrace) E TimeoutException: Message: Wait until Manage vGPU dialog is displayed failed ../.local/lib/python2.7/site-packages/selenium/webdriver/support/wait.py:80: TimeoutException

1 9

oVirt infra daily report - unstable production jobs - 1345
by jenkins＠jenkins.phx.ovirt.org 16 Oct '20

16 Oct '20

Good morning! Attached is the HTML page with the jenkins status report. You can see it also here: - https://jenkins.ovirt.org/job/system_jenkins-report/1345//artifact/exported… Cheers, Jenkins

1 0

[JIRA] (OVIRT-3042) git fetch fails - Network is unreachable (was: [CQ]: 111471,2 (ovirt-engine) failed "ovirt-master" system tests)
by Yedidyah Bar David (oVirt JIRA) 15 Oct '20

15 Oct '20

Yedidyah Bar David created OVIRT-3042: ----------------------------------------- Summary: git fetch fails - Network is unreachable (was: [CQ]: 111471,2 (ovirt-engine) failed "ovirt-master" system tests) Key: OVIRT-3042 URL: https://ovirt-jira.atlassian.net/browse/OVIRT-3042 Project: oVirt - virtualization made easy Issue Type: By-EMAIL Reporter: Yedidyah Bar David Assignee: infra On Thu, Oct 15, 2020 at 10:47 AM Radoslaw Szwajkowski <rszwajko(a)redhat.com> wrote: > > Fixed in https://gerrit.ovirt.org/#/c/111741/ It failed due to infra issue, adding infra-support and changing subject: https://jenkins.ovirt.org/job/ovirt-engine_standard-check-patch/8460/console 10:42:37 stderr: fatal: unable to access 'https://gerrit.ovirt.org/ovirt-engine/': Failed to connect to 2600:1f18:26e2:3333::3333: Network is unreachable I think we already saw several other similar cases recently. Is anyone checking this? Is it a network issue? Routine maintenance? Something else? Perhaps we should patch our scripts to try a few times before failing. Thanks, -- Didi -- This message was sent by Atlassian Jira (v1001.0.0-SNAPSHOT#100149)

1 0

[JIRA] (OVIRT-3040) [FIRING:1] InstanceUnreachable ibm-srv03.ovirt.org kubernetes-nodes-exporter (amd64 linux ibm-srv03.ovirt.org true external bare-metal-external ci)
by Shlomi Zidmi (oVirt JIRA) 15 Oct '20

15 Oct '20

[ https://ovirt-jira.atlassian.net/browse/OVIRT-3040?page=com.atlassian.jira.… ] Shlomi Zidmi commented on OVIRT-3040: ------------------------------------- Node is back online and reachable, however something went wrong with it yesterday and caused it to crash. I’m investigating what was the root cause for this. > [FIRING:1] InstanceUnreachable ibm-srv03.ovirt.org kubernetes-nodes-exporter (amd64 linux ibm-srv03.ovirt.org true external bare-metal-external ci) > --------------------------------------------------------------------------------------------------------------------------------------------------- > > Key: OVIRT-3040 > URL: https://ovirt-jira.atlassian.net/browse/OVIRT-3040 > Project: oVirt - virtualization made easy > Issue Type: Bug > Reporter: Alertmanager_Bot > Assignee: infra > Labels: ALERT{alertname="InstanceUnreachable",instance="ibm-srv03.ovirt.org",job="kubernetes-nodes-exporter"} > > Labels: > - alertname = InstanceUnreachable > - beta_kubernetes_io_arch = amd64 > - beta_kubernetes_io_os = linux > - instance = ibm-srv03.ovirt.org > - job = kubernetes-nodes-exporter > - kubernetes_io_hostname = ibm-srv03.ovirt.org > - node_role_kubernetes_io_compute = true > - region = external > - type = bare-metal-external > - zone = ci > Annotations: > - description = ibm-srv03.ovirt.org of job kubernetes-nodes-exporter has been possibly down for more than 10 minutes. > Source: http://prometheus-0:9090/graph?g0.expr=up%7Bjob%3D%22kubernetes-nodes-expor… -- This message was sent by Atlassian Jira (v1001.0.0-SNAPSHOT#100149)

1 0

[CQ]: 111628, 8 (ovirt-engine) failed "ovirt-master" system tests, but isn't the failure root cause
by oVirt Jenkins 15 Oct '20

15 Oct '20

A system test invoked by the "ovirt-master" change queue including change 111628,8 (ovirt-engine) failed. However, this change seems not to be the root cause for this failure. Change 111471,2 (ovirt-engine) that this change depends on or is based on, was detected as the cause of the testing failures. This change had been removed from the testing queue. Artifacts built from this change will not be released until either change 111471,2 (ovirt-engine) is fixed and this change is updated to refer to or rebased on the fixed version, or this change is modified to no longer depend on it. For further details about the change see: https://gerrit.ovirt.org/#/c/111628/8 For further details about the change that seems to be the root cause behind the testing failures see: https://gerrit.ovirt.org/#/c/111471/2 For failed test results see: https://jenkins.ovirt.org/job/ovirt-master_change-queue-tester/26386/

1 0

[CQ]: 111184, 11 (ovirt-engine) failed "ovirt-master" system tests, but isn't the failure root cause
by oVirt Jenkins 15 Oct '20

15 Oct '20

A system test invoked by the "ovirt-master" change queue including change 111184,11 (ovirt-engine) failed. However, this change seems not to be the root cause for this failure. Change 111471,2 (ovirt-engine) that this change depends on or is based on, was detected as the cause of the testing failures. This change had been removed from the testing queue. Artifacts built from this change will not be released until either change 111471,2 (ovirt-engine) is fixed and this change is updated to refer to or rebased on the fixed version, or this change is modified to no longer depend on it. For further details about the change see: https://gerrit.ovirt.org/#/c/111184/11 For further details about the change that seems to be the root cause behind the testing failures see: https://gerrit.ovirt.org/#/c/111471/2 For failed test results see: https://jenkins.ovirt.org/job/ovirt-master_change-queue-tester/26384/

1 0

[oVirt Jenkins] ovirt-system-tests_upgrade-from-release-suite-4.3_nightly - Build # 420 - Failure!
by jenkins＠jenkins.phx.ovirt.org 15 Oct '20

15 Oct '20

Project: https://jenkins.ovirt.org/job/ovirt-system-tests_upgrade-from-release-suite… Build: https://jenkins.ovirt.org/job/ovirt-system-tests_upgrade-from-release-suite… Build Number: 420 Build Status: Failure Triggered By: Started by timer ------------------------------------- Changes Since Last Success: ------------------------------------- Changes for Build #420 No changes ----------------- Failed Tests: ----------------- No tests ran.

1 1

oVirt infra daily report - unstable production jobs - 1344
by jenkins＠jenkins.phx.ovirt.org 15 Oct '20

15 Oct '20

Good morning! Attached is the HTML page with the jenkins status report. You can see it also here: - https://jenkins.ovirt.org/job/system_jenkins-report/1344//artifact/exported… Cheers, Jenkins

1 0

[JIRA] (OVIRT-3041) review IOPS load on PHX storage servers
by Evgheni Dereveanchin (oVirt JIRA) 14 Oct '20

14 Oct '20

Evgheni Dereveanchin created OVIRT-3041: ------------------------------------------- Summary: review IOPS load on PHX storage servers Key: OVIRT-3041 URL: https://ovirt-jira.atlassian.net/browse/OVIRT-3041 Project: oVirt - virtualization made easy Issue Type: Task Reporter: Evgheni Dereveanchin Assignee: infra While looking at Grafana I stumbled upon strange IOPS load on our storage servers. Looks like /dev/sdc on both is seeing a constant flow of 500 write IOPS with few actual data traffic. Let's investigate this as this may be some VM putting extra load and maybe it is not feeling well. /dev/sda3 should be the storage0[1-2]-data1 storage domain according to the configuration document: https://ovirt-infra-docs.readthedocs.io/en/latest/Phoenix_Lab/Storage_Hosts… -- This message was sent by Atlassian Jira (v1001.0.0-SNAPSHOT#100149)

1 0