- Infra - oVirt List Archives

[JIRA] (OVIRT-3001) [FIRING:1] InstanceUnreachable ibm-srv09.ovirt.org kubernetes-nodes-exporter (amd64 linux ibm-srv09.ovirt.org true external bare-metal-external ci)
by Shlomi Zidmi (oVirt JIRA) 08 Sep '20

08 Sep '20

[ https://ovirt-jira.atlassian.net/browse/OVIRT-3001?page=com.atlassian.jira.… ] Shlomi Zidmi updated OVIRT-3001: -------------------------------- Resolution: Done Status: Done (was: To Do) > [FIRING:1] InstanceUnreachable ibm-srv09.ovirt.org kubernetes-nodes-exporter (amd64 linux ibm-srv09.ovirt.org true external bare-metal-external ci) > --------------------------------------------------------------------------------------------------------------------------------------------------- > > Key: OVIRT-3001 > URL: https://ovirt-jira.atlassian.net/browse/OVIRT-3001 > Project: oVirt - virtualization made easy > Issue Type: Bug > Reporter: Alertmanager_Bot > Assignee: infra > Labels: ALERT{alertname="InstanceUnreachable",instance="ibm-srv09.ovirt.org",job="kubernetes-nodes-exporter"} > > Labels: > - alertname = InstanceUnreachable > - beta_kubernetes_io_arch = amd64 > - beta_kubernetes_io_os = linux > - instance = ibm-srv09.ovirt.org > - job = kubernetes-nodes-exporter > - kubernetes_io_hostname = ibm-srv09.ovirt.org > - node_role_kubernetes_io_compute = true > - region = external > - type = bare-metal-external > - zone = ci > Annotations: > - description = ibm-srv09.ovirt.org of job kubernetes-nodes-exporter has been possibly down for more than 10 minutes. > Source: http://prometheus-0:9090/graph?g0.expr=up%7Bjob%3D%22kubernetes-nodes-expor… -- This message was sent by Atlassian Jira (v1001.0.0-SNAPSHOT#100145)

1 0

[JIRA] (OVIRT-3006) HTTP/500 errors when accessing lists.ovirt.org
by Evgheni Dereveanchin (oVirt JIRA) 08 Sep '20

08 Sep '20

Evgheni Dereveanchin created OVIRT-3006: ------------------------------------------- Summary: HTTP/500 errors when accessing lists.ovirt.org Key: OVIRT-3006 URL: https://ovirt-jira.atlassian.net/browse/OVIRT-3006 Project: oVirt - virtualization made easy Issue Type: Outage Reporter: Evgheni Dereveanchin Assignee: infra I can't access lists.ovirt.org via the browser, getting a 500 Internal Server Error after a considerable time that looks like a timeout. -- This message was sent by Atlassian Jira (v1001.0.0-SNAPSHOT#100145)

1 0

Re: [CI] Builds broken by "Status code: 403 for http://resources.ovirt.org/pub/yum-repo/ovirt-release-master.rpm"
by Nir Soffer 08 Sep '20

08 Sep '20

On Mon, Sep 7, 2020 at 2:30 PM Nir Soffer <nsoffer(a)redhat.com> wrote: > > This is the second time, last time there was an issue during a nightly job that > left the master snapshot repo broken. This is still broken today. In the past sending mail to infra-support created a bug, and someone was handling issue quickly. This seems to be broken now. We depend on http://resources.ovirt.org/pub/yum-repo/ovirt-release-master.rpm for development and CI. When the rpm is missing, CI jobs fail, and developers cannot update their setup. Please fix it as soon as possible. > I have 3 failure builds: > https://jenkins.ovirt.org/job/ovirt-engine_standard-check-patch/7744/ > https://jenkins.ovirt.org/job/ovirt-engine_standard-check-patch/7751/ > https://jenkins.ovirt.org/job/ovirt-engine_standard-check-patch/7755/ > > All seems to fail in: > > [2020-09-07T10:00:56.163Z] + dnf install -y > http://resources.ovirt.org/pub/yum-repo/ovirt-release-master.rpm > [2020-09-07T10:01:08.524Z] Status code: 403 for > http://resources.ovirt.org/pub/yum-repo/ovirt-release-master.rpm (IP: > 66.187.230.40) > > Checking manually show: > > $ curl https://resources.ovirt.org/pub/yum-repo/ovirt-release-master.rpm > <!DOCTYPE HTML PUBLIC "-//IETF//DTD HTML 2.0//EN"> > <html><head> > <title>403 Forbidden</title> > </head><body> > <h1>Forbidden</h1> > <p>You don't have permission to access /pub/yum-repo/ovirt-release-master.rpm > on this server.</p> > </body></html> > > Looking at: > https://resources.ovirt.org/pub/yum-repo/ > > ovirt-elease-master.rpm does not exist. > > So we have 2 issues: > - The file does not exist > - We return "403 Forbidden" instead of "404 Not Found"

3 5

Jenkins build became unstable: ovirt_4.2_publish-rpms_nightly #1099
by jenkins＠jenkins.phx.ovirt.org 08 Sep '20

08 Sep '20

See <https://jenkins.ovirt.org/job/ovirt_4.2_publish-rpms_nightly/1099/display/r…>

1 3

Jenkins build became unstable: ovirt_4.3_publish-rpms_nightly #613
by jenkins＠jenkins.phx.ovirt.org 08 Sep '20

08 Sep '20

See <https://jenkins.ovirt.org/job/ovirt_4.3_publish-rpms_nightly/613/display/re…>

1 3

Jenkins build became unstable: ovirt_master_publish-rpms_nightly #1856
by jenkins＠jenkins.phx.ovirt.org 08 Sep '20

08 Sep '20

See <https://jenkins.ovirt.org/job/ovirt_master_publish-rpms_nightly/1856/displa…>

1 5

Ansible checks
by Marc Dequènes (Duck) 08 Sep '20

08 Sep '20

Quack, ansible-lint added new checks in 4.3.0 and especially one to detect when you create a file and forget to set permissions. Of course most of the time that's not a big deal but sometimes it can deploy bad permissions or even create security problems. One way to make a deployment is to do it manually and when you found the proper things to do you write the rules, right? One problem is that you might forget a few steps but since it's already manually done it deploys nicely until things change for another reason or you need to redeploy elsewhere or another instance. Another problem are these permissions which mean it's deployed but the service cannot access its files because too restrictive permissions or on the contrary it's too open. So it's better to recycle the VM or test in a container first. Currently we use the packaged version of ansible-lint so you did not see any problem _yet_. I have fixed the rules in all roles and also in the infra repo: https://gerrit.ovirt.org/c/111158/ I'm not pointing at people to blame them, and I am myself responsible for many of these mistakes (look at the history in the community-cage-infra-ansible repo or the roles), but I wanted to make sure we think about these problems and try to do better in the future. And btw I also added a YAMLlint check. Most of the time it's not a big deal but sometimes bad indenting causes bugs, especially inside blocks, or make things hardly readable, so I fixed these problems too. Regards. \_o<

1 0

[oVirt Jenkins] ovirt-system-tests_basic-suite-master_nightly - Build # 356 - Failure!
by jenkins＠jenkins.phx.ovirt.org 08 Sep '20

08 Sep '20

Project: https://jenkins.ovirt.org/job/ovirt-system-tests_basic-suite-master_nightly/ Build: https://jenkins.ovirt.org/job/ovirt-system-tests_basic-suite-master_nightly… Build Number: 356 Build Status: Failure Triggered By: Started by timer ------------------------------------- Changes Since Last Success: ------------------------------------- Changes for Build #356 [Marcin Sobczyk] ost_utils: network_utils: Add 'get_ips[6]' functions ----------------- Failed Tests: ----------------- 1 tests failed. FAILED: basic-suite-master.test-scenarios.100_basic_ui_sanity_pytest.test_virtual_machines[firefox] Error Message: TimeoutException: Message: Wait until VM list view is displayed failed Stack Trace: ovirt_driver = <test_utils.navigation.driver.Driver object at 0x7f4944070410> setup_virtual_machines = None save_screenshot = <function save at 0x7f4944aa5d70> save_page_source = <function save at 0x7f49455e4d70> def test_virtual_machines(ovirt_driver, setup_virtual_machines, save_screenshot, save_page_source): try: webadmin_menu = WebAdminLeftMenu(ovirt_driver) > vm_list_view = webadmin_menu.open_vm_list_view() ../basic-suite-master/test-scenarios/100_basic_ui_sanity_pytest.py:266: _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ ../basic-suite-master/test_utils/page_objects/WebAdminLeftMenu.py:22: in open_vm_list_view vm_list_view.wait_for_displayed() ../basic-suite-master/test_utils/page_objects/Displayable.py:14: in wait_for_displayed self.ovirt_driver.wait_until('Wait until ' + self.get_displayable_name() + ' is displayed failed', self.is_displayed) ../basic-suite-master/test_utils/navigation/driver.py:282: in wait_until self._wait_until(message, testlib.SHORT_TIMEOUT, condition_method, *args) ../basic-suite-master/test_utils/navigation/driver.py:288: in _wait_until WebDriverWait(self.driver, timeout).until(ConditionClass(condition_method, *args), message) _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ self = <selenium.webdriver.support.wait.WebDriverWait (session="94efcd6d-2eef-47fb-b98b-8f59d5a01d8e")> method = <test_utils.navigation.driver.ConditionClass object at 0x7f4944ad9410> message = 'Wait until VM list view is displayed failed' def until(self, method, message=''): """Calls the method provided with the driver as an argument until the \ return value is not False.""" screen = None stacktrace = None end_time = time.time() + self._timeout while True: try: value = method(self._driver) if value: return value except self._ignored_exceptions as exc: screen = getattr(exc, 'screen', None) stacktrace = getattr(exc, 'stacktrace', None) time.sleep(self._poll) if time.time() > end_time: break > raise TimeoutException(message, screen, stacktrace) E TimeoutException: Message: Wait until VM list view is displayed failed ../.local/lib/python2.7/site-packages/selenium/webdriver/support/wait.py:80: TimeoutException

1 1

[oVirt Jenkins] ovirt-system-tests_upgrade-from-prevrelease-suite-4.3_nightly - Build # 310 - Failure!
by jenkins＠jenkins.phx.ovirt.org 08 Sep '20

08 Sep '20

Project: https://jenkins.ovirt.org/job/ovirt-system-tests_upgrade-from-prevrelease-s… Build: https://jenkins.ovirt.org/job/ovirt-system-tests_upgrade-from-prevrelease-s… Build Number: 310 Build Status: Failure Triggered By: Started by timer ------------------------------------- Changes Since Last Success: ------------------------------------- Changes for Build #310 [Marcin Sobczyk] ost_utils: network_utils: Add 'get_ips[6]' functions ----------------- Failed Tests: ----------------- No tests ran.

1 1

[oVirt Jenkins] ovirt-system-tests_basic-suite-master_nightly - Build # 352 - Failure!
by jenkins＠jenkins.phx.ovirt.org 08 Sep '20

08 Sep '20

Project: https://jenkins.ovirt.org/job/ovirt-system-tests_basic-suite-master_nightly/ Build: https://jenkins.ovirt.org/job/ovirt-system-tests_basic-suite-master_nightly… Build Number: 352 Build Status: Failure Triggered By: Started by timer ------------------------------------- Changes Since Last Success: ------------------------------------- Changes for Build #352 [Marcin Sobczyk] ost_utils: network_utils: Add 'get_ips[6]' functions [Gal Ben Haim] dockerfile-utils: Better handling for nested repos ----------------- Failed Tests: ----------------- 1 tests failed. FAILED: basic-suite-master.test-scenarios.004_basic_sanity_pytest.test_live_storage_migration Error Message: AssertionError: False != True after 600 seconds Stack Trace: api_v4 = <ovirtsdk4.Connection object at 0x7f8b454470d0> @order_by(_TEST_LIST) def test_live_storage_migration(api_v4): engine = api_v4.system_service() disk_service = test_utils.get_disk_service(engine, DISK0_NAME) correlation_id = 'live_storage_migration' disk_service.move( async=False, filter=False, storage_domain=types.StorageDomain( name=SD_ISCSI_NAME ), query={'correlation_id': correlation_id} ) testlib.assert_true_within_long(lambda: test_utils.all_jobs_finished(engine, correlation_id)) # Assert that the disk is on the correct storage domain, # its status is OK and the snapshot created for the migration # has been merged testlib.assert_true_within_long( > lambda: api_v4.follow_link(disk_service.get().storage_domains[0]).name == SD_ISCSI_NAME ) ../basic-suite-master/test-scenarios/004_basic_sanity_pytest.py:580: _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ /usr/lib/python2.7/site-packages/ovirtlago/testlib.py:286: in assert_true_within_long assert_equals_within_long(func, True, allowed_exceptions) /usr/lib/python2.7/site-packages/ovirtlago/testlib.py:273: in assert_equals_within_long func, value, LONG_TIMEOUT, allowed_exceptions=allowed_exceptions _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ func = <function <lambda> at 0x7f8b45200050>, value = True, timeout = 600 allowed_exceptions = [], initial_wait = 0 def assert_equals_within( func, value, timeout, allowed_exceptions=None, initial_wait=10 ): allowed_exceptions = allowed_exceptions or [] with utils.EggTimer(timeout) as timer: while not timer.elapsed(): try: res = func() if res == value: return except Exception as exc: if _instance_of_any(exc, allowed_exceptions): time.sleep(3) continue LOGGER.exception("Unhandled exception in %s", func) raise if initial_wait == 0: time.sleep(3) else: time.sleep(initial_wait) initial_wait = 0 try: raise AssertionError( > '%s != %s after %s seconds' % (res, value, timeout) ) E AssertionError: False != True after 600 seconds /usr/lib/python2.7/site-packages/ovirtlago/testlib.py:252: AssertionError

1 3