September 2020 - Infra - oVirt List Archives

[JIRA] (OVIRT-3022) poll-upstream-sources pipelines don't work on el8 nodes
by Marcin Sobczyk (oVirt JIRA) 23 Sep '20

23 Sep '20

Marcin Sobczyk created OVIRT-3022: ------------------------------------- Summary: poll-upstream-sources pipelines don't work on el8 nodes Key: OVIRT-3022 URL: https://ovirt-jira.atlassian.net/browse/OVIRT-3022 Project: oVirt - virtualization made easy Issue Type: Bug Reporter: Marcin Sobczyk Assignee: infra Trying to run `poll-upstream-sources` pipeline on an el8 node ends with an [error|https://jenkins.ovirt.org/blue/organizations/jenkins/ost-images_maste…] {noformat} [2020-09-23T02:07:48.761Z] + /home/jenkins/workspace/ost-images_master_standard-poll-upstream-sources/jenkins/stdci_tools/usrc.py --log -d get [2020-09-23T02:07:48.761Z] /usr/bin/env: ‘python’: No such file or directory {noformat} After an attempt of resolving the issue by installing python2 and setting the unversioned python command to point to python2 we fail on dependencies: {noformat} [2020-09-23T09:39:13.566Z] + /home/jenkins/workspace/ost-images_master_standard-poll-upstream-sources/jenkins/stdci_tools/usrc.py --log -d get [2020-09-23T09:39:13.566Z] Traceback (most recent call last): [2020-09-23T09:39:13.566Z] File "/home/jenkins/workspace/ost-images_master_standard-poll-upstream-sources/jenkins/stdci_tools/usrc.py", line 11, in <module> [2020-09-23T09:39:13.566Z] import yaml [2020-09-23T09:39:13.566Z] ImportError: No module named yaml {noformat} A quick look at the `usrc.py` script shows we need at least 3 deps: * python2-six * python2-pyyaml * python2-pyxdg (or simply pyxdg) The last package is not available at all in el8 repos. Please provide a solution to this problem by either moving the script to use py3 or any other means. -- This message was sent by Atlassian Jira (v1001.0.0-SNAPSHOT#100146)

1 0

[JIRA] (OVIRT-3021) Very slow push/pull to/from gerrit
by Nir Soffer (oVirt JIRA) 23 Sep '20

23 Sep '20

[ https://ovirt-jira.atlassian.net/browse/OVIRT-3021?page=com.atlassian.jira.… ] Nir Soffer commented on OVIRT-3021: ----------------------------------- On Wed, Sep 23, 2020 at 1:38 PM Nir Soffer <nsoffer(a)redhat.com> wrote: > > Recently pushing changes to gerrit is very slow. It seems that the issue > is pulling changes during rebase in "git review". > > Sometimes "git review" is very slow, but "git review -R" is very quick. This > show that the issue was getting changes from gerrit during when git-review > try to rebase the current branch on master. > > I noticed that origin/master and gerrit/master do not show the same commit: > > $ git log origin/master | head -1 > commit 6e6e93cb1d237f9b6fdd533d5afdc1b9ba1c197a > > $ git log gerrit/master | head -1 > commit 353e7b1e322aa02d4767b6617ed094be0643b094 > > $ git remote -v > gerrit http://gerrit.ovirt.org/p/vdsm.git (fetch) > gerrit ssh://nsoffer@gerrit.ovirt.org:29418/vdsm.git (push) > origin http://gerrit.ovirt.org/p/vdsm.git (fetch) > origin http://gerrit.ovirt.org/p/vdsm.git (push) > > I've been using this configuration for several years, and the slowness > started only recently, so I guess the issue is not on my side. > > Same issues also on ovirt-engine, using similar configuration: > > $ git remote -v > gerrit http://gerrit.ovirt.org/ovirt-engine (fetch) > gerrit ssh://nsoffer@gerrit.ovirt.org:29418/ovirt-engine.git (push) > origin git://gerrit.ovirt.org/ovirt-engine (fetch) > origin git://gerrit.ovirt.org/ovirt-engine (push) > > Anyone experienced this? > > I guess that restarting the gerrit server will "fix" this issue. More data: Cloning ovirt-engine-api-model: $ git clone git://gerrit.ovirt.org/ovirt-engine-api-model 10.0 KiB/s $ git clone https://gerrit.ovirt.org/ovirt-engine-api-model 22.0 KiB/s $ git clone https://github.com/oVirt/ovirt-engine-api-model.git 136.00 KiB/s Looks like a gerrit issue. > Very slow push/pull to/from gerrit > ---------------------------------- > > Key: OVIRT-3021 > URL: https://ovirt-jira.atlassian.net/browse/OVIRT-3021 > Project: oVirt - virtualization made easy > Issue Type: By-EMAIL > Reporter: Nir Soffer > Assignee: infra > > Recently pushing changes to gerrit is very slow. It seems that the issue > is pulling changes during rebase in "git review". > Sometimes "git review" is very slow, but "git review -R" is very quick. This > show that the issue was getting changes from gerrit during when git-review > try to rebase the current branch on master. > I noticed that origin/master and gerrit/master do not show the same commit: > $ git log origin/master | head -1 > commit 6e6e93cb1d237f9b6fdd533d5afdc1b9ba1c197a > $ git log gerrit/master | head -1 > commit 353e7b1e322aa02d4767b6617ed094be0643b094 > $ git remote -v > gerrit http://gerrit.ovirt.org/p/vdsm.git (fetch) > gerrit ssh://nsoffer@gerrit.ovirt.org:29418/vdsm.git (push) > origin http://gerrit.ovirt.org/p/vdsm.git (fetch) > origin http://gerrit.ovirt.org/p/vdsm.git (push) > I've been using this configuration for several years, and the slowness > started only recently, so I guess the issue is not on my side. > Same issues also on ovirt-engine, using similar configuration: > $ git remote -v > gerrit http://gerrit.ovirt.org/ovirt-engine (fetch) > gerrit ssh://nsoffer@gerrit.ovirt.org:29418/ovirt-engine.git (push) > origin git://gerrit.ovirt.org/ovirt-engine (fetch) > origin git://gerrit.ovirt.org/ovirt-engine (push) > Anyone experienced this? > I guess that restarting the gerrit server will "fix" this issue. > Nir -- This message was sent by Atlassian Jira (v1001.0.0-SNAPSHOT#100146)

1 0

[JIRA] (OVIRT-3021) Very slow push/pull to/from gerrit
by Nir Soffer (oVirt JIRA) 23 Sep '20

23 Sep '20

Nir Soffer created OVIRT-3021: --------------------------------- Summary: Very slow push/pull to/from gerrit Key: OVIRT-3021 URL: https://ovirt-jira.atlassian.net/browse/OVIRT-3021 Project: oVirt - virtualization made easy Issue Type: By-EMAIL Reporter: Nir Soffer Assignee: infra Recently pushing changes to gerrit is very slow. It seems that the issue is pulling changes during rebase in "git review". Sometimes "git review" is very slow, but "git review -R" is very quick. This show that the issue was getting changes from gerrit during when git-review try to rebase the current branch on master. I noticed that origin/master and gerrit/master do not show the same commit: $ git log origin/master | head -1 commit 6e6e93cb1d237f9b6fdd533d5afdc1b9ba1c197a $ git log gerrit/master | head -1 commit 353e7b1e322aa02d4767b6617ed094be0643b094 $ git remote -v gerrit http://gerrit.ovirt.org/p/vdsm.git (fetch) gerrit ssh://nsoffer@gerrit.ovirt.org:29418/vdsm.git (push) origin http://gerrit.ovirt.org/p/vdsm.git (fetch) origin http://gerrit.ovirt.org/p/vdsm.git (push) I've been using this configuration for several years, and the slowness started only recently, so I guess the issue is not on my side. Same issues also on ovirt-engine, using similar configuration: $ git remote -v gerrit http://gerrit.ovirt.org/ovirt-engine (fetch) gerrit ssh://nsoffer@gerrit.ovirt.org:29418/ovirt-engine.git (push) origin git://gerrit.ovirt.org/ovirt-engine (fetch) origin git://gerrit.ovirt.org/ovirt-engine (push) Anyone experienced this? I guess that restarting the gerrit server will "fix" this issue. Nir -- This message was sent by Atlassian Jira (v1001.0.0-SNAPSHOT#100146)

1 0

oVirt infra daily report - unstable production jobs - 1322
by jenkins＠jenkins.phx.ovirt.org 23 Sep '20

23 Sep '20

Good morning! Attached is the HTML page with the jenkins status report. You can see it also here: - https://jenkins.ovirt.org/job/system_jenkins-report/1322//artifact/exported… Cheers, Jenkins

1 0

[JIRA] (OVIRT-3020) monitor mirrorlist URL
by Evgheni Dereveanchin (oVirt JIRA) 22 Sep '20

22 Sep '20

Evgheni Dereveanchin created OVIRT-3020: ------------------------------------------- Summary: monitor mirrorlist URL Key: OVIRT-3020 URL: https://ovirt-jira.atlassian.net/browse/OVIRT-3020 Project: oVirt - virtualization made easy Issue Type: Task Reporter: Evgheni Dereveanchin Assignee: infra We've added a dedicated URL for mirrorlist recently so in some time all oVirt users will use it to fetch the list of mirrors. Due to this, the service is quite important and user-visible. We must add it to monitoring to ensure we always get a response from the relevant release URL. -- This message was sent by Atlassian Jira (v1001.0.0-SNAPSHOT#100146)

1 0

[oVirt Jenkins] ovirt-system-tests_basic-suite-master_nightly - Build # 412 - Failure!
by jenkins＠jenkins.phx.ovirt.org 22 Sep '20

22 Sep '20

Project: https://jenkins.ovirt.org/job/ovirt-system-tests_basic-suite-master_nightly/ Build: https://jenkins.ovirt.org/job/ovirt-system-tests_basic-suite-master_nightly… Build Number: 412 Build Status: Failure Triggered By: Started by timer ------------------------------------- Changes Since Last Success: ------------------------------------- Changes for Build #412 [Marcin Sobczyk] basic: Remove remaining lago deps from '002_bootstrap_pytest' ----------------- Failed Tests: ----------------- 1 tests failed. FAILED: basic-suite-master.test-scenarios.100_basic_ui_sanity_pytest.test_login[firefox] Error Message: test setup failure Stack Trace: engine_fqdn = 'engine', engine_ip = '192.168.201.4' @pytest.fixture(scope="session") def hub_url(engine_fqdn, engine_ip): env_url = _env_hub_url() if env_url is not None: yield env_url else: backend = _grid_backend() if backend == "podman": if os_utils.on_centos(8) and os_utils.inside_mock(): pytest.skip("podman doesn't work in mock") hub_port = network_utils.find_free_port(4444, 4544) with podman.grid(engine_fqdn, engine_ip, hub_port=hub_port) as hub_url: yield hub_url elif backend == "docker": > with docker.grid(engine_fqdn, engine_ip) as hub_url: ../ost_utils/ost_utils/pytest/fixtures/selenium/__init__.py:76: _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ /usr/lib64/python2.7/contextlib.py:17: in __enter__ return self.gen.next() ../ost_utils/ost_utils/selenium/docker.py:118: in grid with _network(network_name): /usr/lib64/python2.7/contextlib.py:17: in __enter__ return self.gen.next() ../ost_utils/ost_utils/selenium/docker.py:61: in _network shell(["docker", "network", "create", network_name]) _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ args = ['docker', 'network', 'create', 'grid'], bytes_output = False kwargs = {}, process = <subprocess.Popen object at 0x7f0bcbce33d0>, out = '' err = 'Error response from daemon: network with name grid already exists ' def shell(args, bytes_output=False, **kwargs): process = subprocess.Popen(args, stdout=subprocess.PIPE, stderr=subprocess.PIPE, **kwargs) out, err = process.communicate() if not bytes_output: out = out.decode("utf-8") err = err.decode("utf-8") if process.returncode: > raise ShellError(process.returncode, out, err) E ShellError: Command failed with rc=1. Stdout: E E Stderr: E Error response from daemon: network with name grid already exists ../ost_utils/ost_utils/shell.py:51: ShellError

1 1

oVirt infra daily report - unstable production jobs - 1321
by jenkins＠jenkins.phx.ovirt.org 22 Sep '20

22 Sep '20

Good morning! Attached is the HTML page with the jenkins status report. You can see it also here: - https://jenkins.ovirt.org/job/system_jenkins-report/1321//artifact/exported… Cheers, Jenkins

1 0

[oVirt Jenkins] ovirt-system-tests_basic-suite-master_nightly - Build # 407 - Failure!
by jenkins＠jenkins.phx.ovirt.org 21 Sep '20

21 Sep '20

Project: https://jenkins.ovirt.org/job/ovirt-system-tests_basic-suite-master_nightly/ Build: https://jenkins.ovirt.org/job/ovirt-system-tests_basic-suite-master_nightly… Build Number: 407 Build Status: Failure Triggered By: Started by timer ------------------------------------- Changes Since Last Success: ------------------------------------- Changes for Build #407 [Marcin Sobczyk] basic: Remove remaining lago deps from '002_bootstrap_pytest' ----------------- Failed Tests: ----------------- 1 tests failed. FAILED: basic-suite-master.test-scenarios.100_basic_ui_sanity_pytest.test_virtual_machines[firefox] Error Message: TimeoutException: Message: Wait until VM list view is displayed failed Stack Trace: ovirt_driver = <test_utils.navigation.driver.Driver object at 0x7ff1fd579410> setup_virtual_machines = None save_screenshot = <function save at 0x7ff1fdf93ed8> save_page_source = <function save at 0x7ff1fead2d70> def test_virtual_machines(ovirt_driver, setup_virtual_machines, save_screenshot, save_page_source): try: webadmin_menu = WebAdminLeftMenu(ovirt_driver) > vm_list_view = webadmin_menu.open_vm_list_view() ../basic-suite-master/test-scenarios/100_basic_ui_sanity_pytest.py:266: _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ ../basic-suite-master/test_utils/page_objects/WebAdminLeftMenu.py:22: in open_vm_list_view vm_list_view.wait_for_displayed() ../basic-suite-master/test_utils/page_objects/Displayable.py:14: in wait_for_displayed self.ovirt_driver.wait_until('Wait until ' + self.get_displayable_name() + ' is displayed failed', self.is_displayed) ../basic-suite-master/test_utils/navigation/driver.py:282: in wait_until self._wait_until(message, testlib.SHORT_TIMEOUT, condition_method, *args) ../basic-suite-master/test_utils/navigation/driver.py:288: in _wait_until WebDriverWait(self.driver, timeout).until(ConditionClass(condition_method, *args), message) _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ self = <selenium.webdriver.support.wait.WebDriverWait (session="fcf0208e-7eb9-4152-b615-3975d274a3d7")> method = <test_utils.navigation.driver.ConditionClass object at 0x7ff1fd55ff50> message = 'Wait until VM list view is displayed failed' def until(self, method, message=''): """Calls the method provided with the driver as an argument until the \ return value is not False.""" screen = None stacktrace = None end_time = time.time() + self._timeout while True: try: value = method(self._driver) if value: return value except self._ignored_exceptions as exc: screen = getattr(exc, 'screen', None) stacktrace = getattr(exc, 'stacktrace', None) time.sleep(self._poll) if time.time() > end_time: break > raise TimeoutException(message, screen, stacktrace) E TimeoutException: Message: Wait until VM list view is displayed failed ../.local/lib/python2.7/site-packages/selenium/webdriver/support/wait.py:80: TimeoutException

1 1

oVirt infra daily report - unstable production jobs - 1320
by jenkins＠jenkins.phx.ovirt.org 21 Sep '20

21 Sep '20

Good morning! Attached is the HTML page with the jenkins status report. You can see it also here: - https://jenkins.ovirt.org/job/system_jenkins-report/1320//artifact/exported… Cheers, Jenkins

1 0

[JIRA] (OVIRT-3019) [FIRING:1] MirrorIsOutOfSync web-ovirt-mirrorchecker.apps.ovirt.org:80 oVirt-mirrors-health http://mirror.rackspace.com/ovirt
by Alertmanager_Bot (oVirt JIRA) 20 Sep '20

20 Sep '20

Alertmanager_Bot created OVIRT-3019: --------------------------------------- Summary: [FIRING:1] MirrorIsOutOfSync web-ovirt-mirrorchecker.apps.ovirt.org:80 oVirt-mirrors-health http://mirror.rackspace.com/ovirt Key: OVIRT-3019 URL: https://ovirt-jira.atlassian.net/browse/OVIRT-3019 Project: oVirt - virtualization made easy Issue Type: Bug Reporter: Alertmanager_Bot Assignee: infra Labels: - alertname = MirrorIsOutOfSync - instance = web-ovirt-mirrorchecker.apps.ovirt.org:80 - job = oVirt-mirrors-health - url = http://mirror.rackspace.com/ovirt Annotations: - description = http://mirror.rackspace.com/ovirt (measured by oVirt-mirrors-health) has not been synced in the past two days. Source: http://prometheus-0:9090/graph?g0.expr=mirror_health_check+%3E+172800&g0.ta… -- This message was sent by Atlassian Jira (v1001.0.0-SNAPSHOT#100146)

1 0