September 2020 - Infra - oVirt List Archives

[JIRA] (OVIRT-2847) Certain substage fails continously with missing useradd in chroot
by Evgheni Dereveanchin (oVirt JIRA) 09 Sep '20

09 Sep '20

[ https://ovirt-jira.atlassian.net/browse/OVIRT-2847?page=com.atlassian.jira.… ] Evgheni Dereveanchin updated OVIRT-2847: ---------------------------------------- Issue Type: Bug (was: By-EMAIL) > Certain substage fails continously with missing useradd in chroot > ----------------------------------------------------------------- > > Key: OVIRT-2847 > URL: https://ovirt-jira.atlassian.net/browse/OVIRT-2847 > Project: oVirt - virtualization made easy > Issue Type: Bug > Reporter: Bell Levin > Assignee: infra > > Hello! > I have had a contentious error in a certain job. > The error seems to be: > ERROR: Could not find useradd in chroot, maybe the install failed? > you can see it in the logs here: > https://jenkins.ovirt.org/job/standard-manual-runner/983/artifact/check-net… > In the check-network.functional-ovs-switch-lib job, in the following runs: > https://jenkins.ovirt.org/blue/organizations/jenkins/standard-manual-runner… > https://jenkins.ovirt.org/blue/organizations/jenkins/standard-manual-runner… > https://jenkins.ovirt.org/blue/organizations/jenkins/standard-manual-runner… > https://jenkins.ovirt.org/blue/organizations/jenkins/standard-manual-runner… > Please advice. > Thanks, > Bell -- This message was sent by Atlassian Jira (v1001.0.0-SNAPSHOT#100145)

1 0

[JIRA] (OVIRT-2847) Certain substage fails continously with missing useradd in chroot
by Evgheni Dereveanchin (oVirt JIRA) 09 Sep '20

09 Sep '20

[ https://ovirt-jira.atlassian.net/browse/OVIRT-2847?page=com.atlassian.jira.… ] Evgheni Dereveanchin updated OVIRT-2847: ---------------------------------------- Resolution: Cannot Reproduce Status: Done (was: To Do) This seems like an old issue with Fedora mocks. Closing since I've not seen this issue in a while > Certain substage fails continously with missing useradd in chroot > ----------------------------------------------------------------- > > Key: OVIRT-2847 > URL: https://ovirt-jira.atlassian.net/browse/OVIRT-2847 > Project: oVirt - virtualization made easy > Issue Type: By-EMAIL > Reporter: Bell Levin > Assignee: infra > > Hello! > I have had a contentious error in a certain job. > The error seems to be: > ERROR: Could not find useradd in chroot, maybe the install failed? > you can see it in the logs here: > https://jenkins.ovirt.org/job/standard-manual-runner/983/artifact/check-net… > In the check-network.functional-ovs-switch-lib job, in the following runs: > https://jenkins.ovirt.org/blue/organizations/jenkins/standard-manual-runner… > https://jenkins.ovirt.org/blue/organizations/jenkins/standard-manual-runner… > https://jenkins.ovirt.org/blue/organizations/jenkins/standard-manual-runner… > https://jenkins.ovirt.org/blue/organizations/jenkins/standard-manual-runner… > Please advice. > Thanks, > Bell -- This message was sent by Atlassian Jira (v1001.0.0-SNAPSHOT#100145)

1 0

[oVirt Jenkins] ovirt-system-tests_basic-suite-master_nightly - Build # 359 - Failure!
by jenkins＠jenkins.phx.ovirt.org 09 Sep '20

09 Sep '20

Project: https://jenkins.ovirt.org/job/ovirt-system-tests_basic-suite-master_nightly/ Build: https://jenkins.ovirt.org/job/ovirt-system-tests_basic-suite-master_nightly… Build Number: 359 Build Status: Failure Triggered By: Started by timer ------------------------------------- Changes Since Last Success: ------------------------------------- Changes for Build #359 [Marcin Sobczyk] basic: Drop outdated code ----------------- Failed Tests: ----------------- 1 tests failed. FAILED: basic-suite-master.test-scenarios.098_ovirt_provider_ovn_pytest.test_use_ovn_provider Error Message: Error: Fault reason is "Operation Failed". Fault detail is "[Cannot add Network. The name of the logical network 'network_1' is already used by an existing logical network in the same data-center. -Please choose a different name.]". HTTP response code is 409. Stack Trace: prefix = <ovirtlago.prefix.OvirtPrefix object at 0x7f8782aba190> api_v4 = <ovirtsdk4.Connection object at 0x7f8781fee590> @versioning.require_version(4, 2) def test_use_ovn_provider(prefix, api_v4): engine = api_v4.system_service() engine_ip = prefix.virt_env.engine_vm().ip() provider_id = network_utils_v4.get_default_ovn_provider_id(engine) token_id = _get_auth_token(engine_ip) _validate_db_empty(token_id, engine_ip) with _disable_auto_sync(api_v4, provider_id): network1_id = _add_network( token_id, engine_ip, NETWORK_1, ) subnet1_id = _add_subnet( token_id, engine_ip, SUBNET_1, network1_id, ) port1_id = _add_port( token_id, engine_ip, PORT_1, network1_id, ) _validate_network(token_id, engine_ip, NETWORK_1, network1_id) _validate_port(token_id, engine_ip, PORT_1, port1_id, network1_id) _validate_subnet(token_id, engine_ip, SUBNET_1, subnet1_id, network1_id) datacenter_id = _get_datacenter_id(api_v4) > _import_network_to_ovirt(api_v4, provider_id, network1_id, datacenter_id) ../basic-suite-master/test-scenarios/098_ovirt_provider_ovn_pytest.py:474: _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ ../basic-suite-master/test-scenarios/098_ovirt_provider_ovn_pytest.py:364: in _import_network_to_ovirt id=datacenter_id /usr/lib64/python2.7/site-packages/ovirtsdk4/services.py:18492: in import_ return self._internal_action(action, 'import', None, headers, query, wait) /usr/lib64/python2.7/site-packages/ovirtsdk4/service.py:299: in _internal_action return future.wait() if wait else future /usr/lib64/python2.7/site-packages/ovirtsdk4/service.py:55: in wait return self._code(response) /usr/lib64/python2.7/site-packages/ovirtsdk4/service.py:296: in callback self._check_fault(response) /usr/lib64/python2.7/site-packages/ovirtsdk4/service.py:134: in _check_fault self._raise_error(response, body.fault) _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ response = <ovirtsdk4.http.Response object at 0x7f8781fee450> detail = <ovirtsdk4.types.Fault object at 0x7f8781fee690> @staticmethod def _raise_error(response, detail=None): """ Creates and raises an error containing the details of the given HTTP response and fault. This method is intended for internal use by other components of the SDK. Refrain from using it directly, as backwards compatibility isn't guaranteed. """ fault = detail if isinstance(detail, types.Fault) else None msg = '' if fault: if fault.reason: if msg: msg += ' ' msg = msg + 'Fault reason is "%s".' % fault.reason if fault.detail: if msg: msg += ' ' msg = msg + 'Fault detail is "%s".' % fault.detail if response: if response.code: if msg: msg += ' ' msg = msg + 'HTTP response code is %s.' % response.code if response.message: if msg: msg += ' ' msg = msg + 'HTTP response message is "%s".' % response.message if isinstance(detail, six.string_types): if msg: msg += ' ' msg = msg + detail + '.' class_ = Error if response is not None: if response.code in [401, 403]: class_ = AuthError elif response.code == 404: class_ = NotFoundError error = class_(msg) error.code = response.code if response else None error.fault = fault > raise error E Error: Fault reason is "Operation Failed". Fault detail is "[Cannot add Network. The name of the logical network 'network_1' is already used by an existing logical network in the same data-center. E -Please choose a different name.]". HTTP response code is 409. /usr/lib64/python2.7/site-packages/ovirtsdk4/service.py:118: Error

1 1

[oVirt Jenkins] ovirt-system-tests_basic-suite-4.3_nightly - Build # 355 - Failure!
by jenkins＠jenkins.phx.ovirt.org 09 Sep '20

09 Sep '20

Project: https://jenkins.ovirt.org/job/ovirt-system-tests_basic-suite-4.3_nightly/ Build: https://jenkins.ovirt.org/job/ovirt-system-tests_basic-suite-4.3_nightly/35… Build Number: 355 Build Status: Failure Triggered By: Started by timer ------------------------------------- Changes Since Last Success: ------------------------------------- Changes for Build #355 [Marcin Sobczyk] ost_utils: network_utils: Add 'get_ips[6]' functions ----------------- Failed Tests: ----------------- 1 tests failed. FAILED: 005_network_by_label.assign_labeled_network Error Message: False != True after 180 seconds Stack Trace: Traceback (most recent call last): File "/usr/lib64/python2.7/unittest/case.py", line 369, in run testMethod() File "/usr/lib/python2.7/site-packages/nose/case.py", line 197, in runTest self.test(*self.arg) File "/usr/lib/python2.7/site-packages/ovirtlago/testlib.py", line 142, in wrapped_test test() File "/usr/lib/python2.7/site-packages/ovirtlago/testlib.py", line 60, in wrapper return func(get_test_prefix(), *args, **kwargs) File "/usr/lib/python2.7/site-packages/ovirtlago/testlib.py", line 79, in wrapper prefix.virt_env.engine_vm().get_api(api_ver=4), *args, **kwargs File "/home/jenkins/agent/workspace/ovirt-system-tests_basic-suite-4.3_nightly/ovirt-system-tests/basic-suite-4.3/test-scenarios/005_network_by_label.py", line 149, in assign_labeled_network host_service, LABELED_NET_NAME)) File "/usr/lib/python2.7/site-packages/ovirtlago/testlib.py", line 282, in assert_true_within_short assert_equals_within_short(func, True, allowed_exceptions) File "/usr/lib/python2.7/site-packages/ovirtlago/testlib.py", line 266, in assert_equals_within_short func, value, SHORT_TIMEOUT, allowed_exceptions=allowed_exceptions File "/usr/lib/python2.7/site-packages/ovirtlago/testlib.py", line 252, in assert_equals_within '%s != %s after %s seconds' % (res, value, timeout) AssertionError: False != True after 180 seconds

1 1

oVirt infra daily report - unstable production jobs - 1308
by jenkins＠jenkins.phx.ovirt.org 09 Sep '20

09 Sep '20

Good morning! Attached is the HTML page with the jenkins status report. You can see it also here: - https://jenkins.ovirt.org/job/system_jenkins-report/1308//artifact/exported… Cheers, Jenkins

1 0

[JIRA] (OVIRT-3000) [FIRING:1] InstanceUnreachable ibm-srv08.ovirt.org kubernetes-nodes-exporter (amd64 linux ibm-srv08.ovirt.org external bare-metal-external ci)
by Shlomi Zidmi (oVirt JIRA) 08 Sep '20

08 Sep '20

[ https://ovirt-jira.atlassian.net/browse/OVIRT-3000?page=com.atlassian.jira.… ] Shlomi Zidmi updated OVIRT-3000: -------------------------------- Resolution: Done Status: Done (was: To Do) > [FIRING:1] InstanceUnreachable ibm-srv08.ovirt.org kubernetes-nodes-exporter (amd64 linux ibm-srv08.ovirt.org external bare-metal-external ci) > ---------------------------------------------------------------------------------------------------------------------------------------------- > > Key: OVIRT-3000 > URL: https://ovirt-jira.atlassian.net/browse/OVIRT-3000 > Project: oVirt - virtualization made easy > Issue Type: Bug > Reporter: Alertmanager_Bot > Assignee: infra > Labels: ALERT{alertname="InstanceUnreachable",instance="ibm-srv08.ovirt.org",job="kubernetes-nodes-exporter"} > > Labels: > - alertname = InstanceUnreachable > - beta_kubernetes_io_arch = amd64 > - beta_kubernetes_io_os = linux > - instance = ibm-srv08.ovirt.org > - job = kubernetes-nodes-exporter > - kubernetes_io_hostname = ibm-srv08.ovirt.org > - region = external > - type = bare-metal-external > - zone = ci > Annotations: > - description = ibm-srv08.ovirt.org of job kubernetes-nodes-exporter has been possibly down for more than 10 minutes. > Source: http://prometheus-0:9090/graph?g0.expr=up%7Bjob%3D%22kubernetes-nodes-expor… -- This message was sent by Atlassian Jira (v1001.0.0-SNAPSHOT#100145)

1 0

[JIRA] (OVIRT-2998) [FIRING:1] InstanceUnreachable ibm-srv07.ovirt.org kubernetes-nodes-exporter (amd64 linux ibm-srv07.ovirt.org external bare-metal-external ci)
by Shlomi Zidmi (oVirt JIRA) 08 Sep '20

08 Sep '20

[ https://ovirt-jira.atlassian.net/browse/OVIRT-2998?page=com.atlassian.jira.… ] Shlomi Zidmi updated OVIRT-2998: -------------------------------- Resolution: Done Status: Done (was: To Do) > [FIRING:1] InstanceUnreachable ibm-srv07.ovirt.org kubernetes-nodes-exporter (amd64 linux ibm-srv07.ovirt.org external bare-metal-external ci) > ---------------------------------------------------------------------------------------------------------------------------------------------- > > Key: OVIRT-2998 > URL: https://ovirt-jira.atlassian.net/browse/OVIRT-2998 > Project: oVirt - virtualization made easy > Issue Type: Bug > Reporter: Alertmanager_Bot > Assignee: infra > Labels: ALERT{alertname="InstanceUnreachable",instance="ibm-srv07.ovirt.org",job="kubernetes-nodes-exporter"} > > Labels: > - alertname = InstanceUnreachable > - beta_kubernetes_io_arch = amd64 > - beta_kubernetes_io_os = linux > - instance = ibm-srv07.ovirt.org > - job = kubernetes-nodes-exporter > - kubernetes_io_hostname = ibm-srv07.ovirt.org > - region = external > - type = bare-metal-external > - zone = ci > Annotations: > - description = ibm-srv07.ovirt.org of job kubernetes-nodes-exporter has been possibly down for more than 10 minutes. > Source: http://prometheus-0:9090/graph?g0.expr=up%7Bjob%3D%22kubernetes-nodes-expor… -- This message was sent by Atlassian Jira (v1001.0.0-SNAPSHOT#100145)

1 0

[JIRA] (OVIRT-3001) [FIRING:1] InstanceUnreachable ibm-srv09.ovirt.org kubernetes-nodes-exporter (amd64 linux ibm-srv09.ovirt.org true external bare-metal-external ci)
by Shlomi Zidmi (oVirt JIRA) 08 Sep '20

08 Sep '20

[ https://ovirt-jira.atlassian.net/browse/OVIRT-3001?page=com.atlassian.jira.… ] Shlomi Zidmi updated OVIRT-3001: -------------------------------- Resolution: Done Status: Done (was: To Do) > [FIRING:1] InstanceUnreachable ibm-srv09.ovirt.org kubernetes-nodes-exporter (amd64 linux ibm-srv09.ovirt.org true external bare-metal-external ci) > --------------------------------------------------------------------------------------------------------------------------------------------------- > > Key: OVIRT-3001 > URL: https://ovirt-jira.atlassian.net/browse/OVIRT-3001 > Project: oVirt - virtualization made easy > Issue Type: Bug > Reporter: Alertmanager_Bot > Assignee: infra > Labels: ALERT{alertname="InstanceUnreachable",instance="ibm-srv09.ovirt.org",job="kubernetes-nodes-exporter"} > > Labels: > - alertname = InstanceUnreachable > - beta_kubernetes_io_arch = amd64 > - beta_kubernetes_io_os = linux > - instance = ibm-srv09.ovirt.org > - job = kubernetes-nodes-exporter > - kubernetes_io_hostname = ibm-srv09.ovirt.org > - node_role_kubernetes_io_compute = true > - region = external > - type = bare-metal-external > - zone = ci > Annotations: > - description = ibm-srv09.ovirt.org of job kubernetes-nodes-exporter has been possibly down for more than 10 minutes. > Source: http://prometheus-0:9090/graph?g0.expr=up%7Bjob%3D%22kubernetes-nodes-expor… -- This message was sent by Atlassian Jira (v1001.0.0-SNAPSHOT#100145)

1 0

[JIRA] (OVIRT-3006) HTTP/500 errors when accessing lists.ovirt.org
by Evgheni Dereveanchin (oVirt JIRA) 08 Sep '20

08 Sep '20

Evgheni Dereveanchin created OVIRT-3006: ------------------------------------------- Summary: HTTP/500 errors when accessing lists.ovirt.org Key: OVIRT-3006 URL: https://ovirt-jira.atlassian.net/browse/OVIRT-3006 Project: oVirt - virtualization made easy Issue Type: Outage Reporter: Evgheni Dereveanchin Assignee: infra I can't access lists.ovirt.org via the browser, getting a 500 Internal Server Error after a considerable time that looks like a timeout. -- This message was sent by Atlassian Jira (v1001.0.0-SNAPSHOT#100145)

1 0

Re: [CI] Builds broken by "Status code: 403 for http://resources.ovirt.org/pub/yum-repo/ovirt-release-master.rpm"
by Nir Soffer 08 Sep '20

08 Sep '20

On Mon, Sep 7, 2020 at 2:30 PM Nir Soffer <nsoffer(a)redhat.com> wrote: > > This is the second time, last time there was an issue during a nightly job that > left the master snapshot repo broken. This is still broken today. In the past sending mail to infra-support created a bug, and someone was handling issue quickly. This seems to be broken now. We depend on http://resources.ovirt.org/pub/yum-repo/ovirt-release-master.rpm for development and CI. When the rpm is missing, CI jobs fail, and developers cannot update their setup. Please fix it as soon as possible. > I have 3 failure builds: > https://jenkins.ovirt.org/job/ovirt-engine_standard-check-patch/7744/ > https://jenkins.ovirt.org/job/ovirt-engine_standard-check-patch/7751/ > https://jenkins.ovirt.org/job/ovirt-engine_standard-check-patch/7755/ > > All seems to fail in: > > [2020-09-07T10:00:56.163Z] + dnf install -y > http://resources.ovirt.org/pub/yum-repo/ovirt-release-master.rpm > [2020-09-07T10:01:08.524Z] Status code: 403 for > http://resources.ovirt.org/pub/yum-repo/ovirt-release-master.rpm (IP: > 66.187.230.40) > > Checking manually show: > > $ curl https://resources.ovirt.org/pub/yum-repo/ovirt-release-master.rpm > <!DOCTYPE HTML PUBLIC "-//IETF//DTD HTML 2.0//EN"> > <html><head> > <title>403 Forbidden</title> > </head><body> > <h1>Forbidden</h1> > <p>You don't have permission to access /pub/yum-repo/ovirt-release-master.rpm > on this server.</p> > </body></html> > > Looking at: > https://resources.ovirt.org/pub/yum-repo/ > > ovirt-elease-master.rpm does not exist. > > So we have 2 issues: > - The file does not exist > - We return "403 Forbidden" instead of "404 Not Found"

3 5