- Devel - oVirt List Archives

[ OST Failure Report ] [ oVirt Master (ovirt-engine) ] [ 08-02-2018 ] [ 001_initialize_engine.initialize_engine ]
by Dafna Ron 08 Feb '18

08 Feb '18

Hi, We are failing test 001_initialize_engine.initialize_engine It seems that the issue is related to the ovirt-engine-dwhd.service not starting. *Link and headline of suspected patches: https://gerrit.ovirt.org/#/c/86920/ <https://gerrit.ovirt.org/#/c/86920/> - * *core: adding key to bundles/AuditLogMessagesLink to Job:http://jenkins.ovirt.org/job/ovirt-master_change-queue-tester/5365/ <http://jenkins.ovirt.org/job/ovirt-master_change-queue-tester/5365/>Link to all logs:http://jenkins.ovirt.org/job/ovirt-master_change-queue-tester/5365/art… <http://jenkins.ovirt.org/job/ovirt-master_change-queue-tester/5365/artifact/>(Relevant) error snippet from the log: <error>Feb 7 22:43:23 lago-basic-suite-master-engine dbus[525]: [system] Reloaded configurationFeb 7 22:43:24 lago-basic-suite-master-engine firewalld[565]: WARNING: ICMP type 'beyond-scope' is not supported by the kernel for ipv6.Feb 7 22:43:24 lago-basic-suite-master-engine firewalld[565]: WARNING: beyond-scope: INVALID_ICMPTYPE: No supported ICMP type., ignoring for run-time.Feb 7 22:43:24 lago-basic-suite-master-engine firewalld[565]: WARNING: ICMP type 'failed-policy' is not supported by the kernel for ipv6.Feb 7 22:43:24 lago-basic-suite-master-engine firewalld[565]: WARNING: failed-policy: INVALID_ICMPTYPE: No supported ICMP type., ignoring for run-time.Feb 7 22:43:24 lago-basic-suite-master-engine firewalld[565]: WARNING: ICMP type 'reject-route' is not supported by the kernel for ipv6.Feb 7 22:43:24 lago-basic-suite-master-engine firewalld[565]: WARNING: reject-route: INVALID_ICMPTYPE: No supported ICMP type., ignoring for run-time.Feb 7 22:43:28 lago-basic-suite-master-engine firewalld[565]: WARNING: ICMP type 'beyond-scope' is not supported by the kernel for ipv6.Feb 7 22:43:28 lago-basic-suite-master-engine firewalld[565]: WARNING: beyond-scope: INVALID_ICMPTYPE: No supported ICMP type., ignoring for run-time.Feb 7 22:43:28 lago-basic-suite-master-engine firewalld[565]: WARNING: ICMP type 'failed-policy' is not supported by the kernel for ipv6.Feb 7 22:43:28 lago-basic-suite-master-engine firewalld[565]: WARNING: failed-policy: INVALID_ICMPTYPE: No supported ICMP type., ignoring for run-time.Feb 7 22:43:28 lago-basic-suite-master-engine firewalld[565]: WARNING: ICMP type 'reject-route' is not supported by the kernel for ipv6.Feb 7 22:43:28 lago-basic-suite-master-engine firewalld[565]: WARNING: reject-route: INVALID_ICMPTYPE: No supported ICMP type., ignoring for run-time.Feb 7 22:43:28 lago-basic-suite-master-engine systemd: Starting oVirt Engine fence_kdump listener...Feb 7 22:43:28 lago-basic-suite-master-engine systemd: Started oVirt Engine fence_kdump listener.Feb 7 22:43:28 lago-basic-suite-master-engine systemd: Reloading.Feb 7 22:43:28 lago-basic-suite-master-engine systemd: Starting oVirt Engine...Feb 7 22:43:28 lago-basic-suite-master-engine journal: 2018-02-07 22:43:28,908-0500 ovirt-engine: INFO _detectJBossVersion:187 Detecting JBoss version. Running: /usr/lib/jvm/jre/bin/java ['ovirt-engine-version', '-server', '-XX:+TieredCompilation', '-Xms1024M', '-Xmx1024M', '-Djava.awt.headless=true', '-Dsun.rmi.dgc.client.gcInterval=3600000', '-Dsun.rmi.dgc.server.gcInterval=3600000', '-Djsse.enableSNIExtension=false', '-XX:+HeapDumpOnOutOfMemoryError', '-XX:HeapDumpPath=/var/log/ovirt-engine/dump', '-Djava.util.logging.manager=org.jboss.logmanager', '-Dlogging.configuration=file:///var/lib/ovirt-engine/jboss_runtime/config/ovirt-engine-logging.properties', '-Dorg.jboss.resolver.warning=true', '-Djboss.modules.system.pkgs=org.jboss.byteman', '-Djboss.server.default.config=ovirt-engine', '-Djboss.home.dir=/usr/share/ovirt-engine-wildfly', '-Djboss.server.base.dir=/usr/share/ovirt-engine', '-Djboss.server.data.dir=/var/lib/ovirt-engine', '-Djboss.server.log.dir=/var/log/ovirt-engine', '-Djboss.server.config.dir=/var/lib/ovirt-engine/jboss_runtime/config', '-Djboss.server.temp.dir=/var/lib/ovirt-engine/jboss_runtime/tmp', '-Djboss.controller.temp.dir=/var/lib/ovirt-engine/jboss_runtime/tmp', '-jar', '/usr/share/ovirt-engine-wildfly/jboss-modules.jar', '-mp', '/usr/share/ovirt-engine-wildfly-overlay/modules:/usr/share/ovirt-engine/modules/common:/usr/share/ovirt-engine-extension-aaa-jdbc/modules:/usr/share/ovirt-engine-extension-aaa-ldap/modules:/usr/share/ovirt-engine-wildfly/modules', '-jaxpmodule', 'javax.xml.jaxp-provider', 'org.jboss.as.standalone', '-v']Feb 7 22:43:29 lago-basic-suite-master-engine journal: 2018-02-07 22:43:29,481-0500 ovirt-engine: INFO _detectJBossVersion:207 Return code: 0, | stdout: '[u'WildFly Full 11.0.0.Final (WildFly Core 3.0.8.Final)'], | stderr: '[]'Feb 7 22:43:29 lago-basic-suite-master-engine systemd: Started oVirt Engine.Feb 7 22:43:29 lago-basic-suite-master-engine systemd: Reloading.Feb 7 22:43:29 lago-basic-suite-master-engine systemd: Starting oVirt Engine Data Warehouse...Feb 7 22:43:29 lago-basic-suite-master-engine systemd: Started oVirt Engine Data Warehouse.Feb 7 22:43:29 lago-basic-suite-master-engine systemd: Reloading.Feb 7 22:43:30 lago-basic-suite-master-engine systemd: Starting oVirt ImageIO Proxy...Feb 7 22:43:30 lago-basic-suite-master-engine systemd: Started oVirt ImageIO Proxy.Feb 7 22:43:30 lago-basic-suite-master-engine systemd: Reloading.Feb 7 22:43:30 lago-basic-suite-master-engine journal: 2018-02-07 22:43:30,301-0500 ovirt-engine-dwhd: ERROR run:554 Error: process terminated with status code 1Feb 7 22:43:30 lago-basic-suite-master-engine systemd: ovirt-engine-dwhd.service: main process exited, code=exited, status=1/FAILUREFeb 7 22:43:30 lago-basic-suite-master-engine systemd: Unit ovirt-engine-dwhd.service entered failed state.Feb 7 22:43:30 lago-basic-suite-master-engine systemd: ovirt-engine-dwhd.service failed.</error>*

1 0

[OST][HC] HE fails to deploy
by Sahina Bose 07 Feb '18

07 Feb '18

Hi all, I see the HE fails to deploy after task in running ansible playbook create_target_vm : TASK [Wait for the engine to come up on the target VM]", with Error engine state=EngineUnexpectedlyDown Is this a known issue that you are working on? thanks! sahina Full HE setup log at http://jenkins.ovirt.org/job/ovirt-system-tests_master_check-patch-el7-x86_…

2 3

ATTN: oVirt Engine 4.2.z has been branched from master
by Tal Nisan 06 Feb '18

06 Feb '18

As the subject says, we've branched today to ovirt-engine-4.2 branch, every change that is required for 4.2.2 and onward should be backported to this branch. Note that we chose now so there will be no confusion on whether your patches are included, if your patch got merge in master before 21:45 IST (20:45 CET) then no backport is needed. Please note that though we branched out the main focus should still be 4.2.2.

1 0

[vdsm][RFC] reconsidering branching out ovirt-4.2
by Francesco Romani 06 Feb '18

06 Feb '18

Hi all, It is time again to reconsider branching out the 4.2 stable branch. So far we decided to *not* branch out, and we are taking tags for ovirt 4.2 releases from master branch. This means we are merging safe and/or stabilization patches only in master. I think it is time to reconsider this decision and branch out for 4.2, because of two reasons: 1. it sends a clearer signal that 4.2 is going in stabilization mode 2. we have requests from virt team, which wants to start working on the next cycle features. If we decide to branch out, I'd start the new branch on monday, February 5 (1 week from now). The discussion is open, please share your acks/nacks for branching out, and for the branching date. I for myself I'm inclined to branch out, so if noone chimes in (!!) I'll execute the above plan. -- Francesco Romani Senior SW Eng., Virtualization R&D Red Hat IRC: fromani github: @fromanirh

5 5

[ACTION-REQUIRED] Making accurate CI for oVirt 4.2
by Barak Korren 06 Feb '18

06 Feb '18

Hi, This message is aimed for project maintainers. Other developers may also find it interesting to have a glimpse at the oVirt-wide test and composition processes. TL;DR: To get accurate CI for oVirt 4.2, most projects need to add 4.2 jobs in YAML. Before I can explain what the current issue is and which action is required, I'm need to provide a brief overview into how oVirt CI works. oVirt CI has two major components to it: 1. The STDCI component which is used to build and test individual projects. Most developers interact with this on a daily basis as it is responding on GitHub and Gerrit events. 2. The "change-queue" (CQ) component which is used to automatically compose the whole of oVirt from its sub projects and run system tests (OST) on it. This component is used to gather the information that is used by the infra team to compose the "OST failure report" you can occasionally see being sent to this list. The change queue is also used to automatically maintain the 'tested' and '*-snapshot' (AKA nightly) repositories. The way the CQ composes oVirt is by looking at the post-merge STDCI 'build-artifacts' jobs, and collecting together artifacts built by jobs that target a specific oVirt version into that version's change queue. Essentially the '*_master_build-artifacts-*' jobs go into the 'ovirt-master' change queue, the '*_4.1_build-artifacts-*' jobs go into the 'ovirt-4.1' change queue etc. Over the course of the oVirt 4.2 development, most project used their 'master' branch, which is typically mapped to '*_master_*' jobs, for developing 4.2 code. So the 'ovirt-master' CQ provided good indication of the state of 4.2 code. As projects started addeing 4.2 branches, we have created an 'ovirt-4.2' CQ to gather them. We did so under the assumption that most projects will branch soon after. The assumption turned up to be wrong as most projects did not yet fork and may not do so in the near future. As some projects did fork, the end result is that currently: ___there is no accurate representation of oVirt 4.2 in CI___ 'ovirt-master' CQ no longer represents oVirt 4.2 as some projects already have some 4.3 code in their 'master' branches. 'ovirt-4.2' CQ does not represent oVirt 4.2 as most projects do not push artifacts into it. To get any benefit from CI, we need to have it represent what we are going to release. This means that at this point we need all projects to have '*_4.2_build-artifacts-*' jobs that map to the code that is intended to be included in oVirt 4.2. Projects can either: 1. Create 4.2 branches and map the new jobs to them. 2. Keep 4.2 development in 'master' and create '4.2' jobs that map to it. Taking route #2 means a commitment to not adding any 4.3 code to the 'master' branch. Please keep it, as rolling back "too new" builds from the various repos and caches we have is very difficult. -- Barak Korren RHV DevOps team , RHCE, RHCi Red Hat EMEA redhat.com | TRIED. TESTED. TRUSTED. | redhat.com/trusted

7 15

Re: [ovirt-devel] [CQ]: 85992, 1 (ovirt-wgt) failed "ovirt-master" system tests
by Sandro Bonazzola 06 Feb '18

06 Feb '18

This failed because the following builds failed: vdsm_master_check-merged-el7-x86_64 <http://jenkins.ovirt.org/job/vdsm_master_check-merged-el7-x86_64/> #3147 <http://jenkins.ovirt.org/job/vdsm_master_check-merged-el7-x86_64/3147/> I think it failed on network/functional/static_ip_test.py ....................+ res=124 But I let VDSM team to review. ovirt-wgt_master_build-artifacts-fc27-x86_64 <http://jenkins.ovirt.org/job/ovirt-wgt_master_build-artifacts-fc27-x86_64/> #2 <http://jenkins.ovirt.org/job/ovirt-wgt_master_build-artifacts-fc27-x86_64/2/> ovirt-wgt_master_build-artifacts-fcraw-x86_64 <http://jenkins.ovirt.org/job/ovirt-wgt_master_build-artifacts-fcraw-x86_64/> #2 <http://jenkins.ovirt.org/job/ovirt-wgt_master_build-artifacts-fcraw-x86_64/…> wgt failed on missing ovirt-guest-agent-windows which was failing due to rich dependency resolution not possible using yum. Fix has been pushed to ovirt-guset-agent: https://gerrit.ovirt.org/#/c/87072/ Once this is merged, guest agent will build on Fedora providing the needed dependency and will fix above 2 jobs. 2018-02-05 22:13 GMT+01:00 oVirt Jenkins <jenkins(a)ovirt.org>: > Change 85992,1 (ovirt-wgt) is probably the reason behind recent system test > failures in the "ovirt-master" change queue and needs to be fixed. > > This change had been removed from the testing queue. Artifacts build from > this > change will not be released until it is fixed. > > For further details about the change see: > https://gerrit.ovirt.org/#/c/85992/1 > > For failed test results see: > http://jenkins.ovirt.org/job/ovirt-master_change-queue-tester/5287/ > _______________________________________________ > Infra mailing list > Infra(a)ovirt.org > http://lists.ovirt.org/mailman/listinfo/infra > -- SANDRO BONAZZOLA ASSOCIATE MANAGER, SOFTWARE ENGINEERING, EMEA ENG VIRTUALIZATION R&D Red Hat EMEA <https://www.redhat.com/> <https://red.ht/sig> TRIED. TESTED. TRUSTED. <https://redhat.com/trusted>

1 0

Re: [ovirt-devel] fcraw Jenkins build fails on import error
by Eyal Edri 05 Feb '18

05 Feb '18

also adding devel, it looks more dev issue than infra. On Mon, Feb 5, 2018 at 2:59 PM, Eyal Edri <eedri(a)redhat.com> wrote: > Adding Sandro, > Are we officially supporting fcraw for engine? > > On Mon, Feb 5, 2018 at 2:26 PM, Tal Nisan <tnisan(a)redhat.com> wrote: > >> >> *http://jenkins.ovirt.org/job/ovirt-engine_master_check-patch-fcraw-x86_64/29/consoleFull <http://jenkins.ovirt.org/job/ovirt-engine_master_check-patch-fcraw-x86_64/2…>* >> >> >> >> *10:53:31* ==================================== ERRORS ====================================*10:53:31* ERROR collecting packaging/setup/tests/ovirt_engine_setup/engine_common/test_database.py *10:53:31* ImportError while importing test module '/home/jenkins/workspace/ovirt-engine_master_check-patch-fcraw-x86_64/ovirt-engine/packaging/setup/tests/ovirt_engine_setup/engine_common/test_database.py'.*10:53:31* Hint: make sure your test modules/packages have valid Python names.*10:53:31* Traceback:*10:53:31* packaging/setup/tests/ovirt_engine_setup/engine_common/test_database.py:19: in <module>*10:53:31* import ovirt_engine_setup.engine_common.database as under_test # isort:skip # noqa: E402*10:53:31* packaging/setup/ovirt_engine_setup/engine_common/database.py:30: in <module>*10:53:31* from otopi import base*10:53:31* E ImportError: No module named otopi*10:53:31* !!!!!!!!!!!!!!!!!!! Interrupted: 1 errors during collection !!!!!!!!!!!!!!!!!!!!*10:53:31* =========================== 1 error in 0.43 seconds ============================ >> >> >> >> >> >> >> _______________________________________________ >> Infra mailing list >> Infra(a)ovirt.org >> http://lists.ovirt.org/mailman/listinfo/infra >> >> > > > -- > > Eyal edri > > > MANAGER > > RHV DevOps > > EMEA VIRTUALIZATION R&D > > > Red Hat EMEA <https://www.redhat.com/> > <https://red.ht/sig> TRIED. TESTED. TRUSTED. <https://redhat.com/trusted> > phone: +972-9-7692018 <+972%209-769-2018> > irc: eedri (on #tlv #rhev-dev #rhev-integ) > -- Eyal edri MANAGER RHV DevOps EMEA VIRTUALIZATION R&D Red Hat EMEA <https://www.redhat.com/> <https://red.ht/sig> TRIED. TESTED. TRUSTED. <https://redhat.com/trusted> phone: +972-9-7692018 irc: eedri (on #tlv #rhev-dev #rhev-integ)

1 0

[IOPROCESS] New release for Fedora
by Nir Soffer 04 Feb '18

04 Feb '18

Hi all, I released ioprocess 1.0.0 for Fedora 27 and 28. If you are using Fedora, please install the new version from the updates-testing and test it. Please share your feedback here: https://bodhi.fedoraproject.org/updates/FEDORA-2018-fbe8141dd2 This version will be available soon from oVirt repositories. Thanks, Nir

1 1

ovirt-engine fails to build on Fedora 27
by Sandro Bonazzola 04 Feb '18

04 Feb '18

Hi, can you please have a look at http://jenkins.ovirt.org/job/ovirt-engine_master_build-artifacts-fc27-x86_6… ? engine seems to fail building the rpm after the maven part completed, during the install process at: 15:53:23 ++ echo ovirt-engine-aaa.pom 15:53:23 ++ sed -e 's/^ovirt-engine-//' -e 's/\.pom//' 15:53:23 + artifact_id=aaa 15:53:23 + '[' -f /home/jenkins/workspace/ovirt-engine_master_build-artifacts-fc27-x86_64/ovirt-engine/rpmbuild/BUILDROOT/ovirt-engine-4.2.1.5-0.0.master.20180202142920.git0bc7f3edd2.fc27.x86_64/usr/share/java/ovirt-engine/aaa.jar ']' 15:53:23 + %add_maven_depmap -f backend JPP.ovirt-engine-aaa.pom ovirt-engine/aaa.jar 15:53:23 /var/tmp/rpm-tmp.YIGxWv: line 161: fg: no job control 15:53:23 error: Bad exit status from /var/tmp/rpm-tmp.YIGxWv (%install) Maybe some broken dep on backend around aaa.jar ? -- SANDRO BONAZZOLA ASSOCIATE MANAGER, SOFTWARE ENGINEERING, EMEA ENG VIRTUALIZATION R&D Red Hat EMEA <https://www.redhat.com/> <https://red.ht/sig> TRIED. TESTED. TRUSTED. <https://redhat.com/trusted>

2 1

OST failures weekly update [28/01/2018-02/02/2018]
by Dafna Ron 02 Feb '18

02 Feb '18

Hello, I would like to summarize the reported OST failures from the previous week. 01-02-2018 Test name: 001_upgrade_engine.test_initialize_engine version: Ovirt master Project otopi Suite: master/upgrade Cause: *https://gerrit.ovirt.org/#/c/86679/ <https://gerrit.ovirt.org/#/c/86679/> - core: Check Sequence before/after* Fix: *ON GOING ISSUE* Test Name: 002_bootstrap.verify_add_all_hosts Version: Ovirt Master Project: vdsm Suite: basic Cause: *https://gerrit.ovirt.org/#/c/69668/ <https://gerrit.ovirt.org/#/c/69668/> - **gluster: Fix error when brick is on a btrfs subvolume* *Fix: No Fix updated* *02*-02-2018 Test Name: 005_network_by_label.assign_hosts_network_label Version: Master Suite: master/upgrade Project: ovirt-image-uploader Cause: host object locked during api request Fix: no fix for failure. re-triggered patch Other incidents: * We have several build-artifacts failing and this is causing change queue to not run - the failure is caused due to clean-up on jobs which will remove fc24 * We are still seeing CPU Unknown errors on the 4.2 branch * there was a jenkins restart this week which cause a few jobs to fail * Jenkins restart may also cause some failures on hosts due to uncompleted clean-up - we are fixing any host found with this issue Thanks, Dafna

1 0