Re: [CQ]: 92610,3 (ovirt-engine) failed "ovirt-4.2" system tests

Seems that the root cause is the fact that CQ tester is using stdci_runner now and stdci_runner is not taking 'extra_sources' into account.

Meanwhile, I've reverted the change so we can re-examine stdci_runner.groovy https://gerrit.ovirt.org/#/c/92714/ On Sun, Jul 1, 2018 at 5:59 PM Daniel Belenky <dbelenky(a)redhat.com&gt; wrote: > Hi, > > There's something that seems odd to me. @Barak Korren > <bkorren(a)redhat.com&gt; @Gal Ben Haim <gbenhaim(a)redhat.com&gt; would > appreciate you to have a look as well. > > According to Alona, the patch that introduced the error is: > https://gerrit.ovirt.org/#/c/92009/ > > It was *tested* (and *deployed*) in https://jenkins.ovirt.org/ > job/ovirt-master_change-queue-tester/8372/ > > But, from yum.log on engine's machine, it seems that an older version of > ovirt-engine was used (not the version that was built with the patch that > introduced the error - hence why it was deployed without failing the tests) > > From https://jenkins.ovirt.org/job/ovirt-master_change- > queue-tester/8372/artifact/exported-artifacts/basic-suit. > el7.x86_64/test_logs/basic-suite-master/post-099_aaa- > ldap.py/lago-basic-suite-master-engine/_var_log/yum.log > > Jun 27 09:47:30 Installed: ovirt-engine-4.3.0-0.0.master.20180627094550.gita6b3aa4.el7.noarch > > Where Alona's patch built ovirt-engine-4.3.0-0.0.master.20180627130008.gita3072d3.el7.noarch.rpm > > As you can see here in engine's log, *a6b3aa4* is older than *a3072d3* > 11aeec678f2 webadmin: Add 'Multi Queues Enabled' to vm dialog > 920bf6a75a1 packaging: Add python-netaddr requirement > *a3072d3*ceba engine: Add "MultiQueuesEnabled" to ovf reader and writer > 4893678a272 engine: Update libvirtVmXml to consider > vmBase.multiQueuesEnabled attribute > 072bad8f4d3 engine: Introduce algorithm for calculating how many queues > asign per vnic > c4dd484c4ee engine: Add multiQueuesEnabled to VmBase > *a6b3aa4*5d18 packaging: setup: Return value 0 when using --help > > > And now, with Alona's patch that supposed to fix the issue (verified via > manual OST), it seems that again, the CQ run is not using Alona's version > of engine but an older one. > > The patch that fixes the issue is https://gerrit.ovirt.org/#/c/92656/ > which was tested here https://jenkins.ovirt.org/job/ > ovirt-master_change-queue-tester/8511/ (but again, the engine was > actually installed from an older version that doesn't include the fix - > hence it failed). > > > Could someone have a look too? If it's indeed the case, it's a severe bug > form our side that we have to address ASAP. > > On Sun, Jul 1, 2018 at 5:15 PM Eyal Edri <eedri(a)redhat.com&gt; wrote: > >> Please prepare an RCA how we hit again a bug in the script if it was >> verified before deployment. >> >> On Sun, Jul 1, 2018 at 4:40 PM Ehud Yonasi <eyonasi(a)redhat.com&gt; wrote: >> >>> Due to a bug in the retention policy, some packages were deleted from >>> tested repo. >>> We're rollbacking from nightly snapshot for all versions, and will >>> update asap. >>> >>> On Sun, Jul 1, 2018 at 4:37 PM Ehud Yonasi <eyonasi(a)redhat.com&gt; wrote: >>> >>>> failed on installing packages. restoring tested repo >>>> >>>> >>>> On Sun, Jul 1, 2018 at 4:34 PM Ehud Yonasi <eyonasi(a)redhat.com&gt; wrote: >>>> >>>>> will trigger the job again >>>>> >>>>> >>>>> On Sun, Jul 1, 2018 at 3:27 PM oVirt Jenkins <jenkins(a)ovirt.org&gt; >>>>> wrote: >>>>> >>>>>> Change 92610,3 (ovirt-engine) is probably the reason behind recent >>>>>> system test >>>>>> failures in the "ovirt-4.2" change queue and needs to be fixed. >>>>>> >>>>>> This change had been removed from the testing queue. Artifacts build >>>>>> from this >>>>>> change will not be released until it is fixed. >>>>>> >>>>>> For further details about the change see: >>>>>> https://gerrit.ovirt.org/#/c/92610/3 >>>>>> >>>>>> For failed test results see: >>>>>> http://jenkins.ovirt.org/job/ovirt-4.2_change-queue-tester/2534/ >>>>>> _______________________________________________ >>>>>> Infra mailing list -- infra(a)ovirt.org >>>>>> To unsubscribe send an email to infra-leave(a)ovirt.org >>>>>> Privacy Statement: https://www.ovirt.org/site/privacy-policy/ >>>>>> oVirt Code of Conduct: https://www.ovirt.org/ >>>>>> community/about/community-guidelines/ >>>>>> List Archives: https://lists.ovirt.org/ >>>>>> archives/list/infra(a)ovirt.org/message/MVIJ6JKUKZSW3XKJTAJNIHPLVIVGWB >>>>>> N2/ >>>>>> >>>>> _______________________________________________ >>> Infra mailing list -- infra(a)ovirt.org >>> To unsubscribe send an email to infra-leave(a)ovirt.org >>> Privacy Statement: https://www.ovirt.org/site/privacy-policy/ >>> oVirt Code of Conduct: https://www.ovirt.org/community/about/community- >>> guidelines/ >>> List Archives: https://lists.ovirt.org/archives/list/infra@ovirt.org/ >>> message/FOCZJSME4AYMZEDMC4GZ63UNVJ5WRSYT/ >>> >> >> >> -- >> >> Eyal edri >> >> >> MANAGER >> >> RHV DevOps >> >> EMEA VIRTUALIZATION R&D >> >> >> Red Hat EMEA <https://www.redhat.com/> >> <https://red.ht/sig> TRIED. TESTED. TRUSTED. >> <https://redhat.com/trusted> >> phone: +972-9-7692018 >> irc: eedri (on #tlv #rhev-dev #rhev-integ) >> _______________________________________________ >> Infra mailing list -- infra(a)ovirt.org >> To unsubscribe send an email to infra-leave(a)ovirt.org >> Privacy Statement: https://www.ovirt.org/site/privacy-policy/ >> oVirt Code of Conduct: https://www.ovirt.org/community/about/community- >> guidelines/ >> List Archives: https://lists.ovirt.org/archives/list/infra@ovirt.org/ >> message/3CGDFQDQMSWAWXUBNPAH4VXBSYGVWCYE/ >> > > > -- > > DANIEL BELENKY > > RHV DEVOPS > -- DANIEL BELENKY RHV DEVOPS