Seems that the root cause is the fact that CQ tester is using stdci_runner now and stdci_runner is not taking 'extra_sources' into account.

Meanwhile, I've reverted the change so we can re-examine  stdci_runner.groovy

https://gerrit.ovirt.org/#/c/92714/

On Sun, Jul 1, 2018 at 5:59 PM Daniel Belenky <dbelenky@redhat.com> wrote:

Hi,

There's something that seems odd to me. @Barak Korren @Gal Ben Haim would appreciate you to have a look as well.

According to Alona, the patch that introduced the error is: https://gerrit.ovirt.org/#/c/92009/

It was tested (and deployed) in https://jenkins.ovirt.org/job/ovirt-master_change-queue-tester/8372/

But, from yum.log on engine's machine, it seems that an older version of ovirt-engine was used (not the version that was built with the patch that introduced the error - hence why it was deployed without failing the tests)

From https://jenkins.ovirt.org/job/ovirt-master_change-queue-tester/8372/artifact/exported-artifacts/basic-suit.el7.x86_64/test_logs/basic-suite-master/post-099_aaa-ldap.py/lago-basic-suite-master-engine/_var_log/yum.log

Jun 27 09:47:30 Installed: ovirt-engine-4.3.0-0.0.master.20180627094550.gita6b3aa4.el7.noarch

Where Alona's patch built ovirt-engine-4.3.0-0.0.master.20180627130008.gita3072d3.el7.noarch.rpm

As you can see here in engine's log, a6b3aa4 is older than a3072d3

11aeec678f2 webadmin: Add 'Multi Queues Enabled' to vm dialog

920bf6a75a1 packaging: Add python-netaddr requirement

a3072d3ceba engine: Add "MultiQueuesEnabled" to ovf reader and writer

4893678a272 engine: Update libvirtVmXml to consider vmBase.multiQueuesEnabled attribute

072bad8f4d3 engine: Introduce algorithm for calculating how many queues asign per vnic

c4dd484c4ee engine: Add multiQueuesEnabled to VmBase

a6b3aa45d18 packaging: setup: Return value 0 when using --help

And now, with Alona's patch that supposed to fix the issue (verified via manual OST), it seems that again, the CQ run is not using Alona's version of engine but an older one.

The patch that fixes the issue is https://gerrit.ovirt.org/#/c/92656/ which was tested here https://jenkins.ovirt.org/job/ovirt-master_change-queue-tester/8511/ (but again, the engine was actually installed from an older version that doesn't include the fix - hence it failed).

Could someone have a look too? If it's indeed the case, it's a severe bug form our side that we have to address ASAP.

On Sun, Jul 1, 2018 at 5:15 PM Eyal Edri <eedri@redhat.com> wrote:

Please prepare an RCA how we hit again a bug in the script if it was verified before deployment. 

On Sun, Jul 1, 2018 at 4:40 PM Ehud Yonasi <eyonasi@redhat.com> wrote:

Due to a bug in the retention policy, some packages were deleted from tested repo. 

We're rollbacking from nightly snapshot for all versions, and will update asap.

On Sun, Jul 1, 2018 at 4:37 PM Ehud Yonasi <eyonasi@redhat.com> wrote:

failed on installing packages. restoring tested repo

On Sun, Jul 1, 2018 at 4:34 PM Ehud Yonasi <eyonasi@redhat.com> wrote:

will trigger the job again

On Sun, Jul 1, 2018 at 3:27 PM oVirt Jenkins <jenkins@ovirt.org> wrote:

Change 92610,3 (ovirt-engine) is probably the reason behind recent system test
failures in the "ovirt-4.2" change queue and needs to be fixed.

This change had been removed from the testing queue. Artifacts build from this
change will not be released until it is fixed.

For further details about the change see:
https://gerrit.ovirt.org/#/c/92610/3

For failed test results see:
http://jenkins.ovirt.org/job/ovirt-4.2_change-queue-tester/2534/
_______________________________________________
Infra mailing list -- infra@ovirt.org
To unsubscribe send an email to infra-leave@ovirt.org
Privacy Statement: https://www.ovirt.org/site/privacy-policy/
oVirt Code of Conduct: https://www.ovirt.org/community/about/community-guidelines/
List Archives: https://lists.ovirt.org/archives/list/infra@ovirt.org/message/MVIJ6JKUKZSW3XKJTAJNIHPLVIVGWBN2/

_______________________________________________
Infra mailing list -- infra@ovirt.org
To unsubscribe send an email to infra-leave@ovirt.org
Privacy Statement: https://www.ovirt.org/site/privacy-policy/
oVirt Code of Conduct: https://www.ovirt.org/community/about/community-guidelines/
List Archives: https://lists.ovirt.org/archives/list/infra@ovirt.org/message/FOCZJSME4AYMZEDMC4GZ63UNVJ5WRSYT/

--

Eyal edri

MANAGER

RHV DevOps

EMEA VIRTUALIZATION R&D

Red Hat EMEA

TRIED. TESTED. TRUSTED.

phone: +972-9-7692018
irc: eedri (on #tlv #rhev-dev #rhev-integ)

_______________________________________________
Infra mailing list -- infra@ovirt.org
To unsubscribe send an email to infra-leave@ovirt.org
Privacy Statement: https://www.ovirt.org/site/privacy-policy/
oVirt Code of Conduct: https://www.ovirt.org/community/about/community-guidelines/
List Archives: https://lists.ovirt.org/archives/list/infra@ovirt.org/message/3CGDFQDQMSWAWXUBNPAH4VXBSYGVWCYE/

--

DANIEL BELENKY

RHV DEVOPS

--

DANIEL BELENKY

RHV DEVOPS