Hi, On 1/28/21 9:43 AM, Arik Hadas wrote:

Hi, Seems like our changes to bios type handing lead to that. Interestingly, OST passed on the patches.. Can you please provide more info on the verification process?

Regards, Marcin

Anyway, we look into it. Thanks for bringing this to our attention!

On Thu, Jan 28, 2021 at 12:13 AM Vojtech Juranek <vjuranek@redhat.com <mailto:vjuranek@redhat.com>> wrote:

Hi, OST fails constantly in test_check_snapshot_with_memory [1] with NPE  in VmDeviceUtils.updateUsbSlots [2]. Build with any additional changes (custom repo) is on [3].

Unfortunately, I wasn't able to find the root cause. Could someone please take a look?

Thanks Vojta

[1] https://jenkins.ovirt.org/job/ovirt-system-tests_manual/7720/consoleFull <https://jenkins.ovirt.org/job/ovirt-system-tests_manual/7720/consoleFull> [2] https://jenkins.ovirt.org/job/ovirt-system-tests_manual/7720/artifact/export... <https://jenkins.ovirt.org/job/ovirt-system-tests_manual/7720/artifact/exported-artifacts/test_logs/basic-suite-master/lago-basic-suite-master-engine/_var_log/ovirt-engine/engine.log> [3] https://jenkins.ovirt.org/job/ovirt-system-tests_manual/7718/parameters/ <https://jenkins.ovirt.org/job/ovirt-system-tests_manual/7718/parameters/>_______________________________________________ Devel mailing list -- devel@ovirt.org <mailto:devel@ovirt.org> To unsubscribe send an email to devel-leave@ovirt.org <mailto:devel-leave@ovirt.org> Privacy Statement: https://www.ovirt.org/privacy-policy.html <https://www.ovirt.org/privacy-policy.html> oVirt Code of Conduct: https://www.ovirt.org/community/about/community-guidelines/ <https://www.ovirt.org/community/about/community-guidelines/> List Archives: https://lists.ovirt.org/archives/list/devel@ovirt.org/message/M5LFINFRHR3T56... <https://lists.ovirt.org/archives/list/devel@ovirt.org/message/M5LFINFRHR3T56UDVBD53EOTUFPDXOPC/>

_______________________________________________ Devel mailing list -- devel@ovirt.org To unsubscribe send an email to devel-leave@ovirt.org Privacy Statement: https://www.ovirt.org/privacy-policy.html oVirt Code of Conduct: https://www.ovirt.org/community/about/community-guidelines/ List Archives: https://lists.ovirt.org/archives/list/devel@ovirt.org/message/6FCPAHKN6ECUTM...

On 1/28/21 10:30 AM, Arik Hadas wrote:

On Thu, Jan 28, 2021 at 11:21 AM Marcin Sobczyk <msobczyk@redhat.com <mailto:msobczyk@redhat.com>> wrote:

Hi,

On 1/28/21 9:43 AM, Arik Hadas wrote: > Hi, > Seems like our changes to bios type handing lead to that. > Interestingly, OST passed on the patches.. Can you please provide more info on the verification process?

Sure. The OST job [1] passed on PS 16 of [2] and there was no change on the patch between PS 16 and PS 17 that got in.

[1] https://jenkins.ovirt.org/view/oVirt%20system%20tests/job/ovirt-system-tests... <https://jenkins.ovirt.org/view/oVirt%20system%20tests/job/ovirt-system-tests_manual/7706/> [2] https://gerrit.ovirt.org/#/c/ovirt-engine/+/111657/ <https://gerrit.ovirt.org/#/c/ovirt-engine/+/111657/>

So here's some post-mortem analysis. The repo used for the manual run [3] was [4]. The build has been already cleaned up by jenkins, so there's no way to peek into what versions of engine were built there. We can estimate though, based on the date of 'check-patch' job, which is Jan 22 4:48 PM. The OST run was done on Jan 25, 2021 11:41 AM. The built packages were simply outdated when the OST run was made. We can actually see that in 'dnf.log' [5]: 2021-01-25T11:45:51Z INFO Dependencies resolved. 2021-01-25T11:45:51Z INFO ====================  Package  Arch  Version  Repository  Size =============== Upgrading:  ...  ovirt-engine  noarch 4.4.5.3-0.0.master.20210125103910.gitd5d5142096e.el8 ovirt-master-tested-el8  13 M The version of ovirt-engine that was available in ovirt-master-tested-el8 repo is from Jan 25. Some conclusions: - we use very fresh versions of packages in OST. If you're planning to test a package of your own please rebase first - if you're trying to test your own package please make sure it's actually used by OST run, you can check that in dnf.log files - in the future we should have an automated way of telling if none of the packages provided by the user didn't land in any of OST's VMs. I filed [6] to address this. Since this is blocking all basic suite runs I posted a patch [7] that disables USB on the VMs we create in the suite. Please review. Regards, Marcin [3] https://jenkins.ovirt.org/view/oVirt%20system%20tests/job/ovirt-system-tests... [4] https://jenkins.ovirt.org/job/ovirt-engine_standard-check-patch/9989/artifac... [5] https://jenkins.ovirt.org/view/oVirt%20system%20tests/job/ovirt-system-tests... [6] https://issues.redhat.com/browse/RHV-40844 [7] https://gerrit.ovirt.org/#/c/ovirt-system-tests/+/113201/

Regards, Marcin

> Anyway, we look into it. > Thanks for bringing this to our attention! > > On Thu, Jan 28, 2021 at 12:13 AM Vojtech Juranek <vjuranek@redhat.com <mailto:vjuranek@redhat.com> > <mailto:vjuranek@redhat.com <mailto:vjuranek@redhat.com>>> wrote: > >     Hi, >     OST fails constantly in test_check_snapshot_with_memory [1] with >     NPE  in >     VmDeviceUtils.updateUsbSlots [2]. Build with any additional >     changes (custom >     repo) is on [3]. > >     Unfortunately, I wasn't able to find the root cause. Could someone >     please take >     a look? > >     Thanks >     Vojta > >     [1] > https://jenkins.ovirt.org/job/ovirt-system-tests_manual/7720/consoleFull <https://jenkins.ovirt.org/job/ovirt-system-tests_manual/7720/consoleFull> >     <https://jenkins.ovirt.org/job/ovirt-system-tests_manual/7720/consoleFull <https://jenkins.ovirt.org/job/ovirt-system-tests_manual/7720/consoleFull>> >     [2] > https://jenkins.ovirt.org/job/ovirt-system-tests_manual/7720/artifact/export... <https://jenkins.ovirt.org/job/ovirt-system-tests_manual/7720/artifact/exported-artifacts/test_logs/basic-suite-master/lago-basic-suite-master-engine/_var_log/ovirt-engine/engine.log> >     <https://jenkins.ovirt.org/job/ovirt-system-tests_manual/7720/artifact/export... <https://jenkins.ovirt.org/job/ovirt-system-tests_manual/7720/artifact/exported-artifacts/test_logs/basic-suite-master/lago-basic-suite-master-engine/_var_log/ovirt-engine/engine.log>> >     [3] > https://jenkins.ovirt.org/job/ovirt-system-tests_manual/7718/parameters/ <https://jenkins.ovirt.org/job/ovirt-system-tests_manual/7718/parameters/> >     <https://jenkins.ovirt.org/job/ovirt-system-tests_manual/7718/parameters/ <https://jenkins.ovirt.org/job/ovirt-system-tests_manual/7718/parameters/>>_______________________________________________ >     Devel mailing list -- devel@ovirt.org <mailto:devel@ovirt.org> <mailto:devel@ovirt.org <mailto:devel@ovirt.org>> >     To unsubscribe send an email to devel-leave@ovirt.org <mailto:devel-leave@ovirt.org> >     <mailto:devel-leave@ovirt.org <mailto:devel-leave@ovirt.org>> >     Privacy Statement: https://www.ovirt.org/privacy-policy.html <https://www.ovirt.org/privacy-policy.html> >     <https://www.ovirt.org/privacy-policy.html <https://www.ovirt.org/privacy-policy.html>> >     oVirt Code of Conduct: > https://www.ovirt.org/community/about/community-guidelines/ <https://www.ovirt.org/community/about/community-guidelines/> >     <https://www.ovirt.org/community/about/community-guidelines/ <https://www.ovirt.org/community/about/community-guidelines/>> >     List Archives: > https://lists.ovirt.org/archives/list/devel@ovirt.org/message/M5LFINFRHR3T56... <https://lists.ovirt.org/archives/list/devel@ovirt.org/message/M5LFINFRHR3T56UDVBD53EOTUFPDXOPC/> >     <https://lists.ovirt.org/archives/list/devel@ovirt.org/message/M5LFINFRHR3T56... <https://lists.ovirt.org/archives/list/devel@ovirt.org/message/M5LFINFRHR3T56UDVBD53EOTUFPDXOPC/>> > > > _______________________________________________ > Devel mailing list -- devel@ovirt.org <mailto:devel@ovirt.org> > To unsubscribe send an email to devel-leave@ovirt.org <mailto:devel-leave@ovirt.org> > Privacy Statement: https://www.ovirt.org/privacy-policy.html <https://www.ovirt.org/privacy-policy.html> > oVirt Code of Conduct: https://www.ovirt.org/community/about/community-guidelines/ <https://www.ovirt.org/community/about/community-guidelines/> > List Archives: https://lists.ovirt.org/archives/list/devel@ovirt.org/message/6FCPAHKN6ECUTM... <https://lists.ovirt.org/archives/list/devel@ovirt.org/message/6FCPAHKN6ECUTMHPPH37UPPMBDUTPDGL/>