Re: [CQ]: 103548,1 (vdsm) failed "ovirt-4.3" system tests
On Thu, Sep 26, 2019 at 3:55 PM oVirt Jenkins <jenkins@ovirt.org> wrote:
Change 103548,1 (vdsm) is probably the reason behind recent system test failures in the "ovirt-4.3" change queue and needs to be fixed.
This change had been removed from the testing queue. Artifacts build from this change will not be released until it is fixed.
This is a bug in the change queue, because For further details about the change see:
this is a travis configuration change, and it cannot break system tests :-) Do we have stats of such failures in the past? For failed test results see:
http://jenkins.ovirt.org/job/ovirt-4.3_change-queue-tester/2188/
Maybe we can add a simple file based condition to tell if a patch need to run system tests? For example in vdsm we have this structure: lib/ static/ tests/ automation/ .travis.ymal ... Changes in tests/ and automation/ cannot break vdsm during runtime, so there is no need to run system tests for patches modifying these directories. Here are changes stats per directory since 4.2: $ git shortlog -sn --since v4.20.0 ./tests ./automation | awk '{sum+=$1} END {print sum}' 4579 $ git shortlog -sn --since v4.20.0 ./lib ./static | awk '{sum+=$1} END {print sum}' 4413 So 50% of the changes cannot break the system and do not require system tests. SLOC Directory SLOC-by-Language (Sorted) 65205 lib python=64977,sh=228 62415 tests python=62317,sh=98 3432 vdsm_hooks python=3432 803 contrib python=740,sh=63 468 automation sh=454,python=14 313 static python=210,sh=103 289 init sh=162,python=127 112 build-aux sh=72,python=40 64 top_dir sh=64 30 doc python=30 30 helpers python=30 1 vdsm_log sh=1 0 docker (none) 0 m4 (none) We can see that the tests are about 50% of the code. This is pretty good for legacy project, but if we look in a more modern project like ovirt-imageio: common: SLOC Directory SLOC-by-Language (Sorted) 3641 test python=3641 2365 ovirt_imageio_common python=2201,ansic=164 103 top_dir python=103 daemon: SLOC Directory SLOC-by-Language (Sorted) 1588 test python=1588 792 ovirt_imageio_daemon python=792 18 top_dir python=18 0 data (none) We see that test code ratio is close to 2/1. So adding a filter per project that can tell if a patch needs system tests very important, and can speed our workflow by fact of 2-3. Adding infra-support to file a bug, we need to work on this. Nir
Hey Nir, It's actually happening because of centos 7.6 mirrors were deleted today. I've proposed [1] to fix it. [1]: https://gerrit.ovirt.org/#/c/103601/ On Thu, Sep 26, 2019 at 4:59 PM Nir Soffer <nsoffer@redhat.com> wrote:
On Thu, Sep 26, 2019 at 3:55 PM oVirt Jenkins <jenkins@ovirt.org> wrote:
Change 103548,1 (vdsm) is probably the reason behind recent system test failures in the "ovirt-4.3" change queue and needs to be fixed.
This change had been removed from the testing queue. Artifacts build from this change will not be released until it is fixed.
This is a bug in the change queue, because
For further details about the change see:
this is a travis configuration change, and it cannot break system tests :-)
Do we have stats of such failures in the past?
For failed test results see:
http://jenkins.ovirt.org/job/ovirt-4.3_change-queue-tester/2188/
Maybe we can add a simple file based condition to tell if a patch need to run system tests?
For example in vdsm we have this structure: lib/ static/ tests/ automation/ .travis.ymal ...
Changes in tests/ and automation/ cannot break vdsm during runtime, so there is no need to run system tests for patches modifying these directories.
Here are changes stats per directory since 4.2:
$ git shortlog -sn --since v4.20.0 ./tests ./automation | awk '{sum+=$1} END {print sum}' 4579 $ git shortlog -sn --since v4.20.0 ./lib ./static | awk '{sum+=$1} END {print sum}' 4413
So 50% of the changes cannot break the system and do not require system tests.
SLOC Directory SLOC-by-Language (Sorted) 65205 lib python=64977,sh=228 62415 tests python=62317,sh=98 3432 vdsm_hooks python=3432 803 contrib python=740,sh=63 468 automation sh=454,python=14 313 static python=210,sh=103 289 init sh=162,python=127 112 build-aux sh=72,python=40 64 top_dir sh=64 30 doc python=30 30 helpers python=30 1 vdsm_log sh=1 0 docker (none) 0 m4 (none)
We can see that the tests are about 50% of the code. This is pretty good for legacy project, but if we look in a more modern project like ovirt-imageio:
common: SLOC Directory SLOC-by-Language (Sorted) 3641 test python=3641 2365 ovirt_imageio_common python=2201,ansic=164 103 top_dir python=103
daemon: SLOC Directory SLOC-by-Language (Sorted) 1588 test python=1588 792 ovirt_imageio_daemon python=792 18 top_dir python=18 0 data (none)
We see that test code ratio is close to 2/1.
So adding a filter per project that can tell if a patch needs system tests very important, and can speed our workflow by fact of 2-3.
Adding infra-support to file a bug, we need to work on this.
Nir _______________________________________________ Devel mailing list -- devel@ovirt.org To unsubscribe send an email to devel-leave@ovirt.org Privacy Statement: https://www.ovirt.org/site/privacy-policy/ oVirt Code of Conduct: https://www.ovirt.org/community/about/community-guidelines/ List Archives: https://lists.ovirt.org/archives/list/devel@ovirt.org/message/IMHBHVRPNCSFFB...
participants (2)
-
Ehud Yonasi -
Nir Soffer