Change 103548,1 (vdsm) is probably the reason behind recent system test
failures in the "ovirt-4.3" change queue and needs to be fixed.
This change had been removed from the testing queue. Artifacts build from this
change will not be released until it is fixed.
This is a bug in the change queue, because
For further details about the change see:
https://gerrit.ovirt.org/#/c/103548/1
this is a travis configuration change, and it cannot break system tests :-)
Do we have stats of such failures in the past?
For failed test results see:
http://jenkins.ovirt.org/job/ovirt-4.3_change-queue-tester/2188/
Maybe we can add a simple file based condition to tell if a patch need to run
system tests?
For example in vdsm we have this structure:
lib/
static/
tests/
automation/
...
Changes in tests/ and automation/ cannot break vdsm during runtime, so there
is no need to run system tests for patches modifying these directories.
Here are changes stats per directory since 4.2:
$ git shortlog -sn --since v4.20.0 ./tests ./automation | awk '{sum+=$1} END {print sum}'
4579
$ git shortlog -sn --since v4.20.0 ./lib ./static | awk '{sum+=$1} END {print sum}'
4413
So 50% of the changes cannot break the system and do not require system tests.
SLOC Directory SLOC-by-Language (Sorted)
65205 lib python=64977,sh=228
62415 tests python=62317,sh=98
3432 vdsm_hooks python=3432
803 contrib python=740,sh=63
468 automation sh=454,python=14
313 static python=210,sh=103
289 init sh=162,python=127
112 build-aux sh=72,python=40
64 top_dir sh=64
30 doc python=30
30 helpers python=30
1 vdsm_log sh=1
0 docker (none)
0 m4 (none)
We can see that the tests are about 50% of the code. This is pretty good for legacy
project, but if we look in a more modern project like ovirt-imageio:
common:
SLOC Directory SLOC-by-Language (Sorted)
3641 test python=3641
2365 ovirt_imageio_common python=2201,ansic=164
103 top_dir python=103
daemon:
SLOC Directory SLOC-by-Language (Sorted)
1588 test python=1588
792 ovirt_imageio_daemon python=792
18 top_dir python=18
0 data (none)
We see that test code ratio is close to 2/1.
So adding a filter per project that can tell if a patch needs system tests very important,
and can speed our workflow by fact of 2-3.
Adding infra-support to file a bug, we need to work on this.
Nir