Hi all,
Thanks to Edward, we have now coverage reports in jenkins.
The way to access the report on jenkins is to use this url:
http://jenkins.ovirt.org/job/vdsm_master_check-patch-fc23-x86_64/<build-number>/artifact/exported-artifacts/htmlcov/index.html
Here is an example, hopefully it will be accessible when you try:
http://jenkins.ovirt.org/job/vdsm_master_check-patch-fc23-x86_64/648/artifact/exported-artifacts/htmlcov/index.html
Todo:
- Easy way to access the report from gerrit
It should be easy to add a link the coverage report in the comment added
by jenkins after a build finish.
- Store the coverage reports for longer time, maybe a week?
- We have only 45% coverage instead of the minimum, 100%.
Note that coverage of 25% can mean *no* code was run by the tests.
The only code
running was the functions and class definitions. Here is an example:
http://jenkins.ovirt.org/job/vdsm_master_check-patch-fc23-x86_64/648/artifact/exported-artifacts/htmlcov/_home_jenkins_workspace_vdsm_master_check-patch-fc23-x86_64_vdsm_vdsm_storage_devicemapper_py.html
Modules that were never imported during the tests have 0% coverage:
http://jenkins.ovirt.org/job/vdsm_master_check-patch-fc23-x86_64/648/artifact/exported-artifacts/htmlcov/_home_jenkins_workspace_vdsm_master_check-patch-fc23-x86_64_vdsm_vdsm_storage_hsm_py.html
- coverage creates lot of useless noise in the jenkins logs, need to slicense
this output.
http://jenkins.ovirt.org/job/vdsm_master_check-patch-fc23-x86_64/648/consoleFull
I did not find a way to do this in nosetests, may need hacking
nose coverage plugin.
- The report includes only the tests in the tests directory.
We have additional tests in lib/vdsm/infra/* which are not
included. We should
move these to the tests directory.
- The report is using absolute paths, but we like shorter relative paths.
I don't see a way to configure nosetests or coverage to generate
relative paths.
May need hacking of the generated html/json in htmlcov.
- Add "make coverage" target for running coverage locally
- An easy way to enable coverage for the functional tests or for running
a single test module.
Can be done using nosetests --cover* options. Should be documented in
tests/README, and maybe automated using a script or Makefile.
When running locally, one would like to have the script open the report
in the browser automatically:
xdg-open tests/htmlcov/index.html
- An easy way to enable coverage when testing flows in vdsm
Petr sent a patch for enabling coverage using vdsm.conf:
https://gerrit.ovirt.org/49168
We discussed adding vdsm-coverage package that will make it easy to setup
Nir
_______________________________________________
Devel mailing list
Devel@ovirt.org
http://lists.ovirt.org/mailman/listinfo/devel