On 11/27 20:57, Nir Soffer wrote: > Todo: > > - Easy way to access the report from gerrit > It should be easy to add a link the coverage report in the comment added > by jenkins after a build finish. Not really, the gerrit comments that jenkins sends can't be changed by the build, that means that you can't put put there a non-static url. There's a quite tricky way that can be done with groovy postscripts and accessing the jenkins inner objects, but that's very likely to break on any plugin upgrade and requires a bit of investigating with objects to change.

> - The report is using absolute paths, but we like shorter relative paths. > > I don't see a way to configure nosetests or coverage to generate > relative paths. > May need hacking of the generated html/json in htmlcov. If you find it please tell me, I have the same issue with other projects, I'll do if I discover a way :)

Hi all, Thanks to Edward, we have now coverage reports in jenkins. The way to access the report on jenkins is to use this url: http://jenkins.ovirt.org/job/vdsm_master_check-patch-fc23-x86_64/ <build-number>/artifact/exported-artifacts/htmlcov/index.html Here is an example, hopefully it will be accessible when you try: http://jenkins.ovirt.org/job/vdsm_master_check-patch-fc23-x86_64/648/arti... Todo: - Easy way to access the report from gerrit It should be easy to add a link the coverage report in the comment added by jenkins after a build finish. - Store the coverage reports for longer time, maybe a week? - We have only 45% coverage instead of the minimum, 100%. Note that coverage of 25% can mean *no* code was run by the tests. The only code running was the functions and class definitions. Here is an example: http://jenkins.ovirt.org/job/vdsm_master_check-patch-fc23-x86_64/648/arti... Modules that were never imported during the tests have 0% coverage: http://jenkins.ovirt.org/job/vdsm_master_check-patch-fc23-x86_64/648/arti... - coverage creates lot of useless noise in the jenkins logs, need to slicense this output. http://jenkins.ovirt.org/job/vdsm_master_check-patch-fc23-x86_64/648/cons... I did not find a way to do this in nosetests, may need hacking nose coverage plugin. - The report includes only the tests in the tests directory. We have additional tests in lib/vdsm/infra/* which are not included. We should move these to the tests directory. - The report is using absolute paths, but we like shorter relative paths. I don't see a way to configure nosetests or coverage to generate relative paths. May need hacking of the generated html/json in htmlcov. - Add "make coverage" target for running coverage locally - An easy way to enable coverage for the functional tests or for running a single test module. Can be done using nosetests --cover* options. Should be documented in tests/README, and maybe automated using a script or Makefile. When running locally, one would like to have the script open the report in the browser automatically: xdg-open tests/htmlcov/index.html - An easy way to enable coverage when testing flows in vdsm Petr sent a patch for enabling coverage using vdsm.conf: https://gerrit.ovirt.org/49168 We discussed adding vdsm-coverage package that will make it easy to setup Nir _______________________________________________ Devel mailing list Devel(a)ovirt.org http://lists.ovirt.org/mailman/listinfo/devel

On 2 December 2015 at 12:21, Allon Mureinik <amureini(a)redhat.com&gt; wrote: > Awesome! > > Infra stakeholders - any chance to have this pushed to the devel list once a > month (or some other reasonable period)? I see a few possible ways to get this: 1. Run the same code in 'check_merged.sh' followed by some code to mail the results. With the whole thing surrounded by some conditional to decide if the time to send a new report had come. (One possibility is every X merged patches since the last release) 2. Devise some extension the the CI standard to let you specify periodic jobs. 3. Send a YAML patch to create a new stand-alone job. (That, experience shows, would result in that job being unmaintained) -- Barak Korren bkorren(a)redhat.com RHEV-CI Team