--WK3l2KTTmXPVedZ6 Content-Type: text/plain; charset=us-ascii Content-Disposition: inline Content-Transfer-Encoding: quoted-printable On 11/27 20:57, Nir Soffer wrote:

Hi all, =20 Thanks to Edward, we have now coverage reports in jenkins. =20 The way to access the report on jenkins is to use this url: http://jenkins.ovirt.org/job/vdsm_master_check-patch-fc23-x86_64/<build-n= umber>/artifact/exported-artifacts/htmlcov/index.html =20 Here is an example, hopefully it will be accessible when you try: http://jenkins.ovirt.org/job/vdsm_master_check-patch-fc23-x86_64/648/arti= fact/exported-artifacts/htmlcov/index.html =20 Todo: =20 - Easy way to access the report from gerrit It should be easy to add a link the coverage report in the comment added by jenkins after a build finish.

Not really, the gerrit comments that jenkins sends can't be changed by the build, that means that you can't put put there a non-static url. There's a quite tricky way that can be done with groovy postscripts and accessing the jenkins inner objects, but that's very likely to break on any plugin upgrade and requires a bit of investigating with objects to change.

=20 - Store the coverage reports for longer time, maybe a week?

If they are generated with the check patch, you always have the latest there (stored as long as your builds). If you want longer archiving, might be a g= ood idea to generate them too in the check-merged.

=20 - We have only 45% coverage instead of the minimum, 100%. =20 Note that coverage of 25% can mean *no* code was run by the tests. The only code running was the functions and class definitions. Here is an example: http://jenkins.ovirt.org/job/vdsm_master_check-patch-fc23-x86_64/648/ar= tifact/exported-artifacts/htmlcov/_home_jenkins_workspace_vdsm_master_check= -patch-fc23-x86_64_vdsm_vdsm_storage_devicemapper_py.html =20 Modules that were never imported during the tests have 0% coverage: http://jenkins.ovirt.org/job/vdsm_master_check-patch-fc23-x86_64/648/ar= tifact/exported-artifacts/htmlcov/_home_jenkins_workspace_vdsm_master_check= -patch-fc23-x86_64_vdsm_vdsm_storage_hsm_py.html =20 - coverage creates lot of useless noise in the jenkins logs, need to slic= ense this output. http://jenkins.ovirt.org/job/vdsm_master_check-patch-fc23-x86_64/648/co= nsoleFull I did not find a way to do this in nosetests, may need hacking nose coverage plugin. =20 - The report includes only the tests in the tests directory. =20 We have additional tests in lib/vdsm/infra/* which are not included. We should move these to the tests directory. =20 - The report is using absolute paths, but we like shorter relative paths. =20 I don't see a way to configure nosetests or coverage to generate relative paths. May need hacking of the generated html/json in htmlcov.

If you find it please tell me, I have the same issue with other projects, I= 'll do if I discover a way :)

=20 - Add "make coverage" target for running coverage locally =20 - An easy way to enable coverage for the functional tests or for running a single test module. =20 Can be done using nosetests --cover* options. Should be documented in tests/README, and maybe automated using a script or Makefile. When running locally, one would like to have the script open the report in the browser automatically: xdg-open tests/htmlcov/index.html =20 - An easy way to enable coverage when testing flows in vdsm =20 Petr sent a patch for enabling coverage using vdsm.conf: https://gerrit.ovirt.org/49168 We discussed adding vdsm-coverage package that will make it easy to set= up =20 Nir

--=20 David Caro Red Hat S.L. Continuous Integration Engineer - EMEA ENG Virtualization R&D Tel.: +420 532 294 605 Email: dcaro@redhat.com IRC: dcaro|dcaroest@{freenode|oftc|redhat} Web: www.redhat.com RHT Global #: 82-62605 --WK3l2KTTmXPVedZ6 Content-Type: application/pgp-signature; name="signature.asc" -----BEGIN PGP SIGNATURE----- Version: GnuPG v1 iQEcBAEBAgAGBQJWWK0AAAoJEEBxx+HSYmnDRZQIAJSjT1HUI0OVV29vhBQ2UqXP jxfiAe+3XjGCBNa6mCmSbv4KSV6mivJYsjk0dq+hh88UzBhJ1IRBhW5nYPx8PaYH y6DdDt8E9IeXWDWjVDdkOMUeDoajcuTaOKoU0HKbYkYx59y6hugtAmYi2r1kU+jT 2ZPweulBTtvq2DzT3IXVkKwqzvR7TdHy9/IGAsENmCrLyG4ZSNlBQwTinwj4qNjC Nrs0/uUewcL/AbL6t7buSSyE8PvB7oaU6KbwZ4dz1mFEWFVZbIa3QFpTrX6k305R l1CWggj0rAisl3ka1ubM3QuvgRESrgjhLWtq9Eqrp8fwYNAFawbiQJmvzP3bZy0= =DV50 -----END PGP SIGNATURE----- --WK3l2KTTmXPVedZ6--