<div dir="ltr"><div class="gmail_default" style="font-family:monospace,monospace">Awesome!</div><div class="gmail_default" style="font-family:monospace,monospace"><br></div><div class="gmail_default" style="font-family:monospace,monospace">Infra stakeholders - any chance to have this pushed to the devel list once a month (or some other reasonable period)?</div><div class="gmail_extra"><br><div class="gmail_quote">On Fri, Nov 27, 2015 at 8:57 PM, Nir Soffer <span dir="ltr"><<a href="mailto:nsoffer@redhat.com" target="_blank">nsoffer@redhat.com</a>></span> wrote:<br><blockquote class="gmail_quote" style="margin:0 0 0 .8ex;border-left:1px #ccc solid;padding-left:1ex">Hi all,<br>
<br>
Thanks to Edward, we have now coverage reports in jenkins.<br>
<br>
The way to access the report on jenkins is to use this url:<br>
<a href="http://jenkins.ovirt.org/job/vdsm_master_check-patch-fc23-x86_64/" rel="noreferrer" target="_blank">http://jenkins.ovirt.org/job/vdsm_master_check-patch-fc23-x86_64/</a><build-number>/artifact/exported-artifacts/htmlcov/index.html<br>
<br>
Here is an example, hopefully it will be accessible when you try:<br>
<a href="http://jenkins.ovirt.org/job/vdsm_master_check-patch-fc23-x86_64/648/artifact/exported-artifacts/htmlcov/index.html" rel="noreferrer" target="_blank">http://jenkins.ovirt.org/job/vdsm_master_check-patch-fc23-x86_64/648/artifact/exported-artifacts/htmlcov/index.html</a><br>
<br>
Todo:<br>
<br>
- Easy way to access the report from gerrit<br>
It should be easy to add a link the coverage report in the comment added<br>
by jenkins after a build finish.<br>
<br>
- Store the coverage reports for longer time, maybe a week?<br>
<br>
- We have only 45% coverage instead of the minimum, 100%.<br>
<br>
Note that coverage of 25% can mean *no* code was run by the tests.<br>
The only code<br>
running was the functions and class definitions. Here is an example:<br>
<a href="http://jenkins.ovirt.org/job/vdsm_master_check-patch-fc23-x86_64/648/artifact/exported-artifacts/htmlcov/_home_jenkins_workspace_vdsm_master_check-patch-fc23-x86_64_vdsm_vdsm_storage_devicemapper_py.html" rel="noreferrer" target="_blank">http://jenkins.ovirt.org/job/vdsm_master_check-patch-fc23-x86_64/648/artifact/exported-artifacts/htmlcov/_home_jenkins_workspace_vdsm_master_check-patch-fc23-x86_64_vdsm_vdsm_storage_devicemapper_py.html</a><br>
<br>
Modules that were never imported during the tests have 0% coverage:<br>
<a href="http://jenkins.ovirt.org/job/vdsm_master_check-patch-fc23-x86_64/648/artifact/exported-artifacts/htmlcov/_home_jenkins_workspace_vdsm_master_check-patch-fc23-x86_64_vdsm_vdsm_storage_hsm_py.html" rel="noreferrer" target="_blank">http://jenkins.ovirt.org/job/vdsm_master_check-patch-fc23-x86_64/648/artifact/exported-artifacts/htmlcov/_home_jenkins_workspace_vdsm_master_check-patch-fc23-x86_64_vdsm_vdsm_storage_hsm_py.html</a><br>
<br>
- coverage creates lot of useless noise in the jenkins logs, need to slicense<br>
this output.<br>
<a href="http://jenkins.ovirt.org/job/vdsm_master_check-patch-fc23-x86_64/648/consoleFull" rel="noreferrer" target="_blank">http://jenkins.ovirt.org/job/vdsm_master_check-patch-fc23-x86_64/648/consoleFull</a><br>
I did not find a way to do this in nosetests, may need hacking<br>
nose coverage plugin.<br>
<br>
- The report includes only the tests in the tests directory.<br>
<br>
We have additional tests in lib/vdsm/infra/* which are not<br>
included. We should<br>
move these to the tests directory.<br>
<br>
- The report is using absolute paths, but we like shorter relative paths.<br>
<br>
I don't see a way to configure nosetests or coverage to generate<br>
relative paths.<br>
May need hacking of the generated html/json in htmlcov.<br>
<br>
- Add "make coverage" target for running coverage locally<br>
<br>
- An easy way to enable coverage for the functional tests or for running<br>
a single test module.<br>
<br>
Can be done using nosetests --cover* options. Should be documented in<br>
tests/README, and maybe automated using a script or Makefile.<br>
When running locally, one would like to have the script open the report<br>
in the browser automatically:<br>
xdg-open tests/htmlcov/index.html<br>
<br>
- An easy way to enable coverage when testing flows in vdsm<br>
<br>
Petr sent a patch for enabling coverage using vdsm.conf:<br>
<a href="https://gerrit.ovirt.org/49168" rel="noreferrer" target="_blank">https://gerrit.ovirt.org/49168</a><br>
We discussed adding vdsm-coverage package that will make it easy to setup<br>
<br>
Nir<br>
_______________________________________________<br>
Devel mailing list<br>
<a href="mailto:Devel@ovirt.org">Devel@ovirt.org</a><br>
<a href="http://lists.ovirt.org/mailman/listinfo/devel" rel="noreferrer" target="_blank">http://lists.ovirt.org/mailman/listinfo/devel</a><br>
<br>
<br>
</blockquote></div><br></div></div>