Jenkins is acting again - scoring patch when test was not run

[Dan Kenigsberg]

On Fri, Feb 28, 2014 at 02:59:39PM -0500, Nir Soffer wrote:
> Hi infra,
> 
> Jenkins gave my patch -1 because it Jenkins was broken:
> http://jenkins.ovirt.org/job/vdsm_master_pep8_gerrit/7364/console
> 
> This is a major error; Jenkins should -1 a patch only if the it could run the
> relevant test, and the test failed.
> 
> In this case the relevant test is running the pep8 tool, and scoring the patch
> is correct if the tool returned with non-zero exit code.
> 
> Any other failure is CI error and should not effect the patch score.
> 
> If you can, it would be nice if jenkins would add a comment on gerrit, telling 
> about errors. In this case, the status should be ERROR instead of FAILURE.
> 
> Examples:
> 
> When jenkins fails to run a test:
> - http://jenkins.ovirt.org/job/vdsm_master_pep8_gerrit/xxxx/ : ERROR
> 
> When tests was run and failed:
> - http://jenkins.ovirt.org/job/vdsm_unit_tests_gerrit_el/xxxx/ : FAILURE
> 
> When test was run and succeeded:
> - http://jenkins.ovirt.org/job/vdsm_unit_tests_gerrit/xxxx/ : SUCCESS

Hear hear!