Jenkins is acting again - scoring patch when test was not run

[Eyal Edri]

----- Original Message -----
> From: "Nir Soffer" <nsoffer at redhat.com>
> To: "infra" <infra at ovirt.org>
> Sent: Friday, February 28, 2014 9:59:39 PM
> Subject: Jenkins is acting again - scoring patch when test was not run
> 
> Hi infra,
> 
> Jenkins gave my patch -1 because it Jenkins was broken:
> http://jenkins.ovirt.org/job/vdsm_master_pep8_gerrit/7364/console
> 
> This is a major error; Jenkins should -1 a patch only if the it could run the
> relevant test, and the test failed.

I've changed the default behaviour of gerrit trigger to only give -1 on 'unstable' 
builds, and not on 'failed' builds.

builds in general should not fail (red) on real errors (if they do, we should fix
relevant job), they should turn unstable (yellow).

and job won't turn unstable on an unexpected infra error (like happened now).


> 
> In this case the relevant test is running the pep8 tool, and scoring the
> patch
> is correct if the tool returned with non-zero exit code.
> 
> Any other failure is CI error and should not effect the patch score.
> 
> If you can, it would be nice if jenkins would add a comment on gerrit,
> telling
> about errors. In this case, the status should be ERROR instead of FAILURE.

i think it will happen now, please let us know if this happens again,
we have the option to modify the default msg per failure.

> 
> Examples:
> 
> When jenkins fails to run a test:
> - http://jenkins.ovirt.org/job/vdsm_master_pep8_gerrit/xxxx/ : ERROR
> 
> When tests was run and failed:
> - http://jenkins.ovirt.org/job/vdsm_unit_tests_gerrit_el/xxxx/ : FAILURE
> 
> When test was run and succeeded:
> - http://jenkins.ovirt.org/job/vdsm_unit_tests_gerrit/xxxx/ : SUCCESS
> 
> Thanks,
> Nir
> _______________________________________________
> Infra mailing list
> Infra at ovirt.org
> http://lists.ovirt.org/mailman/listinfo/infra
>