Jenkins is acting again - scoring patch when test was not run

[Eyal Edri]

----- Original Message -----
> From: "Nir Soffer" <nsoffer at redhat.com>
> To: "Eyal Edri" <eedri at redhat.com>
> Cc: "infra" <infra at ovirt.org>
> Sent: Sunday, March 2, 2014 10:37:42 PM
> Subject: Re: Jenkins is acting again - scoring patch when test was not run
> 
> ----- Original Message -----
> > From: "Eyal Edri" <eedri at redhat.com>
> > To: "Nir Soffer" <nsoffer at redhat.com>
> > Cc: "infra" <infra at ovirt.org>
> > Sent: Sunday, March 2, 2014 10:26:51 PM
> > Subject: Re: Jenkins is acting again - scoring patch when test was not run
> > 
> > 
> > 
> > ----- Original Message -----
> > > From: "Nir Soffer" <nsoffer at redhat.com>
> > > To: "infra" <infra at ovirt.org>
> > > Sent: Friday, February 28, 2014 9:59:39 PM
> > > Subject: Jenkins is acting again - scoring patch when test was not run
> > > 
> > > Hi infra,
> > > 
> > > Jenkins gave my patch -1 because it Jenkins was broken:
> > > http://jenkins.ovirt.org/job/vdsm_master_pep8_gerrit/7364/console
> > > 
> > > This is a major error; Jenkins should -1 a patch only if the it could run
> > > the
> > > relevant test, and the test failed.
> > 
> > I've changed the default behaviour of gerrit trigger to only give -1 on
> > 'unstable'
> > builds, and not on 'failed' builds.
> 
> What is unstable build?

this is a build status that indicates the job logic has failed (e.g unit test failed/pep8 check/etc...),
a build won't be on unstable status unless it is set to do so on specific logic,
leaving all unexpected infra issues to finish as 'failure' (red).

> 
> > 
> > builds in general should not fail (red) on real errors (if they do, we
> > should
> > fix
> > relevant job), they should turn unstable (yellow).
> > 
> > and job won't turn unstable on an unexpected infra error (like happened
> > now).
> > 
> > 
> > > 
> > > In this case the relevant test is running the pep8 tool, and scoring the
> > > patch
> > > is correct if the tool returned with non-zero exit code.
> > > 
> > > Any other failure is CI error and should not effect the patch score.
> > > 
> > > If you can, it would be nice if jenkins would add a comment on gerrit,
> > > telling
> > > about errors. In this case, the status should be ERROR instead of
> > > FAILURE.
> > 
> > i think it will happen now, please let us know if this happens again,
> > we have the option to modify the default msg per failure.
> 
> You can test this by adding a fake error in some build script, and check
> that it did not -1 the related patch.

works - http://gerrit.ovirt.org/#/c/25254/
failed on infra issue, but patch didn't get -1.

> 
> Thanks
> Nir
>