Test succeeded, hudson failed, test marked as FAILURE in gerrit

[Eyal Edri]

----- Original Message -----
> From: "Nir Soffer" <nsoffer at redhat.com>
> To: "David Caro" <dcaroest at redhat.com>
> Cc: "infra" <infra at ovirt.org>
> Sent: Tuesday, July 1, 2014 9:36:59 PM
> Subject: Re: Test succeeded, hudson failed, test marked as FAILURE in gerrit
> 
> 
> 
> ----- Original Message -----
> > From: "David Caro" <dcaroest at redhat.com>
> > To: "Nir Soffer" <nsoffer at redhat.com>
> > Cc: "infra" <infra at ovirt.org>
> > Sent: Tuesday, July 1, 2014 9:03:12 PM
> > Subject: Re: Test succeeded, hudson failed, test marked as FAILURE in
> > gerrit
> > 
> > On Tue 01 Jul 2014 06:33:08 PM CEST, Nir Soffer wrote:
> > > Hi infra,
> > >
> > > Looking in the console log
> > > http://jenkins.ovirt.org/job/vdsm_master_unit-tests_created/10774/console
> > >
> > > We can see that the test succeeded:
> > >
> > >     Ran 551 tests in 264.511s
> > >
> > >     OK (SKIP=41)
> > >
> > > But hudson failed:
> > >
> > >     FATAL: hudson.remoting.RequestAbortedException:
> > >     hudson.remoting.Channel$OrderlyShutdown:
> > >     java.util.concurrent.TimeoutException: Ping started on 1404230202100
> > >     hasn't completed at 1404230442456
> > >
> > > And the test was marked as FAILURE:
> > >
> > >     http://jenkins.ovirt.org/job/vdsm_master_unit-tests_created/10774/ :
> > >     FAILURE
> > >
> > > This is a recurring problem in the CI. We need to eliminate these errors
> > > which
> > > waste developers time, and worse, cause them to ignore failures.
> > >
> > > Please give this issue higher priority.
> > >
> > > Thanks,
> > > Nir
> > > _______________________________________________
> > > Infra mailing list
> > > Infra at ovirt.org
> > > http://lists.ovirt.org/mailman/listinfo/infra
> > 
> > Not long ago we started a thread to try and determine if a job failed
> > due to test error (meaning, one of the tests failed) or infra error
> > (anything else, for example, network error, timeout...).
> > iirc the last proposal was to generate a test file with a strict format
> > that the job will check, and if not found or malformed it will be
> > assumed infra issue, and if well formed, the file itself would
> > determine if it passed of there were any test errors.
> > 
> > But no specific format was proposed, or file for that matter. I would
> > be glad with just two messages (very strict format) in the log, tests
> > ran and passed, and tests ran and did not pass. We are already doing
> > that on a few jobs with good results for other projects.
> > So if you want this to get pushed, please, feel free to provide a
> > deterministic way for ci to detect if the tests did run properly and
> > the result of the run. This also wastes infra time so please provide a
> > solution.
> > 
> > Also, can you open a ticket on https://fedorahosted.org/ovirt/newticket
> > for any specific request? If it's not there, it has no priority. We are
> > trying to open there all the issues so we can manage them better and
> > leave the list for discussions and announcements.
> 
> Right - we need continue with this.
> 
> I suggest we make a call tomorrow to discuss the solution.

nir, please open a ticket on this so this won't get lost again in the emails..

https://fedorahosted.org/ovirt/newticket

> 
> Nir
> _______________________________________________
> Infra mailing list
> Infra at ovirt.org
> http://lists.ovirt.org/mailman/listinfo/infra
>