Re: Test succeeded, hudson failed, test marked as FAILURE in gerrit

From: "Nir Soffer" <nsoffer(a)redhat.com&gt; To: "David Caro" <dcaroest(a)redhat.com&gt; Cc: "infra" <infra(a)ovirt.org&gt; Sent: Tuesday, July 1, 2014 9:36:59 PM Subject: Re: Test succeeded, hudson failed, test marked as FAILURE in gerrit ----- Original Message ----- > From: "David Caro" <dcaroest(a)redhat.com&gt; > To: "Nir Soffer" <nsoffer(a)redhat.com&gt; > Cc: "infra" <infra(a)ovirt.org&gt; > Sent: Tuesday, July 1, 2014 9:03:12 PM > Subject: Re: Test succeeded, hudson failed, test marked as FAILURE in > gerrit > > On Tue 01 Jul 2014 06:33:08 PM CEST, Nir Soffer wrote: > > Hi infra, > > > > Looking in the console log > > http://jenkins.ovirt.org/job/vdsm_master_unit-tests_created/10774/console > > > > We can see that the test succeeded: > > > > Ran 551 tests in 264.511s > > > > OK (SKIP=41) > > > > But hudson failed: > > > > FATAL: hudson.remoting.RequestAbortedException: > > hudson.remoting.Channel$OrderlyShutdown: > > java.util.concurrent.TimeoutException: Ping started on 1404230202100 > > hasn't completed at 1404230442456 > > > > And the test was marked as FAILURE: > > > > http://jenkins.ovirt.org/job/vdsm_master_unit-tests_created/10774/ : > > FAILURE > > > > This is a recurring problem in the CI. We need to eliminate these errors > > which > > waste developers time, and worse, cause them to ignore failures. > > > > Please give this issue higher priority. > > > > Thanks, > > Nir > > _______________________________________________ > > Infra mailing list > > Infra(a)ovirt.org > > http://lists.ovirt.org/mailman/listinfo/infra > > Not long ago we started a thread to try and determine if a job failed > due to test error (meaning, one of the tests failed) or infra error > (anything else, for example, network error, timeout...). > iirc the last proposal was to generate a test file with a strict format > that the job will check, and if not found or malformed it will be > assumed infra issue, and if well formed, the file itself would > determine if it passed of there were any test errors. > > But no specific format was proposed, or file for that matter. I would > be glad with just two messages (very strict format) in the log, tests > ran and passed, and tests ran and did not pass. We are already doing > that on a few jobs with good results for other projects. > So if you want this to get pushed, please, feel free to provide a > deterministic way for ci to detect if the tests did run properly and > the result of the run. This also wastes infra time so please provide a > solution. > > Also, can you open a ticket on https://fedorahosted.org/ovirt/newticket > for any specific request? If it's not there, it has no priority. We are > trying to open there all the issues so we can manage them better and > leave the list for discussions and announcements. Right - we need continue with this. I suggest we make a call tomorrow to discuss the solution.

Nir _______________________________________________ Infra mailing list Infra(a)ovirt.org http://lists.ovirt.org/mailman/listinfo/infra