Test succeeded, hudson failed, test marked as FAILURE in gerrit

[David Caro]

On Tue 01 Jul 2014 06:33:08 PM CEST, Nir Soffer wrote:
> Hi infra,
>
> Looking in the console log
> http://jenkins.ovirt.org/job/vdsm_master_unit-tests_created/10774/console
>
> We can see that the test succeeded:
>
>     Ran 551 tests in 264.511s
>
>     OK (SKIP=41)
>
> But hudson failed:
>
>     FATAL: hudson.remoting.RequestAbortedException: hudson.remoting.Channel$OrderlyShutdown: java.util.concurrent.TimeoutException: Ping started on 1404230202100 hasn't completed at 1404230442456
>
> And the test was marked as FAILURE:
>
>     http://jenkins.ovirt.org/job/vdsm_master_unit-tests_created/10774/ : FAILURE
>
> This is a recurring problem in the CI. We need to eliminate these errors which
> waste developers time, and worse, cause them to ignore failures.
>
> Please give this issue higher priority.
>
> Thanks,
> Nir
> _______________________________________________
> Infra mailing list
> Infra at ovirt.org
> http://lists.ovirt.org/mailman/listinfo/infra

Not long ago we started a thread to try and determine if a job failed 
due to test error (meaning, one of the tests failed) or infra error 
(anything else, for example, network error, timeout...).
iirc the last proposal was to generate a test file with a strict format 
that the job will check, and if not found or malformed it will be 
assumed infra issue, and if well formed, the file itself would 
determine if it passed of there were any test errors.

But no specific format was proposed, or file for that matter. I would 
be glad with just two messages (very strict format) in the log, tests 
ran and passed, and tests ran and did not pass. We are already doing 
that on a few jobs with good results for other projects.
So if you want this to get pushed, please, feel free to provide a 
deterministic way for ci to detect if the tests did run properly and 
the result of the run. This also wastes infra time so please provide a 
solution.

Also, can you open a ticket on https://fedorahosted.org/ovirt/newticket 
for any specific request? If it's not there, it has no priority. We are 
trying to open there all the issues so we can manage them better and 
leave the list for discussions and announcements.

Cheers

--
David Caro

Red Hat S.L.
Continuous Integration Engineer - EMEA ENG Virtualization R&D

Email: dcaro at redhat.com
Web: www.redhat.com
RHT Global #: 82-62605

-------------- next part --------------
A non-text attachment was scrubbed...
Name: signature.asc
Type: application/pgp-signature
Size: 473 bytes
Desc: OpenPGP digital signature
URL: <http://lists.ovirt.org/pipermail/infra/attachments/20140701/4d6b18ac/attachment.sig>