Test succeeded, hudson failed, test marked as FAILURE in gerrit

Hi infra, Looking in the console log http://jenkins.ovirt.org/job/vdsm_master_unit-tests_created/10774/console We can see that the test succeeded: Ran 551 tests in 264.511s OK (SKIP=41) But hudson failed: FATAL: hudson.remoting.RequestAbortedException: hudson.remoting.Channel$OrderlyShutdown: java.util.concurrent.TimeoutException: Ping started on 1404230202100 hasn't completed at 1404230442456 And the test was marked as FAILURE: http://jenkins.ovirt.org/job/vdsm_master_unit-tests_created/10774/ : FAILURE This is a recurring problem in the CI. We need to eliminate these errors which waste developers time, and worse, cause them to ignore failures. Please give this issue higher priority. Thanks, Nir

This is an OpenPGP/MIME signed message (RFC 4880 and 3156) --llMt0RMwT0hi8RoN4CVOv74tgAd35DQis Content-Type: text/plain; charset=UTF-8 Content-Transfer-Encoding: quoted-printable On Tue 01 Jul 2014 06:33:08 PM CEST, Nir Soffer wrote:
Hi infra,
Looking in the console log http://jenkins.ovirt.org/job/vdsm_master_unit-tests_created/10774/conso= le
We can see that the test succeeded:
Ran 551 tests in 264.511s
OK (SKIP=3D41)
But hudson failed:
FATAL: hudson.remoting.RequestAbortedException: hudson.remoting.Cha= nnel$OrderlyShutdown: java.util.concurrent.TimeoutException: Ping started= on 1404230202100 hasn't completed at 1404230442456
And the test was marked as FAILURE:
http://jenkins.ovirt.org/job/vdsm_master_unit-tests_created/10774/ = : FAILURE
This is a recurring problem in the CI. We need to eliminate these error= s which waste developers time, and worse, cause them to ignore failures.
Please give this issue higher priority.
Thanks, Nir _______________________________________________ Infra mailing list Infra@ovirt.org http://lists.ovirt.org/mailman/listinfo/infra
Not long ago we started a thread to try and determine if a job failed=20 due to test error (meaning, one of the tests failed) or infra error=20 (anything else, for example, network error, timeout...). iirc the last proposal was to generate a test file with a strict format=20 that the job will check, and if not found or malformed it will be=20 assumed infra issue, and if well formed, the file itself would=20 determine if it passed of there were any test errors. But no specific format was proposed, or file for that matter. I would=20 be glad with just two messages (very strict format) in the log, tests=20 ran and passed, and tests ran and did not pass. We are already doing=20 that on a few jobs with good results for other projects. So if you want this to get pushed, please, feel free to provide a=20 deterministic way for ci to detect if the tests did run properly and=20 the result of the run. This also wastes infra time so please provide a=20 solution. Also, can you open a ticket on https://fedorahosted.org/ovirt/newticket=20 for any specific request? If it's not there, it has no priority. We are=20 trying to open there all the issues so we can manage them better and=20 leave the list for discussions and announcements. Cheers -- David Caro Red Hat S.L. Continuous Integration Engineer - EMEA ENG Virtualization R&D Email: dcaro@redhat.com Web: www.redhat.com RHT Global #: 82-62605 --llMt0RMwT0hi8RoN4CVOv74tgAd35DQis Content-Type: application/pgp-signature; name="signature.asc" Content-Description: OpenPGP digital signature Content-Disposition: attachment; filename="signature.asc" -----BEGIN PGP SIGNATURE----- Version: GnuPG v1 iQEcBAEBAgAGBQJTsvfgAAoJEEBxx+HSYmnDHbgH/1gm+Ifj8C7fMVCvuG7BYEgn THfHnMCxr5f0zM5VzYkwBgFP4/dmWxQpY2326Qtfktqajn8tZDrJifOu0APn2la7 nvZgEhm2QZGuuYS9FzSKzz3PvcVnGZesM1ldD2RlgzYq6Fp6x6Q3mGoZ1gG/REO8 wsnUNFNdlnFiaB7OU/63DWhuG2a11Ke+i4wRlThHjcDe+IxvPVot4FOySbgHHegz 99E6Q66a3bQChgT2IfXRlX7PTlaQllNA0UsmlLOmUJc4kI54tnMYiQX686uEqRM3 46rPV9QDIZvUKqqbataUOe74wbBur94M9yn7fBIvSaboz+swRx1A6a8vmwqu3MU= =v2yT -----END PGP SIGNATURE----- --llMt0RMwT0hi8RoN4CVOv74tgAd35DQis--

----- Original Message -----
From: "David Caro" <dcaroest@redhat.com> To: "Nir Soffer" <nsoffer@redhat.com> Cc: "infra" <infra@ovirt.org> Sent: Tuesday, July 1, 2014 9:03:12 PM Subject: Re: Test succeeded, hudson failed, test marked as FAILURE in gerrit
On Tue 01 Jul 2014 06:33:08 PM CEST, Nir Soffer wrote:
Hi infra,
Looking in the console log http://jenkins.ovirt.org/job/vdsm_master_unit-tests_created/10774/console
We can see that the test succeeded:
Ran 551 tests in 264.511s
OK (SKIP=41)
But hudson failed:
FATAL: hudson.remoting.RequestAbortedException: hudson.remoting.Channel$OrderlyShutdown: java.util.concurrent.TimeoutException: Ping started on 1404230202100 hasn't completed at 1404230442456
And the test was marked as FAILURE:
http://jenkins.ovirt.org/job/vdsm_master_unit-tests_created/10774/ : FAILURE
This is a recurring problem in the CI. We need to eliminate these errors which waste developers time, and worse, cause them to ignore failures.
Please give this issue higher priority.
Thanks, Nir _______________________________________________ Infra mailing list Infra@ovirt.org http://lists.ovirt.org/mailman/listinfo/infra
Not long ago we started a thread to try and determine if a job failed due to test error (meaning, one of the tests failed) or infra error (anything else, for example, network error, timeout...). iirc the last proposal was to generate a test file with a strict format that the job will check, and if not found or malformed it will be assumed infra issue, and if well formed, the file itself would determine if it passed of there were any test errors.
But no specific format was proposed, or file for that matter. I would be glad with just two messages (very strict format) in the log, tests ran and passed, and tests ran and did not pass. We are already doing that on a few jobs with good results for other projects. So if you want this to get pushed, please, feel free to provide a deterministic way for ci to detect if the tests did run properly and the result of the run. This also wastes infra time so please provide a solution.
Also, can you open a ticket on https://fedorahosted.org/ovirt/newticket for any specific request? If it's not there, it has no priority. We are trying to open there all the issues so we can manage them better and leave the list for discussions and announcements.
Right - we need continue with this. I suggest we make a call tomorrow to discuss the solution. Nir

----- Original Message -----
From: "Nir Soffer" <nsoffer@redhat.com> To: "David Caro" <dcaroest@redhat.com> Cc: "infra" <infra@ovirt.org> Sent: Tuesday, July 1, 2014 9:36:59 PM Subject: Re: Test succeeded, hudson failed, test marked as FAILURE in gerrit
----- Original Message -----
From: "David Caro" <dcaroest@redhat.com> To: "Nir Soffer" <nsoffer@redhat.com> Cc: "infra" <infra@ovirt.org> Sent: Tuesday, July 1, 2014 9:03:12 PM Subject: Re: Test succeeded, hudson failed, test marked as FAILURE in gerrit
On Tue 01 Jul 2014 06:33:08 PM CEST, Nir Soffer wrote:
Hi infra,
Looking in the console log http://jenkins.ovirt.org/job/vdsm_master_unit-tests_created/10774/console
We can see that the test succeeded:
Ran 551 tests in 264.511s
OK (SKIP=41)
But hudson failed:
FATAL: hudson.remoting.RequestAbortedException: hudson.remoting.Channel$OrderlyShutdown: java.util.concurrent.TimeoutException: Ping started on 1404230202100 hasn't completed at 1404230442456
And the test was marked as FAILURE:
http://jenkins.ovirt.org/job/vdsm_master_unit-tests_created/10774/ : FAILURE
This is a recurring problem in the CI. We need to eliminate these errors which waste developers time, and worse, cause them to ignore failures.
Please give this issue higher priority.
Thanks, Nir _______________________________________________ Infra mailing list Infra@ovirt.org http://lists.ovirt.org/mailman/listinfo/infra
Not long ago we started a thread to try and determine if a job failed due to test error (meaning, one of the tests failed) or infra error (anything else, for example, network error, timeout...). iirc the last proposal was to generate a test file with a strict format that the job will check, and if not found or malformed it will be assumed infra issue, and if well formed, the file itself would determine if it passed of there were any test errors.
But no specific format was proposed, or file for that matter. I would be glad with just two messages (very strict format) in the log, tests ran and passed, and tests ran and did not pass. We are already doing that on a few jobs with good results for other projects. So if you want this to get pushed, please, feel free to provide a deterministic way for ci to detect if the tests did run properly and the result of the run. This also wastes infra time so please provide a solution.
Also, can you open a ticket on https://fedorahosted.org/ovirt/newticket for any specific request? If it's not there, it has no priority. We are trying to open there all the issues so we can manage them better and leave the list for discussions and announcements.
Right - we need continue with this.
I suggest we make a call tomorrow to discuss the solution.
nir, please open a ticket on this so this won't get lost again in the emails.. https://fedorahosted.org/ovirt/newticket
Nir _______________________________________________ Infra mailing list Infra@ovirt.org http://lists.ovirt.org/mailman/listinfo/infra
participants (3)
-
David Caro
-
Eyal Edri
-
Nir Soffer