This is an OpenPGP/MIME signed message (RFC 4880 and 3156) --p24AU4JWLLvxM022u0ghKBuqCBEHoClgG Content-Type: text/plain; charset=UTF-8 Content-Transfer-Encoding: quoted-printable On Sun 08 Jun 2014 12:57:24 PM CEST, Nir Soffer wrote:
----- Original Message -----
From: "David Caro" <dcaroest@redhat.com> To: "Nir Soffer" <nsoffer@redhat.com> Cc: "Piotr Kliczewski" <piotr.kliczewski@gmail.com>, fsimonce@redhat.c= om, dcaro@redhat.com, devel@ovirt.org, "Dan Kenigsberg" <danken@redhat.com> Sent: Friday, June 6, 2014 5:16:52 PM Subject: Re: [ovirt-devel] local vdsm build fails
On Fri 06 Jun 2014 03:53:41 PM CEST, Nir Soffer wrote:
----- Original Message -----
From: "Dan Kenigsberg" <danken@redhat.com> To: "Piotr Kliczewski" <piotr.kliczewski@gmail.com>, fsimonce@redhat= =2Ecom, nsoffer@redhat.com, dcaro@redhat.com Cc: devel@ovirt.org Sent: Friday, June 6, 2014 12:15:18 PM Subject: Re: [ovirt-devel] local vdsm build fails
On Fri, Jun 06, 2014 at 09:19:11AM +0200, Piotr Kliczewski wrote:
All,
I pulled the latest vdsm from master and noticed that build is fail= ing.
Here is the patch that causes the failuer:
http://gerrit.ovirt.org/#/c/28226
and looking at jenkins comments I can see that jenkins was failing with the same reason:
http://jenkins.ovirt.org/job/vdsm_master_storage_functional_tests_l= ocalfs_gerrit/1064/console
Nir has already fix that as well. The storage tests were just fine, b= ut a post build script was running cp incorrectly.
David pointed that we need a way to distinguish between test errors a= nd failures. He suggested looking up strings in the test output - we should not go=
there, unless we want to "fix" this many more times in the future.
I suggest to use the these rules:
- SUCCESS - make check returns 0 - FAILURE - make check returns 1 - ERROR - anything else returned by make check or any other script.
I think that make check does work like this, but it should be easy to=
change.
What do you think?
Thanks for your report. Nir has already fixed this in http://gerrit.ovirt.org/28426.
It was introduced in http://gerrit.ovirt.org/#/c/28226/ but missed a=
lso
because we have turned PYFLAKES off in unit test jobs. We must turn = it on in at least one of the tests (or initiate a new jenkins job for `make check-local`).
As a quick fix, David has re-enabled PYFLAKES in http://jenkins.ovirt.org/view/By%20Project/view/vdsm/job/vdsm_master= _unit_tests/configure
Regards, Dan.
Perfect for me, but you should know that it will fail also when strang= e things occur, for example, out of memory, of disk space, slave disconnected, network error, etc.
If you are willing to treat those (the most common infra failures) as devel failures, then no problem on my side,
I'm not - this is why we should separate test failures from test errors= =2E
but I don't want you to start ignoring test errors because it's most probably an infra error (don't get me wrong, it's totally normal to start ignoring an alarm that is not a real problem, as infra members we will try to minimize the infra issues, but it's not yet as stable as we'd like it to be).
This is too late now, people are already ignoring jenkins reports becau= se of the many false negatives :-)
So the return code is not a good solution then, we have to see if it=20 failed, and if it was due to an infra error or a devel error. I think=20 that it's easier to filter for: * A string that means the tests did ran, probably at the end of the log=20 so if there's a connection failure it will be detected as infra issue. * A string that identified if the test failed or passed And if none of those were found, then an infra failure is supposed. -- David Caro Red Hat S.L. Continuous Integration Engineer - EMEA ENG Virtualization R&D Email: dcaro@redhat.com Web: www.redhat.com RHT Global #: 82-62605 --p24AU4JWLLvxM022u0ghKBuqCBEHoClgG Content-Type: application/pgp-signature; name="signature.asc" Content-Description: OpenPGP digital signature Content-Disposition: attachment; filename="signature.asc" -----BEGIN PGP SIGNATURE----- Version: GnuPG v1 iQEcBAEBAgAGBQJTmX/nAAoJEEBxx+HSYmnDENkH/iQWyZH8N2w/UNKhXkJebLXK QZzC8i+8bXJ8Y2iWIuIG9Ktv4NNBKQklQ7vV1mbEoB7ikJSfJSYEaQw5kLOp5Wrt bfWH+qF8T+Hh02NBn+tiXaFURSScorCLA2kTmhtAlyNNbmaw2tlcd7joP3Ykxi1J YpDK3YI6E6WoO/KO9twiKxFqcKoqY3bw28aLiQrQdskXhI6pqH5DrvrGuCq9Ftk8 zejX9Cw4TapPHP+8eawIaqTt+JEkALCbRIL5WbOMOp96YqsSOiYxTD/hwnDqhDJQ tCjD3qtQ7IEPykJqeLhlG2TfcWzyehejvcIXAfCYqeqBssXR+SAy1o99hpKTFzw= =cxTJ -----END PGP SIGNATURE----- --p24AU4JWLLvxM022u0ghKBuqCBEHoClgG--