[ovirt-devel] local vdsm build fails

[David Caro]

On Sun 08 Jun 2014 12:57:24 PM CEST, Nir Soffer wrote:
> ----- Original Message -----
>> From: "David Caro" <dcaroest at redhat.com>
>> To: "Nir Soffer" <nsoffer at redhat.com>
>> Cc: "Piotr Kliczewski" <piotr.kliczewski at gmail.com>, fsimonce at redhat.com, dcaro at redhat.com, devel at ovirt.org, "Dan
>> Kenigsberg" <danken at redhat.com>
>> Sent: Friday, June 6, 2014 5:16:52 PM
>> Subject: Re: [ovirt-devel]  local vdsm build fails
>>
>> On Fri 06 Jun 2014 03:53:41 PM CEST, Nir Soffer wrote:
>>> ----- Original Message -----
>>>> From: "Dan Kenigsberg" <danken at redhat.com>
>>>> To: "Piotr Kliczewski" <piotr.kliczewski at gmail.com>, fsimonce at redhat.com,
>>>> nsoffer at redhat.com, dcaro at redhat.com
>>>> Cc: devel at ovirt.org
>>>> Sent: Friday, June 6, 2014 12:15:18 PM
>>>> Subject: Re: [ovirt-devel]  local vdsm build fails
>>>>
>>>> On Fri, Jun 06, 2014 at 09:19:11AM +0200, Piotr Kliczewski wrote:
>>>>> All,
>>>>>
>>>>> I pulled the latest vdsm from master and noticed that build is failing.
>>>>>
>>>>> Here is the patch that causes the failuer:
>>>>>
>>>>> http://gerrit.ovirt.org/#/c/28226
>>>>>
>>>>> and looking at jenkins comments I can see that jenkins was failing
>>>>> with the same reason:
>>>>>
>>>>> http://jenkins.ovirt.org/job/vdsm_master_storage_functional_tests_localfs_gerrit/1064/console
>>>
>>> Nir has already fix that as well. The storage tests were just fine, but
>>> a post build script was running cp incorrectly.
>>>
>>> David pointed that we need a way to distinguish between test errors and
>>> failures.
>>> He suggested looking up strings in the test output - we should not go
>>> there, unless
>>> we want to "fix" this many more times in the future.
>>>
>>> I suggest to use the these rules:
>>>
>>> - SUCCESS - make check returns 0
>>> - FAILURE - make check returns 1
>>> - ERROR - anything else returned by make check or any other script.
>>>
>>> I think that make check does work like this, but it should be easy to
>>> change.
>>>
>>> What do you think?
>>>
>>>>
>>>> Thanks for your report. Nir has already fixed this in
>>>> http://gerrit.ovirt.org/28426.
>>>>
>>>> It was introduced in http://gerrit.ovirt.org/#/c/28226/ but missed also
>>>> because we have turned PYFLAKES off in unit test jobs. We must turn it on
>>>> in
>>>> at least one of the tests (or initiate a new jenkins job for `make
>>>> check-local`).
>>>>
>>>> As a quick fix, David has re-enabled PYFLAKES in
>>>> http://jenkins.ovirt.org/view/By%20Project/view/vdsm/job/vdsm_master_unit_tests/configure
>>>>
>>>> Regards,
>>>> Dan.
>>>>
>>
>> Perfect for me, but you should know that it will fail also when strange
>> things occur, for example, out of memory, of disk space, slave
>> disconnected, network error, etc.
>>
>> If you are willing to treat those (the most common infra failures) as
>> devel failures, then no problem on my side,
>
> I'm not - this is why we should separate test failures from test errors.
>
>> but I don't want you to
>> start ignoring test errors because it's most probably an infra error
>> (don't get me wrong, it's totally normal to start ignoring an alarm
>> that is not a real problem, as infra members we will try to minimize
>> the infra issues, but it's not yet as stable as we'd like it to be).
>
> This is too late now, people are already ignoring jenkins reports because
> of the many false negatives :-)

So the return code is not a good solution then, we have to see if it 
failed, and if it was due to an infra error or a devel error. I think 
that it's easier to filter for:

* A string that means the tests did ran, probably at the end of the log 
so if there's a connection failure it will be detected as infra issue.
* A string that identified if the test failed or passed

And if none of those were found, then an infra failure is supposed.

--
David Caro

Red Hat S.L.
Continuous Integration Engineer - EMEA ENG Virtualization R&D

Email: dcaro at redhat.com
Web: www.redhat.com
RHT Global #: 82-62605

-------------- next part --------------
A non-text attachment was scrubbed...
Name: signature.asc
Type: application/pgp-signature
Size: 473 bytes
Desc: OpenPGP digital signature
URL: <http://lists.ovirt.org/pipermail/devel/attachments/20140612/d7a6b2d1/attachment-0001.sig>