[ovirt-devel] New test failure on travis

Thu Dec 8 10:11:39 UTC 2016

On 08/12/16 11:26 +0200, Edward Haas wrote:
>On Thu, Dec 8, 2016 at 10:12 AM, Martin Polednik <mpolednik at redhat.com>
>wrote:
>
>> On 08/12/16 09:28 +0200, Edward Haas wrote:
>>
>>> On Wed, Dec 7, 2016 at 11:54 PM, Nir Soffer <nsoffer at redhat.com> wrote:
>>>
>>> broken_on_ci is uses default name="OVIRT_CI", to mark it also for
>>>> travis, we need another broken_on_ci with name="TRAVIS_CI".
>>>>
>>>> Maybe this test should run only if nm is active on the machine?
>>>>
>>>>
>>> We need the test to always run when expected.
>>> If NM is not running, the test will not run (silently) and we will never
>>> know if there is a problem or not.
>>>
>>> It is not convenient to mark each CI type as broken, why the test code
>>> needs to know we have multiple
>>> CI/s?
>>>
>>
>> I believe this is great point - we should just mark the test as broken
>> on *any* CI to create a pressure to get it fixed.
>>
>> Slight off-topic addition: I don't understand why patch marking a test
>> as broken on CI takes more than 5 minutes to get merged in when given
>> pointer to the failure.
>
>
>Because it is a wrong approach. :)
>If a test fails, it is a smell that something bad happens and we may have a
>production problem.
>So before marking and excluding the test, one should feel very guilty that
>this check is no longer covered and better understand why it fails.

Those are 1-in-N case breakages. The fact the test is unstable should
be noted by a maintainer, but shouldn't block any other patches or
series (which is what often happens).

I don't feel any guilt marking bad test as bad.

>
>>
>>
>> Currently, we run on CI tests that are not marked as 'functional'.
>>> Perhaps we need another test type that can be mark not to run on simple
>>> CI.
>>> "power-integration", "super-integration"?
>>>
>>>
>>>
>>>
>>>>
>>>> On Wed, Dec 7, 2016 at 11:23 PM, Dan Kenigsberg <danken at redhat.com>
>>>> wrote:
>>>> > On Wed, Dec 7, 2016 at 2:03 PM, Nir Soffer <nsoffer at redhat.com> wrote:
>>>> >> Looks like we need @brokentest("reason...", name="TRAVIC_CI") on this:
>>>> >
>>>> > Odd, the code already has
>>>> >
>>>> > @broken_on_ci('NetworkManager should not be started on CI nodes')
>>>> >
>>>> >
>>>> >>
>>>> >> See https://travis-ci.org/oVirt/vdsm/jobs/181933329
>>>> >>
>>>> >> ============================================================
>>>> ==========
>>>> >>
>>>> >> ERROR: test suite for <module 'network.nmdbus_test' from
>>>> >> '/vdsm/tests/network/nmdbus_test.py'>
>>>> >>
>>>> >> ------------------------------------------------------------
>>>> ----------
>>>> >>
>>>> >> Traceback (most recent call last):
>>>> >>
>>>> >>   File "/usr/lib/python2.7/site-packages/nose/suite.py", line 209, in
>>>> run
>>>> >>
>>>> >>     self.setUp()
>>>> >>
>>>> >>   File "/usr/lib/python2.7/site-packages/nose/suite.py", line 292, in
>>>> setUp
>>>> >>
>>>> >>     self.setupContext(ancestor)
>>>> >>
>>>> >>   File "/usr/lib/python2.7/site-packages/nose/suite.py", line 315, in
>>>> >> setupContext
>>>> >>
>>>> >>     try_run(context, names)
>>>> >>
>>>> >>   File "/usr/lib/python2.7/site-packages/nose/util.py", line 471, in
>>>> try_run
>>>> >>
>>>> >>     return func()
>>>> >>
>>>> >>   File "/vdsm/tests/testValidation.py", line 191, in wrapper
>>>> >>
>>>> >>     return f(*args, **kwargs)
>>>> >>
>>>> >>   File "/vdsm/tests/testValidation.py", line 97, in wrapper
>>>> >>
>>>> >>     return f(*args, **kwargs)
>>>> >>
>>>> >>   File "/vdsm/tests/network/nmdbus_test.py", line 48, in setup_module
>>>> >>
>>>> >>     NMDbus.init()
>>>> >>
>>>> >>   File "/vdsm/lib/vdsm/network/nm/nmdbus/__init__.py", line 33, in
>>>> init
>>>> >>
>>>> >>     NMDbus.bus = dbus.SystemBus()
>>>> >>
>>>> >>   File "/usr/lib64/python2.7/site-packages/dbus/_dbus.py", line 194,
>>>> in __new__
>>>> >>
>>>> >>     private=private)
>>>> >>
>>>> >>   File "/usr/lib64/python2.7/site-packages/dbus/_dbus.py", line 100,
>>>> in __new__
>>>> >>
>>>> >>     bus = BusConnection.__new__(subclass, bus_type,
>>>> mainloop=mainloop)
>>>> >>
>>>> >>   File "/usr/lib64/python2.7/site-packages/dbus/bus.py", line 122, in
>>>> __new__
>>>> >>
>>>> >>     bus = cls._new_for_bus(address_or_type, mainloop=mainloop)
>>>> >>
>>>> >> DBusException: org.freedesktop.DBus.Error.FileNotFound: Failed to
>>>> >> connect to socket /var/run/dbus/system_bus_socket: No such file or
>>>> >> directory
>>>> >>
>>>> >> -------------------- >> begin captured logging << --------------------
>>>> >>
>>>> >> 2016-12-07 11:48:33,458 DEBUG (MainThread) [root] /usr/bin/taskset
>>>> >> --cpu-list 0-1 /bin/systemctl status NetworkManager (cwd None)
>>>> >> (commands:69)
>>>> >>
>>>> >> 2016-12-07 11:48:33,465 DEBUG (MainThread) [root] FAILED: <err> =
>>>> >> 'Failed to get D-Bus connection: Operation not permitted\n'; <rc> = 1
>>>> >> (commands:93)
>>>> >>
>>>> >> 2016-12-07 11:48:33,465 DEBUG (MainThread) [root] /usr/bin/taskset
>>>> >> --cpu-list 0-1 /bin/systemctl start NetworkManager (cwd None)
>>>> >> (commands:69)
>>>> >>
>>>> >> 2016-12-07 11:48:33,470 DEBUG (MainThread) [root] FAILED: <err> =
>>>> >> 'Failed to get D-Bus connection: Operation not permitted\n'; <rc> = 1
>>>> >> (commands:93)
>>>> >>
>>>> >> --------------------- >> end captured logging << ---------------------
>>>>
>>>>
>> _______________________________________________
>>> Devel mailing list
>>> Devel at ovirt.org
>>> http://lists.ovirt.org/mailman/listinfo/devel
>>>
>>
>>