[ovirt-devel] New test failure on travis

[Nir Soffer]

All travis builds fail since
https://github.com/oVirt/vdsm/commit/c859258bd83212c8ed3bff8436e5158da81bbba0

See https://travis-ci.org/oVirt/vdsm/builds

Edward, please mark this test as broken on travis, or functional, until we have
running network manager in the test environment.

Thanks,
Nir



On Thu, Dec 8, 2016 at 12:11 PM, Martin Polednik <mpolednik at redhat.com> wrote:
> On 08/12/16 11:26 +0200, Edward Haas wrote:
>>
>> On Thu, Dec 8, 2016 at 10:12 AM, Martin Polednik <mpolednik at redhat.com>
>> wrote:
>>
>>> On 08/12/16 09:28 +0200, Edward Haas wrote:
>>>
>>>> On Wed, Dec 7, 2016 at 11:54 PM, Nir Soffer <nsoffer at redhat.com> wrote:
>>>>
>>>> broken_on_ci is uses default name="OVIRT_CI", to mark it also for
>>>>>
>>>>> travis, we need another broken_on_ci with name="TRAVIS_CI".
>>>>>
>>>>> Maybe this test should run only if nm is active on the machine?
>>>>>
>>>>>
>>>> We need the test to always run when expected.
>>>> If NM is not running, the test will not run (silently) and we will never
>>>> know if there is a problem or not.
>>>>
>>>> It is not convenient to mark each CI type as broken, why the test code
>>>> needs to know we have multiple
>>>> CI/s?
>>>>
>>>
>>> I believe this is great point - we should just mark the test as broken
>>> on *any* CI to create a pressure to get it fixed.
>>>
>>> Slight off-topic addition: I don't understand why patch marking a test
>>> as broken on CI takes more than 5 minutes to get merged in when given
>>> pointer to the failure.
>>
>>
>>
>> Because it is a wrong approach. :)
>> If a test fails, it is a smell that something bad happens and we may have
>> a
>> production problem.
>> So before marking and excluding the test, one should feel very guilty that
>> this check is no longer covered and better understand why it fails.
>
>
> Those are 1-in-N case breakages. The fact the test is unstable should
> be noted by a maintainer, but shouldn't block any other patches or
> series (which is what often happens).
>
> I don't feel any guilt marking bad test as bad.
>
>
>>
>>>
>>>
>>> Currently, we run on CI tests that are not marked as 'functional'.
>>>>
>>>> Perhaps we need another test type that can be mark not to run on simple
>>>> CI.
>>>> "power-integration", "super-integration"?
>>>>
>>>>
>>>>
>>>>
>>>>>
>>>>> On Wed, Dec 7, 2016 at 11:23 PM, Dan Kenigsberg <danken at redhat.com>
>>>>> wrote:
>>>>> > On Wed, Dec 7, 2016 at 2:03 PM, Nir Soffer <nsoffer at redhat.com>
>>>>> > wrote:
>>>>> >> Looks like we need @brokentest("reason...", name="TRAVIC_CI") on
>>>>> >> this:
>>>>> >
>>>>> > Odd, the code already has
>>>>> >
>>>>> > @broken_on_ci('NetworkManager should not be started on CI nodes')
>>>>> >
>>>>> >
>>>>> >>
>>>>> >> See https://travis-ci.org/oVirt/vdsm/jobs/181933329
>>>>> >>
>>>>> >> ============================================================
>>>>> ==========
>>>>> >>
>>>>> >> ERROR: test suite for <module 'network.nmdbus_test' from
>>>>> >> '/vdsm/tests/network/nmdbus_test.py'>
>>>>> >>
>>>>> >> ------------------------------------------------------------
>>>>> ----------
>>>>> >>
>>>>> >> Traceback (most recent call last):
>>>>> >>
>>>>> >>   File "/usr/lib/python2.7/site-packages/nose/suite.py", line 209,
>>>>> >> in
>>>>> run
>>>>> >>
>>>>> >>     self.setUp()
>>>>> >>
>>>>> >>   File "/usr/lib/python2.7/site-packages/nose/suite.py", line 292,
>>>>> >> in
>>>>> setUp
>>>>> >>
>>>>> >>     self.setupContext(ancestor)
>>>>> >>
>>>>> >>   File "/usr/lib/python2.7/site-packages/nose/suite.py", line 315,
>>>>> >> in
>>>>> >> setupContext
>>>>> >>
>>>>> >>     try_run(context, names)
>>>>> >>
>>>>> >>   File "/usr/lib/python2.7/site-packages/nose/util.py", line 471, in
>>>>> try_run
>>>>> >>
>>>>> >>     return func()
>>>>> >>
>>>>> >>   File "/vdsm/tests/testValidation.py", line 191, in wrapper
>>>>> >>
>>>>> >>     return f(*args, **kwargs)
>>>>> >>
>>>>> >>   File "/vdsm/tests/testValidation.py", line 97, in wrapper
>>>>> >>
>>>>> >>     return f(*args, **kwargs)
>>>>> >>
>>>>> >>   File "/vdsm/tests/network/nmdbus_test.py", line 48, in
>>>>> >> setup_module
>>>>> >>
>>>>> >>     NMDbus.init()
>>>>> >>
>>>>> >>   File "/vdsm/lib/vdsm/network/nm/nmdbus/__init__.py", line 33, in
>>>>> init
>>>>> >>
>>>>> >>     NMDbus.bus = dbus.SystemBus()
>>>>> >>
>>>>> >>   File "/usr/lib64/python2.7/site-packages/dbus/_dbus.py", line 194,
>>>>> in __new__
>>>>> >>
>>>>> >>     private=private)
>>>>> >>
>>>>> >>   File "/usr/lib64/python2.7/site-packages/dbus/_dbus.py", line 100,
>>>>> in __new__
>>>>> >>
>>>>> >>     bus = BusConnection.__new__(subclass, bus_type,
>>>>> mainloop=mainloop)
>>>>> >>
>>>>> >>   File "/usr/lib64/python2.7/site-packages/dbus/bus.py", line 122,
>>>>> >> in
>>>>> __new__
>>>>> >>
>>>>> >>     bus = cls._new_for_bus(address_or_type, mainloop=mainloop)
>>>>> >>
>>>>> >> DBusException: org.freedesktop.DBus.Error.FileNotFound: Failed to
>>>>> >> connect to socket /var/run/dbus/system_bus_socket: No such file or
>>>>> >> directory
>>>>> >>
>>>>> >> -------------------- >> begin captured logging <<
>>>>> >> --------------------
>>>>> >>
>>>>> >> 2016-12-07 11:48:33,458 DEBUG (MainThread) [root] /usr/bin/taskset
>>>>> >> --cpu-list 0-1 /bin/systemctl status NetworkManager (cwd None)
>>>>> >> (commands:69)
>>>>> >>
>>>>> >> 2016-12-07 11:48:33,465 DEBUG (MainThread) [root] FAILED: <err> =
>>>>> >> 'Failed to get D-Bus connection: Operation not permitted\n'; <rc> =
>>>>> >> 1
>>>>> >> (commands:93)
>>>>> >>
>>>>> >> 2016-12-07 11:48:33,465 DEBUG (MainThread) [root] /usr/bin/taskset
>>>>> >> --cpu-list 0-1 /bin/systemctl start NetworkManager (cwd None)
>>>>> >> (commands:69)
>>>>> >>
>>>>> >> 2016-12-07 11:48:33,470 DEBUG (MainThread) [root] FAILED: <err> =
>>>>> >> 'Failed to get D-Bus connection: Operation not permitted\n'; <rc> =
>>>>> >> 1
>>>>> >> (commands:93)
>>>>> >>
>>>>> >> --------------------- >> end captured logging <<
>>>>> >> ---------------------
>>>>>
>>>>>
>>> _______________________________________________
>>>>
>>>> Devel mailing list
>>>> Devel at ovirt.org
>>>> http://lists.ovirt.org/mailman/listinfo/devel
>>>>
>>>
>>>
>