Looks like we need @brokentest("reason...", name="TRAVIC_CI") on this:

See https://travis-ci.org/oVirt/vdsm/jobs/181933329 ====================================================================== ERROR: test suite for <module 'network.nmdbus_test' from '/vdsm/tests/network/nmdbus_test.py'> ---------------------------------------------------------------------- Traceback (most recent call last): File "/usr/lib/python2.7/site-packages/nose/suite.py", line 209, in run self.setUp() File "/usr/lib/python2.7/site-packages/nose/suite.py", line 292, in setUp self.setupContext(ancestor) File "/usr/lib/python2.7/site-packages/nose/suite.py", line 315, in setupContext try_run(context, names) File "/usr/lib/python2.7/site-packages/nose/util.py", line 471, in try_run return func() File "/vdsm/tests/testValidation.py", line 191, in wrapper return f(*args, **kwargs) File "/vdsm/tests/testValidation.py", line 97, in wrapper return f(*args, **kwargs) File "/vdsm/tests/network/nmdbus_test.py", line 48, in setup_module NMDbus.init() File "/vdsm/lib/vdsm/network/nm/nmdbus/__init__.py", line 33, in init NMDbus.bus = dbus.SystemBus() File "/usr/lib64/python2.7/site-packages/dbus/_dbus.py", line 194, in __new__ private=private) File "/usr/lib64/python2.7/site-packages/dbus/_dbus.py", line 100, in __new__ bus = BusConnection.__new__(subclass, bus_type, mainloop=mainloop) File "/usr/lib64/python2.7/site-packages/dbus/bus.py", line 122, in __new__ bus = cls._new_for_bus(address_or_type, mainloop=mainloop) DBusException: org.freedesktop.DBus.Error.FileNotFound: Failed to connect to socket /var/run/dbus/system_bus_socket: No such file or directory -------------------- >> begin captured logging << -------------------- 2016-12-07 11:48:33,458 DEBUG (MainThread) [root] /usr/bin/taskset --cpu-list 0-1 /bin/systemctl status NetworkManager (cwd None) (commands:69) 2016-12-07 11:48:33,465 DEBUG (MainThread) [root] FAILED: <err> = 'Failed to get D-Bus connection: Operation not permitted\n'; <rc> = 1 (commands:93) 2016-12-07 11:48:33,465 DEBUG (MainThread) [root] /usr/bin/taskset --cpu-list 0-1 /bin/systemctl start NetworkManager (cwd None) (commands:69) 2016-12-07 11:48:33,470 DEBUG (MainThread) [root] FAILED: <err> = 'Failed to get D-Bus connection: Operation not permitted\n'; <rc> = 1 (commands:93) --------------------- >> end captured logging << ---------------------

broken_on_ci is uses default name="OVIRT_CI", to mark it also for travis, we need another broken_on_ci with name="TRAVIS_CI". Maybe this test should run only if nm is active on the machine?

On Wed, Dec 7, 2016 at 11:23 PM, Dan Kenigsberg <danken(a)redhat.com&gt; wrote: > On Wed, Dec 7, 2016 at 2:03 PM, Nir Soffer <nsoffer(a)redhat.com&gt; wrote: >> Looks like we need @brokentest("reason...", name="TRAVIC_CI") on this: > > Odd, the code already has > > @broken_on_ci('NetworkManager should not be started on CI nodes') > > >> >> See https://travis-ci.org/oVirt/vdsm/jobs/181933329 >> >> ====================================================================== >> >> ERROR: test suite for <module 'network.nmdbus_test' from >> '/vdsm/tests/network/nmdbus_test.py'> >> >> ---------------------------------------------------------------------- >> >> Traceback (most recent call last): >> >> File "/usr/lib/python2.7/site-packages/nose/suite.py", line 209, in run >> >> self.setUp() >> >> File "/usr/lib/python2.7/site-packages/nose/suite.py", line 292, in setUp >> >> self.setupContext(ancestor) >> >> File "/usr/lib/python2.7/site-packages/nose/suite.py", line 315, in >> setupContext >> >> try_run(context, names) >> >> File "/usr/lib/python2.7/site-packages/nose/util.py", line 471, in try_run >> >> return func() >> >> File "/vdsm/tests/testValidation.py", line 191, in wrapper >> >> return f(*args, **kwargs) >> >> File "/vdsm/tests/testValidation.py", line 97, in wrapper >> >> return f(*args, **kwargs) >> >> File "/vdsm/tests/network/nmdbus_test.py", line 48, in setup_module >> >> NMDbus.init() >> >> File "/vdsm/lib/vdsm/network/nm/nmdbus/__init__.py", line 33, in init >> >> NMDbus.bus = dbus.SystemBus() >> >> File "/usr/lib64/python2.7/site-packages/dbus/_dbus.py", line 194, in __new__ >> >> private=private) >> >> File "/usr/lib64/python2.7/site-packages/dbus/_dbus.py", line 100, in __new__ >> >> bus = BusConnection.__new__(subclass, bus_type, mainloop=mainloop) >> >> File "/usr/lib64/python2.7/site-packages/dbus/bus.py", line 122, in __new__ >> >> bus = cls._new_for_bus(address_or_type, mainloop=mainloop) >> >> DBusException: org.freedesktop.DBus.Error.FileNotFound: Failed to >> connect to socket /var/run/dbus/system_bus_socket: No such file or >> directory >> >> -------------------- >> begin captured logging << -------------------- >> >> 2016-12-07 11:48:33,458 DEBUG (MainThread) [root] /usr/bin/taskset >> --cpu-list 0-1 /bin/systemctl status NetworkManager (cwd None) >> (commands:69) >> >> 2016-12-07 11:48:33,465 DEBUG (MainThread) [root] FAILED: <err> = >> 'Failed to get D-Bus connection: Operation not permitted\n'; <rc> = 1 >> (commands:93) >> >> 2016-12-07 11:48:33,465 DEBUG (MainThread) [root] /usr/bin/taskset >> --cpu-list 0-1 /bin/systemctl start NetworkManager (cwd None) >> (commands:69) >> >> 2016-12-07 11:48:33,470 DEBUG (MainThread) [root] FAILED: <err> = >> 'Failed to get D-Bus connection: Operation not permitted\n'; <rc> = 1 >> (commands:93) >> >> --------------------- >> end captured logging << ---------------------

On 08/12/16 09:28 +0200, Edward Haas wrote: > On Wed, Dec 7, 2016 at 11:54 PM, Nir Soffer <nsoffer(a)redhat.com&gt; wrote: > > broken_on_ci is uses default name="OVIRT_CI", to mark it also for >> travis, we need another broken_on_ci with name="TRAVIS_CI". >> >> Maybe this test should run only if nm is active on the machine? >> >> > We need the test to always run when expected. > If NM is not running, the test will not run (silently) and we will never > know if there is a problem or not. > > It is not convenient to mark each CI type as broken, why the test code > needs to know we have multiple > CI/s? > I believe this is great point - we should just mark the test as broken on *any* CI to create a pressure to get it fixed. Slight off-topic addition: I don't understand why patch marking a test as broken on CI takes more than 5 minutes to get merged in when given pointer to the failure.

Currently, we run on CI tests that are not marked as 'functional'. > Perhaps we need another test type that can be mark not to run on simple > CI. > "power-integration", "super-integration"? > > > > >> >> On Wed, Dec 7, 2016 at 11:23 PM, Dan Kenigsberg <danken(a)redhat.com&gt; >> wrote: >> > On Wed, Dec 7, 2016 at 2:03 PM, Nir Soffer <nsoffer(a)redhat.com&gt; wrote: >> >> Looks like we need @brokentest("reason...", name="TRAVIC_CI") on this: >> > >> > Odd, the code already has >> > >> > @broken_on_ci('NetworkManager should not be started on CI nodes') >> > >> > >> >> >> >> See https://travis-ci.org/oVirt/vdsm/jobs/181933329 >> >> >> >> ============================================================ >> ========== >> >> >> >> ERROR: test suite for <module 'network.nmdbus_test' from >> >> '/vdsm/tests/network/nmdbus_test.py'> >> >> >> >> ------------------------------------------------------------ >> ---------- >> >> >> >> Traceback (most recent call last): >> >> >> >> File "/usr/lib/python2.7/site-packages/nose/suite.py", line 209, in >> run >> >> >> >> self.setUp() >> >> >> >> File "/usr/lib/python2.7/site-packages/nose/suite.py", line 292, in >> setUp >> >> >> >> self.setupContext(ancestor) >> >> >> >> File "/usr/lib/python2.7/site-packages/nose/suite.py", line 315, in >> >> setupContext >> >> >> >> try_run(context, names) >> >> >> >> File "/usr/lib/python2.7/site-packages/nose/util.py", line 471, in >> try_run >> >> >> >> return func() >> >> >> >> File "/vdsm/tests/testValidation.py", line 191, in wrapper >> >> >> >> return f(*args, **kwargs) >> >> >> >> File "/vdsm/tests/testValidation.py", line 97, in wrapper >> >> >> >> return f(*args, **kwargs) >> >> >> >> File "/vdsm/tests/network/nmdbus_test.py", line 48, in setup_module >> >> >> >> NMDbus.init() >> >> >> >> File "/vdsm/lib/vdsm/network/nm/nmdbus/__init__.py", line 33, in >> init >> >> >> >> NMDbus.bus = dbus.SystemBus() >> >> >> >> File "/usr/lib64/python2.7/site-packages/dbus/_dbus.py", line 194, >> in __new__ >> >> >> >> private=private) >> >> >> >> File "/usr/lib64/python2.7/site-packages/dbus/_dbus.py", line 100, >> in __new__ >> >> >> >> bus = BusConnection.__new__(subclass, bus_type, >> mainloop=mainloop) >> >> >> >> File "/usr/lib64/python2.7/site-packages/dbus/bus.py", line 122, in >> __new__ >> >> >> >> bus = cls._new_for_bus(address_or_type, mainloop=mainloop) >> >> >> >> DBusException: org.freedesktop.DBus.Error.FileNotFound: Failed to >> >> connect to socket /var/run/dbus/system_bus_socket: No such file or >> >> directory >> >> >> >> -------------------- >> begin captured logging << -------------------- >> >> >> >> 2016-12-07 11:48:33,458 DEBUG (MainThread) [root] /usr/bin/taskset >> >> --cpu-list 0-1 /bin/systemctl status NetworkManager (cwd None) >> >> (commands:69) >> >> >> >> 2016-12-07 11:48:33,465 DEBUG (MainThread) [root] FAILED: <err> = >> >> 'Failed to get D-Bus connection: Operation not permitted\n'; <rc> = 1 >> >> (commands:93) >> >> >> >> 2016-12-07 11:48:33,465 DEBUG (MainThread) [root] /usr/bin/taskset >> >> --cpu-list 0-1 /bin/systemctl start NetworkManager (cwd None) >> >> (commands:69) >> >> >> >> 2016-12-07 11:48:33,470 DEBUG (MainThread) [root] FAILED: <err> = >> >> 'Failed to get D-Bus connection: Operation not permitted\n'; <rc> = 1 >> >> (commands:93) >> >> >> >> --------------------- >> end captured logging << --------------------- >> >> _______________________________________________ > Devel mailing list > Devel(a)ovirt.org > http://lists.ovirt.org/mailman/listinfo/devel >

On 08/12/16 11:26 +0200, Edward Haas wrote: > > On Thu, Dec 8, 2016 at 10:12 AM, Martin Polednik <mpolednik(a)redhat.com&gt; > wrote: > >> On 08/12/16 09:28 +0200, Edward Haas wrote: >> >>> On Wed, Dec 7, 2016 at 11:54 PM, Nir Soffer <nsoffer(a)redhat.com&gt; wrote: >>> >>> broken_on_ci is uses default name="OVIRT_CI", to mark it also for >>>> >>>> travis, we need another broken_on_ci with name="TRAVIS_CI". >>>> >>>> Maybe this test should run only if nm is active on the machine? >>>> >>>> >>> We need the test to always run when expected. >>> If NM is not running, the test will not run (silently) and we will never >>> know if there is a problem or not. >>> >>> It is not convenient to mark each CI type as broken, why the test code >>> needs to know we have multiple >>> CI/s? >>> >> >> I believe this is great point - we should just mark the test as broken >> on *any* CI to create a pressure to get it fixed. >> >> Slight off-topic addition: I don't understand why patch marking a test >> as broken on CI takes more than 5 minutes to get merged in when given >> pointer to the failure. > > > > Because it is a wrong approach. :) > If a test fails, it is a smell that something bad happens and we may have > a > production problem. > So before marking and excluding the test, one should feel very guilty that > this check is no longer covered and better understand why it fails. Those are 1-in-N case breakages. The fact the test is unstable should be noted by a maintainer, but shouldn't block any other patches or series (which is what often happens). I don't feel any guilt marking bad test as bad. > >> >> >> Currently, we run on CI tests that are not marked as 'functional'. >>> >>> Perhaps we need another test type that can be mark not to run on simple >>> CI. >>> "power-integration", "super-integration"? >>> >>> >>> >>> >>>> >>>> On Wed, Dec 7, 2016 at 11:23 PM, Dan Kenigsberg <danken(a)redhat.com&gt; >>>> wrote: >>>> > On Wed, Dec 7, 2016 at 2:03 PM, Nir Soffer <nsoffer(a)redhat.com&gt; >>>> > wrote: >>>> >> Looks like we need @brokentest("reason...", name="TRAVIC_CI") on >>>> >> this: >>>> > >>>> > Odd, the code already has >>>> > >>>> > @broken_on_ci('NetworkManager should not be started on CI nodes') >>>> > >>>> > >>>> >> >>>> >> See https://travis-ci.org/oVirt/vdsm/jobs/181933329 >>>> >> >>>> >> ============================================================ >>>> ========== >>>> >> >>>> >> ERROR: test suite for <module 'network.nmdbus_test' from >>>> >> '/vdsm/tests/network/nmdbus_test.py'> >>>> >> >>>> >> ------------------------------------------------------------ >>>> ---------- >>>> >> >>>> >> Traceback (most recent call last): >>>> >> >>>> >> File "/usr/lib/python2.7/site-packages/nose/suite.py", line 209, >>>> >> in >>>> run >>>> >> >>>> >> self.setUp() >>>> >> >>>> >> File "/usr/lib/python2.7/site-packages/nose/suite.py", line 292, >>>> >> in >>>> setUp >>>> >> >>>> >> self.setupContext(ancestor) >>>> >> >>>> >> File "/usr/lib/python2.7/site-packages/nose/suite.py", line 315, >>>> >> in >>>> >> setupContext >>>> >> >>>> >> try_run(context, names) >>>> >> >>>> >> File "/usr/lib/python2.7/site-packages/nose/util.py", line 471, in >>>> try_run >>>> >> >>>> >> return func() >>>> >> >>>> >> File "/vdsm/tests/testValidation.py", line 191, in wrapper >>>> >> >>>> >> return f(*args, **kwargs) >>>> >> >>>> >> File "/vdsm/tests/testValidation.py", line 97, in wrapper >>>> >> >>>> >> return f(*args, **kwargs) >>>> >> >>>> >> File "/vdsm/tests/network/nmdbus_test.py", line 48, in >>>> >> setup_module >>>> >> >>>> >> NMDbus.init() >>>> >> >>>> >> File "/vdsm/lib/vdsm/network/nm/nmdbus/__init__.py", line 33, in >>>> init >>>> >> >>>> >> NMDbus.bus = dbus.SystemBus() >>>> >> >>>> >> File "/usr/lib64/python2.7/site-packages/dbus/_dbus.py", line 194, >>>> in __new__ >>>> >> >>>> >> private=private) >>>> >> >>>> >> File "/usr/lib64/python2.7/site-packages/dbus/_dbus.py", line 100, >>>> in __new__ >>>> >> >>>> >> bus = BusConnection.__new__(subclass, bus_type, >>>> mainloop=mainloop) >>>> >> >>>> >> File "/usr/lib64/python2.7/site-packages/dbus/bus.py", line 122, >>>> >> in >>>> __new__ >>>> >> >>>> >> bus = cls._new_for_bus(address_or_type, mainloop=mainloop) >>>> >> >>>> >> DBusException: org.freedesktop.DBus.Error.FileNotFound: Failed to >>>> >> connect to socket /var/run/dbus/system_bus_socket: No such file or >>>> >> directory >>>> >> >>>> >> -------------------- >> begin captured logging << >>>> >> -------------------- >>>> >> >>>> >> 2016-12-07 11:48:33,458 DEBUG (MainThread) [root] /usr/bin/taskset >>>> >> --cpu-list 0-1 /bin/systemctl status NetworkManager (cwd None) >>>> >> (commands:69) >>>> >> >>>> >> 2016-12-07 11:48:33,465 DEBUG (MainThread) [root] FAILED: <err> = >>>> >> 'Failed to get D-Bus connection: Operation not permitted\n'; <rc> = >>>> >> 1 >>>> >> (commands:93) >>>> >> >>>> >> 2016-12-07 11:48:33,465 DEBUG (MainThread) [root] /usr/bin/taskset >>>> >> --cpu-list 0-1 /bin/systemctl start NetworkManager (cwd None) >>>> >> (commands:69) >>>> >> >>>> >> 2016-12-07 11:48:33,470 DEBUG (MainThread) [root] FAILED: <err> = >>>> >> 'Failed to get D-Bus connection: Operation not permitted\n'; <rc> = >>>> >> 1 >>>> >> (commands:93) >>>> >> >>>> >> --------------------- >> end captured logging << >>>> >> --------------------- >>>> >>>> >> _______________________________________________ >>> >>> Devel mailing list >>> Devel(a)ovirt.org >>> http://lists.ovirt.org/mailman/listinfo/devel >>> >> >>