[ovirt-devel] [VDSM] test_import_modules randomly failing on travis

[Nir Soffer]

On Sat, Nov 26, 2016 at 11:51 AM, Dan Kenigsberg <danken at redhat.com> wrote:
> On Fri, Nov 25, 2016 at 09:56:23PM +0200, Nir Soffer wrote:
>> On Fri, Nov 25, 2016 at 7:29 PM, Nir Soffer <nsoffer at redhat.com> wrote:
>> > Hi all,
>> >
>> > Vdsm travis tests are running file for some weeks. We have several
>> > random failures, probably bad tests that needs to be fixed.
>> >
>> > The most common failure is this - failing only in python 3.
>> >
>> > ======================================================================
>> > ERROR: test_import_modules(('a.py', 'b.py', 'a.pyc', 'a.pyioas'),
>> > ('a', 'b')) (moduleloader_test.ImportModulesTest)
>> > ----------------------------------------------------------------------
>> > Traceback (most recent call last):
>> >   File "/vdsm/tests/testlib.py", line 135, in wrapper
>> >     return f(self, *args)
>> >   File "/vdsm/tests/moduleloader_test.py", line 51, in test_import_modules
>> >     with self._setup_test_modules(files) as module_name:
>> >   File "/usr/lib64/python3.5/contextlib.py", line 59, in __enter__
>> >     return next(self.gen)
>> >   File "/vdsm/tests/moduleloader_test.py", line 42, in _setup_test_modules
>> >     yield importlib.import_module(os.path.basename(path))
>> >   File "/usr/lib64/python3.5/importlib/__init__.py", line 126, in import_module
>> >     return _bootstrap._gcd_import(name[level:], package, level)
>> >   File "<frozen importlib._bootstrap>", line 986, in _gcd_import
>> >   File "<frozen importlib._bootstrap>", line 969, in _find_and_load
>> >   File "<frozen importlib._bootstrap>", line 956, in _find_and_load_unlocked
>> > ImportError: No module named 'tmpozng7rht'
>> >
>> > See complete run:
>> > https://travis-ci.org/nirs/vdsm/jobs/178894634
>> >
>> > Piotr, can you check why this fail and get this test working on travis?
>>
>> I guess nobody have time to fix this now, so I posted
>> https://gerrit.ovirt.org/67333
>> marking these tests as broken on travis until we have time.
>>
>> The build is passing now on travis:
>> https://travis-ci.org/nirs/vdsm/builds/178921340
>
> I've seen it break to day on jenkins as well (it's not only travis!) so
> yeah, unless it is proptly fixed, it has to be marked as broken. It also
> means that https://gerrit.ovirt.org/#/c/67333/3 is too specific...

I found 2 issue that may cause these failures.

With these patches, the tests do not fail any more on travis:
- https://gerrit.ovirt.org/67338 tests: Avoid global state
modification using forked
- https://gerrit.ovirt.org/67339 tests: Remove invalid pyc file tests

See travis build history - builds 148-152 contain both patches and they passed.
Before these patches these tests failed almost on every build.
https://travis-ci.org/nirs/vdsm/builds

Nir