Re: [ovirt-devel] [vdsm] Running VDSM unit tests on Travis CI using Docker

On Wed, Apr 6, 2016 at 2:41 PM, Nir Soffer <nsoffer(a)redhat.com&gt; wrote: > > On Wed, Apr 6, 2016 at 2:33 PM, Edward Haas <ehaas(a)redhat.com&gt; wrote: > > > > > > On Wed, Apr 6, 2016 at 2:23 PM, Nir Soffer <nsoffer(a)redhat.com&gt; wrote: > >> > >> On Wed, Apr 6, 2016 at 2:19 PM, Edward Haas <ehaas(a)redhat.com&gt; wrote: > >> > > >> > > >> > On Wed, Apr 6, 2016 at 1:41 PM, Milan Zamazal <mzamazal(a)redhat.com&gt; > >> > wrote: > >> >> > >> >> Edward Haas <ehaas(a)redhat.com&gt; writes: > >> >> > >> >> > On Wed, Apr 6, 2016 at 11:39 AM, Milan Zamazal > >> >> > <mzamazal(a)redhat.com&gt; > >> >> > wrote: > >> >> > > >> >> > Thank you, Edward, this is useful not only for CI. I use > >> >> > docker > >> >> > for > >> >> > building Vdsm and running its unit tests and this helped me to > >> >> > get > >> >> > the > >> >> > proper updated set of packages after recent changes in Vdsm. > >> >> > > >> >> > BTW, it seems that the following packages should be > >> >> > additionally > >> >> > added > >> >> > for `make check-all': psmisc, which, python-ioprocess > >> >> > > >> >> > > >> >> > Are you saying that make check is passing on your local machine? > >> >> > >> >> When I add the packages given above, `make check-all' (as well as > >> >> `make > >> >> check') works for me except for 4 tests in lib/vdsm/schedule.py that > >> >> produce the following errors with `make check-all': > >> >> > >> >> File "/home/pdm/ovirt/vdsm/vdsm-test/lib/vdsm/schedule.py", line > >> >> 134, > >> >> in schedule > >> >> heapq.heappush(self._calls, (deadline, call)) > >> >> nose.proxy.TypeError: unorderable types: ScheduledCall() < > >> >> ScheduledCall() > >> >> > >> >> File "/home/pdm/ovirt/vdsm/vdsm-test/tests/scheduleTests.py", > >> >> line > >> >> 160, in test_latency > >> >> med = ticker.latency[len(ticker.latency) / 2] > >> >> nose.proxy.TypeError: list indices must be integers, not float > >> >> > >> >> Those are probably Python 3 failures that should be fixed in Vdsm. > >> >> The docker environment works fine for running the unit tests on my > >> >> machine. > >> > > >> > > >> > I ran it on Travis CI with your recommended addition, and I am > >> > getting > >> > this > >> > result: FAILED (SKIP=107, errors=14): > >> > You can view the run here: > >> > https://travis-ci.org/EdDev/vdsm/builds/121117253 > >> > >> Sure, make check in master run tests that should not run on travis. > >> > >> Try the travis branch - after adding ioprocess to the docker image, > >> all tests should pass: > >> https://gerrit.ovirt.org/55738 > >> > >> Nir > > > > > > Ok, will check it as well. > > But its a bit of a lie, many tests are skipped instead of not ran at > > all, > > this needs to be fixed. > > In addition, I find many tests as not unit tests, all tests should pass > > in a > > few seconds not in 2 minutes. > > Storage people always lie :-) > > We have @slowtest for marking slow tests. Unfortunately, some tests are > slow, and there is no value in mocking the thing you want to test. > > Nir Well, I will like to run only the unit tests, and you seem to try and merge the integration tests in the same run.

I think they need to be separated. I would like to see the unit tests pass in a few seconds, have no surprising skips and have no dependency or other services.