[ovirt-devel] [VDSM] Moving to py.test

Hi all, In the last vdsm call we discussed moving to py.test. Here is how to run the current tests with py.test: cd tests PYTHONPATH=../lib:../vdsm py.test -v *_test.py *_tests.py *Tests.py network Currently we have some failures and errors: 8 failed, 1662 passed, 101 skipped, 14 pytest-warnings, 10 error in 86.05 seconds Some py.test nice tricks: Finding the 10 slowest tests: PYTHONPATH=../lib:../vdsm py.test -v --durations=10 vmTests.py *_test.py *_tests.py *Tests.py network Checking code coverage: PYTHONPATH=../lib:../vdsm py.test -v --cov=../vdsm --cov=vdsm *_test.py *_tests.py *Tests.py network Once we finish moving vdsm code to lib/vdsm, renaming the tests to *_test.py, and add pytest.ini, running the test will be simply: py.test Once all the tests pass, we can: - Drop all the special asserts (assertXXX), py.test uses plain asserts https://pytest.org/latest/assert.html#asserting-with-the-assert-statement - Get better error reports https://pytest.org/latest/example/reportingdemo.html - Remove unneeded classes, or inherit from any class https://pytest.org/latest/getting-started.html - Replace @permutations with more powerfull @pytest.mark.parametrize https://pytest.org/latest/parametrize.html#parametrize-basics - Replace slowtest and stresstest with @pytest.mark https://pytest.org/latest/mark.html - Replace monkeypatch.py with simpler and more powerfull pytest monkeypatch funcarg https://pytest.org/latest/monkeypatch.html#monkeypatching-mocking-modules... - Replace NamedTemporaryDir with pytest tmpdir funcarg https://pytest.org/latest/getting-started.html#going-functional-requestin... - Simplify complex context managers with pytest fixtures https://pytest.org/latest/example/simple.html - Drop testrunner.py, most of testlib.py, some of testValidation.py, run_tests*.sh - And more Nir