[ovirt-devel] Porting basic suite to pytest

Hi All, I've been experimenting with porting basic suite from nose to pytest. The rationale behind my efforts were nose being deprecated since a long time [1] and the ability to use pytest's awesome fixtures. Turns out it's not really that hard - with two patches [2][3] I was able to lay the groundwork for non-distruptive and incremental process of porting the basic suite module by module. You can find the steps needed to port a certain module described in [2]. Please find the patches that do the actual porting in this topic [4]. One of the hurdles of using pytest vs nose is that the former executes the test functions in the order they appear in the module. This comes from the reasoning, that tests should be independent, which is not the case for OST. Reordering the test functions to match '_TEST_LIST' for '002_bootstrap' and '004_basic_sanity' modules would make a huge disruption, so I've decided to use a 'pytest-ordering' plugin that allows you to define the order with '(a)pytest.mark.run(order=)' decorators. In other modules, like '003_basic_networking' and '006_migrations', minor refactoring was done, since "test" functions like 'migrate_vm' were used more than once in the module [5][6]. I tried to keep the changes minimal though - my aim was to do the porting - refinements can be done later. The junitxml reports generated by pytest seem to work fine, example of pytest-based module failure can be seen here [7]. Please let me know what do you think. Regards, Marcin [1] https://nose.readthedocs.io/en/latest/#note-to-users [2] https://gerrit.ovirt.org/#/c/107395 [3] https://gerrit.ovirt.org/#/c/107396 [4] https://gerrit.ovirt.org/#/q/topic:basic-suite-on-pytest+(status:open+OR+... [5] https://github.com/oVirt/ovirt-system-tests/blob/73ed33aa6fbd471eb9259397... [6] https://github.com/oVirt/ovirt-system-tests/blob/73ed33aa6fbd471eb9259397... [7] https://jenkins.ovirt.org/job/ovirt-system-tests_standard-check-patch/8224/