[VDSM] storage python 3 tests status
Sharing progress on python 3 work. ## 2 weeks ago git checkout @{two.weeks.ago} tox -e storage-py37 ... 1448 passed, 103 skipped, 100 deselected, 322 xfailed, 236 warnings in 71.40 seconds ## Today git checkout tox -e storage-py37 ... 1643 passed, 103 skipped, 100 deselected, 105 xfailed, 82 xpassed, 107 warnings in 79.72 seconds Change: *passed: +195* *skips: 0* *xfail: -217* *xpass: +107* *warnings: -129* ## XPASS 107 xpass are most likely easy fix, just remove the xfail() mark from the test or parameter. Patches should not introduce new XPASS, you should remove all xfail() marks that a patch fixes. To test that your patch do not introduce new XPASS, add this option to tox.ini: [pytest] xfail_strict = True This converts XPASS to FAIL. These modules need to be fixed: $ egrep '^storage.+_test.py.+X' storage-py37.out storage/filevolume_test.py ...XXXxXXXXxXXX [ 19%] storage/formatconverter_test.py XXXXXxxss [ 20%] storage/merge_test.py XXXXXXxxxxXXxXxxxxxXxxxxxx [ 38%] storage/nbd_test.py ssssssssssssX. [ 49%] storage/sd_manifest_test.py XXXXXXXXXxxxxxxxxxxxxxxxxxxxxxxx............ [ 65%] storage/sdm_amend_volume_test.py xXxXxXxX [ 66%] storage/sdm_copy_data_test.py xXXxxxxXXXXXxxxXXXxxXXxxXXx [ 67%] storage/sdm_merge_test.py xxxxXXX [ 79%] storage/sdm_update_volume_test.py xXxXxxXXxxXXxXxXxXxX....... [ 81%] storage/testlib_test.py xXXXxxXXXXXXXXxxxxxxxxxxxxxXxXxXxX.......... [ 87%] storage/volume_metadata_test.py ..........................X............. [ 90%] ## xfail We must fix these xfail before we can run OST. If we start running OST we will waste days debugging OST. Debugging failing tests is 1000X times faster. $ egrep '^storage.+_test.py.+x' storage-py37.out storage/blocksd_test.py ........x..........sssssssssss............ [ 4%] storage/blockvolume_test.py ...........xxxxxxxxx [ 5%] storage/fileutil_test.py ..xx....ss............................ [ 19%] storage/filevolume_test.py ...XXXxXXXXxXXX [ 19%] storage/formatconverter_test.py XXXXXxxss [ 20%] storage/merge_test.py XXXXXXxxxxXXxXxxxxxXxxxxxx [ 38%] storage/sd_manifest_test.py XXXXXXXXXxxxxxxxxxxxxxxxxxxxxxxx............ [ 65%] storage/sdm_amend_volume_test.py xXxXxXxX [ 66%] storage/sdm_copy_data_test.py xXXxxxxXXXXXxxxXXXxxXXxxXXx [ 67%] storage/sdm_merge_test.py xxxxXXX [ 79%] storage/sdm_update_volume_test.py xXxXxxXXxxXXxXxXxXxX....... [ 81%] storage/testlib_test.py xXXXxxXXXXXXXXxxxxxxxxxxxxxXxXxXxX.......... [ 87%] ## Skips Skips should be used only when test cannot run on specific environment, but I think we have some wrong skips that should have been xfail. $ egrep '^storage.+_test.py.+s' storage-py37.out storage/backends_test.py ss [ 2%] storage/blockdev_test.py ssss....s [ 2%] storage/blocksd_test.py ........x..........sssssssssss............ [ 4%] storage/devicemapper_test.py s. [ 9%] storage/fileutil_test.py ..xx....ss............................ [ 19%] storage/formatconverter_test.py XXXXXxxss [ 20%] storage/loopback_test.py ssss [ 31%] storage/lvm_test.py ....................ssssssssssssssssssss [ 33%] storage/lvmfilter_test.py .....ss.......... [ 35%] storage/managedvolume_test.py ssssssssssssssssss.... [ 36%] storage/misc_test.py .................sssssssss...............ssss...... [ 41%] storage/mount_test.py .........ss........ssssss [ 47%] storage/nbd_test.py ssssssssssssX. [ 49%] storage/udev_multipath_test.py .......................s [ 88%] ## Warnings See "warnings summary" in pytest output. Looks less useful, report issues in packages we use instead of issues in our code. But this may be an issue using old versions of packages. See attached results created with: tox -e storage-py37 > storage-py37.out 2>&1 Nir
On 22 Aug 2019, at 15:51, Nir Soffer <nsoffer@redhat.com> wrote:
Sharing progress on python 3 work.
## 2 weeks ago git checkout @{two.weeks.ago} tox -e storage-py37 ... 1448 passed, 103 skipped, 100 deselected, 322 xfailed, 236 warnings in 71.40 seconds
## Today git checkout tox -e storage-py37 ... 1643 passed, 103 skipped, 100 deselected, 105 xfailed, 82 xpassed, 107 warnings in 79.72 seconds Change: passed: +195 skips: 0 xfail: -217 xpass: +107 warnings: -129
## XPASS
107 xpass are most likely easy fix, just remove the xfail() mark from the test or parameter.
Patches should not introduce new XPASS, you should remove all xfail() marks that a patch fixes. To test that your patch do not introduce new XPASS, add this option to tox.ini:
[pytest] xfail_strict = True
This converts XPASS to FAIL.
These modules need to be fixed:
$ egrep '^storage.+_test.py.+X' storage-py37.out storage/filevolume_test.py ...XXXxXXXXxXXX [ 19%] storage/formatconverter_test.py XXXXXxxss [ 20%] storage/merge_test.py XXXXXXxxxxXXxXxxxxxXxxxxxx [ 38%] storage/nbd_test.py ssssssssssssX. [ 49%] storage/sd_manifest_test.py XXXXXXXXXxxxxxxxxxxxxxxxxxxxxxxx............ [ 65%] storage/sdm_amend_volume_test.py xXxXxXxX [ 66%] storage/sdm_copy_data_test.py xXXxxxxXXXXXxxxXXXxxXXxxXXx [ 67%] storage/sdm_merge_test.py xxxxXXX [ 79%] storage/sdm_update_volume_test.py xXxXxxXXxxXXxXxXxXxX....... [ 81%] storage/testlib_test.py xXXXxxXXXXXXXXxxxxxxxxxxxxxXxXxXxX.......... [ 87%] storage/volume_metadata_test.py ..........................X............. [ 90%]
## xfail
We must fix these xfail before we can run OST. If we start running OST we will waste days debugging OST. Debugging failing tests is 1000X times faster.
I agree about the speed, OST is awfully slow, almost unusable. But other than that, not really, I discourage you from neglecting functional fixes and just focuse on unit tests. It is important to fix actual functional problems Coincidentally, just now I got a working OST on py3/RHEL8 just with a couple of workarounds/skips. So we should soon be able to have this (slow) safeguard in place and we’ll be able to eliminate py2/el7 code. Thanks, michal
$ egrep '^storage.+_test.py.+x' storage-py37.out storage/blocksd_test.py ........x..........sssssssssss............ [ 4%] storage/blockvolume_test.py ...........xxxxxxxxx [ 5%] storage/fileutil_test.py ..xx....ss............................ [ 19%] storage/filevolume_test.py ...XXXxXXXXxXXX [ 19%] storage/formatconverter_test.py XXXXXxxss [ 20%] storage/merge_test.py XXXXXXxxxxXXxXxxxxxXxxxxxx [ 38%] storage/sd_manifest_test.py XXXXXXXXXxxxxxxxxxxxxxxxxxxxxxxx............ [ 65%] storage/sdm_amend_volume_test.py xXxXxXxX [ 66%] storage/sdm_copy_data_test.py xXXxxxxXXXXXxxxXXXxxXXxxXXx [ 67%] storage/sdm_merge_test.py xxxxXXX [ 79%] storage/sdm_update_volume_test.py xXxXxxXXxxXXxXxXxXxX....... [ 81%] storage/testlib_test.py xXXXxxXXXXXXXXxxxxxxxxxxxxxXxXxXxX.......... [ 87%]
## Skips
Skips should be used only when test cannot run on specific environment, but I think we have some wrong skips that should have been xfail.
$ egrep '^storage.+_test.py.+s' storage-py37.out storage/backends_test.py ss [ 2%] storage/blockdev_test.py ssss....s [ 2%] storage/blocksd_test.py ........x..........sssssssssss............ [ 4%] storage/devicemapper_test.py s. [ 9%] storage/fileutil_test.py ..xx....ss............................ [ 19%] storage/formatconverter_test.py XXXXXxxss [ 20%] storage/loopback_test.py ssss [ 31%] storage/lvm_test.py ....................ssssssssssssssssssss [ 33%] storage/lvmfilter_test.py .....ss.......... [ 35%] storage/managedvolume_test.py ssssssssssssssssss.... [ 36%] storage/misc_test.py .................sssssssss...............ssss...... [ 41%] storage/mount_test.py .........ss........ssssss [ 47%] storage/nbd_test.py ssssssssssssX. [ 49%] storage/udev_multipath_test.py .......................s [ 88%]
## Warnings
See "warnings summary" in pytest output.
Looks less useful, report issues in packages we use instead of issues in our code. But this may be an issue using old versions of packages.
See attached results created with:
tox -e storage-py37 > storage-py37.out 2>&1
Nir <storage-py37.out>_______________________________________________ Devel mailing list -- devel@ovirt.org To unsubscribe send an email to devel-leave@ovirt.org Privacy Statement: https://www.ovirt.org/site/privacy-policy/ oVirt Code of Conduct: https://www.ovirt.org/community/about/community-guidelines/ List Archives: https://lists.ovirt.org/archives/list/devel@ovirt.org/message/4C5AQCZPKEHQML...
On Fri, Aug 23, 2019 at 6:41 PM Michal Skrivanek < michal.skrivanek@redhat.com> wrote: ...
I agree about the speed, OST is awfully slow, almost unusable. But other than that, not really, I discourage you from neglecting functional fixes and just focuse on unit tests. It is important to fix actual functional problems
We are focusing on small and fast functional tests doing real flows with (mostly) real storage. I think this give be the best value. Being able to verify changes in seconds means you can move fast - safely. The most important actual issue we have now is tests failing on pyhton 3, for example block storage tests: https://gerrit.ovirt.org/q/topic:py3-blocksd When these tests will pass, there is a chance that block storage will work in OST. A bigger problem is code without any tests like the tasks framework, SPM flows and legacy storage flows. We work now on the tasks framework: https://gerrit.ovirt.org/q/topic:py3_task We also worked on the outOfProcess module which is the foundation of all file based storage domains: https://gerrit.ovirt.org/q/topic:py3-oop This work reveled 2 bugs in ioproess, one fixed now: https://gerrit.ovirt.org/q/topic:ioprocess_access
Coincidentally, just now I got a working OST on py3/RHEL8 just with a couple of workarounds/skips. So we should soon be able to have this (slow) safeguard in place and we’ll be able to eliminate py2/el7 code.
We need to keep python 2.7 code until we finish porting to python 3, and maybe later. It will be hard to support 4.3 if master code is not compatible with python 2. Nir
On Thu, Aug 22, 2019 at 4:51 PM Nir Soffer <nsoffer@redhat.com> wrote:
Sharing progress on python 3 work.
## Aug 22
git checkout
tox -e storage-py37 ... 1643 passed, 103 skipped, 100 deselected, 105 xfailed, 82 xpassed, 107 warnings in 79.72 seconds
# Today (Sep 10) 1724 passed, 105 skipped, 100 deselected, 76 xfailed, 51 xpassed, 98 warnings in 82.43 seconds Change: *passed: +81* *skips: +2* *xfail: -29* *xpass: -31* *warnings: -9* Some tests require root, but I did not run as root, so we see higher skips values. ## Python 2 vs python 3 Looking at travis builds: python 2.7: https://travis-ci.org/oVirt/vdsm/jobs/583262136 1936 passed, 16 skipped, 100 deselected, 5 xfailed in 211.59 seconds python 3.7: https://travis-ci.org/oVirt/vdsm/jobs/583262137 1775 passed, 39 skipped, 100 deselected, 90 xfailed, 52 xpassed, 99 warnings in 198.66 seconds ## Coverage python 2.7: storage: 61% https://jenkins.ovirt.org/job/vdsm_standard-check-patch/11189/artifact/check... combined: 63% https://jenkins.ovirt.org/job/vdsm_standard-check-patch/11189/artifact/check... python 3.7: storage: 59% https://jenkins.ovirt.org/job/vdsm_standard-check-patch/11189/artifact/check... combined: 60% https://jenkins.ovirt.org/job/vdsm_standard-check-patch/11189/artifact/check... Thanks for the awesome work! Nir
participants (2)
-
Michal Skrivanek -
Nir Soffer