On Fri, Feb 15, 2019, 16:36 Jonkery Huang <jonkery.huang@oracle.com wrote:

Hello Nir,

Thanks for the sharing, it is very helpful!

Are these test scripts open sourced and can be re-used by others for testing oVirt?

Sure, everything we do is open source.

These tests are part of vdsm, available here:

https://github.com/ovirt/vdsm

Example tests:

https://github.com/oVirt/vdsm/blob/master/tests/storage/blocksd_test.py

https://github.com/oVirt/vdsm/blob/master/tests/storage/lvm_test.py

The pytest infrastructure:

https://github.com/oVirt/vdsm/blob/master/tests/storage/conftest.py

Nir

Thanks
Jonkery
On 2/14/2019 6:26 AM, Nir Soffer wrote:
I want to share our new block storage tests, running on your laptop, from your editor, creating

real block storage domain with real logical volumes.

One catch, these tests require root - there is no way to create devices without root.

To make it easy to run as root only the tests that need root, they are marked with "root" mark.

Here an example of running the root tests for block storage domain:

$ sudo ~/.local/bin/tox -e storage-py27 tests/storage/blocksd_test.py -- -m root

And for lvm:

$ sudo ~/.local/bin/tox -e storage-py27 tests/storage/lvm_test.py -- -m root

To run all storage tests that require root:

$ sudo ~/.local/bin/tox -e storage-py27 -- -m root tests/storage

Another issue - after running the tests as root, you need to fix ownership of some

files in .tox/, tests/htmlcov*, and /var/tmp/vdsm. You can:

$ sudo chown -R $USER:$USER .tox tests /var/tmp/vdsm

We will improve this later.

Note that I'm running tox installed as user:

$ pip install --user tox

This gets most recent tox with minimal breakage of system python.

With the new tests, our code coverage is now 57%:

https://jenkins.ovirt.org/job/vdsm_standard-check-patch/2888/artifact/check-patch.tests-py27.fc28.x86_64/htmlcov-storage-py27/index.html

- blockSD: 47%

https://jenkins.ovirt.org/job/vdsm_standard-check-patch/2888/artifact/check-patch.tests-py27.fc28.x86_64/htmlcov-storage-py27/_home_jenkins_workspace_vdsm_standard-check-patch_vdsm_lib_vdsm_storage_blockSD_py.html

- lvm: 74%

https://jenkins.ovirt.org/job/vdsm_standard-check-patch/2888/artifact/check-patch.tests-py27.fc28.x86_64/htmlcov-storage-py27/_home_jenkins_workspace_vdsm_standard-check-patch_vdsm_lib_vdsm_storage_lvm_py.html

These tests are rather slow, all the root tests take 26 seconds. But OST takes more then

40 minutes, and cover less code in this area.

OST coverage for lvm mudle: 71%

https://jenkins.ovirt.org/job/ovirt-system-tests_manual/4071/artifact/exported-artifacts/coverage/vdsm/html/_usr_lib_python2_7_site-packages_vdsm_storage_lvm_py.html

To get debug the tests, you can use new option in recent pytest --log-cli-level:

$ sudo ~/.local/bin/tox -e storage-py27 tests/storage/blocksd_test.py -- -m root --log-cli-level=info

Here is an example output from a test creating a storage domain

(use --log-cli-leve=debug if this is not verbose enough)

------------------------------------------------------------------------------- live log call -------------------------------------------------------------------------------

blockSD.py 1034 INFO sdUUID=d4d7649d-4849-4413-bdc2-b7b84f239092 domainName=loop-domain domClass=1 vgUUID=3OJX6U-UDLc-VFtg-2cRO-q3kR-UH2g-Nvf78I storageType=3 version=3, block_size=512, alignment=1048576

blockSD.py 600 INFO size 512 MB (metaratio 262144)

lvm.py 1168 INFO Creating LV (vg=d4d7649d-4849-4413-bdc2-b7b84f239092, lv=metadata, size=512m, activate=True, contiguous=False, initialTags=(), device=None)

lvm.py 1198 WARNING Could not change ownership of one or more volumes in vg (d4d7649d-4849-4413-bdc2-b7b84f239092) - metadata

blockSD.py 522 INFO Create: SORT MAPPING: ['/dev/loop2']

lvm.py 1168 INFO Creating LV (vg=d4d7649d-4849-4413-bdc2-b7b84f239092, lv=inbox, size=16m, activate=True, contiguous=False, initialTags=(), device=None)

lvm.py 1198 WARNING Could not change ownership of one or more volumes in vg (d4d7649d-4849-4413-bdc2-b7b84f239092) - inbox

lvm.py 1168 INFO Creating LV (vg=d4d7649d-4849-4413-bdc2-b7b84f239092, lv=outbox, size=16m, activate=True, contiguous=False, initialTags=(), device=None)

lvm.py 1198 WARNING Could not change ownership of one or more volumes in vg (d4d7649d-4849-4413-bdc2-b7b84f239092) - outbox

lvm.py 1168 INFO Creating LV (vg=d4d7649d-4849-4413-bdc2-b7b84f239092, lv=ids, size=8m, activate=True, contiguous=False, initialTags=(), device=None)

lvm.py 1198 WARNING Could not change ownership of one or more volumes in vg (d4d7649d-4849-4413-bdc2-b7b84f239092) - ids

lvm.py 1168 INFO Creating LV (vg=d4d7649d-4849-4413-bdc2-b7b84f239092, lv=leases, size=2048m, activate=True, contiguous=False, initialTags=(), device=None)

lvm.py 1198 WARNING Could not change ownership of one or more volumes in vg (d4d7649d-4849-4413-bdc2-b7b84f239092) - leases

lvm.py 1168 INFO Creating LV (vg=d4d7649d-4849-4413-bdc2-b7b84f239092, lv=master, size=1024m, activate=True, contiguous=False, initialTags=(), device=None)

lvm.py 1198 WARNING Could not change ownership of one or more volumes in vg (d4d7649d-4849-4413-bdc2-b7b84f239092) - master

lvm.py 1333 INFO Deactivating lvs: vg=d4d7649d-4849-4413-bdc2-b7b84f239092 lvs=['master']

blockdev.py 84 INFO Zeroing device /dev/d4d7649d-4849-4413-bdc2-b7b84f239092/metadata (size=41943040)

utils.py 454 INFO Zero device /dev/d4d7649d-4849-4413-bdc2-b7b84f239092/metadata: 0.00 seconds

blockdev.py 84 INFO Zeroing device /dev/d4d7649d-4849-4413-bdc2-b7b84f239092/inbox (size=1024000)

utils.py 454 INFO Zero device /dev/d4d7649d-4849-4413-bdc2-b7b84f239092/inbox: 0.02 seconds

blockdev.py 84 INFO Zeroing device /dev/d4d7649d-4849-4413-bdc2-b7b84f239092/outbox (size=1024000)

utils.py 454 INFO Zero device /dev/d4d7649d-4849-4413-bdc2-b7b84f239092/outbox: 0.02 seconds

lvm.py 1438 INFO Changing VG tags (vg=d4d7649d-4849-4413-bdc2-b7b84f239092, delTags=[], addTags=['MDT_LEASETIMESEC=60', 'MDT_IOOPTIMEOUTSEC=10', 'MDT_LEASERETRIES=3', 'MDT_LOCKRENEWALINTERVALSEC=5', 'MDT_SDUUID=d4d7649d-4849-4413-bdc2-b7b84f239092', 'MDT_ROLE=Regular', 'MDT_POOL_UUID=', 'MDT_PV0=pv:loop2&44&uuid:wzupcF-uQME-3PIa-4WNf-THJJ-57MS-3UIdxD&44&pestart:0&44&pecount:157&44&mapoffset:0', 'MDT_CLASS=Data', 'MDT__SHA_CKSUM=ee58868dee52c4cc128f0ee89a0c382de1fe6419', 'MDT_LOGBLKSIZE=512', 'MDT_VGUUID=3OJX6U-UDLc-VFtg-2cRO-q3kR-UH2g-Nvf78I', 'MDT_PHYBLKSIZE=512', 'MDT_DESCRIPTION=loop-domain', 'MDT_TYPE=ISCSI', 'MDT_VERSION=3', 'MDT_LOCKPOLICY='])

lvm.py 1438 INFO Changing VG tags (vg=d4d7649d-4849-4413-bdc2-b7b84f239092, delTags=['RHAT_storage_domain_UNREADY'], addTags=['RHAT_storage_domain'])

lvm.py 1325 INFO Activating lvs: vg=d4d7649d-4849-4413-bdc2-b7b84f239092 lvs=['master']

PASSED [ 50%]

Nir
_______________________________________________
Devel mailing list -- devel@ovirt.org
To unsubscribe send an email to devel-leave@ovirt.org
Privacy Statement: https://www.ovirt.org/site/privacy-policy/
oVirt Code of Conduct: https://www.ovirt.org/community/about/community-guidelines/
List Archives: https://lists.ovirt.org/archives/list/devel@ovirt.org/message/BY67R2IBWKIPX3BLTIZ4BDTFP5GXQRG7/