8:26 p.m.
I want to share our new block storage tests, running on your laptop, from
your editor, creating
real block storage domain with real logical volumes.
One catch, these tests require root - there is no way to create devices
without root.
To make it easy to run as root only the tests that need root, they are
marked with "root" mark.
Here an example of running the root tests for block storage domain:
$ sudo ~/.local/bin/tox -e storage-py27 tests/storage/blocksd_test.py
-- -m root
And for lvm:
$ sudo ~/.local/bin/tox -e storage-py27 tests/storage/lvm_test.py -- -m
root
To run all storage tests that require root:
$ sudo ~/.local/bin/tox -e storage-py27 -- -m root tests/storage
Another issue - after running the tests as root, you need to fix ownership
of some
files in .tox/, tests/htmlcov*, and /var/tmp/vdsm. You can:
$ sudo chown -R $USER:$USER .tox tests /var/tmp/vdsm
We will improve this later.
Note that I'm running tox installed as user:
$ pip install --user tox
This gets most recent tox with minimal breakage of system python.
With the new tests, our code coverage is now 57%:
https://jenkins.ovirt.org/job/vdsm_standard-check-patch/2888/artifact/che...
- blockSD: 47%
https://jenkins.ovirt.org/job/vdsm_standard-check-patch/2888/artifact/che...
- lvm: 74%
https://jenkins.ovirt.org/job/vdsm_standard-check-patch/2888/artifact/che...
These tests are rather slow, all the root tests take 26 seconds. But OST
takes more then
40 minutes, and cover less code in this area.
OST coverage for lvm mudle: 71%
https://jenkins.ovirt.org/job/ovirt-system-tests_manual/4071/artifact/exp...
To get debug the tests, you can use new option in recent pytest
--log-cli-level:
$ sudo ~/.local/bin/tox -e storage-py27 tests/storage/blocksd_test.py
-- -m root --log-cli-level=info
Here is an example output from a test creating a storage domain
(use --log-cli-leve=debug if this is not verbose enough)
-------------------------------------------------------------------------------
live log call
-------------------------------------------------------------------------------
blockSD.py 1034 INFO
sdUUID=d4d7649d-4849-4413-bdc2-b7b84f239092 domainName=loop-domain
domClass=1 vgUUID=3OJX6U-UDLc-VFtg-2cRO-q3kR-UH2g-Nvf78I storageType=3
version=3, block_size=512, alignment=1048576
blockSD.py 600 INFO size 512 MB (metaratio 262144)
lvm.py 1168 INFO Creating LV
(vg=d4d7649d-4849-4413-bdc2-b7b84f239092, lv=metadata, size=512m,
activate=True, contiguous=False, initialTags=(), device=None)
lvm.py 1198 WARNING Could not change ownership of one
or more volumes in vg (d4d7649d-4849-4413-bdc2-b7b84f239092) - metadata
blockSD.py 522 INFO Create: SORT MAPPING: ['/dev/loop2']
lvm.py 1168 INFO Creating LV
(vg=d4d7649d-4849-4413-bdc2-b7b84f239092, lv=inbox, size=16m,
activate=True, contiguous=False, initialTags=(), device=None)
lvm.py 1198 WARNING Could not change ownership of one
or more volumes in vg (d4d7649d-4849-4413-bdc2-b7b84f239092) - inbox
lvm.py 1168 INFO Creating LV
(vg=d4d7649d-4849-4413-bdc2-b7b84f239092, lv=outbox, size=16m,
activate=True, contiguous=False, initialTags=(), device=None)
lvm.py 1198 WARNING Could not change ownership of one
or more volumes in vg (d4d7649d-4849-4413-bdc2-b7b84f239092) - outbox
lvm.py 1168 INFO Creating LV
(vg=d4d7649d-4849-4413-bdc2-b7b84f239092, lv=ids, size=8m, activate=True,
contiguous=False, initialTags=(), device=None)
lvm.py 1198 WARNING Could not change ownership of one
or more volumes in vg (d4d7649d-4849-4413-bdc2-b7b84f239092) - ids
lvm.py 1168 INFO Creating LV
(vg=d4d7649d-4849-4413-bdc2-b7b84f239092, lv=leases, size=2048m,
activate=True, contiguous=False, initialTags=(), device=None)
lvm.py 1198 WARNING Could not change ownership of one
or more volumes in vg (d4d7649d-4849-4413-bdc2-b7b84f239092) - leases
lvm.py 1168 INFO Creating LV
(vg=d4d7649d-4849-4413-bdc2-b7b84f239092, lv=master, size=1024m,
activate=True, contiguous=False, initialTags=(), device=None)
lvm.py 1198 WARNING Could not change ownership of one
or more volumes in vg (d4d7649d-4849-4413-bdc2-b7b84f239092) - master
lvm.py 1333 INFO Deactivating lvs:
vg=d4d7649d-4849-4413-bdc2-b7b84f239092 lvs=['master']
blockdev.py 84 INFO Zeroing device
/dev/d4d7649d-4849-4413-bdc2-b7b84f239092/metadata (size=41943040)
utils.py 454 INFO Zero device
/dev/d4d7649d-4849-4413-bdc2-b7b84f239092/metadata: 0.00 seconds
blockdev.py 84 INFO Zeroing device
/dev/d4d7649d-4849-4413-bdc2-b7b84f239092/inbox (size=1024000)
utils.py 454 INFO Zero device
/dev/d4d7649d-4849-4413-bdc2-b7b84f239092/inbox: 0.02 seconds
blockdev.py 84 INFO Zeroing device
/dev/d4d7649d-4849-4413-bdc2-b7b84f239092/outbox (size=1024000)
utils.py 454 INFO Zero device
/dev/d4d7649d-4849-4413-bdc2-b7b84f239092/outbox: 0.02 seconds
lvm.py 1438 INFO Changing VG tags
(vg=d4d7649d-4849-4413-bdc2-b7b84f239092, delTags=[],
addTags=['MDT_LEASETIMESEC=60', 'MDT_IOOPTIMEOUTSEC=10',
'MDT_LEASERETRIES=3', 'MDT_LOCKRENEWALINTERVALSEC=5',
'MDT_SDUUID=d4d7649d-4849-4413-bdc2-b7b84f239092', 'MDT_ROLE=Regular',
'MDT_POOL_UUID=',
'MDT_PV0=pv:loop2&44&uuid:wzupcF-uQME-3PIa-4WNf-THJJ-57MS-3UIdxD&44&pestart:0&44&pecount:157&44&mapoffset:0',
'MDT_CLASS=Data',
'MDT__SHA_CKSUM=ee58868dee52c4cc128f0ee89a0c382de1fe6419',
'MDT_LOGBLKSIZE=512',
'MDT_VGUUID=3OJX6U-UDLc-VFtg-2cRO-q3kR-UH2g-Nvf78I',
'MDT_PHYBLKSIZE=512', 'MDT_DESCRIPTION=loop-domain',
'MDT_TYPE=ISCSI',
'MDT_VERSION=3', 'MDT_LOCKPOLICY='])
lvm.py 1438 INFO Changing VG tags
(vg=d4d7649d-4849-4413-bdc2-b7b84f239092,
delTags=['RHAT_storage_domain_UNREADY'], addTags=['RHAT_storage_domain'])
lvm.py 1325 INFO Activating lvs:
vg=d4d7649d-4849-4413-bdc2-b7b84f239092 lvs=['master']
PASSED
[ 50%]
Nir
11:36 p.m.
Hello Nir,
Thanks for the sharing, it is very helpful!
Are these test scripts open sourced and can be re-used by others for
testing oVirt?
Thanks
Jonkery
On 2/14/2019 6:26 AM, Nir Soffer wrote:
I want to share our new block storage tests, running on your laptop,
from your editor, creating
real block storage domain with real logical volumes.
One catch, these tests require root - there is no way to create
devices without root.
To make it easy to run as root only the tests that need root, they are
marked with "root" mark.
Here an example of running the root tests for block storage domain:
$ sudo ~/.local/bin/tox -e storage-py27
tests/storage/blocksd_test.py -- -m root
And for lvm:
$ sudo ~/.local/bin/tox -e storage-py27 tests/storage/lvm_test.py
-- -m root
To run all storage tests that require root:
$ sudo ~/.local/bin/tox -e storage-py27 -- -m root tests/storage
Another issue - after running the tests as root, you need to fix
ownership of some
files in .tox/, tests/htmlcov*, and /var/tmp/vdsm. You can:
$ sudo chown -R $USER:$USER .tox tests /var/tmp/vdsm
We will improve this later.
Note that I'm running tox installed as user:
$ pip install --user tox
This gets most recent tox with minimal breakage of system python.
With the new tests, our code coverage is now 57%:
https://jenkins.ovirt.org/job/vdsm_standard-check-patch/2888/artifact/che...
- blockSD: 47%
https://jenkins.ovirt.org/job/vdsm_standard-check-patch/2888/artifact/che...
- lvm: 74%
https://jenkins.ovirt.org/job/vdsm_standard-check-patch/2888/artifact/che...
These tests are rather slow, all the root tests take 26 seconds. But
OST takes more then
40 minutes, and cover less code in this area.
OST coverage for lvm mudle: 71%
https://jenkins.ovirt.org/job/ovirt-system-tests_manual/4071/artifact/exp...
To get debug the tests, you can use new option in recent pytest
--log-cli-level:
$ sudo ~/.local/bin/tox -e storage-py27
tests/storage/blocksd_test.py -- -m root --log-cli-level=info
Here is an example output from a test creating a storage domain
(use --log-cli-leve=debug if this is not verbose enough)
-------------------------------------------------------------------------------
live log call
-------------------------------------------------------------------------------
blockSD.py 1034 INFO
sdUUID=d4d7649d-4849-4413-bdc2-b7b84f239092 domainName=loop-domain
domClass=1 vgUUID=3OJX6U-UDLc-VFtg-2cRO-q3kR-UH2g-Nvf78I storageType=3
version=3, block_size=512, alignment=1048576
blockSD.py 600 INFO size 512 MB (metaratio 262144)
lvm.py 1168 INFO Creating LV
(vg=d4d7649d-4849-4413-bdc2-b7b84f239092, lv=metadata, size=512m,
activate=True, contiguous=False, initialTags=(), device=None)
lvm.py 1198 WARNING Could not change ownership of
one or more volumes in vg (d4d7649d-4849-4413-bdc2-b7b84f239092) -
metadata
blockSD.py 522 INFO Create: SORT MAPPING: ['/dev/loop2']
lvm.py 1168 INFO Creating LV
(vg=d4d7649d-4849-4413-bdc2-b7b84f239092, lv=inbox, size=16m,
activate=True, contiguous=False, initialTags=(), device=None)
lvm.py 1198 WARNING Could not change ownership of
one or more volumes in vg (d4d7649d-4849-4413-bdc2-b7b84f239092) - inbox
lvm.py 1168 INFO Creating LV
(vg=d4d7649d-4849-4413-bdc2-b7b84f239092, lv=outbox, size=16m,
activate=True, contiguous=False, initialTags=(), device=None)
lvm.py 1198 WARNING Could not change ownership of
one or more volumes in vg (d4d7649d-4849-4413-bdc2-b7b84f239092) - outbox
lvm.py 1168 INFO Creating LV
(vg=d4d7649d-4849-4413-bdc2-b7b84f239092, lv=ids, size=8m,
activate=True, contiguous=False, initialTags=(), device=None)
lvm.py 1198 WARNING Could not change ownership of
one or more volumes in vg (d4d7649d-4849-4413-bdc2-b7b84f239092) - ids
lvm.py 1168 INFO Creating LV
(vg=d4d7649d-4849-4413-bdc2-b7b84f239092, lv=leases, size=2048m,
activate=True, contiguous=False, initialTags=(), device=None)
lvm.py 1198 WARNING Could not change ownership of
one or more volumes in vg (d4d7649d-4849-4413-bdc2-b7b84f239092) - leases
lvm.py 1168 INFO Creating LV
(vg=d4d7649d-4849-4413-bdc2-b7b84f239092, lv=master, size=1024m,
activate=True, contiguous=False, initialTags=(), device=None)
lvm.py 1198 WARNING Could not change ownership of
one or more volumes in vg (d4d7649d-4849-4413-bdc2-b7b84f239092) - master
lvm.py 1333 INFO Deactivating lvs:
vg=d4d7649d-4849-4413-bdc2-b7b84f239092 lvs=['master']
blockdev.py 84 INFO Zeroing device
/dev/d4d7649d-4849-4413-bdc2-b7b84f239092/metadata (size=41943040)
utils.py 454 INFO Zero device
/dev/d4d7649d-4849-4413-bdc2-b7b84f239092/metadata: 0.00 seconds
blockdev.py 84 INFO Zeroing device
/dev/d4d7649d-4849-4413-bdc2-b7b84f239092/inbox (size=1024000)
utils.py 454 INFO Zero device
/dev/d4d7649d-4849-4413-bdc2-b7b84f239092/inbox: 0.02 seconds
blockdev.py 84 INFO Zeroing device
/dev/d4d7649d-4849-4413-bdc2-b7b84f239092/outbox (size=1024000)
utils.py 454 INFO Zero device
/dev/d4d7649d-4849-4413-bdc2-b7b84f239092/outbox: 0.02 seconds
lvm.py 1438 INFO Changing VG tags
(vg=d4d7649d-4849-4413-bdc2-b7b84f239092, delTags=[],
addTags=['MDT_LEASETIMESEC=60', 'MDT_IOOPTIMEOUTSEC=10',
'MDT_LEASERETRIES=3', 'MDT_LOCKRENEWALINTERVALSEC=5',
'MDT_SDUUID=d4d7649d-4849-4413-bdc2-b7b84f239092', 'MDT_ROLE=Regular',
'MDT_POOL_UUID=',
'MDT_PV0=pv:loop2&44&uuid:wzupcF-uQME-3PIa-4WNf-THJJ-57MS-3UIdxD&44&pestart:0&44&pecount:157&44&mapoffset:0',
'MDT_CLASS=Data',
'MDT__SHA_CKSUM=ee58868dee52c4cc128f0ee89a0c382de1fe6419',
'MDT_LOGBLKSIZE=512',
'MDT_VGUUID=3OJX6U-UDLc-VFtg-2cRO-q3kR-UH2g-Nvf78I',
'MDT_PHYBLKSIZE=512', 'MDT_DESCRIPTION=loop-domain',
'MDT_TYPE=ISCSI',
'MDT_VERSION=3', 'MDT_LOCKPOLICY='])
lvm.py 1438 INFO Changing VG tags
(vg=d4d7649d-4849-4413-bdc2-b7b84f239092,
delTags=['RHAT_storage_domain_UNREADY'],
addTags=['RHAT_storage_domain'])
lvm.py 1325 INFO Activating lvs:
vg=d4d7649d-4849-4413-bdc2-b7b84f239092 lvs=['master']
PASSED [ 50%]
Nir
_______________________________________________
Devel mailing list -- devel(a)ovirt.org
To unsubscribe send an email to devel-leave(a)ovirt.org
Privacy Statement: https://www.ovirt.org/site/privacy-policy/
oVirt Code of Conduct: https://www.ovirt.org/community/about/community-guidelines/
List Archives:
https://lists.ovirt.org/archives/list/devel@ovirt.org/message/BY67R2IBWKI...
12:20 a.m.
On Fri, Feb 15, 2019, 16:36 Jonkery Huang <jonkery.huang(a)oracle.com wrote:
Hello Nir,
Thanks for the sharing, it is very helpful!
Are these test scripts open sourced and can be re-used by others for
testing oVirt?
Sure, everything we do is open source.
These tests are part of vdsm, available here:
https://github.com/ovirt/vdsm
Example tests:
https://github.com/oVirt/vdsm/blob/master/tests/storage/blocksd_test.py
https://github.com/oVirt/vdsm/blob/master/tests/storage/lvm_test.py
The pytest infrastructure:
https://github.com/oVirt/vdsm/blob/master/tests/storage/conftest.py
Nir
Thanks
Jonkery
On 2/14/2019 6:26 AM, Nir Soffer wrote:
I want to share our new block storage tests, running on your laptop, from
your editor, creating
real block storage domain with real logical volumes.
One catch, these tests require root - there is no way to create devices
without root.
To make it easy to run as root only the tests that need root, they are
marked with "root" mark.
Here an example of running the root tests for block storage domain:
$ sudo ~/.local/bin/tox -e storage-py27 tests/storage/blocksd_test.py
-- -m root
And for lvm:
$ sudo ~/.local/bin/tox -e storage-py27 tests/storage/lvm_test.py --
-m root
To run all storage tests that require root:
$ sudo ~/.local/bin/tox -e storage-py27 -- -m root tests/storage
Another issue - after running the tests as root, you need to fix ownership
of some
files in .tox/, tests/htmlcov*, and /var/tmp/vdsm. You can:
$ sudo chown -R $USER:$USER .tox tests /var/tmp/vdsm
We will improve this later.
Note that I'm running tox installed as user:
$ pip install --user tox
This gets most recent tox with minimal breakage of system python.
With the new tests, our code coverage is now 57%:
https://jenkins.ovirt.org/job/vdsm_standard-check-patch/2888/artifact/che...
- blockSD: 47%
https://jenkins.ovirt.org/job/vdsm_standard-check-patch/2888/artifact/che...
- lvm: 74%
https://jenkins.ovirt.org/job/vdsm_standard-check-patch/2888/artifact/che...
These tests are rather slow, all the root tests take 26 seconds. But OST
takes more then
40 minutes, and cover less code in this area.
OST coverage for lvm mudle: 71%
https://jenkins.ovirt.org/job/ovirt-system-tests_manual/4071/artifact/exp...
To get debug the tests, you can use new option in recent pytest
--log-cli-level:
$ sudo ~/.local/bin/tox -e storage-py27 tests/storage/blocksd_test.py
-- -m root --log-cli-level=info
Here is an example output from a test creating a storage domain
(use --log-cli-leve=debug if this is not verbose enough)
-------------------------------------------------------------------------------
live log call
-------------------------------------------------------------------------------
blockSD.py 1034 INFO
sdUUID=d4d7649d-4849-4413-bdc2-b7b84f239092 domainName=loop-domain
domClass=1 vgUUID=3OJX6U-UDLc-VFtg-2cRO-q3kR-UH2g-Nvf78I storageType=3
version=3, block_size=512, alignment=1048576
blockSD.py 600 INFO size 512 MB (metaratio 262144)
lvm.py 1168 INFO Creating LV
(vg=d4d7649d-4849-4413-bdc2-b7b84f239092, lv=metadata, size=512m,
activate=True, contiguous=False, initialTags=(), device=None)
lvm.py 1198 WARNING Could not change ownership of one
or more volumes in vg (d4d7649d-4849-4413-bdc2-b7b84f239092) - metadata
blockSD.py 522 INFO Create: SORT MAPPING:
['/dev/loop2']
lvm.py 1168 INFO Creating LV
(vg=d4d7649d-4849-4413-bdc2-b7b84f239092, lv=inbox, size=16m,
activate=True, contiguous=False, initialTags=(), device=None)
lvm.py 1198 WARNING Could not change ownership of one
or more volumes in vg (d4d7649d-4849-4413-bdc2-b7b84f239092) - inbox
lvm.py 1168 INFO Creating LV
(vg=d4d7649d-4849-4413-bdc2-b7b84f239092, lv=outbox, size=16m,
activate=True, contiguous=False, initialTags=(), device=None)
lvm.py 1198 WARNING Could not change ownership of one
or more volumes in vg (d4d7649d-4849-4413-bdc2-b7b84f239092) - outbox
lvm.py 1168 INFO Creating LV
(vg=d4d7649d-4849-4413-bdc2-b7b84f239092, lv=ids, size=8m, activate=True,
contiguous=False, initialTags=(), device=None)
lvm.py 1198 WARNING Could not change ownership of one
or more volumes in vg (d4d7649d-4849-4413-bdc2-b7b84f239092) - ids
lvm.py 1168 INFO Creating LV
(vg=d4d7649d-4849-4413-bdc2-b7b84f239092, lv=leases, size=2048m,
activate=True, contiguous=False, initialTags=(), device=None)
lvm.py 1198 WARNING Could not change ownership of one
or more volumes in vg (d4d7649d-4849-4413-bdc2-b7b84f239092) - leases
lvm.py 1168 INFO Creating LV
(vg=d4d7649d-4849-4413-bdc2-b7b84f239092, lv=master, size=1024m,
activate=True, contiguous=False, initialTags=(), device=None)
lvm.py 1198 WARNING Could not change ownership of one
or more volumes in vg (d4d7649d-4849-4413-bdc2-b7b84f239092) - master
lvm.py 1333 INFO Deactivating lvs:
vg=d4d7649d-4849-4413-bdc2-b7b84f239092 lvs=['master']
blockdev.py 84 INFO Zeroing device
/dev/d4d7649d-4849-4413-bdc2-b7b84f239092/metadata (size=41943040)
utils.py 454 INFO Zero device
/dev/d4d7649d-4849-4413-bdc2-b7b84f239092/metadata: 0.00 seconds
blockdev.py 84 INFO Zeroing device
/dev/d4d7649d-4849-4413-bdc2-b7b84f239092/inbox (size=1024000)
utils.py 454 INFO Zero device
/dev/d4d7649d-4849-4413-bdc2-b7b84f239092/inbox: 0.02 seconds
blockdev.py 84 INFO Zeroing device
/dev/d4d7649d-4849-4413-bdc2-b7b84f239092/outbox (size=1024000)
utils.py 454 INFO Zero device
/dev/d4d7649d-4849-4413-bdc2-b7b84f239092/outbox: 0.02 seconds
lvm.py 1438 INFO Changing VG tags
(vg=d4d7649d-4849-4413-bdc2-b7b84f239092, delTags=[],
addTags=['MDT_LEASETIMESEC=60', 'MDT_IOOPTIMEOUTSEC=10',
'MDT_LEASERETRIES=3', 'MDT_LOCKRENEWALINTERVALSEC=5',
'MDT_SDUUID=d4d7649d-4849-4413-bdc2-b7b84f239092', 'MDT_ROLE=Regular',
'MDT_POOL_UUID=',
'MDT_PV0=pv:loop2&44&uuid:wzupcF-uQME-3PIa-4WNf-THJJ-57MS-3UIdxD&44&pestart:0&44&pecount:157&44&mapoffset:0',
'MDT_CLASS=Data',
'MDT__SHA_CKSUM=ee58868dee52c4cc128f0ee89a0c382de1fe6419',
'MDT_LOGBLKSIZE=512',
'MDT_VGUUID=3OJX6U-UDLc-VFtg-2cRO-q3kR-UH2g-Nvf78I',
'MDT_PHYBLKSIZE=512', 'MDT_DESCRIPTION=loop-domain',
'MDT_TYPE=ISCSI',
'MDT_VERSION=3', 'MDT_LOCKPOLICY='])
lvm.py 1438 INFO Changing VG tags
(vg=d4d7649d-4849-4413-bdc2-b7b84f239092,
delTags=['RHAT_storage_domain_UNREADY'],
addTags=['RHAT_storage_domain'])
lvm.py 1325 INFO Activating lvs:
vg=d4d7649d-4849-4413-bdc2-b7b84f239092 lvs=['master']
PASSED
[ 50%]
Nir
_______________________________________________
Devel mailing list -- devel(a)ovirt.org
To unsubscribe send an email to devel-leave(a)ovirt.org
Privacy Statement: https://www.ovirt.org/site/privacy-policy/
oVirt Code of Conduct: https://www.ovirt.org/community/about/community-guidelines/
List Archives:
https://lists.ovirt.org/archives/list/devel@ovirt.org/message/BY67R2IBWKI...
1911
days inactive
1912
days old
2 comments
2 participants
participants (2)
-
Jonkery Huang -
Nir Soffer