Re: [Devel] Vdsm functional tests
On 04/03/2014 12:24 PM, Dan Kenigsberg wrote:
On Thu, Apr 03, 2014 at 12:31:44PM -0400, Douglas Schilling Landgraf wrote:
On 04/03/2014 11:08 AM, Dan Kenigsberg wrote:
Functional tests are intended to verify that a running Vdsm instance does what it should, when treated as a black box, over its public API.
They should be comprehensive and representative of a typical field usage of Vdsm. It is a sin to break such a test - but we must be able to know when such a sin is committed.
We currently have the following functional tests modules:
- sosPluginTests.py supervdsmFuncTests.py
Sure, count with me.
Thanks! When do you think you could write a job similar to http://jenkins.ovirt.org/view/By%20Project/view/vdsm/job/vdsm_network_functi... running whenever there's a change in the modules relevant to sosPluginTests and supervdsmFuncTests?
Hi, Instead of to split by domain like (network, infra, storage), why not have a single functional test job? If something fail, it should trigger the volunteers. For while, I started a creation of infra based on the above one. http://jenkins.ovirt.org/job/vdsm_infra_functional_tests -- Cheers Douglas
On Sun, Apr 06, 2014 at 11:47:26PM -0400, Douglas Schilling Landgraf wrote:
On 04/03/2014 12:24 PM, Dan Kenigsberg wrote:
On Thu, Apr 03, 2014 at 12:31:44PM -0400, Douglas Schilling Landgraf wrote:
On 04/03/2014 11:08 AM, Dan Kenigsberg wrote:
Functional tests are intended to verify that a running Vdsm instance does what it should, when treated as a black box, over its public API.
They should be comprehensive and representative of a typical field usage of Vdsm. It is a sin to break such a test - but we must be able to know when such a sin is committed.
We currently have the following functional tests modules:
- sosPluginTests.py supervdsmFuncTests.py
Sure, count with me.
Thanks! When do you think you could write a job similar to http://jenkins.ovirt.org/view/By%20Project/view/vdsm/job/vdsm_network_functi... running whenever there's a change in the modules relevant to sosPluginTests and supervdsmFuncTests?
Hi,
Instead of to split by domain like (network, infra, storage), why not have a single functional test job? If something fail, it should trigger the volunteers.
The benefit of splitting-by-domain is that if infra_functional_tests breaks I don't even need to check the output - I already know who is to blame ;-)
For while, I started a creation of infra based on the above one. http://jenkins.ovirt.org/job/vdsm_infra_functional_tests
Thanks! But I think that it would be better to have a single run of sudo NOSE_WITH_XUNIT=1 ./run_tests.sh ./functional/sosPluginTests.py ./functional/supervdsmFuncTests.py so that a single xml file collects all failures.
On 04/07/2014 05:04 AM, Dan Kenigsberg wrote:
On Sun, Apr 06, 2014 at 11:47:26PM -0400, Douglas Schilling Landgraf wrote:
On 04/03/2014 12:24 PM, Dan Kenigsberg wrote:
On Thu, Apr 03, 2014 at 12:31:44PM -0400, Douglas Schilling Landgraf wrote:
On 04/03/2014 11:08 AM, Dan Kenigsberg wrote:
Functional tests are intended to verify that a running Vdsm instance does what it should, when treated as a black box, over its public API.
They should be comprehensive and representative of a typical field usage of Vdsm. It is a sin to break such a test - but we must be able to know when such a sin is committed.
We currently have the following functional tests modules:
- sosPluginTests.py supervdsmFuncTests.py
Sure, count with me.
Thanks! When do you think you could write a job similar to http://jenkins.ovirt.org/view/By%20Project/view/vdsm/job/vdsm_network_functi... running whenever there's a change in the modules relevant to sosPluginTests and supervdsmFuncTests?
Hi,
Instead of to split by domain like (network, infra, storage), why not have a single functional test job? If something fail, it should trigger the volunteers.
The benefit of splitting-by-domain is that if infra_functional_tests breaks I don't even need to check the output - I already know who is to blame ;-)
For while, I started a creation of infra based on the above one. http://jenkins.ovirt.org/job/vdsm_infra_functional_tests
Thanks! But I think that it would be better to have a single run of sudo NOSE_WITH_XUNIT=1 ./run_tests.sh ./functional/sosPluginTests.py ./functional/supervdsmFuncTests.py so that a single xml file collects all failures.
Done. -- Cheers Douglas
participants (2)
-
Dan Kenigsberg -
Douglas Schilling Landgraf