Re: [Devel] Vdsm functional tests

On Sun, Apr 06, 2014 at 11:47:26PM -0400, Douglas Schilling Landgraf wrote: > On 04/03/2014 12:24 PM, Dan Kenigsberg wrote: >> On Thu, Apr 03, 2014 at 12:31:44PM -0400, Douglas Schilling Landgraf wrote: >>> On 04/03/2014 11:08 AM, Dan Kenigsberg wrote: >>>> Functional tests are intended to verify that a running Vdsm instance >>>> does what it should, when treated as a black box, over its public API. >>>> >>>> They should be comprehensive and representative of a typical field usage >>>> of Vdsm. It is a sin to break such a test - but we must be able to know >>>> when such a sin is committed. >>>> >>>> We currently have the following functional tests modules: >>>> >>>> - sosPluginTests.py >>>> supervdsmFuncTests.py >>>> >>> Sure, count with me. >> >> Thanks! When do you think you could write a job similar to >> http://jenkins.ovirt.org/view/By%20Project/view/vdsm/job/vdsm_network_fun... >> running whenever there's a change in the modules relevant to >> sosPluginTests and supervdsmFuncTests? >> > > Hi, > > Instead of to split by domain like (network, infra, storage), why > not have a single functional test job? If something fail, it should > trigger the volunteers. The benefit of splitting-by-domain is that if infra_functional_tests breaks I don't even need to check the output - I already know who is to blame ;-) > > For while, I started a creation of infra based on the above one. > http://jenkins.ovirt.org/job/vdsm_infra_functional_tests Thanks! But I think that it would be better to have a single run of sudo NOSE_WITH_XUNIT=1 ./run_tests.sh ./functional/sosPluginTests.py ./functional/supervdsmFuncTests.py so that a single xml file collects all failures.