[Devel] Vdsm functional tests

[Dan Kenigsberg]

On Sun, Apr 06, 2014 at 11:47:26PM -0400, Douglas Schilling Landgraf wrote:
> On 04/03/2014 12:24 PM, Dan Kenigsberg wrote:
> >On Thu, Apr 03, 2014 at 12:31:44PM -0400, Douglas Schilling Landgraf wrote:
> >>On 04/03/2014 11:08 AM, Dan Kenigsberg wrote:
> >>>Functional tests are intended to verify that a running Vdsm instance
> >>>does what it should, when treated as a black box, over its public API.
> >>>
> >>>They should be comprehensive and representative of a typical field usage
> >>>of Vdsm. It is a sin to break such a test - but we must be able to know
> >>>when such a sin is committed.
> >>>
> >>>We currently have the following functional tests modules:
> >>>
> >>>- sosPluginTests.py
> >>>   supervdsmFuncTests.py
> >>>
> >>Sure, count with me.
> >
> >Thanks! When do you think you could write a job similar to
> >http://jenkins.ovirt.org/view/By%20Project/view/vdsm/job/vdsm_network_functional_tests/configure
> >running whenever there's a change in the modules relevant to
> >sosPluginTests and supervdsmFuncTests?
> >
> 
> Hi,
> 
> Instead of to split by domain like (network, infra, storage), why
> not have a single functional test job? If something fail, it should
> trigger the volunteers.

The benefit of splitting-by-domain is that if infra_functional_tests
breaks I don't even need to check the output - I already know who is to
blame ;-)

> 
> For while, I started a creation of infra based on the above one.
> http://jenkins.ovirt.org/job/vdsm_infra_functional_tests

Thanks! But I think that it would be better to have a single run of
    sudo NOSE_WITH_XUNIT=1 ./run_tests.sh ./functional/sosPluginTests.py ./functional/supervdsmFuncTests.py
so that a single xml file collects all failures.