[Devel] Vdsm functional tests

[Douglas Schilling Landgraf]

On 04/07/2014 05:04 AM, Dan Kenigsberg wrote:
> On Sun, Apr 06, 2014 at 11:47:26PM -0400, Douglas Schilling Landgraf wrote:
>> On 04/03/2014 12:24 PM, Dan Kenigsberg wrote:
>>> On Thu, Apr 03, 2014 at 12:31:44PM -0400, Douglas Schilling Landgraf wrote:
>>>> On 04/03/2014 11:08 AM, Dan Kenigsberg wrote:
>>>>> Functional tests are intended to verify that a running Vdsm instance
>>>>> does what it should, when treated as a black box, over its public API.
>>>>>
>>>>> They should be comprehensive and representative of a typical field usage
>>>>> of Vdsm. It is a sin to break such a test - but we must be able to know
>>>>> when such a sin is committed.
>>>>>
>>>>> We currently have the following functional tests modules:
>>>>>
>>>>> - sosPluginTests.py
>>>>>    supervdsmFuncTests.py
>>>>>
>>>> Sure, count with me.
>>>
>>> Thanks! When do you think you could write a job similar to
>>> http://jenkins.ovirt.org/view/By%20Project/view/vdsm/job/vdsm_network_functional_tests/configure
>>> running whenever there's a change in the modules relevant to
>>> sosPluginTests and supervdsmFuncTests?
>>>
>>
>> Hi,
>>
>> Instead of to split by domain like (network, infra, storage), why
>> not have a single functional test job? If something fail, it should
>> trigger the volunteers.
>
> The benefit of splitting-by-domain is that if infra_functional_tests
> breaks I don't even need to check the output - I already know who is to
> blame ;-)
>
>>
>> For while, I started a creation of infra based on the above one.
>> http://jenkins.ovirt.org/job/vdsm_infra_functional_tests
>
> Thanks! But I think that it would be better to have a single run of
>      sudo NOSE_WITH_XUNIT=1 ./run_tests.sh ./functional/sosPluginTests.py ./functional/supervdsmFuncTests.py
> so that a single xml file collects all failures.
>
Done.

-- 
Cheers
Douglas