oVirt cluster level test proposal

[Mark Wu]

On Fri 05 Apr 2013 12:21:44 AM CST, Fabian Deutsch wrote:
> Am Montag, den 25.03.2013, 18:06 +0800 schrieb Mark Wu:
>> Hi guys,
>>
>> As per the discussion before, we don't have  integrated functional tests
>> for oVirt.  So I would like to propose a cluster level test plan.
>> Basically, it's composed of igord, cobbler and test cases based on
>> engine REST api. Cobbler stores kickstart files, installation images and
>> test packages repo.
>> Igor is responsible to setup test environment according to the test plan
>> and run the test cases inside test host.  I write down a wiki page to
>> describe the work flow:
>> http://www.ovirt.org/Ovirt_cluster_level_test
>>
>> @Fabian,  according to my investigation on igor,  it should be not
>> difficult to enhance igor to support this proposal.  I would like to get
>> your opinion on it.
>
> Hey Mark,
Fabian, thanks for your good comments. Please see my comments inline.
>
> nice to see this overall proposal which covers all components of oVirt.
> I've looked at your proposal and basically agree and think that it
> should work.
> There are a couple of things which ain't completely clear to me or what
> I'd like to comment:
>
> * It should be possible that Igor provisions a host with RHEL, Fedora or
> CentOS - but I haven't tried it - I suppose there is some work to do to
> get this working.
Yes, exactly.  Actually,  my proposal is extending or enhancing igor to 
cover all components of oVirt.
>
> * Do you want to setup both hosts from scratch at each test (Engine Host
> and Node (or VDSM enabled OS))?
Nope, I would like setup new VM based on existing template (using the 
qcow2 image).
For host,  just re-install related rpm packages. But for oVirt node, 
probably we have to re-install the
host in very test.
>
> * The Engine side host will also need to pull in the igor-client (which
> communicates with the igor daemon, the daemon is not sshing into the
> client [or host under test]).
Yes. If we use igor-client to pull tests in engine hosts, we also need 
ship the package and start
the daemon on boot.  I think running remote commands via ssh can cover 
this part. Or we could
run the igor-client by ssh if necessary.
>
> * We need to decide when an integration test shall be run.
> I'd suggest to run any beta or release candidate against each other.
>
Or weekly or biweekly to find the problem early.
> * A couple of notes on the chart:
>    - koan is not used.
>    - igor doesn't handle the {kickstarts,RPMS} itself
>
Yes, I know currently igor doesn't handle that.  But would you like to 
extend igor to support it?
Or we should leave it to other components.

> * Don't underestimate the work to write the testcases, testsuites and
> plans :)
>
> I nice first step would be to write an igor testplan which sets up Node
> and Engine - maybe it even registers the Node.
> This 'simple' testsuite will force us to prepare the ground for more
> complex testsuites.
Good point!  I going  to start with  this simple testsuite you 
mentioned.
> I also strongly recommend to do he development with VMs instead of real
> hardware.
Yes, exactly!  Unfortunately  in my experience, nested kvm is not 
stable enough.  But I think we need
use it more to push it towards stable.
>
> Greetings
> fabian
>