[node-devel] [RFC] Test Automation (cont.)

[Fabian Deutsch]

Moin,

I've written about the automation of testing about a month ago [1].
Now continuing those thoughts this time it's about how we can establish
a system to do some automated installation and run-time Node testing.

The automated testing of the installation and run-time functionality of
the Node is a big deal, because it's getting more more annoying to do
all the testing manually. Oh - and it shall also help to prevent
regressions and catch bugs in complex setups.

A flexible system is needed which can 
- test several testcases (many cases are condensed into one testsuite)
- on different hardware 
- with different setups (e.g. once with 4 individual disks, then with a
VG spanning all 4 of them, or two VG's each spanning two, and this once
on Fedora and then again on CentOS, etc.)
- such a system should self provision the required hardware and prepare 
  the required setup
The intended workflow and terms are explained in more detail here [2].

I singled out the following milestones on the path to reach the point
where an automated test is triggered by jenkins:
- A Node is able to fetch and run a given testsuite
- A controller can do the provisioning of real or virtual hosts
- The controller is extended so it can provision and run a testsuite on
  a Host
- Jenkins triggers the controller and the controller reports back

I'd like to discuss if I got all requirements right or missed some
cases, thus if there are cases which can not be covered by the outlined
system and if the ordering is to invasive or not.

Maybe you've also got some other thoughts and ideas you would like to
share your 2ct on this topic.

Thanks,
fabian

--
[1] http://lists.ovirt.org/pipermail/node-devel/2012-April/000204.html
[2] http://ovirt.org/wiki/Node_Testing#Workflow
-------------- next part --------------
A non-text attachment was scrubbed...
Name: signature.asc
Type: application/pgp-signature
Size: 836 bytes
Desc: This is a digitally signed message part
URL: <http://lists.ovirt.org/pipermail/node-devel/attachments/20120508/883072b8/attachment.sig>