On Mon, Mar 5, 2018 at 3:01 PM, Yedidyah Bar David <didi@redhat.com> wrote:

Hi all,

Recently I have been pushing various patches to OST in order to verify
specific bugs/fixes I was working on, using them with the "manual"
jenkins job but with no immediate intention to get them merged. Main
reason is that it's not clear what's the best approach to get such
tests merged. Do we want a new suite? How often does it run? How long
does it take? Do we have enough resources? etc.

I'll answer generally, since specifically for each test we'll need to see.

The guidelines that I have (in my head, not written down anywhere) are really:

1. Be efficient in resource - HW mainly. My guideline is quite simple - can it run on my laptop (8GB of RAM).

Not everything can fit of course on my laptop (the performance suite, the upcoming OpenShift on oVirt suite, etc.), but I try.

2. Be quick - whatever we can test in parallel to other tests, is much preferred. We have quite a bit of 'dead time' between tests, we need to use them.

3. Bend the rules, but don't cheat - I move the yum cache repo to /dev/shm, etc. - to make things quicker, but I don't cheat and install deps. ahead of time, etc.

4. Most suites do not run too often (several times a day) - I think it's OK to add to those that run once a day or so.

5. Strive for others (QE!) to contribute to the suite. The more we can collaborate, the better.

6. Generally, we have enough gaps in our positive tests, that I rather not introduce negative tests.

7. Whatever we can do in order to ensure QE does not get a dead-on-arrival or broken functionality build - the better.

Do we have plans re this? Bugs/tickets/etc.?

Bugzilla, but I did not see anything there for quite some time.

Do we want to do something?

Specifically, do we want (eventually) many more suites? This seems
unmanageable. A few suites, perhaps (also) using lago snapshots?
Didn't try that myself, might be useful and relevant.

Some examples:
https://gerrit.ovirt.org/79203
https://gerrit.ovirt.org/79215
https://gerrit.ovirt.org/84813
https://gerrit.ovirt.org/88201
https://gerrit.ovirt.org/88234

And current one - this and the next ones in the stack:
https://gerrit.ovirt.org/88331

Many of those change the default installation and test the 'interesting' combinations - which I think is great.

I'd be happy for a setup with custom '3rd party' certificate, Kerberos auth., ovirtmgmt on a VLAN interface, etc.

Best regards,
--
Didi
_______________________________________________
Devel mailing list
Devel@ovirt.org
http://lists.ovirt.org/mailman/listinfo/devel