Il 5 mar 2018 2:02 PM, "Yedidyah Bar David" <didi@redhat.com> ha scritto: Hi all, Recently I have been pushing various patches to OST in order to verify specific bugs/fixes I was working on, using them with the "manual" jenkins job but with no immediate intention to get them merged. Main reason is that it's not clear what's the best approach to get such tests merged. Do we want a new suite? How often does it run? How long does it take? Do we have enough resources? etc. Do we have plans re this? Bugs/tickets/etc.? Do we want to do something? Specifically, do we want (eventually) many more suites? This seems unmanageable. A few suites, perhaps (also) using lago snapshots? Didn't try that myself, might be useful and relevant. Didn't check since I am on mobile phone but if all the tests are regression tests I think itv would be useful run the test daily. AFAIK we don't have tests running per patch on hosted engine commits so it should be ok adding them to existing test suite Some examples: https://gerrit.ovirt.org/79203 https://gerrit.ovirt.org/79215 https://gerrit.ovirt.org/84813 https://gerrit.ovirt.org/88201 https://gerrit.ovirt.org/88234 And current one - this and the next ones in the stack: https://gerrit.ovirt.org/88331 Best regards, -- Didi _______________________________________________ Infra mailing list Infra@ovirt.org http://lists.ovirt.org/mailman/listinfo/infra

On Mon, Mar 5, 2018 at 3:01 PM, Yedidyah Bar David <didi@redhat.com> wrote:

Hi all,

Recently I have been pushing various patches to OST in order to verify specific bugs/fixes I was working on, using them with the "manual" jenkins job but with no immediate intention to get them merged. Main reason is that it's not clear what's the best approach to get such tests merged. Do we want a new suite? How often does it run? How long does it take? Do we have enough resources? etc.

I'll answer generally, since specifically for each test we'll need to see. The guidelines that I have (in my head, not written down anywhere) are really: 1. Be efficient in resource - HW mainly. My guideline is quite simple - can it run on my laptop (8GB of RAM). Not everything can fit of course on my laptop (the performance suite, the upcoming OpenShift on oVirt suite, etc.), but I try. 2. Be quick - whatever we can test in parallel to other tests, is much preferred. We have quite a bit of 'dead time' between tests, we need to use them. 3. Bend the rules, but don't cheat - I move the yum cache repo to /dev/shm, etc. - to make things quicker, but I don't cheat and install deps. ahead of time, etc. 4. Most suites do not run too often (several times a day) - I think it's OK to add to those that run once a day or so. 5. Strive for others (QE!) to contribute to the suite. The more we can collaborate, the better. 6. Generally, we have enough gaps in our positive tests, that I rather not introduce negative tests. 7. Whatever we can do in order to ensure QE does not get a dead-on-arrival or broken functionality build - the better.

Do we have plans re this? Bugs/tickets/etc.?

Bugzilla, but I did not see anything there for quite some time.

Do we want to do something?

Specifically, do we want (eventually) many more suites? This seems unmanageable. A few suites, perhaps (also) using lago snapshots? Didn't try that myself, might be useful and relevant.

Some examples: https://gerrit.ovirt.org/79203 https://gerrit.ovirt.org/79215 https://gerrit.ovirt.org/84813 https://gerrit.ovirt.org/88201 https://gerrit.ovirt.org/88234

And current one - this and the next ones in the stack: https://gerrit.ovirt.org/88331

Many of those change the default installation and test the 'interesting' combinations - which I think is great. I'd be happy for a setup with custom '3rd party' certificate, Kerberos auth., ovirtmgmt on a VLAN interface, etc. Y.

Best regards, -- Didi _______________________________________________ Devel mailing list Devel@ovirt.org http://lists.ovirt.org/mailman/listinfo/devel