[ovirt-devel] [ovirt-users] ovirt-system-tests hackathon report

Tue Mar 13 18:00:05 UTC 2018

On Mar 13, 2018 6:27 PM, "Sandro Bonazzola" <sbonazzo at redhat.com> wrote:

4 people accepted calendar invite:
- Devin A. Bougie
- Francesco Romani
- Jiri Belka
- suporte, logicworks

4 people tentatively accepted calendar invite:
- Amnon Maimon
- Andreas Bleischwitz
- Arnaud Lauriou
- Stephen Pesini

2 mailing lists accepted calendar invite: users at ovirt.org, devel at ovirt.org
(don't ask me how) so I may have missed someone in above list

4 patches got merged:
Add check for host update to the 1st host. <https://gerrit.ovirt.org/88767>
Merged Yaniv Kaul
<https://gerrit.ovirt.org/#/q/owner:ykaul%2540redhat.com+status:merged>
ovirt-system-tests
<https://gerrit.ovirt.org/#/projects/ovirt-system-tests,dashboards/default>
master
(add_upgrade_check)
<https://gerrit.ovirt.org/#/q/status:merged+project:ovirt-system-tests+branch:master+topic:add_upgrade_check>
4:10
PM
basic-suite-master: add vnic_profile_mappings to register vm
<https://gerrit.ovirt.org/87438> Merged Eitan Raviv
<https://gerrit.ovirt.org/#/q/owner:eraviv%2540redhat.com+status:merged>
ovirt-system-tests
<https://gerrit.ovirt.org/#/projects/ovirt-system-tests,dashboards/default>
master
(register-template-vnic-mapping)
<https://gerrit.ovirt.org/#/q/status:merged+project:ovirt-system-tests+branch:master+topic:register-template-vnic-mapping>
2:50
PM
Revert "ovirt-4.2: Skipping 002_bootstrap.update_default_cluster"
<https://gerrit.ovirt.org/88881> Merged Eyal Edri
<https://gerrit.ovirt.org/#/q/owner:eedri%2540redhat.com+status:merged>
ovirt-system-tests
<https://gerrit.ovirt.org/#/projects/ovirt-system-tests,dashboards/default>
master
<https://gerrit.ovirt.org/#/q/status:merged+project:ovirt-system-tests+branch:master>
11:36
AM
seperate 4.2 tests and utils from master <https://gerrit.ovirt.org/88878>
Merged Eyal Edri
<https://gerrit.ovirt.org/#/q/owner:eedri%2540redhat.com+status:merged>
ovirt-system-tests
<https://gerrit.ovirt.org/#/projects/ovirt-system-tests,dashboards/default>
master
<https://gerrit.ovirt.org/#/q/status:merged+project:ovirt-system-tests+branch:master>
11:35
AM

13 patches has been pushed / reviewed / rebased

Add gdeploy to ovirt-4.2.repo <https://gerrit.ovirt.org/88929>
Daniel Belenky
<https://gerrit.ovirt.org/#/q/owner:dbelenky%2540redhat.com+status:open>
ovirt-system-tests
<https://gerrit.ovirt.org/#/projects/ovirt-system-tests,dashboards/default>
master
<https://gerrit.ovirt.org/#/q/status:open+project:ovirt-system-tests+branch:master>
4:53
PM
Cleanup of test code - next() replaced with any()
<https://gerrit.ovirt.org/88928>
Martin Sivák
<https://gerrit.ovirt.org/#/q/owner:msivak%2540redhat.com+status:open>
ovirt-system-tests
<https://gerrit.ovirt.org/#/projects/ovirt-system-tests,dashboards/default>
master
<https://gerrit.ovirt.org/#/q/status:open+project:ovirt-system-tests+branch:master>
4:51
PM
Add network queues custom property and use it in the vnic profile for VM0
<https://gerrit.ovirt.org/88829>
Yaniv Kaul
<https://gerrit.ovirt.org/#/q/owner:ykaul%2540redhat.com+status:open>
ovirt-system-tests
<https://gerrit.ovirt.org/#/projects/ovirt-system-tests,dashboards/default>
master
(multi_queue_config)
<https://gerrit.ovirt.org/#/q/status:open+project:ovirt-system-tests+branch:master+topic:multi_queue_config>
4:49
PM
new suite: he-basic-iscsi-suite-master <https://gerrit.ovirt.org/85838>
Yuval Turgeman
<https://gerrit.ovirt.org/#/q/owner:yturgema%2540redhat.com+status:open>
ovirt-system-tests
<https://gerrit.ovirt.org/#/projects/ovirt-system-tests,dashboards/default>
master
(he-basic-iscsi-suite-master)
<https://gerrit.ovirt.org/#/q/status:open+project:ovirt-system-tests+branch:master+topic:he-basic-iscsi-suite-master>
4:47
PM
Collect host-deploy bundle from the engine <https://gerrit.ovirt.org/88925>
Yedidyah Bar David
<https://gerrit.ovirt.org/#/q/owner:didi%2540redhat.com+status:open>
ovirt-system-tests
<https://gerrit.ovirt.org/#/projects/ovirt-system-tests,dashboards/default>
master
<https://gerrit.ovirt.org/#/q/status:open+project:ovirt-system-tests+branch:master>
4:41
PM
network-suite-master: Make openstack_client_config fixture available to all
... <https://gerrit.ovirt.org/88029> Merge Conflict Marcin Mirecki
<https://gerrit.ovirt.org/#/q/owner:mmirecki%2540redhat.com+status:open>
ovirt-system-tests
<https://gerrit.ovirt.org/#/projects/ovirt-system-tests,dashboards/default>
master
<https://gerrit.ovirt.org/#/q/status:open+project:ovirt-system-tests+branch:master>
3:39
PM
new suite: he-basic-ng-ansible-suite-master <https://gerrit.ovirt.org/88901>
Sandro Bonazzola
<https://gerrit.ovirt.org/#/q/owner:sbonazzo%2540redhat.com+status:open>
ovirt-system-tests
<https://gerrit.ovirt.org/#/projects/ovirt-system-tests,dashboards/default>
master
(he-basic-ng-ansible-suite-master)
<https://gerrit.ovirt.org/#/q/status:open+project:ovirt-system-tests+branch:master+topic:he-basic-ng-ansible-suite-master>
3:37
PM
Enable and move additional tests to 002 <https://gerrit.ovirt.org/88711>
Yaniv Kaul
<https://gerrit.ovirt.org/#/q/owner:ykaul%2540redhat.com+status:open>
ovirt-system-tests
<https://gerrit.ovirt.org/#/projects/ovirt-system-tests,dashboards/default>
master
(move_more_to_002)
<https://gerrit.ovirt.org/#/q/status:open+project:ovirt-system-tests+branch:master+topic:move_more_to_002>
3:08
PM
common: ovirt-4.2.repo <https://gerrit.ovirt.org/88904>
Sandro Bonazzola
<https://gerrit.ovirt.org/#/q/owner:sbonazzo%2540redhat.com+status:open>
ovirt-system-tests
<https://gerrit.ovirt.org/#/projects/ovirt-system-tests,dashboards/default>
master
<https://gerrit.ovirt.org/#/q/status:open+project:ovirt-system-tests+branch:master>
2:34
PM
networking: Introducing test_stateless_vm_duplicate_
mac_addr_vnic_does_not_be... <https://gerrit.ovirt.org/86946>
Leon Goldberg
<https://gerrit.ovirt.org/#/q/owner:lgoldber%2540redhat.com+status:open>
ovirt-system-tests
<https://gerrit.ovirt.org/#/projects/ovirt-system-tests,dashboards/default>
master
<https://gerrit.ovirt.org/#/q/status:open+project:ovirt-system-tests+branch:master>
2:08
PM
hc: Updating gdeploy conf to create vdo volumes
<https://gerrit.ovirt.org/88899>
Sahina Bose
<https://gerrit.ovirt.org/#/q/owner:sabose%2540redhat.com+status:open>
ovirt-system-tests
<https://gerrit.ovirt.org/#/projects/ovirt-system-tests,dashboards/default>
master
<https://gerrit.ovirt.org/#/q/status:open+project:ovirt-system-tests+branch:master>
12:55
PM
master: add USB to the sanity VM <https://gerrit.ovirt.org/88731>
Michal Skrivanek
<https://gerrit.ovirt.org/#/q/owner:michal.skrivanek%2540redhat.com+status:open>
ovirt-system-tests
<https://gerrit.ovirt.org/#/projects/ovirt-system-tests,dashboards/default>
master
<https://gerrit.ovirt.org/#/q/status:open+project:ovirt-system-tests+branch:master>
12:54
PM
Test hosted-engine cleanup <https://gerrit.ovirt.org/88331>
Yedidyah Bar David
<https://gerrit.ovirt.org/#/q/owner:didi%2540redhat.com+status:open>
ovirt-system-tests
<https://gerrit.ovirt.org/#/projects/ovirt-system-tests,dashboards/default>
master
<https://gerrit.ovirt.org/#/q/status:open+project:ovirt-system-tests+branch:master>
9:39
AM

Nice list of patches!

Feedback from the event:
- "if we want to add many more tests to OST, and I think we do, we need to
do some change there to allow that. Current framework is simply not
scalable enough"

More specific and constructive ideas are welcome. We know we want to move
to pytest, for example.

- not joining the hackathon because "I'd be like an elephant in a porcelain
shop"
- "I'm not sure I'm OK with the flood of suites that we have - the more we
have, the harder it is to sync and maintain but more importantly - to run."
- "We can't keep adding new suite for each parameter we want to test, it
adds overhead to monitoring, resources and maintenance."

I tend to agree, but not sure how to elegantly solve it.

- invite wasn't clear enough. I found people on #ovirt on Freenode and on
Red Hat IRC servers and redirected them to OFTC IRC server (my fault,
hopefully managed to workaround it by talking to people)

Lessons learned:
- Calendar invites to mailing lists doesn't work well, need a different way
to track mailing list members joining the events.
- Invites needs to be pedantic on how to join the event, not leaving space
for interpretation and misunderstanding.
- We need a contribution guide to ovirt-system-test: we need to make people
comfortable in trying to add a new test and we need to ensure that we won't
reject a day of work because the patch doesn't match core contributors
plannings on number of suites, resources and so on

Agree.
Y.

- The ovirt-system-tests check patch script is not good enough. It triggers
too many sequential suites on every single patch pushed, and fails due to
timeout taking more than 6 hours to complete.
- The way ovirt-system-test collects rpms from defined repos is not smart
enough: it doesn't take the latest version of a given package, just the
first found in sequential order of the repos,

Thanks everyone who participated to the event! if you have time please
continue improving ovirt-system-test even if today event is almost
completed!

-- 

SANDRO BONAZZOLA

ASSOCIATE MANAGER, SOFTWARE ENGINEERING, EMEA ENG VIRTUALIZATION R&D

Red Hat EMEA <https://www.redhat.com/>
<https://red.ht/sig>
TRIED. TESTED. TRUSTED. <https://redhat.com/trusted>

_______________________________________________
Users mailing list
Users at ovirt.org
http://lists.ovirt.org/mailman/listinfo/users
-------------- next part --------------
An HTML attachment was scrubbed...
URL: <http://lists.ovirt.org/pipermail/devel/attachments/20180313/570af85a/attachment-0001.html>