[ovirt-users] ovirt-system-tests hackathon report

Tue Mar 13 16:26:04 UTC 2018

4 people accepted calendar invite:
- Devin A. Bougie
- Francesco Romani
- Jiri Belka
- suporte, logicworks

4 people tentatively accepted calendar invite:
- Amnon Maimon
- Andreas Bleischwitz
- Arnaud Lauriou
- Stephen Pesini

2 mailing lists accepted calendar invite: users at ovirt.org, devel at ovirt.org
(don't ask me how) so I may have missed someone in above list

4 patches got merged:
Add check for host update to the 1st host. <https://gerrit.ovirt.org/88767>
Merged Yaniv Kaul
<https://gerrit.ovirt.org/#/q/owner:ykaul%2540redhat.com+status:merged>
ovirt-system-tests
<https://gerrit.ovirt.org/#/projects/ovirt-system-tests,dashboards/default>
master
(add_upgrade_check)
<https://gerrit.ovirt.org/#/q/status:merged+project:ovirt-system-tests+branch:master+topic:add_upgrade_check>
4:10
PM
basic-suite-master: add vnic_profile_mappings to register vm
<https://gerrit.ovirt.org/87438> Merged Eitan Raviv
<https://gerrit.ovirt.org/#/q/owner:eraviv%2540redhat.com+status:merged>
ovirt-system-tests
<https://gerrit.ovirt.org/#/projects/ovirt-system-tests,dashboards/default>
master
(register-template-vnic-mapping)
<https://gerrit.ovirt.org/#/q/status:merged+project:ovirt-system-tests+branch:master+topic:register-template-vnic-mapping>
2:50
PM
Revert "ovirt-4.2: Skipping 002_bootstrap.update_default_cluster"
<https://gerrit.ovirt.org/88881> Merged Eyal Edri
<https://gerrit.ovirt.org/#/q/owner:eedri%2540redhat.com+status:merged>
ovirt-system-tests
<https://gerrit.ovirt.org/#/projects/ovirt-system-tests,dashboards/default>
master
<https://gerrit.ovirt.org/#/q/status:merged+project:ovirt-system-tests+branch:master>
11:36
AM
seperate 4.2 tests and utils from master <https://gerrit.ovirt.org/88878>
Merged Eyal Edri
<https://gerrit.ovirt.org/#/q/owner:eedri%2540redhat.com+status:merged>
ovirt-system-tests
<https://gerrit.ovirt.org/#/projects/ovirt-system-tests,dashboards/default>
master
<https://gerrit.ovirt.org/#/q/status:merged+project:ovirt-system-tests+branch:master>
11:35
AM

13 patches has been pushed / reviewed / rebased

Add gdeploy to ovirt-4.2.repo <https://gerrit.ovirt.org/88929>
Daniel Belenky
<https://gerrit.ovirt.org/#/q/owner:dbelenky%2540redhat.com+status:open>
ovirt-system-tests
<https://gerrit.ovirt.org/#/projects/ovirt-system-tests,dashboards/default>
master
<https://gerrit.ovirt.org/#/q/status:open+project:ovirt-system-tests+branch:master>
4:53
PM
Cleanup of test code - next() replaced with any()
<https://gerrit.ovirt.org/88928>
Martin Sivák
<https://gerrit.ovirt.org/#/q/owner:msivak%2540redhat.com+status:open>
ovirt-system-tests
<https://gerrit.ovirt.org/#/projects/ovirt-system-tests,dashboards/default>
master
<https://gerrit.ovirt.org/#/q/status:open+project:ovirt-system-tests+branch:master>
4:51
PM
Add network queues custom property and use it in the vnic profile for VM0
<https://gerrit.ovirt.org/88829>
Yaniv Kaul
<https://gerrit.ovirt.org/#/q/owner:ykaul%2540redhat.com+status:open>
ovirt-system-tests
<https://gerrit.ovirt.org/#/projects/ovirt-system-tests,dashboards/default>
master
(multi_queue_config)
<https://gerrit.ovirt.org/#/q/status:open+project:ovirt-system-tests+branch:master+topic:multi_queue_config>
4:49
PM
new suite: he-basic-iscsi-suite-master <https://gerrit.ovirt.org/85838>
Yuval Turgeman
<https://gerrit.ovirt.org/#/q/owner:yturgema%2540redhat.com+status:open>
ovirt-system-tests
<https://gerrit.ovirt.org/#/projects/ovirt-system-tests,dashboards/default>
master
(he-basic-iscsi-suite-master)
<https://gerrit.ovirt.org/#/q/status:open+project:ovirt-system-tests+branch:master+topic:he-basic-iscsi-suite-master>
4:47
PM
Collect host-deploy bundle from the engine <https://gerrit.ovirt.org/88925>
Yedidyah Bar David
<https://gerrit.ovirt.org/#/q/owner:didi%2540redhat.com+status:open>
ovirt-system-tests
<https://gerrit.ovirt.org/#/projects/ovirt-system-tests,dashboards/default>
master
<https://gerrit.ovirt.org/#/q/status:open+project:ovirt-system-tests+branch:master>
4:41
PM
network-suite-master: Make openstack_client_config fixture available to all
... <https://gerrit.ovirt.org/88029> Merge Conflict Marcin Mirecki
<https://gerrit.ovirt.org/#/q/owner:mmirecki%2540redhat.com+status:open>
ovirt-system-tests
<https://gerrit.ovirt.org/#/projects/ovirt-system-tests,dashboards/default>
master
<https://gerrit.ovirt.org/#/q/status:open+project:ovirt-system-tests+branch:master>
3:39
PM
new suite: he-basic-ng-ansible-suite-master <https://gerrit.ovirt.org/88901>
Sandro Bonazzola
<https://gerrit.ovirt.org/#/q/owner:sbonazzo%2540redhat.com+status:open>
ovirt-system-tests
<https://gerrit.ovirt.org/#/projects/ovirt-system-tests,dashboards/default>
master
(he-basic-ng-ansible-suite-master)
<https://gerrit.ovirt.org/#/q/status:open+project:ovirt-system-tests+branch:master+topic:he-basic-ng-ansible-suite-master>
3:37
PM
Enable and move additional tests to 002 <https://gerrit.ovirt.org/88711>
Yaniv Kaul
<https://gerrit.ovirt.org/#/q/owner:ykaul%2540redhat.com+status:open>
ovirt-system-tests
<https://gerrit.ovirt.org/#/projects/ovirt-system-tests,dashboards/default>
master
(move_more_to_002)
<https://gerrit.ovirt.org/#/q/status:open+project:ovirt-system-tests+branch:master+topic:move_more_to_002>
3:08
PM
common: ovirt-4.2.repo <https://gerrit.ovirt.org/88904>
Sandro Bonazzola
<https://gerrit.ovirt.org/#/q/owner:sbonazzo%2540redhat.com+status:open>
ovirt-system-tests
<https://gerrit.ovirt.org/#/projects/ovirt-system-tests,dashboards/default>
master
<https://gerrit.ovirt.org/#/q/status:open+project:ovirt-system-tests+branch:master>
2:34
PM
networking: Introducing
test_stateless_vm_duplicate_mac_addr_vnic_does_not_be...
<https://gerrit.ovirt.org/86946>
Leon Goldberg
<https://gerrit.ovirt.org/#/q/owner:lgoldber%2540redhat.com+status:open>
ovirt-system-tests
<https://gerrit.ovirt.org/#/projects/ovirt-system-tests,dashboards/default>
master
<https://gerrit.ovirt.org/#/q/status:open+project:ovirt-system-tests+branch:master>
2:08
PM
hc: Updating gdeploy conf to create vdo volumes
<https://gerrit.ovirt.org/88899>
Sahina Bose
<https://gerrit.ovirt.org/#/q/owner:sabose%2540redhat.com+status:open>
ovirt-system-tests
<https://gerrit.ovirt.org/#/projects/ovirt-system-tests,dashboards/default>
master
<https://gerrit.ovirt.org/#/q/status:open+project:ovirt-system-tests+branch:master>
12:55
PM
master: add USB to the sanity VM <https://gerrit.ovirt.org/88731>
Michal Skrivanek
<https://gerrit.ovirt.org/#/q/owner:michal.skrivanek%2540redhat.com+status:open>
ovirt-system-tests
<https://gerrit.ovirt.org/#/projects/ovirt-system-tests,dashboards/default>
master
<https://gerrit.ovirt.org/#/q/status:open+project:ovirt-system-tests+branch:master>
12:54
PM
Test hosted-engine cleanup <https://gerrit.ovirt.org/88331>
Yedidyah Bar David
<https://gerrit.ovirt.org/#/q/owner:didi%2540redhat.com+status:open>
ovirt-system-tests
<https://gerrit.ovirt.org/#/projects/ovirt-system-tests,dashboards/default>
master
<https://gerrit.ovirt.org/#/q/status:open+project:ovirt-system-tests+branch:master>
9:39
AM

Feedback from the event:
- "if we want to add many more tests to OST, and I think we do, we need to
do some change there to allow that. Current framework is simply not
scalable enough"
- not joining the hackathon because "I'd be like an elephant in a porcelain
shop"
- "I'm not sure I'm OK with the flood of suites that we have - the more we
have, the harder it is to sync and maintain but more importantly - to run."
- "We can't keep adding new suite for each parameter we want to test, it
adds overhead to monitoring, resources and maintenance."
- invite wasn't clear enough. I found people on #ovirt on Freenode and on
Red Hat IRC servers and redirected them to OFTC IRC server (my fault,
hopefully managed to workaround it by talking to people)

Lessons learned:
- Calendar invites to mailing lists doesn't work well, need a different way
to track mailing list members joining the events.
- Invites needs to be pedantic on how to join the event, not leaving space
for interpretation and misunderstanding.
- We need a contribution guide to ovirt-system-test: we need to make people
comfortable in trying to add a new test and we need to ensure that we won't
reject a day of work because the patch doesn't match core contributors
plannings on number of suites, resources and so on
- The ovirt-system-tests check patch script is not good enough. It triggers
too many sequential suites on every single patch pushed, and fails due to
timeout taking more than 6 hours to complete.
- The way ovirt-system-test collects rpms from defined repos is not smart
enough: it doesn't take the latest version of a given package, just the
first found in sequential order of the repos,

Thanks everyone who participated to the event! if you have time please
continue improving ovirt-system-test even if today event is almost
completed!

-- 

SANDRO BONAZZOLA

ASSOCIATE MANAGER, SOFTWARE ENGINEERING, EMEA ENG VIRTUALIZATION R&D

Red Hat EMEA <https://www.redhat.com/>
<https://red.ht/sig>
TRIED. TESTED. TRUSTED. <https://redhat.com/trusted>
-------------- next part --------------
An HTML attachment was scrubbed...
URL: <http://lists.ovirt.org/pipermail/users/attachments/20180313/cfab7836/attachment.html>