[ovirt-devel] ovirt-system-tests hackathon report

4 people accepted calendar invite: - Devin A. Bougie - Francesco Romani - Jiri Belka - suporte, logicworks 4 people tentatively accepted calendar invite: - Amnon Maimon - Andreas Bleischwitz - Arnaud Lauriou - Stephen Pesini 2 mailing lists accepted calendar invite: users(a)ovirt.org, devel(a)ovirt.org (don't ask me how) so I may have missed someone in above list 4 patches got merged: Add check for host update to the 1st host. <https://gerrit.ovirt.org/88767> Merged Yaniv Kaul <https://gerrit.ovirt.org/#/q/owner:ykaul%2540redhat.com+status:merged> ovirt-system-tests <https://gerrit.ovirt.org/#/projects/ovirt-system-tests,dashboards/default... master (add_upgrade_check) <https://gerrit.ovirt.org/#/q/status:merged+project:ovirt-system-tests+bra... 4:10 PM basic-suite-master: add vnic_profile_mappings to register vm <https://gerrit.ovirt.org/87438> Merged Eitan Raviv <https://gerrit.ovirt.org/#/q/owner:eraviv%2540redhat.com+status:merged> ovirt-system-tests <https://gerrit.ovirt.org/#/projects/ovirt-system-tests,dashboards/default... master (register-template-vnic-mapping) <https://gerrit.ovirt.org/#/q/status:merged+project:ovirt-system-tests+bra... 2:50 PM Revert "ovirt-4.2: Skipping 002_bootstrap.update_default_cluster" <https://gerrit.ovirt.org/88881> Merged Eyal Edri <https://gerrit.ovirt.org/#/q/owner:eedri%2540redhat.com+status:merged> ovirt-system-tests <https://gerrit.ovirt.org/#/projects/ovirt-system-tests,dashboards/default... master <https://gerrit.ovirt.org/#/q/status:merged+project:ovirt-system-tests+bra... 11:36 AM seperate 4.2 tests and utils from master <https://gerrit.ovirt.org/88878> Merged Eyal Edri <https://gerrit.ovirt.org/#/q/owner:eedri%2540redhat.com+status:merged> ovirt-system-tests <https://gerrit.ovirt.org/#/projects/ovirt-system-tests,dashboards/default... master <https://gerrit.ovirt.org/#/q/status:merged+project:ovirt-system-tests+bra... 11:35 AM 13 patches has been pushed / reviewed / rebased Add gdeploy to ovirt-4.2.repo <https://gerrit.ovirt.org/88929> Daniel Belenky <https://gerrit.ovirt.org/#/q/owner:dbelenky%2540redhat.com+status:open> ovirt-system-tests <https://gerrit.ovirt.org/#/projects/ovirt-system-tests,dashboards/default... master <https://gerrit.ovirt.org/#/q/status:open+project:ovirt-system-tests+branc... 4:53 PM Cleanup of test code - next() replaced with any() <https://gerrit.ovirt.org/88928> Martin Sivák <https://gerrit.ovirt.org/#/q/owner:msivak%2540redhat.com+status:open> ovirt-system-tests <https://gerrit.ovirt.org/#/projects/ovirt-system-tests,dashboards/default... master <https://gerrit.ovirt.org/#/q/status:open+project:ovirt-system-tests+branc... 4:51 PM Add network queues custom property and use it in the vnic profile for VM0 <https://gerrit.ovirt.org/88829> Yaniv Kaul <https://gerrit.ovirt.org/#/q/owner:ykaul%2540redhat.com+status:open> ovirt-system-tests <https://gerrit.ovirt.org/#/projects/ovirt-system-tests,dashboards/default... master (multi_queue_config) <https://gerrit.ovirt.org/#/q/status:open+project:ovirt-system-tests+branc... 4:49 PM new suite: he-basic-iscsi-suite-master <https://gerrit.ovirt.org/85838> Yuval Turgeman <https://gerrit.ovirt.org/#/q/owner:yturgema%2540redhat.com+status:open> ovirt-system-tests <https://gerrit.ovirt.org/#/projects/ovirt-system-tests,dashboards/default... master (he-basic-iscsi-suite-master) <https://gerrit.ovirt.org/#/q/status:open+project:ovirt-system-tests+branc... 4:47 PM Collect host-deploy bundle from the engine <https://gerrit.ovirt.org/88925> Yedidyah Bar David <https://gerrit.ovirt.org/#/q/owner:didi%2540redhat.com+status:open> ovirt-system-tests <https://gerrit.ovirt.org/#/projects/ovirt-system-tests,dashboards/default... master <https://gerrit.ovirt.org/#/q/status:open+project:ovirt-system-tests+branc... 4:41 PM network-suite-master: Make openstack_client_config fixture available to all ... <https://gerrit.ovirt.org/88029> Merge Conflict Marcin Mirecki <https://gerrit.ovirt.org/#/q/owner:mmirecki%2540redhat.com+status:open> ovirt-system-tests <https://gerrit.ovirt.org/#/projects/ovirt-system-tests,dashboards/default... master <https://gerrit.ovirt.org/#/q/status:open+project:ovirt-system-tests+branc... 3:39 PM new suite: he-basic-ng-ansible-suite-master <https://gerrit.ovirt.org/88901> Sandro Bonazzola <https://gerrit.ovirt.org/#/q/owner:sbonazzo%2540redhat.com+status:open> ovirt-system-tests <https://gerrit.ovirt.org/#/projects/ovirt-system-tests,dashboards/default... master (he-basic-ng-ansible-suite-master) <https://gerrit.ovirt.org/#/q/status:open+project:ovirt-system-tests+branc... 3:37 PM Enable and move additional tests to 002 <https://gerrit.ovirt.org/88711> Yaniv Kaul <https://gerrit.ovirt.org/#/q/owner:ykaul%2540redhat.com+status:open> ovirt-system-tests <https://gerrit.ovirt.org/#/projects/ovirt-system-tests,dashboards/default... master (move_more_to_002) <https://gerrit.ovirt.org/#/q/status:open+project:ovirt-system-tests+branc... 3:08 PM common: ovirt-4.2.repo <https://gerrit.ovirt.org/88904> Sandro Bonazzola <https://gerrit.ovirt.org/#/q/owner:sbonazzo%2540redhat.com+status:open> ovirt-system-tests <https://gerrit.ovirt.org/#/projects/ovirt-system-tests,dashboards/default... master <https://gerrit.ovirt.org/#/q/status:open+project:ovirt-system-tests+branc... 2:34 PM networking: Introducing test_stateless_vm_duplicate_mac_addr_vnic_does_not_be... <https://gerrit.ovirt.org/86946> Leon Goldberg <https://gerrit.ovirt.org/#/q/owner:lgoldber%2540redhat.com+status:open> ovirt-system-tests <https://gerrit.ovirt.org/#/projects/ovirt-system-tests,dashboards/default... master <https://gerrit.ovirt.org/#/q/status:open+project:ovirt-system-tests+branc... 2:08 PM hc: Updating gdeploy conf to create vdo volumes <https://gerrit.ovirt.org/88899> Sahina Bose <https://gerrit.ovirt.org/#/q/owner:sabose%2540redhat.com+status:open> ovirt-system-tests <https://gerrit.ovirt.org/#/projects/ovirt-system-tests,dashboards/default... master <https://gerrit.ovirt.org/#/q/status:open+project:ovirt-system-tests+branc... 12:55 PM master: add USB to the sanity VM <https://gerrit.ovirt.org/88731> Michal Skrivanek <https://gerrit.ovirt.org/#/q/owner:michal.skrivanek%2540redhat.com+status... ovirt-system-tests <https://gerrit.ovirt.org/#/projects/ovirt-system-tests,dashboards/default... master <https://gerrit.ovirt.org/#/q/status:open+project:ovirt-system-tests+branc... 12:54 PM Test hosted-engine cleanup <https://gerrit.ovirt.org/88331> Yedidyah Bar David <https://gerrit.ovirt.org/#/q/owner:didi%2540redhat.com+status:open> ovirt-system-tests <https://gerrit.ovirt.org/#/projects/ovirt-system-tests,dashboards/default... master <https://gerrit.ovirt.org/#/q/status:open+project:ovirt-system-tests+branc... 9:39 AM Feedback from the event: - "if we want to add many more tests to OST, and I think we do, we need to do some change there to allow that. Current framework is simply not scalable enough" - not joining the hackathon because "I'd be like an elephant in a porcelain shop" - "I'm not sure I'm OK with the flood of suites that we have - the more we have, the harder it is to sync and maintain but more importantly - to run." - "We can't keep adding new suite for each parameter we want to test, it adds overhead to monitoring, resources and maintenance." - invite wasn't clear enough. I found people on #ovirt on Freenode and on Red Hat IRC servers and redirected them to OFTC IRC server (my fault, hopefully managed to workaround it by talking to people) Lessons learned: - Calendar invites to mailing lists doesn't work well, need a different way to track mailing list members joining the events. - Invites needs to be pedantic on how to join the event, not leaving space for interpretation and misunderstanding. - We need a contribution guide to ovirt-system-test: we need to make people comfortable in trying to add a new test and we need to ensure that we won't reject a day of work because the patch doesn't match core contributors plannings on number of suites, resources and so on - The ovirt-system-tests check patch script is not good enough. It triggers too many sequential suites on every single patch pushed, and fails due to timeout taking more than 6 hours to complete. - The way ovirt-system-test collects rpms from defined repos is not smart enough: it doesn't take the latest version of a given package, just the first found in sequential order of the repos, Thanks everyone who participated to the event! if you have time please continue improving ovirt-system-test even if today event is almost completed! -- SANDRO BONAZZOLA ASSOCIATE MANAGER, SOFTWARE ENGINEERING, EMEA ENG VIRTUALIZATION R&D Red Hat EMEA <https://www.redhat.com/> <https://red.ht/sig> TRIED. TESTED. TRUSTED. <https://redhat.com/trusted>