[ovirt-devel] [ovirt-users] ovirt-system-tests hackathon report

Tue Mar 13 20:48:17 UTC 2018

I missed it :( indeed calendar invite to the list didn't work.

On Tue, 13 Mar 2018 at 21:58 Eyal Edri <eedri at redhat.com> wrote:

> On Tue, Mar 13, 2018 at 8:00 PM, Yaniv Kaul <ykaul at redhat.com> wrote:
>
>>
>>
>> On Mar 13, 2018 6:27 PM, "Sandro Bonazzola" <sbonazzo at redhat.com> wrote:
>>
>> 4 people accepted calendar invite:
>> - Devin A. Bougie
>> - Francesco Romani
>> - Jiri Belka
>> - suporte, logicworks
>>
>> 4 people tentatively accepted calendar invite:
>> - Amnon Maimon
>> - Andreas Bleischwitz
>> - Arnaud Lauriou
>> - Stephen Pesini
>>
>> 2 mailing lists accepted calendar invite: users at ovirt.org,
>> devel at ovirt.org (don't ask me how) so I may have missed someone in above
>> list
>>
>>
>> 4 patches got merged:
>> Add check for host update to the 1st host.
>> <https://gerrit.ovirt.org/88767> Merged Yaniv Kaul
>> <https://gerrit.ovirt.org/#/q/owner:ykaul%2540redhat.com+status:merged>
>> ovirt-system-tests
>> <https://gerrit.ovirt.org/#/projects/ovirt-system-tests,dashboards/default> master
>> (add_upgrade_check)
>> <https://gerrit.ovirt.org/#/q/status:merged+project:ovirt-system-tests+branch:master+topic:add_upgrade_check> 4:10
>> PM
>> basic-suite-master: add vnic_profile_mappings to register vm
>> <https://gerrit.ovirt.org/87438> Merged Eitan Raviv
>> <https://gerrit.ovirt.org/#/q/owner:eraviv%2540redhat.com+status:merged>
>> ovirt-system-tests
>> <https://gerrit.ovirt.org/#/projects/ovirt-system-tests,dashboards/default> master
>> (register-template-vnic-mapping)
>> <https://gerrit.ovirt.org/#/q/status:merged+project:ovirt-system-tests+branch:master+topic:register-template-vnic-mapping> 2:50
>> PM
>> Revert "ovirt-4.2: Skipping 002_bootstrap.update_default_cluster"
>> <https://gerrit.ovirt.org/88881> Merged Eyal Edri
>> <https://gerrit.ovirt.org/#/q/owner:eedri%2540redhat.com+status:merged>
>> ovirt-system-tests
>> <https://gerrit.ovirt.org/#/projects/ovirt-system-tests,dashboards/default>
>> master
>> <https://gerrit.ovirt.org/#/q/status:merged+project:ovirt-system-tests+branch:master> 11:36
>> AM
>> seperate 4.2 tests and utils from master <https://gerrit.ovirt.org/88878>
>> Merged Eyal Edri
>> <https://gerrit.ovirt.org/#/q/owner:eedri%2540redhat.com+status:merged>
>> ovirt-system-tests
>> <https://gerrit.ovirt.org/#/projects/ovirt-system-tests,dashboards/default>
>> master
>> <https://gerrit.ovirt.org/#/q/status:merged+project:ovirt-system-tests+branch:master> 11:35
>> AM
>>
>> 13 patches has been pushed / reviewed / rebased
>>
>> Add gdeploy to ovirt-4.2.repo <https://gerrit.ovirt.org/88929>
>> Daniel Belenky
>> <https://gerrit.ovirt.org/#/q/owner:dbelenky%2540redhat.com+status:open>
>> ovirt-system-tests
>> <https://gerrit.ovirt.org/#/projects/ovirt-system-tests,dashboards/default>
>> master
>> <https://gerrit.ovirt.org/#/q/status:open+project:ovirt-system-tests+branch:master> 4:53
>> PM
>> Cleanup of test code - next() replaced with any()
>> <https://gerrit.ovirt.org/88928>
>> Martin Sivák
>> <https://gerrit.ovirt.org/#/q/owner:msivak%2540redhat.com+status:open>
>> ovirt-system-tests
>> <https://gerrit.ovirt.org/#/projects/ovirt-system-tests,dashboards/default>
>> master
>> <https://gerrit.ovirt.org/#/q/status:open+project:ovirt-system-tests+branch:master> 4:51
>> PM
>> Add network queues custom property and use it in the vnic profile for VM0
>> <https://gerrit.ovirt.org/88829>
>> Yaniv Kaul
>> <https://gerrit.ovirt.org/#/q/owner:ykaul%2540redhat.com+status:open>
>> ovirt-system-tests
>> <https://gerrit.ovirt.org/#/projects/ovirt-system-tests,dashboards/default> master
>> (multi_queue_config)
>> <https://gerrit.ovirt.org/#/q/status:open+project:ovirt-system-tests+branch:master+topic:multi_queue_config> 4:49
>> PM
>> new suite: he-basic-iscsi-suite-master <https://gerrit.ovirt.org/85838>
>> Yuval Turgeman
>> <https://gerrit.ovirt.org/#/q/owner:yturgema%2540redhat.com+status:open>
>> ovirt-system-tests
>> <https://gerrit.ovirt.org/#/projects/ovirt-system-tests,dashboards/default> master
>> (he-basic-iscsi-suite-master)
>> <https://gerrit.ovirt.org/#/q/status:open+project:ovirt-system-tests+branch:master+topic:he-basic-iscsi-suite-master> 4:47
>> PM
>> Collect host-deploy bundle from the engine
>> <https://gerrit.ovirt.org/88925>
>> Yedidyah Bar David
>> <https://gerrit.ovirt.org/#/q/owner:didi%2540redhat.com+status:open>
>> ovirt-system-tests
>> <https://gerrit.ovirt.org/#/projects/ovirt-system-tests,dashboards/default>
>> master
>> <https://gerrit.ovirt.org/#/q/status:open+project:ovirt-system-tests+branch:master> 4:41
>> PM
>> network-suite-master: Make openstack_client_config fixture available to
>> all ... <https://gerrit.ovirt.org/88029> Merge Conflict Marcin Mirecki
>> <https://gerrit.ovirt.org/#/q/owner:mmirecki%2540redhat.com+status:open>
>> ovirt-system-tests
>> <https://gerrit.ovirt.org/#/projects/ovirt-system-tests,dashboards/default>
>> master
>> <https://gerrit.ovirt.org/#/q/status:open+project:ovirt-system-tests+branch:master> 3:39
>> PM
>> new suite: he-basic-ng-ansible-suite-master
>> <https://gerrit.ovirt.org/88901>
>> Sandro Bonazzola
>> <https://gerrit.ovirt.org/#/q/owner:sbonazzo%2540redhat.com+status:open>
>> ovirt-system-tests
>> <https://gerrit.ovirt.org/#/projects/ovirt-system-tests,dashboards/default> master
>> (he-basic-ng-ansible-suite-master)
>> <https://gerrit.ovirt.org/#/q/status:open+project:ovirt-system-tests+branch:master+topic:he-basic-ng-ansible-suite-master> 3:37
>> PM
>> Enable and move additional tests to 002 <https://gerrit.ovirt.org/88711>
>> Yaniv Kaul
>> <https://gerrit.ovirt.org/#/q/owner:ykaul%2540redhat.com+status:open>
>> ovirt-system-tests
>> <https://gerrit.ovirt.org/#/projects/ovirt-system-tests,dashboards/default> master
>> (move_more_to_002)
>> <https://gerrit.ovirt.org/#/q/status:open+project:ovirt-system-tests+branch:master+topic:move_more_to_002> 3:08
>> PM
>> common: ovirt-4.2.repo <https://gerrit.ovirt.org/88904>
>> Sandro Bonazzola
>> <https://gerrit.ovirt.org/#/q/owner:sbonazzo%2540redhat.com+status:open>
>> ovirt-system-tests
>> <https://gerrit.ovirt.org/#/projects/ovirt-system-tests,dashboards/default>
>> master
>> <https://gerrit.ovirt.org/#/q/status:open+project:ovirt-system-tests+branch:master> 2:34
>> PM
>> networking: Introducing
>> test_stateless_vm_duplicate_mac_addr_vnic_does_not_be...
>> <https://gerrit.ovirt.org/86946>
>> Leon Goldberg
>> <https://gerrit.ovirt.org/#/q/owner:lgoldber%2540redhat.com+status:open>
>> ovirt-system-tests
>> <https://gerrit.ovirt.org/#/projects/ovirt-system-tests,dashboards/default>
>> master
>> <https://gerrit.ovirt.org/#/q/status:open+project:ovirt-system-tests+branch:master> 2:08
>> PM
>> hc: Updating gdeploy conf to create vdo volumes
>> <https://gerrit.ovirt.org/88899>
>> Sahina Bose
>> <https://gerrit.ovirt.org/#/q/owner:sabose%2540redhat.com+status:open>
>> ovirt-system-tests
>> <https://gerrit.ovirt.org/#/projects/ovirt-system-tests,dashboards/default>
>> master
>> <https://gerrit.ovirt.org/#/q/status:open+project:ovirt-system-tests+branch:master> 12:55
>> PM
>> master: add USB to the sanity VM <https://gerrit.ovirt.org/88731>
>> Michal Skrivanek
>> <https://gerrit.ovirt.org/#/q/owner:michal.skrivanek%2540redhat.com+status:open>
>> ovirt-system-tests
>> <https://gerrit.ovirt.org/#/projects/ovirt-system-tests,dashboards/default>
>> master
>> <https://gerrit.ovirt.org/#/q/status:open+project:ovirt-system-tests+branch:master> 12:54
>> PM
>> Test hosted-engine cleanup <https://gerrit.ovirt.org/88331>
>> Yedidyah Bar David
>> <https://gerrit.ovirt.org/#/q/owner:didi%2540redhat.com+status:open>
>> ovirt-system-tests
>> <https://gerrit.ovirt.org/#/projects/ovirt-system-tests,dashboards/default>
>> master
>> <https://gerrit.ovirt.org/#/q/status:open+project:ovirt-system-tests+branch:master> 9:39
>> AM
>>
>>
>> Nice list of patches!
>>
>>
>>
>> Feedback from the event:
>> - "if we want to add many more tests to OST, and I think we do, we need
>> to do some change there to allow that. Current framework is simply not
>> scalable enough"
>>
>>
>> More specific and constructive ideas are welcome. We know we want to move
>> to pytest, for example.
>>
>
> +1, there is a lot of things we need to improve in OST, the network suite
> is a good example of going forward with PyTest, we have other idea for
> improvements like splitting the suites into multiple projects using a new
> feature in std-ci,
> dropping the reposync file or making it optional ( actually
> ovirt-demo-tool is already doing that, using release rpms ).
>
> If there are more ideas, we can consider doing an infra hackathon where we
> can focus in improving infrastructure and moving to PyTest most of the
> tests ( at least in master ) .
>
>
>>
>> - not joining the hackathon because "I'd be like an elephant in a
>> porcelain shop"
>> - "I'm not sure I'm OK with the flood of suites that we have - the more
>> we have, the harder it is to sync and maintain but more importantly - to
>> run."
>> - "We can't keep adding new suite for each parameter we want to test, it
>> adds overhead to monitoring, resources and maintenance."
>>
>>
>> I tend to agree, but not sure how to elegantly solve it.
>>
>
> We need to think on a new design for it which includes scalability and
> multiple maintainers, the requirements and usage has changed significantly
> over the years,
> I also don't have an easy solution for it, yet at least.
>
>
>>
>> - invite wasn't clear enough. I found people on #ovirt on Freenode and on
>> Red Hat IRC servers and redirected them to OFTC IRC server (my fault,
>> hopefully managed to workaround it by talking to people)
>>
>>
>> Lessons learned:
>> - Calendar invites to mailing lists doesn't work well, need a different
>> way to track mailing list members joining the events.
>> - Invites needs to be pedantic on how to join the event, not leaving
>> space for interpretation and misunderstanding.
>> - We need a contribution guide to ovirt-system-test: we need to make
>> people comfortable in trying to add a new test and we need to ensure that
>> we won't reject a day of work because the patch doesn't match core
>> contributors plannings on number of suites, resources and so on
>>
>>
> This could have been better if the single OST maintainer could be part of
> the planning and also participate and give feedback and assistance,
> unfourtunately it was decided to do the hackhaton without him present.
> I agree we need to improve our contribution guide, and work has started
> towards it, but any specific feedback or tickets on what can be improved
> will surely help.
>
>
>>
>> Agree.
>> Y.
>>
>> - The ovirt-system-tests check patch script is not good enough. It
>> triggers too many sequential suites on every single patch pushed, and fails
>> due to timeout taking more than 6 hours to complete.
>>
>>
> This is very close to be much better, we're 1-2 weeks away from
> implemeting a new feature in STD CI V2 which will allow us to replace the
> existing 'change resolver' and eventually run all suites in check-patch in
> parallel, dramatically reducing runtime.
>
>
>> - The way ovirt-system-test collects rpms from defined repos is not smart
>> enough: it doesn't take the latest version of a given package, just the
>> first found in sequential order of the repos,
>>
>>
> Its not that its not smart, its by design, repoman uses "only-latest"
> option to get the first RPM he finds from the repos, I'm pretty sure there
> was a good reason behind it when it was written a few years ago, I'll try
> to remember and update.
> In any case, the relevant patch where this was needed had a different
> problem with dynamic replacement of repos which we need to think on a
> solution for.
>
>
>>
>> Thanks everyone who participated to the event! if you have time please
>> continue improving ovirt-system-test even if today event is almost
>> completed!
>>
>>
>> --
>>
>> SANDRO BONAZZOLA
>>
>> ASSOCIATE MANAGER, SOFTWARE ENGINEERING, EMEA ENG VIRTUALIZATION R&D
>>
>> Red Hat EMEA <https://www.redhat.com/>
>> <https://red.ht/sig>
>> TRIED. TESTED. TRUSTED. <https://redhat.com/trusted>
>>
>>
>> _______________________________________________
>> Users mailing list
>> Users at ovirt.org
>> http://lists.ovirt.org/mailman/listinfo/users
>>
>>
>>
>> _______________________________________________
>> Infra mailing list
>> Infra at ovirt.org
>> http://lists.ovirt.org/mailman/listinfo/infra
>>
>>
>
>
> --
>
> Eyal edri
>
>
> MANAGER
>
> RHV DevOps
>
> EMEA VIRTUALIZATION R&D
>
>
> Red Hat EMEA <https://www.redhat.com/>
> <https://red.ht/sig> TRIED. TESTED. TRUSTED. <https://redhat.com/trusted>
> phone: +972-9-7692018 <+972%209-769-2018>
> irc: eedri (on #tlv #rhev-dev #rhev-integ)
> _______________________________________________
> Devel mailing list
> Devel at ovirt.org
> http://lists.ovirt.org/mailman/listinfo/devel
-------------- next part --------------
An HTML attachment was scrubbed...
URL: <http://lists.ovirt.org/pipermail/infra/attachments/20180313/eed269f2/attachment-0001.html>