[ovirt-devel] Re: oVirt Community - open discussion around repositories and workflows

On 2. 12. 2021, at 12:51, Milan Zamazal <mzamazal(a)redhat.com&gt; wrote: Michal Skrivanek <michal.skrivanek(a)redhat.com&gt; writes: >> On 1. 12. 2021, at 16:57, Nir Soffer <nsoffer(a)redhat.com&gt; wrote: >> >> On Wed, Dec 1, 2021 at 11:38 AM Milan Zamazal <mzamazal(a)redhat.com&gt; wrote: >>> >>> Michal Skrivanek <michal.skrivanek(a)redhat.com&gt; writes: >>> >>>> Hi all, >>>> so far we haven't encounter any blocking issue with this effort, I >>>> wanted to propose to decide on oVirt development moving to GitHub, >>>> COPR and CBS. Recent issue with decommissioning of our CI datacenter >>>> is a good reminder why we are doing that... >>>> What do we want to do? >>>> 1) move "ovirt-master-snapshot" compose to COPR >>>> it is feasible for all projects except ovirt-node and >>>> appliance due to COPR limitations, for these two we plan to use a >>>> self-hosted runner in github env. >>>> it replaces the "build-artifacts" stdci stage >>>> 2) move release to CentOS Community Build System to simplify our oVirt releases >>>> replaces our custom releng-tools process and aligns us better >>>> with CentOS that is our main (and only) platform we support. >>>> 3) move development from Gerrit to GitHub >>>> this is a very visible change and affects every oVirt >>>> developer. We need a way how to test posted patches and the current >>>> stdci "check-patch" stage is overly complex and slow to run, we lack >>>> people for stdci maintenance in general (bluntly, it's a dead >>>> project). Out of the various options that exist we ended up converging >>>> to Github. Why? Just because it's the most simple thing to do for us, >>>> with least amount of effort, least amount of additional people and hw >>>> resources, with a manageable learning curve. It comes at a price - it >>>> only works if we switch our primary development from Gerrit to Github >>>> for all the remaining projects. It is a big change to our processes, >>>> but I believe we have to go through that transition in order to solve >>>> our CI troubles for good. We started preparing guide and templates to >>>> use so that we keep a uniform "look and feel" for all sub-projects, is >>>> shall be ready soon. >>>> >>>> I'd like us to move from "POC" stage to "production", and actively >>>> start working on the above, start moving project after project. >>>> Let me ask for a final round of thoughts, comments, objections, we are ready to go ahead. >>> >>> Hi, >>> >>> the Vdsm maintainers have discussed the possibility of moving Vdsm >>> development to GitHub and we consider it a reasonable and feasible >>> option. Although GitHub is not on par with Gerrit as for code reviews, >>> having a more reliable common development platform outweighs the >>> disadvantages. There is already an ongoing work on having a fully >>> usable Vdsm CI on GitHub. >>> >>> One thing related to the move is that we would like to retain the >>> history of code reviews from Gerrit. The comments there contain >>> valuable information that we wouldn't like to lose. Is there a way to >>> export the public Gerrit contents, once we make a switch to GitHub for >>> each particular project, to something that could be reasonably used for >>> patch archaeology when needed? >> >> I think keeping a readonly instance would be best, one all projects >> migrated to github. > > >> >> I hope there is a way to export the data to static html so it will be >> available forever without running an actual gerrit instance. > > no idea...it can definitely be scraped patch after patch..but it's > going to be really huge and again, it will keep running, there's no > plan to shut it down or anything. > gerrit.ovirt.org will stay up And working / properly maintained? Can you guarantee it will remain usable for the relevant purposes? If yes then it would be indeed the best option.

> for as long as it's needed and relevant. If it ever comes to shutting > it down I don't think there's going to be anyone caring about the > comments > >> >> Nir >> >>> >>>> It's not going to be easy, but I firmly believe it will greatly >>>> improve maintainability of oVirt and reduce overhead that we all >>>> struggle with for years. >>>> >>>> Thanks, >>>> michal >>>> >>>>> On 10. 11. 2021, at 9:17, Sandro Bonazzola <sbonazzo(a)redhat.com&gt; wrote: >>>>> >>>>> Hi, here's an update on what has been done so far and how it is going. >>>>> >>>>> COPR >>>>> All the oVirt active subprojects are now built on COPR except oVirt >>>>> Engine Appliance and oVirt Node: I'm still looking into how to build >>>>> them on COPR. >>>>> >>>>> Of those subprojects only the following are not yet built >>>>> automatically on patch merge event as they have pending patches for >>>>> enabling the automation: >>>>> - ovirt-engine-nodejs-modules: >>>>> https://gerrit.ovirt.org/c/ovirt-engine-nodejs-modules/+/117506 >>>>> <https://gerrit.ovirt.org/c/ovirt-engine-nodejs-modules/+/117506>- >>>>> ovirt-engine-ui-extensions: >>>>> https://gerrit.ovirt.org/c/ovirt-engine-ui-extensions/+/117512 >>>>> <https://gerrit.ovirt.org/c/ovirt-engine-ui-extensions/+/117512> >>>>> - ovirt-web-ui: https://github.com/oVirt/ovirt-web-ui/pull/1532 >>>>> <https://github.com/oVirt/ovirt-web-ui/pull/1532> >>>>> >>>>> You can see the build status for the whole project here: >>>>> https://copr.fedorainfracloud.org/coprs/ovirt/ovirt-master-snapshot/monitor/ >>>>> <https://copr.fedorainfracloud.org/coprs/ovirt/ovirt-master-snapshot/monit... >>>>> If you are maintaining an oVirt project and you want to enable >>>>> builds for CentOS Stream 9 or other architectures supported by copr >>>>> please let me know. >>>>> >>>>> So far, the COPR infrastructure seems reliable and working well. >>>>> >>>>> GitHub >>>>> The following projects are developed on GitHub only: >>>>> - ovirt-ansible-collection >>>>> - ovirt-cockpit-sso >>>>> - ovirt-web-ui >>>>> - python-ovirt-engine-sdk4 >>>>> - ovirt-engine-sdk-go >>>>> >>>>> Within this list: >>>>> - ovirt-engine-sdk-go is not being built in COPR as the rpm is not >>>>> needed for developing with go and the automation is already handled >>>>> on GitHub actions only. >>>>> - ovirt-cockpit-sso is still triggering jenkins jobs but it's ready >>>>> to drop them as PR are now tested with github actions too and builds >>>>> are handled in COPR. >>>>> >>>>> So far, moving the development to GitHub only seems to be working >>>>> well and I would suggest the maintainers of the oVirt subprojects to >>>>> consider moving to GitHub only as well. >>>>> +Sanja Bonic <mailto:sbonic@redhat.com> can help you enabling GitHub >>>>> actions for your oVirt projects so please ping her if you need help. >>>>> >>>>> CentOS Community Build >>>>> >>>>> I'm going to try building the same projects currently being built in >>>>> COPR also within the CentOS Community Build system in the coming >>>>> weeks. >>>>> If you are already a CentOS Virtualization SIG member and you want >>>>> to help with this effort please let me know what you are going to >>>>> build there so we won't duplicate the work. >>>>> If you are an oVirt project maintainer I would recommend you to join >>>>> CentOS Virtualization SIG so you'll be independent releasing your >>>>> package builds. >>>>> >>>>> >>>>> >>>>> >>>>> Il giorno mar 2 nov 2021 alle ore 12:06 Sandro Bonazzola >>>>> <sbonazzo(a)redhat.com <mailto:sbonazzo@redhat.com>> ha scritto: >>>>> Hi, after researching for a while and discussing part of it with a >>>>> few other developers on IRC and calls, I'd like to update on current >>>>> efforts. >>>>> Recently I sent several patches to allow building of the merged >>>>> patches (equivalent of build-artifacts stage in oVirt Standard CI) >>>>> using copr. >>>>> How does this work: >>>>> Within the git repository a Makefile needs to be created in `.copr/Makefile`. >>>>> The makefile needs to provide a `srpm` target with the instructions on how to generate a .src.rpm. >>>>> That makefile will be executed with something like: `make -f >>>>> .copr/Makefile srpm outdir="/tmp/outdir/"` on copr infrastructure >>>>> where the outdir will point to the place where the src.rpm will be >>>>> stored to be sent to the mock instances for the different targets >>>>> (el8, el9, x86_64, aarch64, ppc64le). >>>>> An example of the patch providing this makefile can be seen here: >>>>> https://gerrit.ovirt.org/c/ovirt-provider-ovn/+/117396/1/.copr/Makefile >>>>> <https://gerrit.ovirt.org/c/ovirt-provider-ovn/+/117396/1/.copr/Makefile> >>>>> Please note the src.rpm will be built on Fedora 34 (as of today, 35 may be used soon). >>>>> >>>>> On GitHub, a webhook needs to be added in the repository >>>>> configuration pointing to the copr API trigger. An administrator of >>>>> the github/oVirt organization is needed in order to do that. >>>>> I can handle the webhook setup for your project within the oVirt GitHub, please ping me as needed. >>>>> >>>>> The result of the latest builds can be displayed on GitHub README as >>>>> in this patch: >>>>> https://gerrit.ovirt.org/c/ovirt-provider-ovn/+/117396/1/README.adoc >>>>> <https://gerrit.ovirt.org/c/ovirt-provider-ovn/+/117396/1/README.adoc>(... >>>>> or this one https://gerrit.ovirt.org/c/vdsm/+/117368/3/README.md >>>>> <https://gerrit.ovirt.org/c/vdsm/+/117368/3/README.md> (Markdown) >>>>> >>>>> All the builds will be executed only once the patch is merged. The >>>>> build will happen in >>>>> https://copr.fedorainfracloud.org/coprs/ovirt/ovirt-master-snapshot/ >>>>> <https://copr.fedorainfracloud.org/coprs/ovirt/ovirt-master-snapshot/>This >>>>> is going to replace the existing >>>>> https://resources.ovirt.org/pub/ovirt-master-snapshot >>>>> <https://resources.ovirt.org/pub/ovirt-master-snapshot> repository >>>>> structure. >>>>> Advantages: >>>>> - builds will be signed >>>>> - composes will be immediately available and not waiting till the next day to be in the nightly >>>>> >>>>> On copr, in order to add a package to the compose you need to have >>>>> admin role on >>>>> https://copr.fedorainfracloud.org/coprs/ovirt/ovirt-master-snapshot/permi... >>>>> <https://copr.fedorainfracloud.org/coprs/ovirt/ovirt-master-snapshot/permi... >>>>> I can handle for you the addition of the package to the compose, ping me as needed. >>>>> >>>>> Several packages have been already updated to match this flow, you >>>>> can see current status here: >>>>> https://copr.fedorainfracloud.org/coprs/ovirt/ovirt-master-snapshot/monitor/ >>>>> <https://copr.fedorainfracloud.org/coprs/ovirt/ovirt-master-snapshot/monit... >>>>> If you don't see your oVirt subproject there, please help getting it ready. >>>>> I'm going to track status here: >>>>> https://github.com/oVirt/ovirt-site/wiki/Building-oVirt-on-COPR >>>>> <https://github.com/oVirt/ovirt-site/wiki/Building-oVirt-on-COPR> >>>>> (not yet started, give me a day :-) ) >>>>> >>>>> Please note this cover only for the "build-artifacts" stage and is >>>>> meant to provide builds of the project on a per patch way. >>>>> >>>>> We are aiming at having all projects built in copr by the end of November 2021. >>>>> Once this will be completed we can drop build-artifacts related code >>>>> from the repos and free jenkins resources. >>>>> >>>>> For official releases I'm looking into using the CentOS Community >>>>> Build System https://cbs.centos.org/koji/ >>>>> <https://cbs.centos.org/koji/>;. >>>>> Maintainers willing to take care of the build and release of their >>>>> own packages are welcome to join the CentOS Virtualization SIG >>>>> https://wiki.centos.org/SpecialInterestGroup/Virtualization >>>>> <https://wiki.centos.org/SpecialInterestGroup/Virtualization> . This >>>>> should replace the releng-tools flow we are currently using for >>>>> shipping releases on https://resources.ovirt.org/pub/ >>>>> <https://resources.ovirt.org/pub/> >>>>> >>>>> >>>>> >>>>> Il giorno gio 21 ott 2021 alle ore 14:38 Sandro Bonazzola >>>>> <sbonazzo(a)redhat.com <mailto:sbonazzo@redhat.com>> ha scritto: >>>>> Dear community members, >>>>> >>>>> We would like to take some next steps in improving usability and >>>>> general decision making for our community and are interested in your >>>>> thoughts and suggestions. >>>>> >>>>> The first step is a decision regarding our version control and >>>>> workflows associated with it. Contributions to new features and >>>>> general improvements for oVirt outside of Red Hat have been rare and >>>>> we would like to hear from you if we could simplify this process for >>>>> you. One suggestion we have is that we could simplify the >>>>> contribution process and improve the contributor experience by >>>>> moving our repositories to GitHub or GitLab. >>>>> >>>>> Currently, our Gerrit (gerrit.ovirt.org <http://gerrit.ovirt.org/>) >>>>> instance is mirrored to GitHub and we already have several >>>>> repositories that are exclusively developed there, including their >>>>> CI running on GitHub Actions, for example: >>>>> * https://github.com/oVirt/go-ovirt-client <https://github.com/oVirt/go-ovirt-client> >>>>> * https://github.com/oVirt/512-byte-vm <https://github.com/oVirt/512-byte-vm> >>>>> >>>>> There are also various related projects that run on GitLab, such as: >>>>> * https://gitlab.com/qemu-project <https://gitlab.com/qemu-project> >>>>> * https://gitlab.com/libvirt <https://gitlab.com/libvirt> >>>>> >>>>> One recent project that moved from Gerrit to GitLab with a very >>>>> similar discussion is mediawiki: >>>>> https://www.mediawiki.org/wiki/GitLab_consultation/Discussion_summary >>>>> <https://www.mediawiki.org/wiki/GitLab_consultation/Discussion_summary> >>>>> >>>>> Once we hear more of your thoughts around this and if the decision >>>>> falls on moving to either GitHub or GitLab, our plan would be to >>>>> start changing existing workflows and CI project by project as it >>>>> makes sense. Both platforms offer similar features. Two key points >>>>> that stand out for us are that: >>>>> * GitLab is open source while GitHub isn't >>>>> * GitHub is more visible which allows contributors to amplify their >>>>> contributions and raise their profile by contributing to various big >>>>> projects, including oVirt >>>>> >>>>> We are looking forward to hearing your thoughts, >>>>> >>>>> -- >>>>> Sandro Bonazzola >>>>> MANAGER, SOFTWARE ENGINEERING, EMEA R&D RHV >>>>> Red Hat EMEA <https://www.redhat.com/> >>>>> sbonazzo(a)redhat.com <mailto:sbonazzo@redhat.com> >>>>> <https://www.redhat.com/> >>>>> Red Hat respects your work life balance. Therefore there is no need >>>>> to answer this email out of your office hours. >>>>> >>>>> >>>>> >>>>> >>>>> -- >>>>> Sandro Bonazzola >>>>> MANAGER, SOFTWARE ENGINEERING, EMEA R&D RHV >>>>> Red Hat EMEA <https://www.redhat.com/> >>>>> sbonazzo(a)redhat.com <mailto:sbonazzo@redhat.com> >>>>> <https://www.redhat.com/> >>>>> Red Hat respects your work life balance. Therefore there is no need >>>>> to answer this email out of your office hours. >>>>> >>>>> >>>>> >>>>> >>>>> -- >>>>> Sandro Bonazzola >>>>> MANAGER, SOFTWARE ENGINEERING, EMEA R&D RHV >>>>> Red Hat EMEA <https://www.redhat.com/> >>>>> sbonazzo(a)redhat.com <mailto:sbonazzo@redhat.com> >>>>> <https://www.redhat.com/> >>>>> Red Hat respects your work life balance. Therefore there is no need >>>>> to answer this email out of your office hours. >>>>> >>>>> >>>>> _______________________________________________ >>>>> Devel mailing list -- devel(a)ovirt.org <mailto:devel@ovirt.org> >>>>> To unsubscribe send an email to devel-leave(a)ovirt.org <mailto:devel-leave@ovirt.org> >>>>> Privacy Statement: https://www.ovirt.org/privacy-policy.html >>>>> <https://www.ovirt.org/privacy-policy.html> >>>>> oVirt Code of Conduct: >>>>> https://www.ovirt.org/community/about/community-guidelines/ >>>>> <https://www.ovirt.org/community/about/community-guidelines/> >>>>> List Archives: >>>>> https://lists.ovirt.org/archives/list/devel@ovirt.org/message/23RJ2FN52ZB... >>>>> <https://lists.ovirt.org/archives/list/devel@ovirt.org/message/23RJ2FN52ZB... >>>> _______________________________________________ >>>> Devel mailing list -- devel(a)ovirt.org >>>> To unsubscribe send an email to devel-leave(a)ovirt.org >>>> Privacy Statement: https://www.ovirt.org/privacy-policy.html >>>> oVirt Code of Conduct: https://www.ovirt.org/community/about/community-guidelines/ >>>> List Archives: >>>> https://lists.ovirt.org/archives/list/devel@ovirt.org/message/SBCBQWZCR5Y... >>> _______________________________________________ >>> Devel mailing list -- devel(a)ovirt.org >>> To unsubscribe send an email to devel-leave(a)ovirt.org >>> Privacy Statement: https://www.ovirt.org/privacy-policy.html >>> oVirt Code of Conduct: https://www.ovirt.org/community/about/community-guidelines/ >>> List Archives: >>> https://lists.ovirt.org/archives/list/devel@ovirt.org/message/ILNA5W6BEFN... >>