[ovirt-devel] Re: Switching to GH actions build: some not-yet-clear parts and some tips

On 6. 12. 2021, at 12:16, Martin Perina <mperina(a)redhat.com&gt; wrote: Hi, I've tried to move ovirt-engine-extensions-api <https://github.com/ovirt/ovirt-engine-extensions-api> to github and setup GH action to build to have some sort of a basic template for all our Java based projects: https://github.com/oVirt/ovirt-engine-extensions-api/blob/master/.github/... <https://github.com/oVirt/ovirt-engine-extensions-api/blob/master/.github/... During the work I discovered some parts which should be further discussed: 1. Most of projects requires packages from ovirt-master-release RPM repositories to build itself - In past we have been using https://resources.ovirt.org/pub/yum-repo/ovirt-release-master.rpm <https://resources.ovirt.org/pub/yum-repo/ovirt-release-master.rpm> to fetch the latest version of this RPM - Do we plan to keep it and have this URL as the main source for oVirt master repositories? - ovirt-engine-extensions-api projects requires only Virt SIG repo so I've applied below hack to provide it: https://github.com/oVirt/ovirt-engine-extensions-api/blob/master/.automat... <https://github.com/oVirt/ovirt-engine-extensions-api/blob/master/.automat... - I don't like the solution, it would be much nicer to have a constant URL for the latest ovirt-release-master RPM

2. Providing RPM repositories for different OS inside single build artifact - I wanted to build the project for both CentOS Stream 8 and 9, but I figured out that upload-artifact action merges results for all different OS builds into a single directory, so created repositories are mixed together. - I've use GH strategy option and used centos-stream-8 for CS8 and centos-stream-9 for CS9 builds: https://github.com/oVirt/ovirt-engine-extensions-api/blob/master/.github/... <https://github.com/oVirt/ovirt-engine-extensions-api/blob/master/.github/... - Do we already have that documented so an OST plugin to use those artifacts per OS can be created? If not, where to put it?

3. Using maven cache for Java project builds - When build maven projects in jenkins.ovirt.org <http://jenkins.ovirt.org/> we were using artifactory.ovirt.org <http://artifactory.ovirt.org/> to cache maven artifacts (and also save download bandwidth to PHX datacenter) - It doesn't make to use artifactory.ovirt.org <http://artifactory.ovirt.org/> inside GH actions, so I've tried to define a GH action specific cache: https://github.com/oVirt/ovirt-engine-extensions-api/blob/master/.github/... <https://github.com/oVirt/ovirt-engine-extensions-api/blob/master/.github/... 4. Caching required dependencies - Above maven dependencies helps for maven invocations before RPM build but it doesn't help to fetch all required RPM dependencies for pro RPM maven based build - Right now it's almost 500 GB of packages to download and install on top of the default container and it for ovirt-engine-extensions-api it takes 50 % of the whole build time just to download and install those packages (I know it's just 1 minute, but anyway :-) - I've noticed that VDSM is using its own customized container <https://github.com/oVirt/vdsm/blob/master/.github/workflows/ci.yml#L8> - Wouldn't it be worthwhile to also create customized containers for Java based projects?

5. Differences between CS8 and CS9 - There are again some disturbing changes between CS8 and CS9: - PowerTools repo from CS8 is now called CRB in CS9 - pki-deps javapackages-tools modules from CS8 don't exist in CS9 - Core DNF plugins are not installed by default in CS9 container - To bypass those differences I've create 2 different shell scripts and plug them into a single workflow job: https://github.com/oVirt/ovirt-engine-extensions-api/blob/master/.github/... <https://github.com/oVirt/ovirt-engine-extensions-api/blob/master/.github/... https://github.com/oVirt/ovirt-engine-extensions-api/blob/master/.automat... <https://github.com/oVirt/ovirt-engine-extensions-api/blob/master/.automat... https://github.com/oVirt/ovirt-engine-extensions-api/blob/master/.automat... <https://github.com/oVirt/ovirt-engine-extensions-api/blob/master/.automat... 6. git commands cannot be easily used inside GH action - GH doesn't checkout the project completely during checkout action, so .git directory is not present - In the past I've been using 'git archive' <https://gerrit.ovirt.org/gitweb?p=ovirt-engine-extensions-api.git;a=blob;... to create .tar.gz source archive, but on GH it needed to be switched to a classic tar <https://github.com/oVirt/ovirt-engine-extensions-api/blob/master/.automat... - In the past I've been using 'git rev-list' <https://gerrit.ovirt.org/gitweb?p=ovirt-engine-extensions-api.git;a=blob;... to fetch short git hash for snapshot builds, but on GH it needed to be switched to below hack <https://github.com/oVirt/ovirt-engine-extensions-api/blob/master/.github/...

Regards, Martin -- Martin Perina Manager, Software Engineering Red Hat Czech s.r.o. _______________________________________________ Devel mailing list -- devel(a)ovirt.org To unsubscribe send an email to devel-leave(a)ovirt.org Privacy Statement: https://www.ovirt.org/privacy-policy.html oVirt Code of Conduct: https://www.ovirt.org/community/about/community-guidelines/ List Archives: https://lists.ovirt.org/archives/list/devel@ovirt.org/message/PBUC7OKVHJJ...