On 6. 12. 2021, at 12:16, Martin Perina <mperina@redhat.com> wrote:
Hi,
I've tried to move ovirt-engine-extensions-api <https://github.com/ovirt/ovirt-engine-extensions-api> to github and setup GH action to build to have some sort of a basic template for all our Java based projects:
https://github.com/oVirt/ovirt-engine-extensions-api/blob/master/.github/wor... <https://github.com/oVirt/ovirt-engine-extensions-api/blob/master/.github/workflows/build.yml>
During the work I discovered some parts which should be further discussed:
1. Most of projects requires packages from ovirt-master-release RPM repositories to build itself - In past we have been using https://resources.ovirt.org/pub/yum-repo/ovirt-release-master.rpm <https://resources.ovirt.org/pub/yum-repo/ovirt-release-master.rpm> to fetch the latest version of this RPM - Do we plan to keep it and have this URL as the main source for oVirt master repositories? - ovirt-engine-extensions-api projects requires only Virt SIG repo so I've applied below hack to provide it: https://github.com/oVirt/ovirt-engine-extensions-api/blob/master/.automation... <https://github.com/oVirt/ovirt-engine-extensions-api/blob/master/.automation/prepare-env-cs8.sh#L10> - I don't like the solution, it would be much nicer to have a constant URL for the latest ovirt-release-master RPM
I'd say the canonical way to use it should be dnf copr enable ovirt/ovirt-master-snapshot dnf install ovirt-release-master
2. Providing RPM repositories for different OS inside single build artifact - I wanted to build the project for both CentOS Stream 8 and 9, but I figured out that upload-artifact action merges results for all different OS builds into a single directory, so created repositories are mixed together. - I've use GH strategy option and used centos-stream-8 for CS8 and centos-stream-9 for CS9 builds: https://github.com/oVirt/ovirt-engine-extensions-api/blob/master/.github/wor... <https://github.com/oVirt/ovirt-engine-extensions-api/blob/master/.github/workflows/build.yml#L14> - Do we already have that documented so an OST plugin to use those artifacts per OS can be created? If not, where to put it?
not yet. but we wanted to "develop" a unified artifact structiure with repodata so taht it can be then consumed as a repo elsewhere
3. Using maven cache for Java project builds - When build maven projects in jenkins.ovirt.org <http://jenkins.ovirt.org/> we were using artifactory.ovirt.org <http://artifactory.ovirt.org/> to cache maven artifacts (and also save download bandwidth to PHX datacenter) - It doesn't make to use artifactory.ovirt.org <http://artifactory.ovirt.org/> inside GH actions, so I've tried to define a GH action specific cache: https://github.com/oVirt/ovirt-engine-extensions-api/blob/master/.github/wor... <https://github.com/oVirt/ovirt-engine-extensions-api/blob/master/.github/workflows/build.yml#L37>
4. Caching required dependencies - Above maven dependencies helps for maven invocations before RPM build but it doesn't help to fetch all required RPM dependencies for pro RPM maven based build - Right now it's almost 500 GB of packages to download and install on top of the default container and it for ovirt-engine-extensions-api it takes 50 % of the whole build time just to download and install those packages (I know it's just 1 minute, but anyway :-) - I've noticed that VDSM is using its own customized container <https://github.com/oVirt/vdsm/blob/master/.github/workflows/ci.yml#L8> - Wouldn't it be worthwhile to also create customized containers for Java based projects?
I think our own container with these dependencies would be great
5. Differences between CS8 and CS9 - There are again some disturbing changes between CS8 and CS9: - PowerTools repo from CS8 is now called CRB in CS9 - pki-deps javapackages-tools modules from CS8 don't exist in CS9 - Core DNF plugins are not installed by default in CS9 container - To bypass those differences I've create 2 different shell scripts and plug them into a single workflow job: https://github.com/oVirt/ovirt-engine-extensions-api/blob/master/.github/wor... <https://github.com/oVirt/ovirt-engine-extensions-api/blob/master/.github/workflows/build.yml#L33> https://github.com/oVirt/ovirt-engine-extensions-api/blob/master/.automation... <https://github.com/oVirt/ovirt-engine-extensions-api/blob/master/.automation/prepare-env-cs8.sh> https://github.com/oVirt/ovirt-engine-extensions-api/blob/master/.automation... <https://github.com/oVirt/ovirt-engine-extensions-api/blob/master/.automation/prepare-env-cs9.sh>
6. git commands cannot be easily used inside GH action - GH doesn't checkout the project completely during checkout action, so .git directory is not present - In the past I've been using 'git archive' <https://gerrit.ovirt.org/gitweb?p=ovirt-engine-extensions-api.git;a=blob;f=automation/build-artifacts.sh;h=965012c1195eaf288e149bde71e64b2e9647b2c4;hb=refs/heads/master#l45> to create .tar.gz source archive, but on GH it needed to be switched to a classic tar <https://github.com/oVirt/ovirt-engine-extensions-api/blob/master/.automation/build-artifacts.sh#L17> - In the past I've been using 'git rev-list' <https://gerrit.ovirt.org/gitweb?p=ovirt-engine-extensions-api.git;a=blob;f=automation/build-artifacts.sh;h=965012c1195eaf288e149bde71e64b2e9647b2c4;hb=refs/heads/master#l39> to fetch short git hash for snapshot builds, but on GH it needed to be switched to below hack <https://github.com/oVirt/ovirt-engine-extensions-api/blob/master/.github/workflows/build.yml#L47>
did you try options from https://github.com/actions/checkout#usage? Thanks, michal
Regards, Martin
-- Martin Perina Manager, Software Engineering Red Hat Czech s.r.o. _______________________________________________ Devel mailing list -- devel@ovirt.org To unsubscribe send an email to devel-leave@ovirt.org Privacy Statement: https://www.ovirt.org/privacy-policy.html oVirt Code of Conduct: https://www.ovirt.org/community/about/community-guidelines/ List Archives: https://lists.ovirt.org/archives/list/devel@ovirt.org/message/PBUC7OKVHJJPDL...