6:43 a.m.
Il giorno lun 6 dic 2021 alle ore 12:18 Martin Perina <mperina(a)redhat.com>
ha scritto:
Hi,
I've tried to move ovirt-engine-extensions-api
<https://github.com/ovirt/ovirt-engine-extensions-api> to github and
setup GH action to build to have some sort of a basic template for all our
Java based projects:
https://github.com/oVirt/ovirt-engine-extensions-api/blob/master/.github/...
During the work I discovered some parts which should be further discussed:
1. Most of projects requires packages from ovirt-master-release RPM
repositories to build itself
- In past we have been using
https://resources.ovirt.org/pub/yum-repo/ovirt-release-master.rpm to
fetch the latest version of this RPM
- Do we plan to keep it and have this URL as the main source for oVirt
master repositories?
No.
Please follow
https://ovirt.org/develop/dev-process/install-nightly-snapshot.html
- ovirt-engine-extensions-api projects requires only Virt SIG
repo so
I've applied below hack to provide it:
https://github.com/oVirt/ovirt-engine-extensions-api/blob/master/.automat...
- I don't like the solution, it would be much nicer to have a constant
URL for the latest ovirt-release-master RPM
Instrucitons in
https://ovirt.org/develop/dev-process/install-nightly-snapshot.html whould
be stable.
For CentOS Virt SIG specifically I haven't yet built the 4.5 release rpms
for CentOS . Once they'll be ready you should be able to just dnf install
the centos release ovirt-4.5 rpm to enable them. I'll try to handle it
before Christmas.
2. Providing RPM repositories for different OS inside single build artifact
- I wanted to build the project for both CentOS Stream 8 and 9, but I
figured out that upload-artifact action merges results for all different OS
builds into a single directory, so created repositories are mixed together.
This may have a solution, try using the wildcard pattern
https://github.com/actions/upload-artifact#upload-using-a-wildcard-pattern
- I've use GH strategy option and used centos-stream-8 for
CS8 and
centos-stream-9 for CS9 builds:
https://github.com/oVirt/ovirt-engine-extensions-api/blob/master/.github/...
- Do we already have that documented so an OST plugin to use those
artifacts per OS can be created? If not, where to put it?
Didn't see one yet.
3. Using maven cache for Java project builds
- When build maven projects in jenkins.ovirt.org we were using
artifactory.ovirt.org to cache maven artifacts (and also save download
bandwidth to PHX datacenter)
- It doesn't make to use artifactory.ovirt.org inside GH actions, so
I've tried to define a GH action specific cache:
https://github.com/oVirt/ovirt-engine-extensions-api/blob/master/.github/...
make a lot of sense.
4. Caching required dependencies
- Above maven dependencies helps for maven invocations before RPM
build but it doesn't help to fetch all required RPM dependencies for pro
RPM maven based build
- Right now it's almost 500 GB of packages to download and install on
top of the default container and it for ovirt-engine-extensions-api it
takes 50 % of the whole build
time just to download and install those packages (I know it's just 1
minute, but anyway :-)
- I've noticed that VDSM is using its own customized container
<https://github.com/oVirt/vdsm/blob/master/.github/workflows/ci.yml#L8>
- Wouldn't it be worthwhile to also create customized containers for
Java based projects?
I got suggestion from @Yaniv Kaul <ykaul(a)redhat.com> to do something as in
Gluster: https://github.com/gluster/Gluster-Builds - they are building
Gluster RPMs by running a container (the 'build container'), which has the
required dependencies and they updates once a month, or when there's a new
dependency.
If someone volunteer to maintain those containers, it may save build time
and bandwidth.
5. Differences between CS8 and CS9
- There are again some disturbing changes between CS8 and CS9:
- PowerTools repo from CS8 is now called CRB in CS9
- pki-deps javapackages-tools modules from CS8 don't exist in CS9
- Core DNF plugins are not installed by default in CS9 container
- To bypass those differences I've create 2 different shell scripts
and plug them into a single workflow job:
https://github.com/oVirt/ovirt-engine-extensions-api/blob/master/.github/...
https://github.com/oVirt/ovirt-engine-extensions-api/blob/master/.automat...
https://github.com/oVirt/ovirt-engine-extensions-api/blob/master/.automat...
6. git commands cannot be easily used inside GH action
- GH doesn't checkout the project completely during checkout action,
so .git directory is not present
- In the past I've been using 'git archive'
<https://gerrit.ovirt.org/gitweb?p=ovirt-engine-extensions-api.git;a=blob;...
to create .tar.gz source archive, but on GH it needed to be switched to a classic
tar
<https://github.com/oVirt/ovirt-engine-extensions-api/blob/master/.automat...
- In the past I've been using 'git rev-list'
<https://gerrit.ovirt.org/gitweb?p=ovirt-engine-extensions-api.git;a=blob;...
to fetch short git hash for snapshot builds, but on GH it needed to be
switched to below hack
<https://github.com/oVirt/ovirt-engine-extensions-api/blob/master/.github/...
You can dnf install git before running checkout action and then configure
checkout action to download full git content.
```
- uses: actions/checkout@v2
with:
fetch-depth: 0
```
Regards,
Martin
--
Martin Perina
Manager, Software Engineering
Red Hat Czech s.r.o.
_______________________________________________
Devel mailing list -- devel(a)ovirt.org
To unsubscribe send an email to devel-leave(a)ovirt.org
Privacy Statement: https://www.ovirt.org/privacy-policy.html
oVirt Code of Conduct:
https://www.ovirt.org/community/about/community-guidelines/
List Archives:
https://lists.ovirt.org/archives/list/devel@ovirt.org/message/PBUC7OKVHJJ...
--
Sandro Bonazzola
MANAGER, SOFTWARE ENGINEERING, EMEA R&D RHV
Red Hat EMEA <https://www.redhat.com/>
sbonazzo(a)redhat.com
<https://www.redhat.com/>
*Red Hat respects your work life balance. Therefore there is no need to
answer this email out of your office hours.*