Package building and build process

[Alon Bar-Lev]

----- Original Message -----
> From: "David Caro" <dcaroest at redhat.com>
> To: "Alon Bar-Lev" <alonbl at redhat.com>, "infra" <infra at ovirt.org>
> Sent: Monday, May 5, 2014 7:10:31 PM
> Subject: Re: Package building and build process
> 
> On 05/05/2014 05:51 PM, Alon Bar-Lev wrote:
> > 
> > 
> > ----- Original Message -----
> >> From: "David Caro" <dcaroest at redhat.com>
> >> To: "infra" <infra at ovirt.org>
> >> Sent: Monday, May 5, 2014 6:39:53 PM
> >> Subject: Package building and build process
> >>
> >> Hi!
> >>
> >> Lately we had some issues when trying to build some of the projects, and
> >> we
> >> wanted to start a discussion to clarify what do we want and how to get
> >> there.
> >>
> >> We can split the thread if it gets too large for all the subjects.
> >>
> >> The main points to discuss:
> >>
> >> * How the source tarballs should be built
> >>   · Should we use a jenkins job for it, so the maintainer just downloads
> >>   the
> >>     tarball from it
> >>   · Should we force the maintainer to build it and supply the tarball
> >>   himself
> >>   · Should we allow both
> > 
> > Not sure why you manage this discussion.
> > You just forgot to mention that up until now it worked perfectly ok, we had
> > a job to build releases out of tarballs, and all worked perfectly.
> > It was broken when you decided not to install certain dependencies on build
> > slaves.
> 
> Arising from different version of some projects requiring different version
> of
> different packages and different test requiring different sources for the
> same
> packages (testing against nightly, stable, pep8 latest, pep8 epel, ...) we
> had
> to introduce some job isolation, and reserving one slave for each job that is
> running became unproductive and hard to maintain (at first, we had some
> slaves
> for each project, then started installing and removing packages on each job,
> then not only installing and removing packages but also repositories,
> isolating
> the build on the same slave became a necessity).
> 
> > 
> > Anyway...
> > 
> > This is up to maintainer to decide.
> 
> It's up to the CI/infra/release team to provide the platform to build the
> projects for the maintainers to use.

As a service to the group.

> 
> > Build infrastructure should be able to get upstream tarball (any upstream
> > not just ovirt) and produce binaries for all our supported platforms as a
> > service, just like koji, brew or any downstream service.
> > The root resource of open source is the source tarball, this is the release
> > not the binary.
> That's what the next point is about.
> 
> > People here at ovirt tend to confuse what open source is.
> > 
> >>
> >> * Separate source release from binary release
> >>
> >> * Which tools to use to build the rpms
> >>   · plain rpmbuild
> >>   · Mock
> >>   · Docker
> >>   · copr
> >>   · Open Build System
> >>   · koji
> > 
> > Once upstream releases source tarball, this is not important.
> > This is the entire idea of open *SOURCE*.
> It might not be important to you, but we want to provide the easiest way for
> the
> users to use oVirt, not only for someone to build it, having a binary release
> for the major platforms helps TREMENDOUSLY on getting users to use the
> product.

Once again... and again... and again....

The release is the source tarball.
We provide the service of providing binaries as well.
This is fine, this is a service.

> 
> > 
> >> * How to organize the rpm/tar build jobs
> >>   · One job for all the projects
> >>   · One job per-project
> > 
> > If we cannot provide single job for generic tarball build, we have a huge
> > problem.
> I don't agree.

You do not need to agree, you should provide it as a service.

> 
> > 
> > I worked very hard with all maintainers (except vdsm) to be able to achieve
> > the standard sequence of:
> > 
> > # rpmbuild -tb <tarball>
> 
> You are not taking into account all the dependencies that rpmbuild -tb
> <tarball>
> has, each project has it's own, and they are incompatible between projects
> and
> between versions. And not just building the rpms, building the tarballs also
> have some requirements that must be met. So building a package is not just
> rpmbuild, but setup the repos, install packages, then rpmbuild, then cleanup
> packages and restore repos.

rpmbuild -ts <tarball>
yum-builddep <srpm>

solves this as well, adding optional repositories can be a parameter to the job.

all was already done nothing to reinvent.

> 
> > 
> > Regards,
> > Alon Bar-Lev.
> > 
> >>
> >> PD. Opened an etherpad to keep the points
> >> http://etherpad.ovirt.org/p/build_discussion
> >> --
> >> David Caro
> >>
> >> Red Hat S.L.
> >> Continuous Integration Engineer - EMEA ENG Virtualization R&D
> >>
> >> Email: dcaro at redhat.com
> >> Web: www.redhat.com
> >> RHT Global #: 82-62605
> >>
> >>
> >> _______________________________________________
> >> Infra mailing list
> >> Infra at ovirt.org
> >> http://lists.ovirt.org/mailman/listinfo/infra
> >>
> 
> 
> --
> David Caro
> 
> Red Hat S.L.
> Continuous Integration Engineer - EMEA ENG Virtualization R&D
> 
> Email: dcaro at redhat.com
> Web: www.redhat.com
> RHT Global #: 82-62605
> 
>