Package building and build process

Mon May 5 16:50:42 UTC 2014

On 05/05/2014 06:26 PM, Alon Bar-Lev wrote:
> 
> 
> ----- Original Message -----
>> From: "David Caro" <dcaroest at redhat.com>
>> To: "Alon Bar-Lev" <alonbl at redhat.com>, "infra" <infra at ovirt.org>
>> Sent: Monday, May 5, 2014 7:10:31 PM
>> Subject: Re: Package building and build process
>>
>> On 05/05/2014 05:51 PM, Alon Bar-Lev wrote:
>>>
>>>
>>> ----- Original Message -----
>>>> From: "David Caro" <dcaroest at redhat.com>
>>>> To: "infra" <infra at ovirt.org>
>>>> Sent: Monday, May 5, 2014 6:39:53 PM
>>>> Subject: Package building and build process
>>>>
>>>> Hi!
>>>>
>>>> Lately we had some issues when trying to build some of the projects, and
>>>> we
>>>> wanted to start a discussion to clarify what do we want and how to get
>>>> there.
>>>>
>>>> We can split the thread if it gets too large for all the subjects.
>>>>
>>>> The main points to discuss:
>>>>
>>>> * How the source tarballs should be built
>>>>   · Should we use a jenkins job for it, so the maintainer just downloads
>>>>   the
>>>>     tarball from it
>>>>   · Should we force the maintainer to build it and supply the tarball
>>>>   himself
>>>>   · Should we allow both
>>>
>>> Not sure why you manage this discussion.
>>> You just forgot to mention that up until now it worked perfectly ok, we had
>>> a job to build releases out of tarballs, and all worked perfectly.
>>> It was broken when you decided not to install certain dependencies on build
>>> slaves.
>>
>> Arising from different version of some projects requiring different version
>> of
>> different packages and different test requiring different sources for the
>> same
>> packages (testing against nightly, stable, pep8 latest, pep8 epel, ...) we
>> had
>> to introduce some job isolation, and reserving one slave for each job that is
>> running became unproductive and hard to maintain (at first, we had some
>> slaves
>> for each project, then started installing and removing packages on each job,
>> then not only installing and removing packages but also repositories,
>> isolating
>> the build on the same slave became a necessity).
>>
>>>
>>> Anyway...
>>>
>>> This is up to maintainer to decide.
>>
>> It's up to the CI/infra/release team to provide the platform to build the
>> projects for the maintainers to use.
> 
> As a service to the group.

Yes, as a service to the group and ultimately to the project I'm part of the
people that decides how to manage and provide that infrastructure.

> 
>>
>>> Build infrastructure should be able to get upstream tarball (any upstream
>>> not just ovirt) and produce binaries for all our supported platforms as a
>>> service, just like koji, brew or any downstream service.
>>> The root resource of open source is the source tarball, this is the release
>>> not the binary.
>> That's what the next point is about.
>>
>>> People here at ovirt tend to confuse what open source is.
>>>
>>>>
>>>> * Separate source release from binary release
>>>>
>>>> * Which tools to use to build the rpms
>>>>   · plain rpmbuild
>>>>   · Mock
>>>>   · Docker
>>>>   · copr
>>>>   · Open Build System
>>>>   · koji
>>>
>>> Once upstream releases source tarball, this is not important.
>>> This is the entire idea of open *SOURCE*.
>> It might not be important to you, but we want to provide the easiest way for
>> the
>> users to use oVirt, not only for someone to build it, having a binary release
>> for the major platforms helps TREMENDOUSLY on getting users to use the
>> product.
> 
> Once again... and again... and again....
> 
> The release is the source tarball.
I already acked this.

> We provide the service of providing binaries as well.
> This is fine, this is a service.
> 
>>
>>>
>>>> * How to organize the rpm/tar build jobs
>>>>   · One job for all the projects
>>>>   · One job per-project
>>>
>>> If we cannot provide single job for generic tarball build, we have a huge
>>> problem.
>> I don't agree.
> 
> You do not need to agree, you should provide it as a service.
Yes, I need to agree, as I've been appointed by the group as part of the
decision making process that has responsibility over that service, you are not.
And as such you haven't earned the right to tell anyone in this team what they
should and what they should not do, you are welcome to share your opinion, but
don't expect anyone to follow your orders.

> 
>>
>>>
>>> I worked very hard with all maintainers (except vdsm) to be able to achieve
>>> the standard sequence of:
>>>
>>> # rpmbuild -tb <tarball>
>>
>> You are not taking into account all the dependencies that rpmbuild -tb
>> <tarball>
>> has, each project has it's own, and they are incompatible between projects
>> and
>> between versions. And not just building the rpms, building the tarballs also
>> have some requirements that must be met. So building a package is not just
>> rpmbuild, but setup the repos, install packages, then rpmbuild, then cleanup
>> packages and restore repos.
> 
> rpmbuild -ts <tarball>
> yum-builddep <srpm>
> 
> solves this as well, adding optional repositories can be a parameter to the job.

I did not say that it was not doable, only that we might want to replicate it
per-project for the sake of organization and easy management.

> 
> all was already done nothing to reinvent.
> 
>>
>>>
>>> Regards,
>>> Alon Bar-Lev.
>>>
>>>>
>>>> PD. Opened an etherpad to keep the points
>>>> http://etherpad.ovirt.org/p/build_discussion
>>>> --
>>>> David Caro
>>>>
>>>> Red Hat S.L.
>>>> Continuous Integration Engineer - EMEA ENG Virtualization R&D
>>>>
>>>> Email: dcaro at redhat.com
>>>> Web: www.redhat.com
>>>> RHT Global #: 82-62605
>>>>
>>>>
>>>> _______________________________________________
>>>> Infra mailing list
>>>> Infra at ovirt.org
>>>> http://lists.ovirt.org/mailman/listinfo/infra
>>>>
>>
>>
>> --
>> David Caro
>>
>> Red Hat S.L.
>> Continuous Integration Engineer - EMEA ENG Virtualization R&D
>>
>> Email: dcaro at redhat.com
>> Web: www.redhat.com
>> RHT Global #: 82-62605
>>
>>

-- 
David Caro

Red Hat S.L.
Continuous Integration Engineer - EMEA ENG Virtualization R&D

Email: dcaro at redhat.com
Web: www.redhat.com
RHT Global #: 82-62605

-------------- next part --------------
A non-text attachment was scrubbed...
Name: signature.asc
Type: application/pgp-signature
Size: 473 bytes
Desc: OpenPGP digital signature
URL: <http://lists.ovirt.org/pipermail/infra/attachments/20140505/cd04221a/attachment.sig>