release repo structure and 3.3.2

Itamar Heim iheim at redhat.com
Thu Jan 16 11:53:48 UTC 2014 

On 01/16/2014 01:49 PM, Sandro Bonazzola wrote:
> Il 16/01/2014 10:13, David Caro ha scritto:
>> El jue 16 ene 2014 01:39:50 CET, Alon Bar-Lev escribió:
>>>
>>> I won't argue more, obviously, you do not understand the point of distributed management.
>> Thanks Alon, always so useful.
>>>
>>> But just think why don't we build large monolithic software.
>> Anyone thinks the same way Alon does?
>> If so, can anyone explain me he's position? (As it seems he's not
>> willing to do so)
>>
>> Also any other opinions? Other Ideas?
>
> IMHO we should keep our repositories as our downstream distro do.
>
> For releases:
> releases/Fedora/19/3.3.0
> releases/Fedora/19/3.3.1
> releases/Fedora/19/3.3.2
> releases/Fedora/19/3.3.3
> releases/Fedora/20/3.4.0

doesn't our upgrade process inside a minor version requires the repo to 
have both new and old versions of the rpm?

> ....
> and
> updates/19/
> updates/20/
>
> for all other packages which are not tied to ovirt release cycle.
>
> I would like also that packages handled downstream as vdsm, -sdk-python and others would be removed from our upstream repository since they're handled
> downstream.
> on new z-stream release ovirt-release will be bumped
> yum update from releases/Fedora/19/3.4.0 will add 3.4.1 as new enabled repository
> yum update from releases/Fedora/19/3.4.1 will remove 3.4.0 and add 3.4.2 and conflict with any <= 3.4.0
> yum update from releases/Fedora/19/3.4.2 will remove 3.4.1 and add 3.4.2 and conflict with any <= 3.4.1
> and so on.
>
> while otopi, log-collector and so on can safely be updated along all 3.4.z cycle.
>
>
>
>>
>>>
>>> ----- Original Message -----
>>>> From: "David Caro" <dcaroest at redhat.com>
>>>> To: "Alon Bar-Lev" <alonbl at redhat.com>
>>>> Cc: "Sandro Bonazzola" <sbonazzo at redhat.com>, "infra" <infra at ovirt.org>, "Kiril Nesenko" <kiril at redhat.com>
>>>> Sent: Thursday, January 16, 2014 2:35:16 AM
>>>> Subject: Re: release repo structure and 3.3.2
>>>>
>>>> El jue 16 ene 2014 01:04:33 CET, Alon Bar-Lev escribió:
>>>>>
>>>>>
>>>>> ----- Original Message -----
>>>>>> From: "David Caro" <dcaroest at redhat.com>
>>>>>> To: "Alon Bar-Lev" <alonbl at redhat.com>
>>>>>> Cc: "Sandro Bonazzola" <sbonazzo at redhat.com>, "infra" <infra at ovirt.org>,
>>>>>> "Kiril Nesenko" <kiril at redhat.com>
>>>>>> Sent: Thursday, January 16, 2014 1:58:08 AM
>>>>>> Subject: Re: release repo structure and 3.3.2
>>>>>>
>>>>>> El mié 15 ene 2014 19:04:04 CET, Alon Bar-Lev escribió:
>>>>>>>
>>>>>>>
>>>>>>> ----- Original Message -----
>>>>>>>> From: "David Caro" <dcaroest at redhat.com>
>>>>>>>> To: "Alon Bar-Lev" <alonbl at redhat.com>
>>>>>>>> Cc: "Sandro Bonazzola" <sbonazzo at redhat.com>, "infra" <infra at ovirt.org>,
>>>>>>>> "Kiril Nesenko" <kiril at redhat.com>
>>>>>>>> Sent: Wednesday, January 15, 2014 5:47:59 PM
>>>>>>>> Subject: Re: release repo structure and 3.3.2
>>>>>>>>
>>>>>>>> El mié 15 ene 2014 16:30:00 CET, Alon Bar-Lev escribió:
>>>>>>>>>
>>>>>>>>>
>>>>>>>>> ----- Original Message -----
>>>>>>>>>> From: "David Caro" <dcaroest at redhat.com>
>>>>>>>>>> To: "Sandro Bonazzola" <sbonazzo at redhat.com>, "Alon Bar-Lev"
>>>>>>>>>> <alonbl at redhat.com>, "infra" <infra at ovirt.org>
>>>>>>>>>> Cc: "Kiril Nesenko" <kiril at redhat.com>
>>>>>>>>>> Sent: Wednesday, January 15, 2014 5:26:32 PM
>>>>>>>>>> Subject: Re: release repo structure and 3.3.2
>>>>>>>>>>
>>>>>>>>>> El 07/01/14 15:31, Sandro Bonazzola escribió:
>>>>>>>>>>> Il 01/01/2014 10:42, Alon Bar-Lev ha scritto:
>>>>>>>>>>>> Hi,
>>>>>>>>>>>>
>>>>>>>>>>>> For some reason there 3.3.2 z-stream was released in its own
>>>>>>>>>>>> repository
>>>>>>>>>>>> so
>>>>>>>>>>>> people that are subscribed to stable[1] did not get it.
>>>>>>>>>>>
>>>>>>>>>>> Why not?
>>>>>>>>>>> stable release had ovirt-release-10 which enabled both stable and
>>>>>>>>>>> 3.3.2
>>>>>>>>>>> repository by yum updating it.
>>>>>>>>>>>
>>>>>>>>>>>>
>>>>>>>>>>>> There is no much sense in releasing fix release that people do not
>>>>>>>>>>>> get
>>>>>>>>>>>> in
>>>>>>>>>>>> simple "yum update".
>>>>>>>>>>>>
>>>>>>>>>>>> Also the following is now broken of most packages' spec:
>>>>>>>>>>>> Source0:
>>>>>>>>>>>> http://ovirt.org/releases/stable/src/@PACKAGE_NAME@-@PACKAGE_VERSION@.tar.gz
>>>>>>>>>>>>
>>>>>>>>>>>> For each minor we should have rolling repository, to reduce noise
>>>>>>>>>>>> and
>>>>>>>>>>>> provide service.
>>>>>>>>>>>>
>>>>>>>>>>>> All released tarballs (sources) should be stored at fixed location
>>>>>>>>>>>> to
>>>>>>>>>>>> allow distro specific code to fetch, the location must be synced
>>>>>>>>>>>> with
>>>>>>>>>>>> what we publish.
>>>>>>>>>>>>
>>>>>>>>>>>> Immediate action is to move the 3.3.2 content into the stable
>>>>>>>>>>>> directory.
>>>>>>>>>>>
>>>>>>>>>>> So previous request of having each release in its own repository has
>>>>>>>>>>> been
>>>>>>>>>>> retired?
>>>>>>>>>>> Or is it combined?
>>>>>>>>>>> Do we want stable to be a rolling repository and have also a
>>>>>>>>>>> repository
>>>>>>>>>>> for
>>>>>>>>>>> each version?
>>>>>>>>>>> I'm not against having rolling packages in just one stable
>>>>>>>>>>> repository,
>>>>>>>>>>> I
>>>>>>>>>>> just want to understand what is the desired structure of the
>>>>>>>>>>> repositories.
>>>>>>>>>>
>>>>>>>>>> I am, having a stable repository with rolling rpms is a lot more hard
>>>>>>>>>> to
>>>>>>>>>> manage
>>>>>>>>>> and maintain than having separated individual complete repos.
>>>>>>>>>>
>>>>>>>>>> Because what we are actually delivering is not a specific rpm, but the
>>>>>>>>>> whole
>>>>>>>>>> set, that is, one repository with the set of rpms that were tested
>>>>>>>>>> together
>>>>>>>>>> and
>>>>>>>>>> validated. If at any point you want to mix them, you still can adding
>>>>>>>>>> the
>>>>>>>>>> other
>>>>>>>>>> repos.
>>>>>>>>>>
>>>>>>>>>> For updates just updating the directory where the 'stable' link points
>>>>>>>>>> gets
>>>>>>>>>> it done.
>>>>>>>>>>
>>>>>>>>>> For rollbacks you'll have to configure the old repo. That is not as
>>>>>>>>>> annoying
>>>>>>>>>> as
>>>>>>>>>> it might seem, because when you enable the stable repo, you want to
>>>>>>>>>> have
>>>>>>>>>> the
>>>>>>>>>> stable version, that changes with time. If you want to rollback to a
>>>>>>>>>> previous
>>>>>>>>>> version then just use that versions specific repo. At much we can
>>>>>>>>>> provide
>>>>>>>>>> a
>>>>>>>>>> link
>>>>>>>>>> like 'previous_stable' so if you want to rollback to the previous
>>>>>>>>>> version
>>>>>>>>>> you
>>>>>>>>>> can use --enablerepo=previous_version easily, but if you want to keep
>>>>>>>>>> using
>>>>>>>>>> that, you should point directly to the specific version you want tot
>>>>>>>>>> use.
>>>>>>>>>>
>>>>>>>>>> Creating a new repository using is almost as cheap (on hard disk
>>>>>>>>>> space)
>>>>>>>>>> as
>>>>>>>>>> having a rolling repository, if you use hard links, so we can create
>>>>>>>>>> lot's
>>>>>>>>>> of
>>>>>>>>>> them, specially for small changes from one to another.
>>>>>>>>>>
>>>>>>>>>> The only drawback that I see is when you have to release a minor
>>>>>>>>>> change
>>>>>>>>>> in
>>>>>>>>>> one
>>>>>>>>>> the the rpms, for example, to fix a critical bug, the repo will not
>>>>>>>>>> include
>>>>>>>>>> the
>>>>>>>>>> old package, but I'm not sure if that's really a drawback... if you
>>>>>>>>>> really
>>>>>>>>>> need
>>>>>>>>>> that package without the critical fix (you should not) you can have it
>>>>>>>>>> changing
>>>>>>>>>> to that specific repository. The internal naming of the repos does not
>>>>>>>>>> really
>>>>>>>>>> matter, having to point to the repo 3.3.3-beta.2 to get the second
>>>>>>>>>> 'respin'
>>>>>>>>>> of
>>>>>>>>>> the 3.3.3 beta repo is not a big issue I think.
>>>>>>>>>>
>>>>>>>>>> The advantages are many, the most importants I see:
>>>>>>>>>>   - Easy management:
>>>>>>>>>>     * no need to go version hunting in the repo to remove/add rpms
>>>>>>>>>>     * you should never get a repo with version combinations that are
>>>>>>>>>>     not
>>>>>>>>>>     tested
>>>>>>>>>>     * it's a lot easier to get rid of old repos, and to move them
>>>>>>>>>>     around
>>>>>>>>>>     as
>>>>>>>>>>     they
>>>>>>>>>>       are independent
>>>>>>>>>>     * no broken links, right now stable repo is full of links to other
>>>>>>>>>>     repos,
>>>>>>>>>>     so
>>>>>>>>>>       removing those repos leave the links broken, you have to go
>>>>>>>>>>       checking
>>>>>>>>>>       if
>>>>>>>>>>       someone links to them (or their internal directories) if you have
>>>>>>>>>>       to
>>>>>>>>>>       clean
>>>>>>>>>>       up old versions
>>>>>>>>>>   - Testing, it's a lot easier to reproduce any error found, as you can
>>>>>>>>>>     just use the same repo and you'll get the same version set.
>>>>>>>>>>
>>>>>>>>>> What do you think?
>>>>>>>>>
>>>>>>>>
>>>>>>>>> And you do not allow quick fix of issues found in various of packages.
>>>>>>>> Why not? You can create a new repo based in the previous one that
>>>>>>>> includes the fixed packages. It's cheap!
>>>>>>>
>>>>>>> who is you?
>>>>>> In this case you is the person/process/chimpanzee that is in charge of
>>>>>> publishing the fixed packages to the correct environment
>>>>>>
>>>>>>> how do I push fix to users for z-stream of packages as otopi,
>>>>>>> ovirt-host-deploy, log collector and such?
>>>>>> Exactly the same way you do it for engine or vdsm
>>>>>>
>>>>>>> why is these components' release cycle should be at same schedule of
>>>>>>> ovirt-engine which is heavy and slow?
>>>>>> It should not.
>>>>>>
>>>>>>>
>>>>>>>>>
>>>>>>>>> Although there is /some/ sense in syncing minor releases, I do not see
>>>>>>>>> any
>>>>>>>>> reason of syncing z-stream.
>>>>>>>>>
>>>>>>>>
>>>>>>>>> I think that you do not trust individual maintainer to provide
>>>>>>>>> z-streams.
>>>>>>>>>
>>>>>>>>> A change in z-stream should not be exposed (unless is fixing) an
>>>>>>>>> external
>>>>>>>>> interface.
>>>>>>>>
>>>>>>>> I don't think it should be hidden neither, just make clear that those
>>>>>>>> are not builds to be used widely, maybe just putting it under another
>>>>>>>> directory (not releases). Where only promoted repos can go (meaning,
>>>>>>>> not everyone can put repos there).
>>>>>>>> For example:
>>>>>>>> repos/releases -> for repos that have been tested and we want to publish
>>>>>>>> repos/testing -> for any temporary generated repo, that is not fully
>>>>>>>> tested and not ready for be used widely
>>>>>>>>
>>>>>>>
>>>>>>> why not released? only because engine is slow? I do not understand.
>>>>>> I don't even understand your question. We got lost at some point. I'll
>>>>>> try to explain a little more what I said before, maybe that will
>>>>>> clarify the issue to you.
>>>>>> You said that a change in z-stream should not be exposed, for that I
>>>>>> understand for that that a package that is meant to go to a z-stream
>>>>>> should not be exposed to the general public. I think that it should,
>>>>>> but it must be clear to the public that it's not yet ready (I suppose
>>>>>> that's the reason you don't want it public), so they use it at their
>>>>>> own risk. And separating the repos into two seems a good wat for making
>>>>>> that clear (another one is adding a suffix to the repo name for
>>>>>> example).
>>>>>>
>>>>>>>
>>>>>>>> That way you make sure that if anyone is using a repo that is not fully
>>>>>>>> tested, is because he wants to, but you don't forbid it.
>>>>>>>
>>>>>>> why do you think that someone is releasing untested packages?
>>>>>> I do not think that someone is releasing untested packages. That
>>>>>> sentence comes from the hypothetical situation where a repository that
>>>>>> is not meant to be used by the general public (I said untested, but it
>>>>>> could be for any other reason) is made public using a different url
>>>>>> than the repos that are meant to be widely used.
>>>>>>
>>>>>> Part of the advantages of that system is the ease to run tests on
>>>>>> specific version sets (repos). That we do not do right now (at least
>>>>>> *upstream*) but I think would be done in the near future.
>>>>>
>>>>> I will try to explain again.
>>>>>
>>>>> There is no actual relationship between packages, these could have been
>>>>> provided asynchronous by multiple sources and maintainers regardless of
>>>>> the ovrit project, just like libvirt or sanlock or any other 10000
>>>>> dependencies we have outside of the scope of the project.
>>>>
>>>> That's not true, the relation is that they are provided by the same
>>>> repository and that they are maintained by the same community. Yes,
>>>> they could have been provided by multiple sources and maintainers, but
>>>> they were not.
>>>>
>>>>> Trying to control the release cycle only because we have two fat components
>>>>> is something that should be avoided.
>>>> The model I exposed does not care if you create a new repository each
>>>> hour changing just one package or you create the repository on time a
>>>> year. What it's true is that it will be recreated when ANY (one or
>>>> more) of the packages included changes.
>>>>
>>>>>
>>>>> So far we have successfully released packages async with no regressions nor
>>>>> issues, and quickly solved user issues. There is absolutely no reason to
>>>>> stop this offering.
>>>> Yes, that in the last weeks sandro and me (mostly me) spent more than
>>>> two days trying to create a couple releases with the old process. It's
>>>> hard to maintain and I personally prefer focusing on another tasks than
>>>> searching rpm versions and trying to figure out what can be
>>>> deleted/moved and what can't. A proved way to improve things is to
>>>> change them, try new ways, if you do not change you are waiting for the
>>>> environment to do so, and that's usually really hard to achieve.
>>>> Nothing suggests that adopting the process that I explained will affect
>>>> the user experience substantially (if think otherwise, please
>>>> elaborate).
>>>>
>>>>>
>>>>>>
>>>>>>>
>>>>>>>>>
>>>>>>>>> Alon
>>>>>>>>>
>>>>>>>>>>
>>>>>>>>>>>
>>>>>>>>>>>> Regards,
>>>>>>>>>>>> Alon Bar-Lev.
>>>>>>>>>>>>
>>>>>>>>>>>> [1] http://resources.ovirt.org/releases/stable/
>>>>>>>>>>>>
>>>>>>>>>>>
>>>>>>>>>>>
>>>>>>>>>>
>>>>>>>>>>
>>>>>>>>>> --
>>>>>>>>>> David Caro
>>>>>>>>>>
>>>>>>>>>> Red Hat S.L.
>>>>>>>>>> Continuous Integration Engineer - EMEA ENG Virtualization R&D
>>>>>>>>>>
>>>>>>>>>> Email: dcaro at redhat.com
>>>>>>>>>> Web: www.redhat.com
>>>>>>>>>> RHT Global #: 82-62605
>>>>>>>>>>
>>>>>>>>>>
>>>>>>>>
>>>>>>>>
>>>>>>>>
>>>>>>>> --
>>>>>>>> David Caro
>>>>>>>>
>>>>>>>> Red Hat S.L.
>>>>>>>> Continuous Integration Engineer - EMEA ENG Virtualization R&D
>>>>>>>>
>>>>>>>> Email: dcaro at redhat.com
>>>>>>>> Web: www.redhat.com
>>>>>>>> RHT Global #: 82-62605
>>>>>>>>
>>>>>>>>
>>>>>>
>>>>>>
>>>>>>
>>>>>> --
>>>>>> David Caro
>>>>>>
>>>>>> Red Hat S.L.
>>>>>> Continuous Integration Engineer - EMEA ENG Virtualization R&D
>>>>>>
>>>>>> Email: dcaro at redhat.com
>>>>>> Web: www.redhat.com
>>>>>> RHT Global #: 82-62605
>>>>>>
>>>>>>
>>>>
>>>>
>>>>
>>>> --
>>>> David Caro
>>>>
>>>> Red Hat S.L.
>>>> Continuous Integration Engineer - EMEA ENG Virtualization R&D
>>>>
>>>> Email: dcaro at redhat.com
>>>> Web: www.redhat.com
>>>> RHT Global #: 82-62605
>>>>
>>>>
>>
>>
>>
>> --
>> David Caro
>>
>> Red Hat S.L.
>> Continuous Integration Engineer - EMEA ENG Virtualization R&D
>>
>> Email: dcaro at redhat.com
>> Web: www.redhat.com
>> RHT Global #: 82-62605
>>
>
>

More information about the Infra mailing list