release repo structure and 3.3.2

[Alon Bar-Lev]

I won't argue more, obviously, you do not understand the point of distributed management.

But just think why don't we build large monolithic software.

----- Original Message -----
> From: "David Caro" <dcaroest at redhat.com>
> To: "Alon Bar-Lev" <alonbl at redhat.com>
> Cc: "Sandro Bonazzola" <sbonazzo at redhat.com>, "infra" <infra at ovirt.org>, "Kiril Nesenko" <kiril at redhat.com>
> Sent: Thursday, January 16, 2014 2:35:16 AM
> Subject: Re: release repo structure and 3.3.2
> 
> El jue 16 ene 2014 01:04:33 CET, Alon Bar-Lev escribió:
> >
> >
> > ----- Original Message -----
> >> From: "David Caro" <dcaroest at redhat.com>
> >> To: "Alon Bar-Lev" <alonbl at redhat.com>
> >> Cc: "Sandro Bonazzola" <sbonazzo at redhat.com>, "infra" <infra at ovirt.org>,
> >> "Kiril Nesenko" <kiril at redhat.com>
> >> Sent: Thursday, January 16, 2014 1:58:08 AM
> >> Subject: Re: release repo structure and 3.3.2
> >>
> >> El mié 15 ene 2014 19:04:04 CET, Alon Bar-Lev escribió:
> >>>
> >>>
> >>> ----- Original Message -----
> >>>> From: "David Caro" <dcaroest at redhat.com>
> >>>> To: "Alon Bar-Lev" <alonbl at redhat.com>
> >>>> Cc: "Sandro Bonazzola" <sbonazzo at redhat.com>, "infra" <infra at ovirt.org>,
> >>>> "Kiril Nesenko" <kiril at redhat.com>
> >>>> Sent: Wednesday, January 15, 2014 5:47:59 PM
> >>>> Subject: Re: release repo structure and 3.3.2
> >>>>
> >>>> El mié 15 ene 2014 16:30:00 CET, Alon Bar-Lev escribió:
> >>>>>
> >>>>>
> >>>>> ----- Original Message -----
> >>>>>> From: "David Caro" <dcaroest at redhat.com>
> >>>>>> To: "Sandro Bonazzola" <sbonazzo at redhat.com>, "Alon Bar-Lev"
> >>>>>> <alonbl at redhat.com>, "infra" <infra at ovirt.org>
> >>>>>> Cc: "Kiril Nesenko" <kiril at redhat.com>
> >>>>>> Sent: Wednesday, January 15, 2014 5:26:32 PM
> >>>>>> Subject: Re: release repo structure and 3.3.2
> >>>>>>
> >>>>>> El 07/01/14 15:31, Sandro Bonazzola escribió:
> >>>>>>> Il 01/01/2014 10:42, Alon Bar-Lev ha scritto:
> >>>>>>>> Hi,
> >>>>>>>>
> >>>>>>>> For some reason there 3.3.2 z-stream was released in its own
> >>>>>>>> repository
> >>>>>>>> so
> >>>>>>>> people that are subscribed to stable[1] did not get it.
> >>>>>>>
> >>>>>>> Why not?
> >>>>>>> stable release had ovirt-release-10 which enabled both stable and
> >>>>>>> 3.3.2
> >>>>>>> repository by yum updating it.
> >>>>>>>
> >>>>>>>>
> >>>>>>>> There is no much sense in releasing fix release that people do not
> >>>>>>>> get
> >>>>>>>> in
> >>>>>>>> simple "yum update".
> >>>>>>>>
> >>>>>>>> Also the following is now broken of most packages' spec:
> >>>>>>>> Source0:
> >>>>>>>> http://ovirt.org/releases/stable/src/@PACKAGE_NAME@-@PACKAGE_VERSION@.tar.gz
> >>>>>>>>
> >>>>>>>> For each minor we should have rolling repository, to reduce noise
> >>>>>>>> and
> >>>>>>>> provide service.
> >>>>>>>>
> >>>>>>>> All released tarballs (sources) should be stored at fixed location
> >>>>>>>> to
> >>>>>>>> allow distro specific code to fetch, the location must be synced
> >>>>>>>> with
> >>>>>>>> what we publish.
> >>>>>>>>
> >>>>>>>> Immediate action is to move the 3.3.2 content into the stable
> >>>>>>>> directory.
> >>>>>>>
> >>>>>>> So previous request of having each release in its own repository has
> >>>>>>> been
> >>>>>>> retired?
> >>>>>>> Or is it combined?
> >>>>>>> Do we want stable to be a rolling repository and have also a
> >>>>>>> repository
> >>>>>>> for
> >>>>>>> each version?
> >>>>>>> I'm not against having rolling packages in just one stable
> >>>>>>> repository,
> >>>>>>> I
> >>>>>>> just want to understand what is the desired structure of the
> >>>>>>> repositories.
> >>>>>>
> >>>>>> I am, having a stable repository with rolling rpms is a lot more hard
> >>>>>> to
> >>>>>> manage
> >>>>>> and maintain than having separated individual complete repos.
> >>>>>>
> >>>>>> Because what we are actually delivering is not a specific rpm, but the
> >>>>>> whole
> >>>>>> set, that is, one repository with the set of rpms that were tested
> >>>>>> together
> >>>>>> and
> >>>>>> validated. If at any point you want to mix them, you still can adding
> >>>>>> the
> >>>>>> other
> >>>>>> repos.
> >>>>>>
> >>>>>> For updates just updating the directory where the 'stable' link points
> >>>>>> gets
> >>>>>> it done.
> >>>>>>
> >>>>>> For rollbacks you'll have to configure the old repo. That is not as
> >>>>>> annoying
> >>>>>> as
> >>>>>> it might seem, because when you enable the stable repo, you want to
> >>>>>> have
> >>>>>> the
> >>>>>> stable version, that changes with time. If you want to rollback to a
> >>>>>> previous
> >>>>>> version then just use that versions specific repo. At much we can
> >>>>>> provide
> >>>>>> a
> >>>>>> link
> >>>>>> like 'previous_stable' so if you want to rollback to the previous
> >>>>>> version
> >>>>>> you
> >>>>>> can use --enablerepo=previous_version easily, but if you want to keep
> >>>>>> using
> >>>>>> that, you should point directly to the specific version you want tot
> >>>>>> use.
> >>>>>>
> >>>>>> Creating a new repository using is almost as cheap (on hard disk
> >>>>>> space)
> >>>>>> as
> >>>>>> having a rolling repository, if you use hard links, so we can create
> >>>>>> lot's
> >>>>>> of
> >>>>>> them, specially for small changes from one to another.
> >>>>>>
> >>>>>> The only drawback that I see is when you have to release a minor
> >>>>>> change
> >>>>>> in
> >>>>>> one
> >>>>>> the the rpms, for example, to fix a critical bug, the repo will not
> >>>>>> include
> >>>>>> the
> >>>>>> old package, but I'm not sure if that's really a drawback... if you
> >>>>>> really
> >>>>>> need
> >>>>>> that package without the critical fix (you should not) you can have it
> >>>>>> changing
> >>>>>> to that specific repository. The internal naming of the repos does not
> >>>>>> really
> >>>>>> matter, having to point to the repo 3.3.3-beta.2 to get the second
> >>>>>> 'respin'
> >>>>>> of
> >>>>>> the 3.3.3 beta repo is not a big issue I think.
> >>>>>>
> >>>>>> The advantages are many, the most importants I see:
> >>>>>>  - Easy management:
> >>>>>>    * no need to go version hunting in the repo to remove/add rpms
> >>>>>>    * you should never get a repo with version combinations that are
> >>>>>>    not
> >>>>>>    tested
> >>>>>>    * it's a lot easier to get rid of old repos, and to move them
> >>>>>>    around
> >>>>>>    as
> >>>>>>    they
> >>>>>>      are independent
> >>>>>>    * no broken links, right now stable repo is full of links to other
> >>>>>>    repos,
> >>>>>>    so
> >>>>>>      removing those repos leave the links broken, you have to go
> >>>>>>      checking
> >>>>>>      if
> >>>>>>      someone links to them (or their internal directories) if you have
> >>>>>>      to
> >>>>>>      clean
> >>>>>>      up old versions
> >>>>>>  - Testing, it's a lot easier to reproduce any error found, as you can
> >>>>>>    just use the same repo and you'll get the same version set.
> >>>>>>
> >>>>>> What do you think?
> >>>>>
> >>>>
> >>>>> And you do not allow quick fix of issues found in various of packages.
> >>>> Why not? You can create a new repo based in the previous one that
> >>>> includes the fixed packages. It's cheap!
> >>>
> >>> who is you?
> >> In this case you is the person/process/chimpanzee that is in charge of
> >> publishing the fixed packages to the correct environment
> >>
> >>> how do I push fix to users for z-stream of packages as otopi,
> >>> ovirt-host-deploy, log collector and such?
> >> Exactly the same way you do it for engine or vdsm
> >>
> >>> why is these components' release cycle should be at same schedule of
> >>> ovirt-engine which is heavy and slow?
> >> It should not.
> >>
> >>>
> >>>>>
> >>>>> Although there is /some/ sense in syncing minor releases, I do not see
> >>>>> any
> >>>>> reason of syncing z-stream.
> >>>>>
> >>>>
> >>>>> I think that you do not trust individual maintainer to provide
> >>>>> z-streams.
> >>>>>
> >>>>> A change in z-stream should not be exposed (unless is fixing) an
> >>>>> external
> >>>>> interface.
> >>>>
> >>>> I don't think it should be hidden neither, just make clear that those
> >>>> are not builds to be used widely, maybe just putting it under another
> >>>> directory (not releases). Where only promoted repos can go (meaning,
> >>>> not everyone can put repos there).
> >>>> For example:
> >>>> repos/releases -> for repos that have been tested and we want to publish
> >>>> repos/testing -> for any temporary generated repo, that is not fully
> >>>> tested and not ready for be used widely
> >>>>
> >>>
> >>> why not released? only because engine is slow? I do not understand.
> >> I don't even understand your question. We got lost at some point. I'll
> >> try to explain a little more what I said before, maybe that will
> >> clarify the issue to you.
> >> You said that a change in z-stream should not be exposed, for that I
> >> understand for that that a package that is meant to go to a z-stream
> >> should not be exposed to the general public. I think that it should,
> >> but it must be clear to the public that it's not yet ready (I suppose
> >> that's the reason you don't want it public), so they use it at their
> >> own risk. And separating the repos into two seems a good wat for making
> >> that clear (another one is adding a suffix to the repo name for
> >> example).
> >>
> >>>
> >>>> That way you make sure that if anyone is using a repo that is not fully
> >>>> tested, is because he wants to, but you don't forbid it.
> >>>
> >>> why do you think that someone is releasing untested packages?
> >> I do not think that someone is releasing untested packages. That
> >> sentence comes from the hypothetical situation where a repository that
> >> is not meant to be used by the general public (I said untested, but it
> >> could be for any other reason) is made public using a different url
> >> than the repos that are meant to be widely used.
> >>
> >> Part of the advantages of that system is the ease to run tests on
> >> specific version sets (repos). That we do not do right now (at least
> >> *upstream*) but I think would be done in the near future.
> >
> > I will try to explain again.
> >
> > There is no actual relationship between packages, these could have been
> > provided asynchronous by multiple sources and maintainers regardless of
> > the ovrit project, just like libvirt or sanlock or any other 10000
> > dependencies we have outside of the scope of the project.
> 
> That's not true, the relation is that they are provided by the same
> repository and that they are maintained by the same community. Yes,
> they could have been provided by multiple sources and maintainers, but
> they were not.
> 
> > Trying to control the release cycle only because we have two fat components
> > is something that should be avoided.
> The model I exposed does not care if you create a new repository each
> hour changing just one package or you create the repository on time a
> year. What it's true is that it will be recreated when ANY (one or
> more) of the packages included changes.
> 
> >
> > So far we have successfully released packages async with no regressions nor
> > issues, and quickly solved user issues. There is absolutely no reason to
> > stop this offering.
> Yes, that in the last weeks sandro and me (mostly me) spent more than
> two days trying to create a couple releases with the old process. It's
> hard to maintain and I personally prefer focusing on another tasks than
> searching rpm versions and trying to figure out what can be
> deleted/moved and what can't. A proved way to improve things is to
> change them, try new ways, if you do not change you are waiting for the
> environment to do so, and that's usually really hard to achieve.
> Nothing suggests that adopting the process that I explained will affect
> the user experience substantially (if think otherwise, please
> elaborate).
> 
> >
> >>
> >>>
> >>>>>
> >>>>> Alon
> >>>>>
> >>>>>>
> >>>>>>>
> >>>>>>>> Regards,
> >>>>>>>> Alon Bar-Lev.
> >>>>>>>>
> >>>>>>>> [1] http://resources.ovirt.org/releases/stable/
> >>>>>>>>
> >>>>>>>
> >>>>>>>
> >>>>>>
> >>>>>>
> >>>>>> --
> >>>>>> David Caro
> >>>>>>
> >>>>>> Red Hat S.L.
> >>>>>> Continuous Integration Engineer - EMEA ENG Virtualization R&D
> >>>>>>
> >>>>>> Email: dcaro at redhat.com
> >>>>>> Web: www.redhat.com
> >>>>>> RHT Global #: 82-62605
> >>>>>>
> >>>>>>
> >>>>
> >>>>
> >>>>
> >>>> --
> >>>> David Caro
> >>>>
> >>>> Red Hat S.L.
> >>>> Continuous Integration Engineer - EMEA ENG Virtualization R&D
> >>>>
> >>>> Email: dcaro at redhat.com
> >>>> Web: www.redhat.com
> >>>> RHT Global #: 82-62605
> >>>>
> >>>>
> >>
> >>
> >>
> >> --
> >> David Caro
> >>
> >> Red Hat S.L.
> >> Continuous Integration Engineer - EMEA ENG Virtualization R&D
> >>
> >> Email: dcaro at redhat.com
> >> Web: www.redhat.com
> >> RHT Global #: 82-62605
> >>
> >>
> 
> 
> 
> --
> David Caro
> 
> Red Hat S.L.
> Continuous Integration Engineer - EMEA ENG Virtualization R&D
> 
> Email: dcaro at redhat.com
> Web: www.redhat.com
> RHT Global #: 82-62605
> 
>