Re: release repo structure and 3.3.2

On 01/16/2014 01:49 PM, Sandro Bonazzola wrote: > Il 16/01/2014 10:13, David Caro ha scritto: >> El jue 16 ene 2014 01:39:50 CET, Alon Bar-Lev escribió: >>> >>> I won't argue more, obviously, you do not understand the point of distributed management. >> Thanks Alon, always so useful. >>> >>> But just think why don't we build large monolithic software. >> Anyone thinks the same way Alon does? >> If so, can anyone explain me he's position? (As it seems he's not >> willing to do so) >> >> Also any other opinions? Other Ideas? > > IMHO we should keep our repositories as our downstream distro do. > > For releases: > releases/Fedora/19/3.3.0 > releases/Fedora/19/3.3.1 > releases/Fedora/19/3.3.2 > releases/Fedora/19/3.3.3 > releases/Fedora/20/3.4.0 doesn't our upgrade process inside a minor version requires the repo to have both new and old versions of the rpm?

> .... > and > updates/19/ > updates/20/ > > for all other packages which are not tied to ovirt release cycle. > > I would like also that packages handled downstream as vdsm, -sdk-python and others would be removed from our upstream repository since they're handled > downstream. > on new z-stream release ovirt-release will be bumped > yum update from releases/Fedora/19/3.4.0 will add 3.4.1 as new enabled repository > yum update from releases/Fedora/19/3.4.1 will remove 3.4.0 and add 3.4.2 and conflict with any <= 3.4.0 > yum update from releases/Fedora/19/3.4.2 will remove 3.4.1 and add 3.4.2 and conflict with any <= 3.4.1 > and so on. > > while otopi, log-collector and so on can safely be updated along all 3.4.z cycle. > > > >> >>> >>> ----- Original Message ----- >>>> From: "David Caro" <dcaroest(a)redhat.com&gt; >>>> To: "Alon Bar-Lev" <alonbl(a)redhat.com&gt; >>>> Cc: "Sandro Bonazzola" <sbonazzo(a)redhat.com&gt;, "infra" <infra(a)ovirt.org&gt;, "Kiril Nesenko" <kiril(a)redhat.com&gt; >>>> Sent: Thursday, January 16, 2014 2:35:16 AM >>>> Subject: Re: release repo structure and 3.3.2 >>>> >>>> El jue 16 ene 2014 01:04:33 CET, Alon Bar-Lev escribió: >>>>> >>>>> >>>>> ----- Original Message ----- >>>>>> From: "David Caro" <dcaroest(a)redhat.com&gt; >>>>>> To: "Alon Bar-Lev" <alonbl(a)redhat.com&gt; >>>>>> Cc: "Sandro Bonazzola" <sbonazzo(a)redhat.com&gt;, "infra" <infra(a)ovirt.org&gt;, >>>>>> "Kiril Nesenko" <kiril(a)redhat.com&gt; >>>>>> Sent: Thursday, January 16, 2014 1:58:08 AM >>>>>> Subject: Re: release repo structure and 3.3.2 >>>>>> >>>>>> El mié 15 ene 2014 19:04:04 CET, Alon Bar-Lev escribió: >>>>>>> >>>>>>> >>>>>>> ----- Original Message ----- >>>>>>>> From: "David Caro" <dcaroest(a)redhat.com&gt; >>>>>>>> To: "Alon Bar-Lev" <alonbl(a)redhat.com&gt; >>>>>>>> Cc: "Sandro Bonazzola" <sbonazzo(a)redhat.com&gt;, "infra" <infra(a)ovirt.org&gt;, >>>>>>>> "Kiril Nesenko" <kiril(a)redhat.com&gt; >>>>>>>> Sent: Wednesday, January 15, 2014 5:47:59 PM >>>>>>>> Subject: Re: release repo structure and 3.3.2 >>>>>>>> >>>>>>>> El mié 15 ene 2014 16:30:00 CET, Alon Bar-Lev escribió: >>>>>>>>> >>>>>>>>> >>>>>>>>> ----- Original Message ----- >>>>>>>>>> From: "David Caro" <dcaroest(a)redhat.com&gt; >>>>>>>>>> To: "Sandro Bonazzola" <sbonazzo(a)redhat.com&gt;, "Alon Bar-Lev" >>>>>>>>>> <alonbl(a)redhat.com&gt;, "infra" <infra(a)ovirt.org&gt; >>>>>>>>>> Cc: "Kiril Nesenko" <kiril(a)redhat.com&gt; >>>>>>>>>> Sent: Wednesday, January 15, 2014 5:26:32 PM >>>>>>>>>> Subject: Re: release repo structure and 3.3.2 >>>>>>>>>> >>>>>>>>>> El 07/01/14 15:31, Sandro Bonazzola escribió: >>>>>>>>>>> Il 01/01/2014 10:42, Alon Bar-Lev ha scritto: >>>>>>>>>>>> Hi, >>>>>>>>>>>> >>>>>>>>>>>> For some reason there 3.3.2 z-stream was released in its own >>>>>>>>>>>> repository >>>>>>>>>>>> so >>>>>>>>>>>> people that are subscribed to stable[1] did not get it. >>>>>>>>>>> >>>>>>>>>>> Why not? >>>>>>>>>>> stable release had ovirt-release-10 which enabled both stable and >>>>>>>>>>> 3.3.2 >>>>>>>>>>> repository by yum updating it. >>>>>>>>>>> >>>>>>>>>>>> >>>>>>>>>>>> There is no much sense in releasing fix release that people do not >>>>>>>>>>>> get >>>>>>>>>>>> in >>>>>>>>>>>> simple "yum update". >>>>>>>>>>>> >>>>>>>>>>>> Also the following is now broken of most packages' spec: >>>>>>>>>>>> Source0: >>>>>>>>>>>> http://ovirt.org/releases/stable/src/@PACKAGE_NAME@-@PACKAGE_VERSION@.tar.gz >>>>>>>>>>>> >>>>>>>>>>>> For each minor we should have rolling repository, to reduce noise >>>>>>>>>>>> and >>>>>>>>>>>> provide service. >>>>>>>>>>>> >>>>>>>>>>>> All released tarballs (sources) should be stored at fixed location >>>>>>>>>>>> to >>>>>>>>>>>> allow distro specific code to fetch, the location must be synced >>>>>>>>>>>> with >>>>>>>>>>>> what we publish. >>>>>>>>>>>> >>>>>>>>>>>> Immediate action is to move the 3.3.2 content into the stable >>>>>>>>>>>> directory. >>>>>>>>>>> >>>>>>>>>>> So previous request of having each release in its own repository has >>>>>>>>>>> been >>>>>>>>>>> retired? >>>>>>>>>>> Or is it combined? >>>>>>>>>>> Do we want stable to be a rolling repository and have also a >>>>>>>>>>> repository >>>>>>>>>>> for >>>>>>>>>>> each version? >>>>>>>>>>> I'm not against having rolling packages in just one stable >>>>>>>>>>> repository, >>>>>>>>>>> I >>>>>>>>>>> just want to understand what is the desired structure of the >>>>>>>>>>> repositories. >>>>>>>>>> >>>>>>>>>> I am, having a stable repository with rolling rpms is a lot more hard >>>>>>>>>> to >>>>>>>>>> manage >>>>>>>>>> and maintain than having separated individual complete repos. >>>>>>>>>> >>>>>>>>>> Because what we are actually delivering is not a specific rpm, but the >>>>>>>>>> whole >>>>>>>>>> set, that is, one repository with the set of rpms that were tested >>>>>>>>>> together >>>>>>>>>> and >>>>>>>>>> validated. If at any point you want to mix them, you still can adding >>>>>>>>>> the >>>>>>>>>> other >>>>>>>>>> repos. >>>>>>>>>> >>>>>>>>>> For updates just updating the directory where the 'stable' link points >>>>>>>>>> gets >>>>>>>>>> it done. >>>>>>>>>> >>>>>>>>>> For rollbacks you'll have to configure the old repo. That is not as >>>>>>>>>> annoying >>>>>>>>>> as >>>>>>>>>> it might seem, because when you enable the stable repo, you want to >>>>>>>>>> have >>>>>>>>>> the >>>>>>>>>> stable version, that changes with time. If you want to rollback to a >>>>>>>>>> previous >>>>>>>>>> version then just use that versions specific repo. At much we can >>>>>>>>>> provide >>>>>>>>>> a >>>>>>>>>> link >>>>>>>>>> like 'previous_stable' so if you want to rollback to the previous >>>>>>>>>> version >>>>>>>>>> you >>>>>>>>>> can use --enablerepo=previous_version easily, but if you want to keep >>>>>>>>>> using >>>>>>>>>> that, you should point directly to the specific version you want tot >>>>>>>>>> use. >>>>>>>>>> >>>>>>>>>> Creating a new repository using is almost as cheap (on hard disk >>>>>>>>>> space) >>>>>>>>>> as >>>>>>>>>> having a rolling repository, if you use hard links, so we can create >>>>>>>>>> lot's >>>>>>>>>> of >>>>>>>>>> them, specially for small changes from one to another. >>>>>>>>>> >>>>>>>>>> The only drawback that I see is when you have to release a minor >>>>>>>>>> change >>>>>>>>>> in >>>>>>>>>> one >>>>>>>>>> the the rpms, for example, to fix a critical bug, the repo will not >>>>>>>>>> include >>>>>>>>>> the >>>>>>>>>> old package, but I'm not sure if that's really a drawback... if you >>>>>>>>>> really >>>>>>>>>> need >>>>>>>>>> that package without the critical fix (you should not) you can have it >>>>>>>>>> changing >>>>>>>>>> to that specific repository. The internal naming of the repos does not >>>>>>>>>> really >>>>>>>>>> matter, having to point to the repo 3.3.3-beta.2 to get the second >>>>>>>>>> 'respin' >>>>>>>>>> of >>>>>>>>>> the 3.3.3 beta repo is not a big issue I think. >>>>>>>>>> >>>>>>>>>> The advantages are many, the most importants I see: >>>>>>>>>> - Easy management: >>>>>>>>>> * no need to go version hunting in the repo to remove/add rpms >>>>>>>>>> * you should never get a repo with version combinations that are >>>>>>>>>> not >>>>>>>>>> tested >>>>>>>>>> * it's a lot easier to get rid of old repos, and to move them >>>>>>>>>> around >>>>>>>>>> as >>>>>>>>>> they >>>>>>>>>> are independent >>>>>>>>>> * no broken links, right now stable repo is full of links to other >>>>>>>>>> repos, >>>>>>>>>> so >>>>>>>>>> removing those repos leave the links broken, you have to go >>>>>>>>>> checking >>>>>>>>>> if >>>>>>>>>> someone links to them (or their internal directories) if you have >>>>>>>>>> to >>>>>>>>>> clean >>>>>>>>>> up old versions >>>>>>>>>> - Testing, it's a lot easier to reproduce any error found, as you can >>>>>>>>>> just use the same repo and you'll get the same version set. >>>>>>>>>> >>>>>>>>>> What do you think? >>>>>>>>> >>>>>>>> >>>>>>>>> And you do not allow quick fix of issues found in various of packages. >>>>>>>> Why not? You can create a new repo based in the previous one that >>>>>>>> includes the fixed packages. It's cheap! >>>>>>> >>>>>>> who is you? >>>>>> In this case you is the person/process/chimpanzee that is in charge of >>>>>> publishing the fixed packages to the correct environment >>>>>> >>>>>>> how do I push fix to users for z-stream of packages as otopi, >>>>>>> ovirt-host-deploy, log collector and such? >>>>>> Exactly the same way you do it for engine or vdsm >>>>>> >>>>>>> why is these components' release cycle should be at same schedule of >>>>>>> ovirt-engine which is heavy and slow? >>>>>> It should not. >>>>>> >>>>>>> >>>>>>>>> >>>>>>>>> Although there is /some/ sense in syncing minor releases, I do not see >>>>>>>>> any >>>>>>>>> reason of syncing z-stream. >>>>>>>>> >>>>>>>> >>>>>>>>> I think that you do not trust individual maintainer to provide >>>>>>>>> z-streams. >>>>>>>>> >>>>>>>>> A change in z-stream should not be exposed (unless is fixing) an >>>>>>>>> external >>>>>>>>> interface. >>>>>>>> >>>>>>>> I don't think it should be hidden neither, just make clear that those >>>>>>>> are not builds to be used widely, maybe just putting it under another >>>>>>>> directory (not releases). Where only promoted repos can go (meaning, >>>>>>>> not everyone can put repos there). >>>>>>>> For example: >>>>>>>> repos/releases -> for repos that have been tested and we want to publish >>>>>>>> repos/testing -> for any temporary generated repo, that is not fully >>>>>>>> tested and not ready for be used widely >>>>>>>> >>>>>>> >>>>>>> why not released? only because engine is slow? I do not understand. >>>>>> I don't even understand your question. We got lost at some point. I'll >>>>>> try to explain a little more what I said before, maybe that will >>>>>> clarify the issue to you. >>>>>> You said that a change in z-stream should not be exposed, for that I >>>>>> understand for that that a package that is meant to go to a z-stream >>>>>> should not be exposed to the general public. I think that it should, >>>>>> but it must be clear to the public that it's not yet ready (I suppose >>>>>> that's the reason you don't want it public), so they use it at their >>>>>> own risk. And separating the repos into two seems a good wat for making >>>>>> that clear (another one is adding a suffix to the repo name for >>>>>> example). >>>>>> >>>>>>> >>>>>>>> That way you make sure that if anyone is using a repo that is not fully >>>>>>>> tested, is because he wants to, but you don't forbid it. >>>>>>> >>>>>>> why do you think that someone is releasing untested packages? >>>>>> I do not think that someone is releasing untested packages. That >>>>>> sentence comes from the hypothetical situation where a repository that >>>>>> is not meant to be used by the general public (I said untested, but it >>>>>> could be for any other reason) is made public using a different url >>>>>> than the repos that are meant to be widely used. >>>>>> >>>>>> Part of the advantages of that system is the ease to run tests on >>>>>> specific version sets (repos). That we do not do right now (at least >>>>>> *upstream*) but I think would be done in the near future. >>>>> >>>>> I will try to explain again. >>>>> >>>>> There is no actual relationship between packages, these could have been >>>>> provided asynchronous by multiple sources and maintainers regardless of >>>>> the ovrit project, just like libvirt or sanlock or any other 10000 >>>>> dependencies we have outside of the scope of the project. >>>> >>>> That's not true, the relation is that they are provided by the same >>>> repository and that they are maintained by the same community. Yes, >>>> they could have been provided by multiple sources and maintainers, but >>>> they were not. >>>> >>>>> Trying to control the release cycle only because we have two fat components >>>>> is something that should be avoided. >>>> The model I exposed does not care if you create a new repository each >>>> hour changing just one package or you create the repository on time a >>>> year. What it's true is that it will be recreated when ANY (one or >>>> more) of the packages included changes. >>>> >>>>> >>>>> So far we have successfully released packages async with no regressions nor >>>>> issues, and quickly solved user issues. There is absolutely no reason to >>>>> stop this offering. >>>> Yes, that in the last weeks sandro and me (mostly me) spent more than >>>> two days trying to create a couple releases with the old process. It's >>>> hard to maintain and I personally prefer focusing on another tasks than >>>> searching rpm versions and trying to figure out what can be >>>> deleted/moved and what can't. A proved way to improve things is to >>>> change them, try new ways, if you do not change you are waiting for the >>>> environment to do so, and that's usually really hard to achieve. >>>> Nothing suggests that adopting the process that I explained will affect >>>> the user experience substantially (if think otherwise, please >>>> elaborate). >>>> >>>>> >>>>>> >>>>>>> >>>>>>>>> >>>>>>>>> Alon >>>>>>>>> >>>>>>>>>> >>>>>>>>>>> >>>>>>>>>>>> Regards, >>>>>>>>>>>> Alon Bar-Lev. >>>>>>>>>>>> >>>>>>>>>>>> [1] http://resources.ovirt.org/releases/stable/ >>>>>>>>>>>> >>>>>>>>>>> >>>>>>>>>>> >>>>>>>>>> >>>>>>>>>> >>>>>>>>>> -- >>>>>>>>>> David Caro >>>>>>>>>> >>>>>>>>>> Red Hat S.L. >>>>>>>>>> Continuous Integration Engineer - EMEA ENG Virtualization R&D >>>>>>>>>> >>>>>>>>>> Email: dcaro(a)redhat.com >>>>>>>>>> Web: www.redhat.com >>>>>>>>>> RHT Global #: 82-62605 >>>>>>>>>> >>>>>>>>>> >>>>>>>> >>>>>>>> >>>>>>>> >>>>>>>> -- >>>>>>>> David Caro >>>>>>>> >>>>>>>> Red Hat S.L. >>>>>>>> Continuous Integration Engineer - EMEA ENG Virtualization R&D >>>>>>>> >>>>>>>> Email: dcaro(a)redhat.com >>>>>>>> Web: www.redhat.com >>>>>>>> RHT Global #: 82-62605 >>>>>>>> >>>>>>>> >>>>>> >>>>>> >>>>>> >>>>>> -- >>>>>> David Caro >>>>>> >>>>>> Red Hat S.L. >>>>>> Continuous Integration Engineer - EMEA ENG Virtualization R&D >>>>>> >>>>>> Email: dcaro(a)redhat.com >>>>>> Web: www.redhat.com >>>>>> RHT Global #: 82-62605 >>>>>> >>>>>> >>>> >>>> >>>> >>>> -- >>>> David Caro >>>> >>>> Red Hat S.L. >>>> Continuous Integration Engineer - EMEA ENG Virtualization R&D >>>> >>>> Email: dcaro(a)redhat.com >>>> Web: www.redhat.com >>>> RHT Global #: 82-62605 >>>> >>>> >> >> >> >> -- >> David Caro >> >> Red Hat S.L. >> Continuous Integration Engineer - EMEA ENG Virtualization R&D >> >> Email: dcaro(a)redhat.com >> Web: www.redhat.com >> RHT Global #: 82-62605 >> > >