Re: release repo structure and 3.3.2

Il 16/01/2014 10:13, David Caro ha scritto: > El jue 16 ene 2014 01:39:50 CET, Alon Bar-Lev escribió: >> >> I won't argue more, obviously, you do not understand the point of distributed management. > Thanks Alon, always so useful. >> >> But just think why don't we build large monolithic software. > Anyone thinks the same way Alon does? > If so, can anyone explain me he's position? (As it seems he's not > willing to do so) > > Also any other opinions? Other Ideas? IMHO we should keep our repositories as our downstream distro do. For releases: releases/Fedora/19/3.3.0 releases/Fedora/19/3.3.1 releases/Fedora/19/3.3.2 releases/Fedora/19/3.3.3 releases/Fedora/20/3.4.0

.... and updates/19/ updates/20/ for all other packages which are not tied to ovirt release cycle. I would like also that packages handled downstream as vdsm, -sdk-python and others would be removed from our upstream repository since they're handled downstream. on new z-stream release ovirt-release will be bumped yum update from releases/Fedora/19/3.4.0 will add 3.4.1 as new enabled repository yum update from releases/Fedora/19/3.4.1 will remove 3.4.0 and add 3.4.2 and conflict with any <= 3.4.0 yum update from releases/Fedora/19/3.4.2 will remove 3.4.1 and add 3.4.2 and conflict with any <= 3.4.1 and so on. while otopi, log-collector and so on can safely be updated along all 3.4.z cycle. > >> >> ----- Original Message ----- >>> From: "David Caro" <dcaroest(a)redhat.com&gt; >>> To: "Alon Bar-Lev" <alonbl(a)redhat.com&gt; >>> Cc: "Sandro Bonazzola" <sbonazzo(a)redhat.com&gt;, "infra" <infra(a)ovirt.org&gt;, "Kiril Nesenko" <kiril(a)redhat.com&gt; >>> Sent: Thursday, January 16, 2014 2:35:16 AM >>> Subject: Re: release repo structure and 3.3.2 >>> >>> El jue 16 ene 2014 01:04:33 CET, Alon Bar-Lev escribió: >>>> >>>> >>>> ----- Original Message ----- >>>>> From: "David Caro" <dcaroest(a)redhat.com&gt; >>>>> To: "Alon Bar-Lev" <alonbl(a)redhat.com&gt; >>>>> Cc: "Sandro Bonazzola" <sbonazzo(a)redhat.com&gt;, "infra" <infra(a)ovirt.org&gt;, >>>>> "Kiril Nesenko" <kiril(a)redhat.com&gt; >>>>> Sent: Thursday, January 16, 2014 1:58:08 AM >>>>> Subject: Re: release repo structure and 3.3.2 >>>>> >>>>> El mié 15 ene 2014 19:04:04 CET, Alon Bar-Lev escribió: >>>>>> >>>>>> >>>>>> ----- Original Message ----- >>>>>>> From: "David Caro" <dcaroest(a)redhat.com&gt; >>>>>>> To: "Alon Bar-Lev" <alonbl(a)redhat.com&gt; >>>>>>> Cc: "Sandro Bonazzola" <sbonazzo(a)redhat.com&gt;, "infra" <infra(a)ovirt.org&gt;, >>>>>>> "Kiril Nesenko" <kiril(a)redhat.com&gt; >>>>>>> Sent: Wednesday, January 15, 2014 5:47:59 PM >>>>>>> Subject: Re: release repo structure and 3.3.2 >>>>>>> >>>>>>> El mié 15 ene 2014 16:30:00 CET, Alon Bar-Lev escribió: >>>>>>>> >>>>>>>> >>>>>>>> ----- Original Message ----- >>>>>>>>> From: "David Caro" <dcaroest(a)redhat.com&gt; >>>>>>>>> To: "Sandro Bonazzola" <sbonazzo(a)redhat.com&gt;, "Alon Bar-Lev" >>>>>>>>> <alonbl(a)redhat.com&gt;, "infra" <infra(a)ovirt.org&gt; >>>>>>>>> Cc: "Kiril Nesenko" <kiril(a)redhat.com&gt; >>>>>>>>> Sent: Wednesday, January 15, 2014 5:26:32 PM >>>>>>>>> Subject: Re: release repo structure and 3.3.2 >>>>>>>>> >>>>>>>>> El 07/01/14 15:31, Sandro Bonazzola escribió: >>>>>>>>>> Il 01/01/2014 10:42, Alon Bar-Lev ha scritto: >>>>>>>>>>> Hi, >>>>>>>>>>> >>>>>>>>>>> For some reason there 3.3.2 z-stream was released in its own >>>>>>>>>>> repository >>>>>>>>>>> so >>>>>>>>>>> people that are subscribed to stable[1] did not get it. >>>>>>>>>> >>>>>>>>>> Why not? >>>>>>>>>> stable release had ovirt-release-10 which enabled both stable and >>>>>>>>>> 3.3.2 >>>>>>>>>> repository by yum updating it. >>>>>>>>>> >>>>>>>>>>> >>>>>>>>>>> There is no much sense in releasing fix release that people do not >>>>>>>>>>> get >>>>>>>>>>> in >>>>>>>>>>> simple "yum update". >>>>>>>>>>> >>>>>>>>>>> Also the following is now broken of most packages' spec: >>>>>>>>>>> Source0: >>>>>>>>>>> http://ovirt.org/releases/stable/src/@PACKAGE_NAME@-@PACKAGE_VERSION@.tar.gz >>>>>>>>>>> >>>>>>>>>>> For each minor we should have rolling repository, to reduce noise >>>>>>>>>>> and >>>>>>>>>>> provide service. >>>>>>>>>>> >>>>>>>>>>> All released tarballs (sources) should be stored at fixed location >>>>>>>>>>> to >>>>>>>>>>> allow distro specific code to fetch, the location must be synced >>>>>>>>>>> with >>>>>>>>>>> what we publish. >>>>>>>>>>> >>>>>>>>>>> Immediate action is to move the 3.3.2 content into the stable >>>>>>>>>>> directory. >>>>>>>>>> >>>>>>>>>> So previous request of having each release in its own repository has >>>>>>>>>> been >>>>>>>>>> retired? >>>>>>>>>> Or is it combined? >>>>>>>>>> Do we want stable to be a rolling repository and have also a >>>>>>>>>> repository >>>>>>>>>> for >>>>>>>>>> each version? >>>>>>>>>> I'm not against having rolling packages in just one stable >>>>>>>>>> repository, >>>>>>>>>> I >>>>>>>>>> just want to understand what is the desired structure of the >>>>>>>>>> repositories. >>>>>>>>> >>>>>>>>> I am, having a stable repository with rolling rpms is a lot more hard >>>>>>>>> to >>>>>>>>> manage >>>>>>>>> and maintain than having separated individual complete repos. >>>>>>>>> >>>>>>>>> Because what we are actually delivering is not a specific rpm, but the >>>>>>>>> whole >>>>>>>>> set, that is, one repository with the set of rpms that were tested >>>>>>>>> together >>>>>>>>> and >>>>>>>>> validated. If at any point you want to mix them, you still can adding >>>>>>>>> the >>>>>>>>> other >>>>>>>>> repos. >>>>>>>>> >>>>>>>>> For updates just updating the directory where the 'stable' link points >>>>>>>>> gets >>>>>>>>> it done. >>>>>>>>> >>>>>>>>> For rollbacks you'll have to configure the old repo. That is not as >>>>>>>>> annoying >>>>>>>>> as >>>>>>>>> it might seem, because when you enable the stable repo, you want to >>>>>>>>> have >>>>>>>>> the >>>>>>>>> stable version, that changes with time. If you want to rollback to a >>>>>>>>> previous >>>>>>>>> version then just use that versions specific repo. At much we can >>>>>>>>> provide >>>>>>>>> a >>>>>>>>> link >>>>>>>>> like 'previous_stable' so if you want to rollback to the previous >>>>>>>>> version >>>>>>>>> you >>>>>>>>> can use --enablerepo=previous_version easily, but if you want to keep >>>>>>>>> using >>>>>>>>> that, you should point directly to the specific version you want tot >>>>>>>>> use. >>>>>>>>> >>>>>>>>> Creating a new repository using is almost as cheap (on hard disk >>>>>>>>> space) >>>>>>>>> as >>>>>>>>> having a rolling repository, if you use hard links, so we can create >>>>>>>>> lot's >>>>>>>>> of >>>>>>>>> them, specially for small changes from one to another. >>>>>>>>> >>>>>>>>> The only drawback that I see is when you have to release a minor >>>>>>>>> change >>>>>>>>> in >>>>>>>>> one >>>>>>>>> the the rpms, for example, to fix a critical bug, the repo will not >>>>>>>>> include >>>>>>>>> the >>>>>>>>> old package, but I'm not sure if that's really a drawback... if you >>>>>>>>> really >>>>>>>>> need >>>>>>>>> that package without the critical fix (you should not) you can have it >>>>>>>>> changing >>>>>>>>> to that specific repository. The internal naming of the repos does not >>>>>>>>> really >>>>>>>>> matter, having to point to the repo 3.3.3-beta.2 to get the second >>>>>>>>> 'respin' >>>>>>>>> of >>>>>>>>> the 3.3.3 beta repo is not a big issue I think. >>>>>>>>> >>>>>>>>> The advantages are many, the most importants I see: >>>>>>>>> - Easy management: >>>>>>>>> * no need to go version hunting in the repo to remove/add rpms >>>>>>>>> * you should never get a repo with version combinations that are >>>>>>>>> not >>>>>>>>> tested >>>>>>>>> * it's a lot easier to get rid of old repos, and to move them >>>>>>>>> around >>>>>>>>> as >>>>>>>>> they >>>>>>>>> are independent >>>>>>>>> * no broken links, right now stable repo is full of links to other >>>>>>>>> repos, >>>>>>>>> so >>>>>>>>> removing those repos leave the links broken, you have to go >>>>>>>>> checking >>>>>>>>> if >>>>>>>>> someone links to them (or their internal directories) if you have >>>>>>>>> to >>>>>>>>> clean >>>>>>>>> up old versions >>>>>>>>> - Testing, it's a lot easier to reproduce any error found, as you can >>>>>>>>> just use the same repo and you'll get the same version set. >>>>>>>>> >>>>>>>>> What do you think? >>>>>>>> >>>>>>> >>>>>>>> And you do not allow quick fix of issues found in various of packages. >>>>>>> Why not? You can create a new repo based in the previous one that >>>>>>> includes the fixed packages. It's cheap! >>>>>> >>>>>> who is you? >>>>> In this case you is the person/process/chimpanzee that is in charge of >>>>> publishing the fixed packages to the correct environment >>>>> >>>>>> how do I push fix to users for z-stream of packages as otopi, >>>>>> ovirt-host-deploy, log collector and such? >>>>> Exactly the same way you do it for engine or vdsm >>>>> >>>>>> why is these components' release cycle should be at same schedule of >>>>>> ovirt-engine which is heavy and slow? >>>>> It should not. >>>>> >>>>>> >>>>>>>> >>>>>>>> Although there is /some/ sense in syncing minor releases, I do not see >>>>>>>> any >>>>>>>> reason of syncing z-stream. >>>>>>>> >>>>>>> >>>>>>>> I think that you do not trust individual maintainer to provide >>>>>>>> z-streams. >>>>>>>> >>>>>>>> A change in z-stream should not be exposed (unless is fixing) an >>>>>>>> external >>>>>>>> interface. >>>>>>> >>>>>>> I don't think it should be hidden neither, just make clear that those >>>>>>> are not builds to be used widely, maybe just putting it under another >>>>>>> directory (not releases). Where only promoted repos can go (meaning, >>>>>>> not everyone can put repos there). >>>>>>> For example: >>>>>>> repos/releases -> for repos that have been tested and we want to publish >>>>>>> repos/testing -> for any temporary generated repo, that is not fully >>>>>>> tested and not ready for be used widely >>>>>>> >>>>>> >>>>>> why not released? only because engine is slow? I do not understand. >>>>> I don't even understand your question. We got lost at some point. I'll >>>>> try to explain a little more what I said before, maybe that will >>>>> clarify the issue to you. >>>>> You said that a change in z-stream should not be exposed, for that I >>>>> understand for that that a package that is meant to go to a z-stream >>>>> should not be exposed to the general public. I think that it should, >>>>> but it must be clear to the public that it's not yet ready (I suppose >>>>> that's the reason you don't want it public), so they use it at their >>>>> own risk. And separating the repos into two seems a good wat for making >>>>> that clear (another one is adding a suffix to the repo name for >>>>> example). >>>>> >>>>>> >>>>>>> That way you make sure that if anyone is using a repo that is not fully >>>>>>> tested, is because he wants to, but you don't forbid it. >>>>>> >>>>>> why do you think that someone is releasing untested packages? >>>>> I do not think that someone is releasing untested packages. That >>>>> sentence comes from the hypothetical situation where a repository that >>>>> is not meant to be used by the general public (I said untested, but it >>>>> could be for any other reason) is made public using a different url >>>>> than the repos that are meant to be widely used. >>>>> >>>>> Part of the advantages of that system is the ease to run tests on >>>>> specific version sets (repos). That we do not do right now (at least >>>>> *upstream*) but I think would be done in the near future. >>>> >>>> I will try to explain again. >>>> >>>> There is no actual relationship between packages, these could have been >>>> provided asynchronous by multiple sources and maintainers regardless of >>>> the ovrit project, just like libvirt or sanlock or any other 10000 >>>> dependencies we have outside of the scope of the project. >>> >>> That's not true, the relation is that they are provided by the same >>> repository and that they are maintained by the same community. Yes, >>> they could have been provided by multiple sources and maintainers, but >>> they were not. >>> >>>> Trying to control the release cycle only because we have two fat components >>>> is something that should be avoided. >>> The model I exposed does not care if you create a new repository each >>> hour changing just one package or you create the repository on time a >>> year. What it's true is that it will be recreated when ANY (one or >>> more) of the packages included changes. >>> >>>> >>>> So far we have successfully released packages async with no regressions nor >>>> issues, and quickly solved user issues. There is absolutely no reason to >>>> stop this offering. >>> Yes, that in the last weeks sandro and me (mostly me) spent more than >>> two days trying to create a couple releases with the old process. It's >>> hard to maintain and I personally prefer focusing on another tasks than >>> searching rpm versions and trying to figure out what can be >>> deleted/moved and what can't. A proved way to improve things is to >>> change them, try new ways, if you do not change you are waiting for the >>> environment to do so, and that's usually really hard to achieve. >>> Nothing suggests that adopting the process that I explained will affect >>> the user experience substantially (if think otherwise, please >>> elaborate). >>> >>>> >>>>> >>>>>> >>>>>>>> >>>>>>>> Alon >>>>>>>> >>>>>>>>> >>>>>>>>>> >>>>>>>>>>> Regards, >>>>>>>>>>> Alon Bar-Lev. >>>>>>>>>>> >>>>>>>>>>> [1] http://resources.ovirt.org/releases/stable/ >>>>>>>>>>> >>>>>>>>>> >>>>>>>>>> >>>>>>>>> >>>>>>>>> >>>>>>>>> -- >>>>>>>>> David Caro >>>>>>>>> >>>>>>>>> Red Hat S.L. >>>>>>>>> Continuous Integration Engineer - EMEA ENG Virtualization R&D >>>>>>>>> >>>>>>>>> Email: dcaro(a)redhat.com >>>>>>>>> Web: www.redhat.com >>>>>>>>> RHT Global #: 82-62605 >>>>>>>>> >>>>>>>>> >>>>>>> >>>>>>> >>>>>>> >>>>>>> -- >>>>>>> David Caro >>>>>>> >>>>>>> Red Hat S.L. >>>>>>> Continuous Integration Engineer - EMEA ENG Virtualization R&D >>>>>>> >>>>>>> Email: dcaro(a)redhat.com >>>>>>> Web: www.redhat.com >>>>>>> RHT Global #: 82-62605 >>>>>>> >>>>>>> >>>>> >>>>> >>>>> >>>>> -- >>>>> David Caro >>>>> >>>>> Red Hat S.L. >>>>> Continuous Integration Engineer - EMEA ENG Virtualization R&D >>>>> >>>>> Email: dcaro(a)redhat.com >>>>> Web: www.redhat.com >>>>> RHT Global #: 82-62605 >>>>> >>>>> >>> >>> >>> >>> -- >>> David Caro >>> >>> Red Hat S.L. >>> Continuous Integration Engineer - EMEA ENG Virtualization R&D >>> >>> Email: dcaro(a)redhat.com >>> Web: www.redhat.com >>> RHT Global #: 82-62605 >>> >>> > > > > -- > David Caro > > Red Hat S.L. > Continuous Integration Engineer - EMEA ENG Virtualization R&D > > Email: dcaro(a)redhat.com > Web: www.redhat.com > RHT Global #: 82-62605 >