On Fri, Feb 3, 2017 at 3:59 PM, Sandro Bonazzola <sbonazzo(a)redhat.com&gt; wrote: I completely agree with Sandro. Its not the responsibility of infra team or integration team to make sure a certain oVirt project is building / compiling / testing for any supported OS, this lays solely on the maintainer or the team developing the project. As Sandro mentioned, the infra team is providing simple tools for building / testing / publishing any oVirt project which uses the standard CI and continues to evolve and simplify the process even more so it will be very easy for anyone to add those. If the current tools / framework is missing something or mis-behaving, you can also contact the infra team or open a ticket as Sandro mentioned above and we'll do our best to find the best solution for it. -- Eyal Edri Associate Manager RHV DevOps EMEA ENG Virtualization R&D Red Hat Israel phone: +972-9-7692018 irc: eedri (on #tlv #rhev-dev #rhev-integ)

On 3 February 2017 at 15:59, Sandro Bonazzola <sbonazzo(a)redhat.com&gt; wrote: ... There are currently two pieces of information about the project that are needed to be known in order to build and release it, but are currently specified in YAML in the 'jenkins' repo, rather then within the project's own repo: 1. Which platforms should the project be built on 2. Which oVirt releases should particular build of the project be included in. The fact that this is specified in the 'jenkins' repo **does not place this outside the maintainers` responsibility**. Things are done this way currently only because of a technical limitation with the way Standard-CI jobs are currently created. We actually have an initiative to move this information to the project repos. I've started with asking on devel list about how to specify this as part of Standard-CI [1]. But have received little topical response so far. [1]: http://lists.ovirt.org/pipermail/devel/2017-January/029161.html -- Barak Korren bkorren(a)redhat.com RHCE, RHCi, RHV-DevOps Team https://ifireball.wordpress.com/

Having said that, I would be perfectly fine with a single repository that tracks the release configuration, but only if it also held the git repository link and hash/branch name that is supposed to be released. It would be in some way a "distgit repo". Something like Sandro has for builds. [ovirt-4.0.x-ci] git://gerrit.ovirt.org/project.git#ovirt-4.0.x git://gerrit.ovirt.org/project2.git#v4.0.x git://github.com/Me/repo#master [ovirt-4.0.6] git://gerrit.ovirt.org/project.git#ovirt-4.0.6 git://gerrit.ovirt.org/project2.git#v4.0.6 http://github.com/Me/repo/releases/x.y.z.zip Any release would then be reviewed by the CI team and that would be fine for me. It would allow any branch name or versioning convention and would not pollute the sources. It would also be gerrit agnostic. Martin On Sun, Feb 5, 2017 at 10:17 AM, Martin Sivak <msivak(a)redhat.com&gt; wrote: > Hi, > >> The fact that this is specified in the 'jenkins' repo **does not place >> this outside the maintainers` responsibility**. > > >> We actually have an initiative to move this information to the project >> repos. I've started with asking on devel list about how to specify >> this as part of Standard-CI [1]. But have received little topical >> response so far. >> [1]: http://lists.ovirt.org/pipermail/devel/2017-January/029161.html > > You've got enough responses already to propose a different schema than > fixed branch names. Just give us a config file. Seriously, stop > reinventing the wheel and take a look at how others are doing it > (distgit, travis, tito, ...). > > Martin > >> >> -- >> Barak Korren >> bkorren(a)redhat.com >> RHCE, RHCi, RHV-DevOps Team >> https://ifireball.wordpress.com/ >> _______________________________________________ >> Devel mailing list >> Devel(a)ovirt.org >> http://lists.ovirt.org/mailman/listinfo/devel

On 5 February 2017 at 19:25, Nir Soffer <nsoffer(a)redhat.com&gt; wrote: Please: http://ovirt-infra-docs.readthedocs.io/en/latest/CI/Build_and_test_standa... Nope. The examples in the doc above haven't change much, you can just do more. Just submit a patch the the Jenkins repo - it gets tested and you get detailed report. For example: http://jenkins.ovirt.org/job/jenkins_master_check-patch-el7-x86_64/797/ar... Some files are unnecessarily messy, but there is no real duplication in the format itself. Nobody asked you to be responsible for the whole repo. Just a few files in it that contain information the integration/infra teams cannot possibly know. See thread I mentioned above. We got docs too. And this will make a 12 job matrix: project: name: 'vdsm_standard' project: vdsm trigger: 'on-change' version: - master: branch: master - 4.1: branch: ovirt-4.1 - 4.0: branch: ovirt-4.0 - 3.6: branch: ovirt-3.6 distro: - el7 - fc24 - fc25 Not really that different. The real complexity start showing up because you don't really want the full matrix, and also you want PPC64LE sometimes, etc. etc. Point being, the real requirements are more complex then what you do on Travis. Sent a patch to Gerrit, also see thread mentioned above and: https://ovirt-jira.atlassian.net/browse/OVIRT-1013 Take a look at 'Jenkins Blue Ocean", we'll deploy that eventually. Then again, the existing Jenkins UI is far more flexible then any of the modern flashy UIs I've seen. I guess this is a KDE v.s. Gnome kind of thing... What really needs to happen is that the relevant info needs to end up in a Gerrit comment. -- Barak Korren bkorren(a)redhat.com RHCE, RHCi, RHV-DevOps Team https://ifireball.wordpress.com/

I am using copr for building mom for a single reason. We have no distgit equivalent where I would be able to mark an arbitrary git hash as a release (using my tag and branch structure) and so copr gives me the package release experience I want. Otherwise Jenkins would be fine with me. If we are getting closer to something like that then great and I will use it when it is ready. Martin On Fri, Feb 10, 2017 at 9:57 AM, Sandro Bonazzola <sbonazzo(a)redhat.com&gt; wrote:

But that is not what I am after. I want to be able to say: Use this git hash from this repository for 4.1 and 4.0 for any upcoming release until I say otherwise (without having to use some special tag). This is not about build procedure, but about release management. The build procedure can be then part of the automation as you say, but I won't be the one executing it, Jenkins will do that for me any time you decide to do a compose. Martin On Fri, Feb 10, 2017 at 10:35 AM, Sandro Bonazzola <sbonazzo(a)redhat.com&gt; wrote: > > > On Fri, Feb 10, 2017 at 10:07 AM, Martin Sivak <msivak(a)redhat.com&gt; wrote: >> >> I am using copr for building mom for a single reason. We have no >> distgit equivalent where I would be able to mark an arbitrary git hash >> as a release (using my tag and branch structure) and so copr gives me >> the package release experience I want. Otherwise Jenkins would be fine >> with me. >> >> If we are getting closer to something like that then great and I will >> use it when it is ready. > > > to just: > spectool -g automation/mom.spec > rpmbuild \ > -ba \ > --define="_sourcedir ${PWD}" \ > --define="_srcrpmdir ${PWD}" \ > --define="_rpmdir ${PWD}" \ > automation/mom.spec > > instead of > > ./autogen.sh > ./configure --prefix=/usr > make dist > rpmbuild -ta *.tar.gz > > when jenkins will run it will build like in copr. > > >> >> Martin >> >> On Fri, Feb 10, 2017 at 9:57 AM, Sandro Bonazzola <sbonazzo(a)redhat.com&gt; >> wrote: >> > >> > >> > On Sun, Feb 5, 2017 at 6:25 PM, Nir Soffer <nsoffer(a)redhat.com&gt; wrote: >> >> >> >> On Sun, Feb 5, 2017 at 11:33 AM, Martin Sivak <msivak(a)redhat.com&gt; >> >> wrote: >> >> >> A repo where I do not have commit rights means I can't take full >> >> >> responsibility. >> >> >> A repo that is not atomically synchronized with my sources means I >> >> >> can't take full responsibility. >> >> > >> >> > Having said that, I would be perfectly fine with a single repository >> >> > that tracks the release configuration, but only if it also held the >> >> > git repository link and hash/branch name that is supposed to be >> >> > released. It would be in some way a "distgit repo". Something like >> >> > Sandro has for builds. >> >> > >> >> > [ovirt-4.0.x-ci] >> >> > git://gerrit.ovirt.org/project.git#ovirt-4.0.x >> >> > git://gerrit.ovirt.org/project2.git#v4.0.x >> >> > git://github.com/Me/repo#master >> >> > >> >> > [ovirt-4.0.6] >> >> > git://gerrit.ovirt.org/project.git#ovirt-4.0.6 >> >> > git://gerrit.ovirt.org/project2.git#v4.0.6 >> >> > http://github.com/Me/repo/releases/x.y.z.zip >> >> > >> >> > Any release would then be reviewed by the CI team and that would be >> >> > fine for me. It would allow any branch name or versioning convention >> >> > and would not pollute the sources. It would also be gerrit agnostic. >> >> >> >> Well said! >> >> >> >> May pain points with jenkins project: >> >> >> >> - No documentation >> >> - The format is not stable, each time you edit the format is different >> >> - No way to test changes, only infra guys can test changes, sometimes >> >> even infra guys cannot test changes and they simply merge them >> >> for testing >> >> - The project format is full of duplication >> >> - No commit right, cannot be responsible for something I cannot change >> >> >> >> Compare with travis: >> >> >> >> - Everting defined in *my* project in .travis.yml >> >> - Configuration format is well documented >> >> https://docs.travis-ci.com/ >> >> - Configuration format is well designed, can do lot of work with very >> >> little configuration >> >> For example - this yaml define matrix of 3 builds: >> >> https://github.com/oVirt/vdsm/blob/master/.travis.yml >> >> - Easy to test changes before merging (push to your fork on github) >> >> - Very nice web ui, e.g. >> >> https://travis-ci.org/oVirt/vdsm/builds/198571421 >> >> https://travis-ci.org/oVirt/vdsm/jobs/198571422 >> >> >> > >> > I've no strong feelings on this subject. I personally don't know travis >> > so I >> > can't compare it to anything else. >> > I'm a jenkins Standard-CI user and I tend to be happy with what we have. >> > That said, despite I would prefer all ovirt projects to be aligned with >> > the >> > same workflow, I'm perfectly fine with whatever CI testing / building >> > system >> > is implemented or used for each project provided that: >> > - it works >> > - it produces rpms which can be added to ovirt-system-test for >> > functional >> > testing >> > - it produces source archives which can be published on release >> > - it produces rpms which can be installed on release for all supported >> > platforms and arches. >> > - it doesn't require creative ways to get artifacts to be released >> > >> > >> > >> > >> >> >> >> Nir >> >> >> >> > >> >> > Martin >> >> > >> >> > >> >> > On Sun, Feb 5, 2017 at 10:17 AM, Martin Sivak <msivak(a)redhat.com&gt; >> >> > wrote: >> >> >> Hi, >> >> >> >> >> >>> The fact that this is specified in the 'jenkins' repo **does not >> >> >>> place >> >> >>> this outside the maintainers` responsibility**. >> >> >> >> >> >> A repo where I do not have commit rights means I can't take full >> >> >> responsibility. >> >> >> A repo that is not atomically synchronized with my sources means I >> >> >> can't take full responsibility. >> >> >> >> >> >>> We actually have an initiative to move this information to the >> >> >>> project >> >> >>> repos. I've started with asking on devel list about how to specify >> >> >>> this as part of Standard-CI [1]. But have received little topical >> >> >>> response so far. >> >> >>> [1]: >> >> >>> http://lists.ovirt.org/pipermail/devel/2017-January/029161.html >> >> >> >> >> >> You've got enough responses already to propose a different schema >> >> >> than >> >> >> fixed branch names. Just give us a config file. Seriously, stop >> >> >> reinventing the wheel and take a look at how others are doing it >> >> >> (distgit, travis, tito, ...). >> >> >> >> >> >> Martin >> >> >> >> >> >>> >> >> >>> -- >> >> >>> Barak Korren >> >> >>> bkorren(a)redhat.com >> >> >>> RHCE, RHCi, RHV-DevOps Team >> >> >>> https://ifireball.wordpress.com/ >> >> >>> _______________________________________________ >> >> >>> Devel mailing list >> >> >>> Devel(a)ovirt.org >> >> >>> http://lists.ovirt.org/mailman/listinfo/devel >> >> > _______________________________________________ >> >> > Devel mailing list >> >> > Devel(a)ovirt.org >> >> > http://lists.ovirt.org/mailman/listinfo/devel >> >> _______________________________________________ >> >> Devel mailing list >> >> Devel(a)ovirt.org >> >> http://lists.ovirt.org/mailman/listinfo/devel >> > >> > >> > >> > >> > -- >> > Sandro Bonazzola >> > Better technology. Faster innovation. Powered by community >> > collaboration. >> > See how it works at redhat.com > > > > > -- > Sandro Bonazzola > Better technology. Faster innovation. Powered by community collaboration. > See how it works at redhat.com

What you are not seeing is that this is partially manual and spread out across multiple repositories. I do not want to add Jenkins build anywhere. I want to tell Jenkins what it should build to get the latest release (hash/tag) and nightly (branch) and be done with it. You can then rebuild it as often as you want, change supported distributions or change configuration without me having to touch it, because you have the exact git hash. The same with source code release. The git hash / tag also gives tells you what commit you should use for the tarball. In a short version, I want to deliver the source code and the spec (+ automation for the CI) and leave Jenkins games to you. Martin On Fri, Feb 10, 2017 at 11:48 AM, Sandro Bonazzola <sbonazzo(a)redhat.com&gt; wrote: > > > On Fri, Feb 10, 2017 at 10:42 AM, Martin Sivak <msivak(a)redhat.com&gt; wrote: >> >> > I think you already can do that. >> > When you want to build for a release you can create automation/mom.spec >> > and change automaton/build-artifacts.sh >> >> But that is not what I am after. I want to be able to say: Use this >> git hash from this repository for 4.1 and 4.0 for any upcoming release >> until I say otherwise (without having to use some special tag). >> >> This is not about build procedure, but about release management. The >> build procedure can be then part of the automation as you say, but I >> won't be the one executing it, Jenkins will do that for me any time >> you decide to do a compose. > > > Not sure to follow. > jenkins will execute the build right after you'll merge the change. > after that, you can add the jenkins build to ovirt-<release>.conf > and it will be published for that release. > All following releases will keep that version until you add it to another > ovirt-<new-release>.conf, since the files are incremental. > > The only time this may fail is between major releases like 4.0 -> 4.1 or 4.1 > -> 4.2 > when beta composes are done by taking latest snapshots in > ovirt-<betarelease>.conf > which I usually ask to review before merging like I did for 4.1: > https://gerrit.ovirt.org/#/c/67645/ > > > > >> >> >> Martin >> >> On Fri, Feb 10, 2017 at 10:35 AM, Sandro Bonazzola <sbonazzo(a)redhat.com&gt; >> wrote: >> > >> > >> > On Fri, Feb 10, 2017 at 10:07 AM, Martin Sivak <msivak(a)redhat.com&gt; >> > wrote: >> >> >> >> I am using copr for building mom for a single reason. We have no >> >> distgit equivalent where I would be able to mark an arbitrary git hash >> >> as a release (using my tag and branch structure) and so copr gives me >> >> the package release experience I want. Otherwise Jenkins would be fine >> >> with me. >> >> >> >> If we are getting closer to something like that then great and I will >> >> use it when it is ready. >> > >> > >> > I think you already can do that. >> > When you want to build for a release you can create automation/mom.spec >> > and change automaton/build-artifacts.sh >> > to just: >> > spectool -g automation/mom.spec >> > rpmbuild \ >> > -ba \ >> > --define="_sourcedir ${PWD}" \ >> > --define="_srcrpmdir ${PWD}" \ >> > --define="_rpmdir ${PWD}" \ >> > automation/mom.spec >> > >> > instead of >> > >> > ./autogen.sh >> > ./configure --prefix=/usr >> > make dist >> > rpmbuild -ta *.tar.gz >> > >> > when jenkins will run it will build like in copr. >> > >> > >> >> >> >> Martin >> >> >> >> On Fri, Feb 10, 2017 at 9:57 AM, Sandro Bonazzola <sbonazzo(a)redhat.com&gt; >> >> wrote: >> >> > >> >> > >> >> > On Sun, Feb 5, 2017 at 6:25 PM, Nir Soffer <nsoffer(a)redhat.com&gt; >> >> > wrote: >> >> >> >> >> >> On Sun, Feb 5, 2017 at 11:33 AM, Martin Sivak <msivak(a)redhat.com&gt; >> >> >> wrote: >> >> >> >> A repo where I do not have commit rights means I can't take full >> >> >> >> responsibility. >> >> >> >> A repo that is not atomically synchronized with my sources means >> >> >> >> I >> >> >> >> can't take full responsibility. >> >> >> > >> >> >> > Having said that, I would be perfectly fine with a single >> >> >> > repository >> >> >> > that tracks the release configuration, but only if it also held >> >> >> > the >> >> >> > git repository link and hash/branch name that is supposed to be >> >> >> > released. It would be in some way a "distgit repo". Something like >> >> >> > Sandro has for builds. >> >> >> > >> >> >> > [ovirt-4.0.x-ci] >> >> >> > git://gerrit.ovirt.org/project.git#ovirt-4.0.x >> >> >> > git://gerrit.ovirt.org/project2.git#v4.0.x >> >> >> > git://github.com/Me/repo#master >> >> >> > >> >> >> > [ovirt-4.0.6] >> >> >> > git://gerrit.ovirt.org/project.git#ovirt-4.0.6 >> >> >> > git://gerrit.ovirt.org/project2.git#v4.0.6 >> >> >> > http://github.com/Me/repo/releases/x.y.z.zip >> >> >> > >> >> >> > Any release would then be reviewed by the CI team and that would >> >> >> > be >> >> >> > fine for me. It would allow any branch name or versioning >> >> >> > convention >> >> >> > and would not pollute the sources. It would also be gerrit >> >> >> > agnostic. >> >> >> >> >> >> Well said! >> >> >> >> >> >> May pain points with jenkins project: >> >> >> >> >> >> - No documentation >> >> >> - The format is not stable, each time you edit the format is >> >> >> different >> >> >> - No way to test changes, only infra guys can test changes, >> >> >> sometimes >> >> >> even infra guys cannot test changes and they simply merge them >> >> >> for testing >> >> >> - The project format is full of duplication >> >> >> - No commit right, cannot be responsible for something I cannot >> >> >> change >> >> >> >> >> >> Compare with travis: >> >> >> >> >> >> - Everting defined in *my* project in .travis.yml >> >> >> - Configuration format is well documented >> >> >> https://docs.travis-ci.com/ >> >> >> - Configuration format is well designed, can do lot of work with >> >> >> very >> >> >> little configuration >> >> >> For example - this yaml define matrix of 3 builds: >> >> >> https://github.com/oVirt/vdsm/blob/master/.travis.yml >> >> >> - Easy to test changes before merging (push to your fork on github) >> >> >> - Very nice web ui, e.g. >> >> >> https://travis-ci.org/oVirt/vdsm/builds/198571421 >> >> >> https://travis-ci.org/oVirt/vdsm/jobs/198571422 >> >> >> >> >> > >> >> > I've no strong feelings on this subject. I personally don't know >> >> > travis >> >> > so I >> >> > can't compare it to anything else. >> >> > I'm a jenkins Standard-CI user and I tend to be happy with what we >> >> > have. >> >> > That said, despite I would prefer all ovirt projects to be aligned >> >> > with >> >> > the >> >> > same workflow, I'm perfectly fine with whatever CI testing / building >> >> > system >> >> > is implemented or used for each project provided that: >> >> > - it works >> >> > - it produces rpms which can be added to ovirt-system-test for >> >> > functional >> >> > testing >> >> > - it produces source archives which can be published on release >> >> > - it produces rpms which can be installed on release for all >> >> > supported >> >> > platforms and arches. >> >> > - it doesn't require creative ways to get artifacts to be released >> >> > >> >> > >> >> > >> >> > >> >> >> >> >> >> Nir >> >> >> >> >> >> > >> >> >> > Martin >> >> >> > >> >> >> > >> >> >> > On Sun, Feb 5, 2017 at 10:17 AM, Martin Sivak <msivak(a)redhat.com&gt; >> >> >> > wrote: >> >> >> >> Hi, >> >> >> >> >> >> >> >>> The fact that this is specified in the 'jenkins' repo **does not >> >> >> >>> place >> >> >> >>> this outside the maintainers` responsibility**. >> >> >> >> >> >> >> >> A repo where I do not have commit rights means I can't take full >> >> >> >> responsibility. >> >> >> >> A repo that is not atomically synchronized with my sources means >> >> >> >> I >> >> >> >> can't take full responsibility. >> >> >> >> >> >> >> >>> We actually have an initiative to move this information to the >> >> >> >>> project >> >> >> >>> repos. I've started with asking on devel list about how to >> >> >> >>> specify >> >> >> >>> this as part of Standard-CI [1]. But have received little >> >> >> >>> topical >> >> >> >>> response so far. >> >> >> >>> [1]: >> >> >> >>> http://lists.ovirt.org/pipermail/devel/2017-January/029161.html >> >> >> >> >> >> >> >> You've got enough responses already to propose a different schema >> >> >> >> than >> >> >> >> fixed branch names. Just give us a config file. Seriously, stop >> >> >> >> reinventing the wheel and take a look at how others are doing it >> >> >> >> (distgit, travis, tito, ...). >> >> >> >> >> >> >> >> Martin >> >> >> >> >> >> >> >>> >> >> >> >>> -- >> >> >> >>> Barak Korren >> >> >> >>> bkorren(a)redhat.com >> >> >> >>> RHCE, RHCi, RHV-DevOps Team >> >> >> >>> https://ifireball.wordpress.com/ >> >> >> >>> _______________________________________________ >> >> >> >>> Devel mailing list >> >> >> >>> Devel(a)ovirt.org >> >> >> >>> http://lists.ovirt.org/mailman/listinfo/devel >> >> >> > _______________________________________________ >> >> >> > Devel mailing list >> >> >> > Devel(a)ovirt.org >> >> >> > http://lists.ovirt.org/mailman/listinfo/devel >> >> >> _______________________________________________ >> >> >> Devel mailing list >> >> >> Devel(a)ovirt.org >> >> >> http://lists.ovirt.org/mailman/listinfo/devel >> >> > >> >> > >> >> > >> >> > >> >> > -- >> >> > Sandro Bonazzola >> >> > Better technology. Faster innovation. Powered by community >> >> > collaboration. >> >> > See how it works at redhat.com >> > >> > >> > >> > >> > -- >> > Sandro Bonazzola >> > Better technology. Faster innovation. Powered by community >> > collaboration. >> > See how it works at redhat.com > > > > > -- > Sandro Bonazzola > Better technology. Faster innovation. Powered by community collaboration. > See how it works at redhat.com