[Engine-devel] Proposal for new commit msg design for engine commits

[Mike Kolesnik]

----- Original Message -----
> 
> 
> ----- Original Message -----
> > From: "Itamar Heim" <iheim at redhat.com>
> > To: "Eyal Edri" <eedri at redhat.com>
> > Cc: "engine-devel" <engine-devel at ovirt.org>, "infra" <infra at ovirt.org>
> > Sent: Wednesday, July 10, 2013 10:37:03 PM
> > Subject: Re: [Engine-devel] Proposal for new commit msg design for engine
> > 	commits
> > 
> > On 07/10/2013 10:27 PM, Eyal Edri wrote:
> > >
> > >
> > > ----- Original Message -----
> > >> From: "Fabian Deutsch" <fabiand at redhat.com>
> > >> To: "Alon Bar-Lev" <alonbl at redhat.com>
> > >> Cc: "engine-devel" <engine-devel at ovirt.org>, "infra" <infra at ovirt.org>
> > >> Sent: Tuesday, July 9, 2013 3:54:06 PM
> > >> Subject: Re: [Engine-devel] Proposal for new commit msg design for
> > >> engine
> > >> commits
> > >>
> > >> Am Dienstag, den 09.07.2013, 08:49 -0400 schrieb Alon Bar-Lev:
> > >>>
> > >>>
> > >>> ----- Original Message -----
> > >>>> From: "Yair Zaslavsky" <yzaslavs at redhat.com>
> > >>>> To: "Alon Bar-Lev" <alonbl at redhat.com>
> > >>>> Cc: "Eyal Edri" <eedri at redhat.com>, "engine-devel"
> > >>> <engine-devel at ovirt.org>, "infra" <infra at ovirt.org>
> > >>>> Sent: Tuesday, July 9, 2013 3:42:24 PM
> > >>>> Subject: Re: [Engine-devel] Proposal for new commit msg design for
> > >>> engine     commits
> > >>>>
> > >>>>
> > >>>>
> > >>>> ----- Original Message -----
> > >>>>> From: "Alon Bar-Lev" <alonbl at redhat.com>
> > >>>>> To: "Eyal Edri" <eedri at redhat.com>
> > >>>>> Cc: "engine-devel" <engine-devel at ovirt.org>, "infra"
> > >>> <infra at ovirt.org>
> > >>>>> Sent: Tuesday, July 9, 2013 3:33:57 PM
> > >>>>> Subject: Re: [Engine-devel] Proposal for new commit msg design for
> > >>> engine
> > >>>>>      commits
> > >>>>>
> > >>>>>
> > >>>>>
> > >>>>> ----- Original Message -----
> > >>>>>> From: "Eyal Edri" <eedri at redhat.com>
> > >>>>>> To: "engine-devel" <engine-devel at ovirt.org>
> > >>>>>> Cc: "infra" <infra at ovirt.org>
> > >>>>>> Sent: Tuesday, July 9, 2013 12:38:51 PM
> > >>>>>> Subject: Proposal for new commit msg design for engine commits
> > >>>>>>
> > >>>>>> Hi,
> > >>>>>>
> > >>>>>> You all probably know and familiar with 'ovirt-engine' git hook
> > >>> for
> > >>>>>> commit
> > >>>>>> msg template [1].
> > >>>>>> this helps understand the general area of the patch in the
> > >>> project but it
> > >>>>>> lacks additional info that might
> > >>>>>> be valuable for scaling automatic tests in Jenkins CI.
> > >>>>>>
> > >>>>>> Let me explain:
> > >>>>>>
> > >>>>>> Infra team is working hard on expanding oVirt CI infrastructure
> > >>> and
> > >>>>>> adding
> > >>>>>> more tests in jenkins (per commit/patch).
> > >>>>>> Adding important meta-data per patch can significatly improve
> > >>> the ability
> > >>>>>> to
> > >>>>>> run specific tests for each patch/commit,
> > >>>>>> and not waste valuable resources on Jenkins jobs that are not
> > >>> relevant to
> > >>>>>> the
> > >>>>>> code in the patch.
> > >>>>>>
> > >>>>>> So the idea is to add/expand current metadata per patch, in the
> > >>> form of:
> > >>>>>> (either)
> > >>>>>>   1. expanding current header template to include more data like
> > >>> 'network'
> > >>>>>>   ,
> > >>>>>>   'setup', 'tools', 'virt'
> > >>>>>
> > >>>>> Please do not expand header, it is too short anyway.
> > >>>>>
> > >>>>>>   2. adding a new label with relevant tags for the patch, called
> > >>> e.g
> > >>>>>>   'METADATA: network, rest, virt'
> > >>>>>
> > >>>>> Having:
> > >>>>>
> > >>>>> CI-Tests: xxx
> > >>>>> CI-Tests: yyy
> > >>>>> CI-Tests: zzz
> > >>>>>
> > >>>>> Is much better.
> > >>>>
> > >>>> I'm not sure we should have CI-Test - as we might use this for
> > >>> something else
> > >>>> besides CI.
> > >>>> Region_of_Interest as Dan suggests sounds better IMHO.
> > >>>
> > >>> I don't care how this is to be called.
> > >>> However, I do not think that commit message is the place for
> > >>> instructing CI to do anything.
> > >>> Commit message stays for good, it should contain information that is
> > >>> required a year from now.
> > >>> It has nothing to do with tests and such.
> > >>
> > >> I agree with Alon here that the Ci informations don't belong in the
> > >> commit msg.
> > >> My opinion is that a testcase should know what it covers. This
> > >> information from the testcase can then be used by any party to determin
> > >> if the testcase should be run on a specific commit (which yields
> > >> informations about the changed paths, files, owner, author, etc ...
> > >> which might be valuable).
> > >
> > > i think you're missing the point here.
> > > can you explain how do you propose a test case will know "what it
> > > covers"?
> > >
> > > let's take an example:
> > > let's say a new commit comes from ovirt-engine:
> > > http://gerrit.ovirt.org/#/c/16668/
> > > commit msg: "core: Use images instead of volumes at CDA message".
> > >
> > > now you have 1000 test cases (could be system or functional test).
> > > (let's assume that your infra can't support running 1000 tests per
> > > patch/commit).
> > >
> > > Some of these test suits checks network flow, some virt
> > > (migration/template
> > > for e.g), some host install, others storage flows and so on... ).
> > > you have one repo to clone (ovirt-engine, let's keep vdsm a side for a
> > > min), and to compile the project from for the tests.
> > >
> > > now given this scenario, please explain how will you know which test from
> > > the 1000 you have you'll run on it.
> > > do you believe that according to the author/path/filename you'll know if
> > > that patch involves storage or virt scenario?
> > >
> > > i don't think there's an alternative to a metadata to assist mapping the
> > > patch to a relevant "topic" in the code.
> > > whether it exists as a git note or a label in the commit, that's another
> > > matter and probably less important.
> > 
> > we could use gerrit labels per test topic?
> 
> I don't think it should be by patch bases, but common logic based of
> metadata, if we do this within source tree maintainer can choose which tests
> he wish to have by simply commit metadata into the source tree.
> 
> Asking for a special test can always be done in jenkins post commit, or by
> some generic field in gerrit.
> 
> However, the standard test suites is something that should be applied via
> policy of the component in question using approved metadata.
> 

+1

It's up to the maintainers to decide which tests cover their code, not up to the committer.