----- Original Message -----
From: "David Caro" <dcaroest(a)redhat.com>
To: "Nir Soffer" <nsoffer(a)redhat.com>
Cc: "Eyal Edri" <eedri(a)redhat.com>, "Oved Ourfali"
<ovedo(a)redhat.com>, "infra" <infra(a)ovirt.org>, devel(a)ovirt.org
Sent: Wednesday, December 10, 2014 4:59:36 PM
Subject: Re: [ovirt-devel] Creating a new gerrit flag
On 12/10, Nir Soffer wrote:
>
>
> ----- Original Message -----
> > From: "Eyal Edri" <eedri(a)redhat.com>
> > To: devel(a)ovirt.org
> > Cc: "Oved Ourfali" <ovedo(a)redhat.com>, "infra"
<infra(a)ovirt.org>
> > Sent: Wednesday, December 10, 2014 10:40:47 AM
> > Subject: Re: [ovirt-devel] Creating a new gerrit flag
> >
> >
> >
> > ----- Original Message -----
> > > From: "Oved Ourfali" <ovedo(a)redhat.com>
> > > To: "David Caro" <dcaroest(a)redhat.com>
> > > Cc: devel(a)ovirt.org
> > > Sent: Wednesday, December 10, 2014 8:30:30 AM
> > > Subject: Re: [ovirt-devel] Creating a new gerrit flag
> > >
> > >
> > >
> > > ----- Original Message -----
> > > > From: "David Caro" <dcaroest(a)redhat.com>
> > > > To: "Oved Ourfali" <ovedo(a)redhat.com>
> > > > Cc: devel(a)ovirt.org
> > > > Sent: Tuesday, December 9, 2014 7:02:44 PM
> > > > Subject: Re: [ovirt-devel] Creating a new gerrit flag
> > > >
> > > > On 12/09, Oved Ourfali wrote:
> > > > >
> > > > >
> > > > > ----- Original Message -----
> > > > > > From: "David Caro" <dcaroest(a)redhat.com>
> > > > > > To: "Oved Ourfali" <ovedo(a)redhat.com>
> > > > > > Cc: "Sven Kieske" <s.kieske(a)mittwald.de>,
devel(a)ovirt.org
> > > > > > Sent: Tuesday, December 9, 2014 3:40:30 PM
> > > > > > Subject: Re: [ovirt-devel] Creating a new gerrit flag
> > > > > >
> > > > > > On 12/09, Oved Ourfali wrote:
> > > > > > >
> > > > > > >
> > > > > > > ----- Original Message -----
> > > > > > > > From: "Sven Kieske"
<s.kieske(a)mittwald.de>
> > > > > > > > To: devel(a)ovirt.org
> > > > > > > > Sent: Tuesday, December 9, 2014 3:21:43 PM
> > > > > > > > Subject: Re: [ovirt-devel] Creating a new gerrit
flag
> > > > > > > >
> > > > > > > >
> > > > > > > >
> > > > > > > > On 09/12/14 13:47, Oved Ourfali wrote:
> > > > > > > > > safe up to 95% or so.
> > > > > > > >
> > > > > > > > You just made up that number.
> > > > > > > > I don't really understand why you would want
> > > > > > > > to downgrade your code quality by circumventing
tests.
> > > > > > > >
> > > > > > > > Maybe someone can elaborate on this a bit?
> > > > > > > >
> > > > > > >
> > > > > > > It doesn't downgrade the code quality.
> > > > > > > It is just a way to ensure developers can both merge
changes,
> > > > > > > and
> > > > > > > do
> > > > > > > it
> > > > > > > as
> > > > > > > safely as possible without relying on post-submit
tools.
> > > > > > > The number is indeed invented... as I don't have
real
> > > > > > > statistics,
> > > > > > > but
> > > > > > > it
> > > > > > > comes to say that it would be safe most of the time.
> > > > > > > After the patch is merged, if CI will fail, it is the
> > > > > > > responsibility
> > > > > > > of
> > > > > > > the
> > > > > > > developer to check that out and fix that.
> > > > > >
> > > > > > This thread was started to avoid getting to that point, as
> > > > > > getting a
> > > > > > failed patch inside the code means breaking all the other
tests
> > > > > > that
> > > > > > run on top of it and that blocks all the development, not
only
> > > > > > that
> > > > > > specific patch.
> > > > > >
> > > > >
> > > > > The issue that started the discussion was an issue in which
there
> > > > > was a
> > > > > Tests "-1" flag, and it was ignored.
> > > > > My suggestion will enforce that it won't be ignored.
> > > > > In more rare cases, in which the rebase is the source of the
tests
> > > > > issue,
> > > > > then you'll find about it later.
> > > >
> > > > I started the discussion, and I started it because a developer
> > > > complained about not being able to merge a patch because it was
> > > > failing the tests due to an already merged patch that was making all
> > > > the builds to fail. And was trying to get a solution to avoid
getting
> > > > to that point where a patch is merged while breaking the tests.
> > > >
> > > >
> > > > So in summary, you are suggestion this:
> > > >
> > > > Creating a new flag 'tested' with values +1, 0 and -1 that
only
> > > > jenkins
> > > > and managers can set
> > > >
> > > > Block form submitting any patches that have a -1
> > > >
> > > > Carry the value of that flag to following patches only if the flag
> > > > was
> > > > -1
> > > >
> > >
> >
> > +1, we need a way to block bad patches from being merged, even with a
> > rebase
> > in gerrit.
> > going forward we're planning a few changes to the way jenkins jobs are
> > run on
> > ovirt ci, which will help
> > reduce noise and imrove resources usages.
> >
> > 1. moving into a flow process, where critical jobs like unit
> > tests/checkstyle
> > will run first and only then other heavy jobs will run
> > (integration/rpms/findbugs)
>
> This is already implemented in vdsm for few months - running "make check"
> will run the fast tests first and will not run the slower tests if a fast
> test
> failed.
Please change to be able to run only fast tests or only slow tests,
that way we can separate the job into two and give feedback about the
fast tests before the slows have finished running.
These are the available targets (from faster to slower):
- gitignore - check that certain files are ignored
- pyflakes - check common Python errors (e.g. unused imports)
- pep8 - style check
- check - run the fast checks above and if successful, the unittests
Environment variables:
NOSE_SLOW_TESTS=1 - enable slow tests (we have only few)
NOSE_STRESS_TESTS=1 - enable stress tests (probably not useful for the CI)
Note that the environment variables are used only for the tests in
vdsm/tests there are few tests in various sub directories that do
not use the test infrastructure in vdsm/tests.
- check-all - run make check enbaling both slow and stress tests
Do you need a separate target for the unittests?
Nir