Finding the "last successful" OST CI run of something

[Barak Korren]

On 10 December 2017 at 15:43, Yedidyah Bar David <didi at redhat.com> wrote:
> Hi,
>
> I understand that OST (or the job(s) that run it in CI) has some logic
> to decide what tests need to be ran based on the components that were
> changed since the last run.

First you need to be clear about what you're asking about. There is
OST, which is the lago-based set of testing suits that can be run
under various conditions, and there is CQ, which collects changes made
to the whole of oVirt, runs them through certain OST suits and
collects packages that passed successfully in the 'tested' repo.

If we're talking about CQ - it always runs the same suits regardless
of what has changed.

> Suppose that I want to see how the last good run of something looked
> like. Say the engine, or vdsm, or hosted-engine. Is there a simple way
> to find that out?

The short answer is no. There is no simple 100% reliable way ATM. The
assumption right no is that if you`ve not been contacted by one of our
team members, you can assume your changes either passed or are still
queued to be tested.

The reason for that is that CQ collects changes in batches and runs
them together. Those batches and broken down to smaller bathes only
when failures are detected.
It is possible to trace an individual change through the CQ to find
out if it passed or not, but a good understanding of how CQ works is
required for that.

One heuristic way to know what passed is to just look at the 'tested'
repo. It is typically trivial to go from an RPM name to the Git commit
that generated it.

At some point we will make CQ report back to Gerrit. This will make
knowing what happened to an individual change trivial. There has also
been requests to make CQ tag projects as changes to them pass - but
there needs to be an opt-in mechanism for this from project owners, as
people can be weary of having an automated system write to their Git
repos.

>
> Also, we might risk here not doing some test because we think it's not
> needed, and perhaps we should run all tests at least once every X
> (say, 1 day). Do we already? How can I see logs of this?

As said before - CQ runs all the suits that were deemed stable enough
for CQ on all changes, no decision logic is involved.
Some of the other suits are scheduled to run on a daily basis, but
those typically run against the released or snapshot repos rather then
the pre-tested set of packages.

-- 
Barak Korren
RHV DevOps team , RHCE, RHCi
Red Hat EMEA
redhat.com | TRIED. TESTED. TRUSTED. | redhat.com/trusted