8:43 a.m.
Hi,
I understand that OST (or the job(s) that run it in CI) has some logic
to decide what tests need to be ran based on the components that were
changed since the last run.
Suppose that I want to see how the last good run of something looked
like. Say the engine, or vdsm, or hosted-engine. Is there a simple way
to find that out?
Also, we might risk here not doing some test because we think it's not
needed, and perhaps we should run all tests at least once every X
(say, 1 day). Do we already? How can I see logs of this?
Thanks,
--
Didi
9:03 a.m.
On Sun, Dec 10, 2017 at 3:43 PM, Yedidyah Bar David <didi(a)redhat.com> wrote:
Hi,
I understand that OST (or the job(s) that run it in CI) has some logic
to decide what tests need to be ran based on the components that were
changed since the last run.
Indeed. OST have a tool that we call Change Resolver. What is basically
does is to extract the changed files from the git commit, and return a list
of suites that include this file or a symlink to it in it's directory.
Also it has some logic to decide what to do for some edge cases (files
outside of suites directory/files that could not be resolved)
Suppose that I want to see how the last good run of something looked
like. Say the engine, or vdsm, or hosted-engine. Is there a simple way
to find that out?
I think that the best place to look for those runs of OST is to go to the
Change Queue Testers (the one for the version you look for) and search for
a job with your requested component (engine, vdsm, ...)
Also, we might risk here not doing some test because we think it's not
needed, and perhaps we should run all tests at least once every X
(say, 1 day). Do we already? How can I see logs of this?
We have timed jobs for all the suites that are not part of the Change Queue.
http://jenkins.ovirt.org/view/oVirt%20system%20tests/
--
DANIEL BELENKY
RHV DEVOPS
<https://red.ht/sig>
10:17 a.m.
On 10 December 2017 at 15:43, Yedidyah Bar David <didi(a)redhat.com> wrote:
Hi,
I understand that OST (or the job(s) that run it in CI) has some logic
to decide what tests need to be ran based on the components that were
changed since the last run.
First you need to be clear about what you're asking about. There is
OST, which is the lago-based set of testing suits that can be run
under various conditions, and there is CQ, which collects changes made
to the whole of oVirt, runs them through certain OST suits and
collects packages that passed successfully in the 'tested' repo.
If we're talking about CQ - it always runs the same suits regardless
of what has changed.
Suppose that I want to see how the last good run of something looked
like. Say the engine, or vdsm, or hosted-engine. Is there a simple way
to find that out?
The short answer is no. There is no simple 100% reliable way ATM. The
assumption right no is that if you`ve not been contacted by one of our
team members, you can assume your changes either passed or are still
queued to be tested.
The reason for that is that CQ collects changes in batches and runs
them together. Those batches and broken down to smaller bathes only
when failures are detected.
It is possible to trace an individual change through the CQ to find
out if it passed or not, but a good understanding of how CQ works is
required for that.
One heuristic way to know what passed is to just look at the 'tested'
repo. It is typically trivial to go from an RPM name to the Git commit
that generated it.
At some point we will make CQ report back to Gerrit. This will make
knowing what happened to an individual change trivial. There has also
been requests to make CQ tag projects as changes to them pass - but
there needs to be an opt-in mechanism for this from project owners, as
people can be weary of having an automated system write to their Git
repos.
Also, we might risk here not doing some test because we think it's not
needed, and perhaps we should run all tests at least once every X
(say, 1 day). Do we already? How can I see logs of this?
As said before - CQ runs all the suits that were deemed stable enough
for CQ on all changes, no decision logic is involved.
Some of the other suits are scheduled to run on a daily basis, but
those typically run against the released or snapshot repos rather then
the pre-tested set of packages.
--
Barak Korren
RHV DevOps team , RHCE, RHCi
Red Hat EMEA
redhat.com | TRIED. TESTED. TRUSTED. | redhat.com/trusted
2343
days inactive
2343
days old
2 comments
3 participants
participants (3)
-
Barak Korren -
Daniel Belenky -
Yedidyah Bar David