Hi,

I have another question about our approach to moving to stdci v2 - after some talk with Martin, he suggested to make a switch from v1 to v2 just by adding 'stdci.yaml' in master and our stable branches and turning off old CI. Then, we would have the same old build process just with v2.

That makes sense to me - right now I'm working with refining 'stdci.yaml' for master (cleaning things up, splitting into substages, etc.), but the problem is that for each patch I post *two* pipelines are being launched - the old CI and the new one. IMHO this is abusing the infrastructure.

By enabling v2 first and only then focusing on doing refinements we could avoid that.

But the real question is - do we want the new CI to be optimized (substages etc.) for stable branches? I don't feel really comfortable with messing with stable's build process...

Marcin

On Mon, Nov 26, 2018 at 3:00 PM Marcin Sobczyk <msobczyk@redhat.com> wrote:

Hi,

I'm currently working on paralleling our stdci v2.

I've already extracted 'linters' stage, more patches (and more
substages) are on the way.

This part i.e. :

if git diff-tree --no-commit-id --name-only -r HEAD | egrep --quiet
'vdsm.spec.in|Makefile.am|automation' ; then
     ./automation/build-artifacts.sh"
...

seems to be an excellent candidate for extraction to a separate substage.

The question is - how should we proceed with tests? I can create
substage for each of:

tox -e "tests,{storage,lib,network,virt}"

But the original 'check-patch' combined the coverage reports into one -
we would lose that.

My long term goal is to get rid of all the ugly bash code in the makefile, and

run everything via tox, but as first step I think we can split the work by

running:

    make tests

In the tests substage, instead of "make check" today.

Will do so.

Does it change anything about coverage?

We'll get separate coverage reports for py27 x el7 and {py27, py36} x fc28.

Theoretically we can split also to storage/network/virt/infra jobs but I think

this will consume too many resources and harm other projects sharing 

the slaves.

 

There is a possibility that we could work on something that gathers
coverage data from multiple sources (tests, OST) as a completely
separate jenkins job or smth, but that will be a bigger effort.
What do you think about it?

Marcin