On Sun, Dec 11, 2016 at 2:49 PM, Gil Shinar <gshinar@redhat.com> wrote:

If I understand the issue correctly, the failures are for fc23 and not for el7. build artifacts 23, the one that bumped the version, failed for fc23 [1] because of test failures.

I don't think it has to do with the distribution, my guess it is about the timing of the jobs, as the version bump code is the same for all distros. And we don't talk about job failures, the issue was the job was building 1.2.9 and not 1.2.10.

[1] http://jenkins.ovirt.org/job/vdsm-jsonrpc-java_4.0_ build-artifacts-fc23-x86_64/23/console

On Sun, Dec 11, 2016 at 11:18 AM, Eyal Edri <eedri@redhat.com> wrote:

...

Adding infra as well.

I see a very strange thing happening on the build artifacts jobs:

On [1] we see a successful build of 1.2.10 built in build-artifacts, but on the 2 patches merged after it, its back to 1.2.9 [2]. Is it possible that the 2 patches merged after the version bump weren't rebased on the version branch and were built using older code? I think what we're seeing is a race of 3 patches merged all at the same time ( jenkins shows same datetime on the builds ) and it might be that the 2 builds that show older version run before the version bump was done, even if it shows otherwise on CI.

I've run manually the job again and it nows produces 1.2.10 as expected [3], so the job should work once the rpms are deployed ( 30 min ~ )

I believe this is one of the things Zuul will solve, I can't think on something we can do at the moment to prevents such issues,

Barak - any ideas?

The project has 'fast forward only' mode in Gerrit.

[1] http://jenkins.ovirt.org/job/vdsm-jsonrpc-java_4.0_build -artifacts-el7-x86_64/23/ [2] http://jenkins.ovirt.org/job/vdsm-jsonrpc-java_4.0_build -artifacts-el7-x86_64/24/ [3] http://jenkins.ovirt.org/job/vdsm-jsonrpc-java_4.0_build -artifacts-el7-x86_64/26/console

On Sun, Dec 11, 2016 at 11:01 AM, Piotr Kliczewski <pkliczew@redhat.com> wrote:

...

Yes, version bump was merged.

11 gru 2016 09:57 "Eyal Edri" <eedri@redhat.com> napisał(a):

...

Was the version bumped pushed to the repo itself in Gerrit? I.e - does build-artifacts builds now the new version 1.2.10?

On Fri, Dec 9, 2016 at 9:33 PM, Sandro Bonazzola <sbonazzo@redhat.com> wrote:

...

Il 09/Dic/2016 18:45, "Piotr Kliczewski" <pkliczew@redhat.com> ha scritto:

Here [1] is the manual build. We need to check publisher.

Manual builds are not published by nightly publisher. Manual builds are meant for releases or for testing. I haven't the laptop with me tonight and tomorrow. Didi, Lev can you handle Sunday morning?

Thanks, Piotr

[1] http://jenkins.ovirt.org/job/vdsm-jsonrpc-java_any_build -artifacts-manual/30/

9 gru 2016 18:36 "Piotr Kliczewski" <pkliczew@redhat.com> napisał(a):

...

Both us and ds rpms were built. Is it possible to check why new version was not published to the repo?

9 gru 2016 18:07 "Martin Perina" <mperina@redhat.com> napisał(a):

> Piotr, are you able to publish new vdsm-jsonrpc-java 1.2.10 into > ovirt-4.0-snapshot repo [1]? Or only someone from integration team can do > that? > > Thanks > > Martin > > [1] http://plain.resources.ovirt.org/pub/ovirt-4.0-snapshot/rpm > > > On Fri, Dec 9, 2016 at 4:54 PM, Yaniv Kaul <ykaul@redhat.com> wrote: > >> See [1]: >> >> + yum install --nogpgcheck -y --downloaddir=/dev/shm ovirt-engine ovirt-log-collector 'ovirt-engine-extension-aaa-ldap*'*13:55:25* Error: Package: ovirt-engine-backend-4.0.7-0.0.master.20161208233116.git3dff5ce.el7.centos.noarch (alocalsync)*13:55:25* Requires: vdsm-jsonrpc-java >= 1.2.10*13:55:25* Available: vdsm-jsonrpc-java-1.2.9-1.20161208102442.gite5c0c8e.el7.centos.noarch (alocalsync)*13:55:25* vdsm-jsonrpc-java = 1.2.9-1.20161208102442.gite5c0c8e.el7.centos >> >> >> >> >> [1] http://jenkins.ovirt.org/job/ovirt-system-tests_master_check-patch-fc24-x86_... >> >> >> _______________________________________________ >> Devel mailing list >> Devel@ovirt.org >> http://lists.phx.ovirt.org/mailman/listinfo/devel >> > >

---

Devel mailing list Devel@ovirt.org http://lists.phx.ovirt.org/mailman/listinfo/devel

_______________________________________________ Devel mailing list Devel@ovirt.org http://lists.phx.ovirt.org/mailman/listinfo/devel

-- Eyal Edri Associate Manager RHV DevOps EMEA ENG Virtualization R&D Red Hat Israel

phone: +972-9-7692018 irc: eedri (on #tlv #rhev-dev #rhev-integ)

-- Eyal Edri Associate Manager RHV DevOps EMEA ENG Virtualization R&D Red Hat Israel

phone: +972-9-7692018 irc: eedri (on #tlv #rhev-dev #rhev-integ)

_______________________________________________ Infra mailing list Infra@ovirt.org http://lists.phx.ovirt.org/mailman/listinfo/infra

-- Eyal Edri Associate Manager RHV DevOps EMEA ENG Virtualization R&D Red Hat Israel phone: +972-9-7692018 irc: eedri (on #tlv #rhev-dev #rhev-integ)

On Sun, Dec 11, 2016 at 4:00 PM, Barak Korren <bkorren@redhat.com> wrote:

On 11 December 2016 at 11:18, Eyal Edri <eedri@redhat.com> wrote:

...

Adding infra as well.

I see a very strange thing happening on the build artifacts jobs:

On [1] we see a successful build of 1.2.10 built in build-artifacts, but on the 2 patches merged after it, its back to 1.2.9 [2]. Is it possible that the 2 patches merged after the version bump weren't rebased on the version branch and were built using older code?

Is build_artifacts running before or after merge? This is impossible if its running after.

...

The project has 'fast forward only' mode in Gerrit.

If build_artifacts runs on pre-merge code that even this will not help.

Bottom line - build_artifacs should never ever run on unmerged patches.

AFAIK all of these builds were triggered on merged patches.

-- Barak Korren bkorren@redhat.com RHCE, RHCi, RHV-DevOps Team https://ifireball.wordpress.com/

-- Eyal Edri Associate Manager RHV DevOps EMEA ENG Virtualization R&D Red Hat Israel phone: +972-9-7692018 irc: eedri (on #tlv #rhev-dev #rhev-integ)

I think what happened is: 1. The two patches https://gerrit.ovirt.org/#/c/67976 and https://gerrit.ovirt.org/#/c/67878/(the version bump) were merged approximately at the same time. 2. The build_artifacts job for the first patch ended after the build_artifacts job of the version bump, thus that was the 'last successful build' in Jenkins(build number 25) 3. The nightly publisher job took that RPM(from job 25), which was before the version bump. so this is not a git issue(just a bug in the nightly publisher flow) http://jenkins.ovirt.org/job/vdsm-jsonrpc-java_4.0_build-artifacts-el7-x86_6... On Sun, Dec 11, 2016 at 4:11 PM, Barak Korren <bkorren@redhat.com> wrote:

On 11 December 2016 at 16:01, Eyal Edri <eedri@redhat.com> wrote:

...

AFAIK all of these builds were triggered on merged patches.

We need to see which code its cloning - I suspect it is still trying to clone the patch instead of the master branch, this may indeed be a bug in how we configure the git plugin to work with the Gerrit trigger. (It just occurred to me it may need to be configured differently fro pre and post-merge jobs)

We will need to investigate this more deeply (compare git hashes jobs get before and after merge, etc.)

But if its a ff-only repo, this means that you can't merge a patch that needs a rebase, so if patches were rebased, and build_artifacts runs post-merge, it should be impossible to get it to run on non-rebased code.

-- Barak Korren bkorren@redhat.com RHCE, RHCi, RHV-DevOps Team https://ifireball.wordpress.com/ _______________________________________________ Devel mailing list Devel@ovirt.org http://lists.phx.ovirt.org/mailman/listinfo/devel