On Wed, 6 Feb 2019 at 11:15, Simone Tiraboschi <stirabos@redhat.com> wrote:


On Wed, Feb 6, 2019 at 10:00 AM Dan Kenigsberg <danken@redhat.com> wrote:
On Wed, Feb 6, 2019 at 10:54 AM Simone Tiraboschi <stirabos@redhat.com> wrote:
>
>
>
> On Wed, Feb 6, 2019 at 9:45 AM Dan Kenigsberg <danken@redhat.com> wrote:
>>
>> On Wed, Feb 6, 2019 at 10:16 AM Simone Tiraboschi <stirabos@redhat.com> wrote:
>> >
>> >
>> >
>> > On Tue, Feb 5, 2019 at 7:07 PM Dafna Ron <dron@redhat.com> wrote:
>> >>
>> >> Hi,
>> >>
>> >> Please note that ovirt-ansible-hosted-engine-setup has a versioning problem with the package and is causing bootstrap to fail for upgrade suite [1]
>> >>
>> >> This is effecting all projects, its been reported to the developers and should be fixed as soon as possible.
>> >>
>> >> you can view CQ status here:
>> >> https://jenkins.ovirt.org/view/Change%20queue%20jobs/job/ovirt-master_change-queue-tester/
>> >>
>> >> [1] http://pastebin.test.redhat.com/708086
>>
>> It is unfair to refer to an internal pastebin here. It is also not
>> very sensible, as it is quite short.
>>
>> 2019-02-05 11:23:51,390-0500 ERROR
>> otopi.plugins.otopi.packagers.yumpackager yumpackager.error:85 Yum
>> [u'ovirt-hosted-engine-setup-2.3.5-0.0.master.20190205110929.gitfdbc215.el7.noarch
>> requires ovirt-ansible-hosted-engine-setup >= 1.0.10']
>> 2019-02-05 11:23:51,390-0500 DEBUG otopi.context
>> context._executeMethod:142 method exception
>> Traceback (most recent call last):
>>   File "/tmp/ovirt-6fV8LBWX5i/pythonlib/otopi/context.py", line 132,
>> in _executeMethod
>>     method['method']()
>>   File "/tmp/ovirt-6fV8LBWX5i/otopi-plugins/otopi/packagers/yumpackager.py",
>> line 248, in _packages
>>     self.processTransaction()
>>   File "/tmp/ovirt-6fV8LBWX5i/otopi-plugins/otopi/packagers/yumpackager.py",
>> line 262, in processTransaction
>>     if self._miniyum.buildTransaction():
>>   File "/tmp/ovirt-6fV8LBWX5i/pythonlib/otopi/miniyum.py", line 920,
>> in buildTransaction
>>     raise yum.Errors.YumBaseError(msg)
>> YumBaseError: [u'ovirt-hosted-engine-setup-2.3.5-0.0.master.20190205110929.gitfdbc215.el7.noarch
>> requires ovirt-ansible-hosted-engine-setup >= 1.0.10']
>> 2019-02-05 11:23:51,391-0500 ERROR otopi.context
>> context._executeMethod:151 Failed to execute stage 'Package
>> installation': [u'ovirt-hosted-engine-setup-2.3.5-0.0.master.20190205110929.gitfdbc215.el7.noarch
>> requires ovirt-ansible-hosted-engine-setup >= 1.0.10']
>> 2019-02-05 11:23:51,413-0500 DEBUG
>> otopi.plugins.otopi.debug.debug_failure.debug_failure
>> debug_failure._notification:100 tcp connections:
>>
>> >>
>> >
>> > The issue is that on github we already have
>> > VERSION="1.0.10"
>> > as we can see in
>> > https://github.com/oVirt/ovirt-ansible-hosted-engine-setup/blob/master/build.sh#L3
>> >
>> > And this has been bumped before the commit that now is reported as broken.
>> >
>> > CI instead is still building the package as 1.0.9 ignoring the commit that bumped the version.
>> > Honestly I don't know how I can fix it if the version value is already the desired one in the source code.
>>
>> I don't see your ovirt-ansible-hosted-engine-setup-1.0.10, only
>> https://plain.resources.ovirt.org/pub/ovirt-master-snapshot/rpm/el7/noarch/ovirt-ansible-hosted-engine-setup-1.0.9-0.1.master.20190129095419.el7.noarch.rpm
>> Not even under "tested":
>> https://plain.resources.ovirt.org/repos/ovirt/tested/master/rpm/el7/noarch/ovirt-ansible-hosted-engine-setup-1.0.9-0.1.master.20190129095419.el7.noarch.rpm
>>
>> Simone, can you doublecheck that its artifacts have been built and
>> have been accepted by the change queue?
>
>
> It has been built here once as 1.0.10:
> https://jenkins.ovirt.org/job/oVirt_ovirt-ansible-hosted-engine-setup_standard-on-ghpush/91/
>
> then on the next commit, CI started building it again as 1.0.9 although in the source code we have 1.0.10 and so this issue.

I don't understand the issue yet (that's not surprising as I do not
know what is that "ghpush" job). Which CI job has built the wrong
version? can you share its logs? who owns it?

In the git log I see:
commit b5a6c1db135d81d75f3330160e7ef4a84c97fd60 (HEAD -> master, upstream/master, origin/master, origin/HEAD, nolog)
Author: Simone Tiraboschi <stirabos@redhat.com>
Date:   Tue Feb 5 10:56:58 2019 +0100

    Avoid using no_log when we have to pass back values to otopi

commit 96974fad1ee6aee33f8183e49240f8a2a7a617d4
Author: Simone Tiraboschi <stirabos@redhat.com>
Date:   Thu Jan 31 16:39:58 2019 +0100

    use dynamic inclusion to avoid tag inheritance

commit 4a9a23fb8e88acba5af4febed43d9e4b02e7a2c5
Author: Simone Tiraboschi <stirabos@redhat.com>
Date:   Thu Jan 31 15:15:04 2019 +0100

    Force facts gathering on partial executions

commit 7428b54a5ba8458379b1a27d116f9504bb830e69
Author: Simone Tiraboschi <stirabos@redhat.com>
Date:   Wed Jan 30 17:01:01 2019 +0100

    Use static imports and tags
    



Version has been bumped to 1.0.10 on commit 7428b54a5ba8458379b1a27d116f9504bb830e69 since it introduces a backward incompatible change and we need to track it.

7428b54a5ba8458379b1a27d116f9504bb830e69 failed CI tests due to an issue on a different package found yesterday.

So 7428b54a5ba8458379b1a27d116f9504bb830e69 got ignored and now CI is building from commit b5a6c1db135d81d75f3330160e7ef4a84c97fd60 (the last one) rebased on something before 7428b54a5ba8458379b1a27d116f9504bb830e69 which is not what we have in git so now, after b5a6c1db135d81d75f3330160e7ef4a84c97fd60 (last commit) the package builds in CI as 1.0.9 although in the code we have 1.0.10 and so the issue.


We never ignore commits, certainly not merged ones...

We can fall back to older builds on system test failures and throw away newer build if we suspect they cause the failure, if which case the builds need to be resubmitted, but this logic happens at the build leve not the commit level, there is no commit reordering or dropping anywhere.


--
Barak Korren
RHV DevOps team , RHCE, RHCi
Red Hat EMEA
redhat.com | TRIED. TESTED. TRUSTED. | redhat.com/trusted