package versioning problem causing failure on CQ ovirt-master on all projects
Hi, Please note that ovirt-ansible-hosted-engine-setup has a versioning problem with the package and is causing bootstrap to fail for upgrade suite [1] This is effecting all projects, its been reported to the developers and should be fixed as soon as possible. you can view CQ status here: https://jenkins.ovirt.org/view/Change%20queue%20jobs/job/ovirt-master_change... [1] http://pastebin.test.redhat.com/708086 Thanks, Dafna
On Tue, Feb 5, 2019 at 8:09 PM Dafna Ron <dron@redhat.com> wrote:
Hi,
Please note that ovirt-ansible-hosted-engine-setup has a versioning problem with the package and is causing bootstrap to fail for upgrade suite [1]
This is effecting all projects, its been reported to the developers and should be fixed as soon as possible.
It this the same issue? + install_cmd='yum install --nogpgcheck --downloaddir=/dev/shm -y' + yum install --nogpgcheck --downloaddir=/dev/shm -y ntp net-snmp ovirt-engine ovirt-log-collector 'ovirt-engine-extension-aaa-ldap*' otopi-debug-plugins cronie Error: Package: ovirt-engine-4.3.0.5-0.0.master.20190205084851.gitaaebfc9.el7.noarch (alocalsync) Requires: ovirt-web-ui + ret=1 + echo 'install failed with status 1' + exit 1 Seen here: https://jenkins.ovirt.org/blue/organizations/jenkins/ovirt-system-tests_manu... Looks like another variant of the issue we had few days ago.
you can view CQ status here:
https://jenkins.ovirt.org/view/Change%20queue%20jobs/job/ovirt-master_change...
[1] http://pastebin.test.redhat.com/708086
Thanks, Dafna
_______________________________________________ Devel mailing list -- devel@ovirt.org To unsubscribe send an email to devel-leave@ovirt.org Privacy Statement: https://www.ovirt.org/site/privacy-policy/ oVirt Code of Conduct: https://www.ovirt.org/community/about/community-guidelines/ List Archives: https://lists.ovirt.org/archives/list/devel@ovirt.org/message/L7REKAWSA72ZE5...
On Tue, Feb 5, 2019 at 4:36 PM Nir Soffer <nsoffer@redhat.com> wrote:
On Tue, Feb 5, 2019 at 8:09 PM Dafna Ron <dron@redhat.com> wrote:
Hi,
Please note that ovirt-ansible-hosted-engine-setup has a versioning problem with the package and is causing bootstrap to fail for upgrade suite [1]
This is effecting all projects, its been reported to the developers and should be fixed as soon as possible.
It this the same issue?
+ install_cmd='yum install --nogpgcheck --downloaddir=/dev/shm -y' + yum install --nogpgcheck --downloaddir=/dev/shm -y ntp net-snmp ovirt-engine ovirt-log-collector 'ovirt-engine-extension-aaa-ldap*' otopi-debug-plugins cronie Error: Package: ovirt-engine-4.3.0.5-0.0.master.20190205084851.gitaaebfc9.el7.noarch (alocalsync) Requires: ovirt-web-ui
Dafna, let me know if I need to do something. ovirt-web-ui lives in copr and doesn't use stdci -- not sure if this is related at all.
+ ret=1 + echo 'install failed with status 1' + exit 1
Seen here:
https://jenkins.ovirt.org/blue/organizations/jenkins/ovirt-system-tests_manu...
Looks like another variant of the issue we had few days ago.
you can view CQ status here:
https://jenkins.ovirt.org/view/Change%20queue%20jobs/job/ovirt-master_change...
[1] http://pastebin.test.redhat.com/708086
Thanks, Dafna
_______________________________________________ Devel mailing list -- devel@ovirt.org To unsubscribe send an email to devel-leave@ovirt.org Privacy Statement: https://www.ovirt.org/site/privacy-policy/ oVirt Code of Conduct: https://www.ovirt.org/community/about/community-guidelines/ List Archives: https://lists.ovirt.org/archives/list/devel@ovirt.org/message/L7REKAWSA72ZE5...
_______________________________________________ Devel mailing list -- devel@ovirt.org To unsubscribe send an email to devel-leave@ovirt.org Privacy Statement: https://www.ovirt.org/site/privacy-policy/ oVirt Code of Conduct: https://www.ovirt.org/community/about/community-guidelines/ List Archives: https://lists.ovirt.org/archives/list/devel@ovirt.org/message/FFAPHIFMTP6WBW...
-- GREG SHEREMETA SENIOR SOFTWARE ENGINEER - TEAM LEAD - RHV UX Red Hat NA <https://www.redhat.com/> gshereme@redhat.com IRC: gshereme <https://red.ht/sig>
On Tue, Feb 5, 2019 at 7:07 PM Dafna Ron <dron@redhat.com> wrote:
Hi,
Please note that ovirt-ansible-hosted-engine-setup has a versioning problem with the package and is causing bootstrap to fail for upgrade suite [1]
This is effecting all projects, its been reported to the developers and should be fixed as soon as possible.
you can view CQ status here:
https://jenkins.ovirt.org/view/Change%20queue%20jobs/job/ovirt-master_change...
The issue is that on github we already have VERSION="1.0.10" as we can see in https://github.com/oVirt/ovirt-ansible-hosted-engine-setup/blob/master/build... And this has been bumped before the commit that now is reported as broken. CI instead is still building the package as 1.0.9 ignoring the commit that bumped the version. Honestly I don't know how I can fix it if the version value is already the desired one in the source code.
Thanks, Dafna
On Wed, Feb 6, 2019 at 10:16 AM Simone Tiraboschi <stirabos@redhat.com> wrote:
On Tue, Feb 5, 2019 at 7:07 PM Dafna Ron <dron@redhat.com> wrote:
Hi,
Please note that ovirt-ansible-hosted-engine-setup has a versioning problem with the package and is causing bootstrap to fail for upgrade suite [1]
This is effecting all projects, its been reported to the developers and should be fixed as soon as possible.
you can view CQ status here: https://jenkins.ovirt.org/view/Change%20queue%20jobs/job/ovirt-master_change...
It is unfair to refer to an internal pastebin here. It is also not very sensible, as it is quite short. 2019-02-05 11:23:51,390-0500 ERROR otopi.plugins.otopi.packagers.yumpackager yumpackager.error:85 Yum [u'ovirt-hosted-engine-setup-2.3.5-0.0.master.20190205110929.gitfdbc215.el7.noarch requires ovirt-ansible-hosted-engine-setup >= 1.0.10'] 2019-02-05 11:23:51,390-0500 DEBUG otopi.context context._executeMethod:142 method exception Traceback (most recent call last): File "/tmp/ovirt-6fV8LBWX5i/pythonlib/otopi/context.py", line 132, in _executeMethod method['method']() File "/tmp/ovirt-6fV8LBWX5i/otopi-plugins/otopi/packagers/yumpackager.py", line 248, in _packages self.processTransaction() File "/tmp/ovirt-6fV8LBWX5i/otopi-plugins/otopi/packagers/yumpackager.py", line 262, in processTransaction if self._miniyum.buildTransaction(): File "/tmp/ovirt-6fV8LBWX5i/pythonlib/otopi/miniyum.py", line 920, in buildTransaction raise yum.Errors.YumBaseError(msg) YumBaseError: [u'ovirt-hosted-engine-setup-2.3.5-0.0.master.20190205110929.gitfdbc215.el7.noarch requires ovirt-ansible-hosted-engine-setup >= 1.0.10'] 2019-02-05 11:23:51,391-0500 ERROR otopi.context context._executeMethod:151 Failed to execute stage 'Package installation': [u'ovirt-hosted-engine-setup-2.3.5-0.0.master.20190205110929.gitfdbc215.el7.noarch requires ovirt-ansible-hosted-engine-setup >= 1.0.10'] 2019-02-05 11:23:51,413-0500 DEBUG otopi.plugins.otopi.debug.debug_failure.debug_failure debug_failure._notification:100 tcp connections:
The issue is that on github we already have VERSION="1.0.10" as we can see in https://github.com/oVirt/ovirt-ansible-hosted-engine-setup/blob/master/build...
And this has been bumped before the commit that now is reported as broken.
CI instead is still building the package as 1.0.9 ignoring the commit that bumped the version. Honestly I don't know how I can fix it if the version value is already the desired one in the source code.
I don't see your ovirt-ansible-hosted-engine-setup-1.0.10, only https://plain.resources.ovirt.org/pub/ovirt-master-snapshot/rpm/el7/noarch/o... Not even under "tested": https://plain.resources.ovirt.org/repos/ovirt/tested/master/rpm/el7/noarch/o... Simone, can you doublecheck that its artifacts have been built and have been accepted by the change queue?
On Wed, Feb 6, 2019 at 9:45 AM Dan Kenigsberg <danken@redhat.com> wrote:
On Wed, Feb 6, 2019 at 10:16 AM Simone Tiraboschi <stirabos@redhat.com> wrote:
On Tue, Feb 5, 2019 at 7:07 PM Dafna Ron <dron@redhat.com> wrote:
Hi,
Please note that ovirt-ansible-hosted-engine-setup has a versioning
problem with the package and is causing bootstrap to fail for upgrade suite [1]
This is effecting all projects, its been reported to the developers and
should be fixed as soon as possible.
you can view CQ status here:
https://jenkins.ovirt.org/view/Change%20queue%20jobs/job/ovirt-master_change...
It is unfair to refer to an internal pastebin here. It is also not very sensible, as it is quite short.
2019-02-05 11:23:51,390-0500 ERROR otopi.plugins.otopi.packagers.yumpackager yumpackager.error:85 Yum
[u'ovirt-hosted-engine-setup-2.3.5-0.0.master.20190205110929.gitfdbc215.el7.noarch requires ovirt-ansible-hosted-engine-setup >= 1.0.10'] 2019-02-05 11:23:51,390-0500 DEBUG otopi.context context._executeMethod:142 method exception Traceback (most recent call last): File "/tmp/ovirt-6fV8LBWX5i/pythonlib/otopi/context.py", line 132, in _executeMethod method['method']() File "/tmp/ovirt-6fV8LBWX5i/otopi-plugins/otopi/packagers/yumpackager.py", line 248, in _packages self.processTransaction() File "/tmp/ovirt-6fV8LBWX5i/otopi-plugins/otopi/packagers/yumpackager.py", line 262, in processTransaction if self._miniyum.buildTransaction(): File "/tmp/ovirt-6fV8LBWX5i/pythonlib/otopi/miniyum.py", line 920, in buildTransaction raise yum.Errors.YumBaseError(msg) YumBaseError: [u'ovirt-hosted-engine-setup-2.3.5-0.0.master.20190205110929.gitfdbc215.el7.noarch requires ovirt-ansible-hosted-engine-setup >= 1.0.10'] 2019-02-05 11:23:51,391-0500 ERROR otopi.context context._executeMethod:151 Failed to execute stage 'Package installation': [u'ovirt-hosted-engine-setup-2.3.5-0.0.master.20190205110929.gitfdbc215.el7.noarch requires ovirt-ansible-hosted-engine-setup >= 1.0.10'] 2019-02-05 11:23:51,413-0500 DEBUG otopi.plugins.otopi.debug.debug_failure.debug_failure debug_failure._notification:100 tcp connections:
The issue is that on github we already have VERSION="1.0.10" as we can see in
https://github.com/oVirt/ovirt-ansible-hosted-engine-setup/blob/master/build...
And this has been bumped before the commit that now is reported as
broken.
CI instead is still building the package as 1.0.9 ignoring the commit
Honestly I don't know how I can fix it if the version value is already
that bumped the version. the desired one in the source code.
I don't see your ovirt-ansible-hosted-engine-setup-1.0.10, only
https://plain.resources.ovirt.org/pub/ovirt-master-snapshot/rpm/el7/noarch/o... Not even under "tested":
https://plain.resources.ovirt.org/repos/ovirt/tested/master/rpm/el7/noarch/o...
Simone, can you doublecheck that its artifacts have been built and have been accepted by the change queue?
It has been built here once as 1.0.10: https://jenkins.ovirt.org/job/oVirt_ovirt-ansible-hosted-engine-setup_standa... then on the next commit, CI started building it again as 1.0.9 although in the source code we have 1.0.10 and so this issue.
On Wed, Feb 6, 2019 at 10:54 AM Simone Tiraboschi <stirabos@redhat.com> wrote:
On Wed, Feb 6, 2019 at 9:45 AM Dan Kenigsberg <danken@redhat.com> wrote:
On Wed, Feb 6, 2019 at 10:16 AM Simone Tiraboschi <stirabos@redhat.com> wrote:
On Tue, Feb 5, 2019 at 7:07 PM Dafna Ron <dron@redhat.com> wrote:
Hi,
Please note that ovirt-ansible-hosted-engine-setup has a versioning problem with the package and is causing bootstrap to fail for upgrade suite [1]
This is effecting all projects, its been reported to the developers and should be fixed as soon as possible.
you can view CQ status here: https://jenkins.ovirt.org/view/Change%20queue%20jobs/job/ovirt-master_change...
It is unfair to refer to an internal pastebin here. It is also not very sensible, as it is quite short.
2019-02-05 11:23:51,390-0500 ERROR otopi.plugins.otopi.packagers.yumpackager yumpackager.error:85 Yum [u'ovirt-hosted-engine-setup-2.3.5-0.0.master.20190205110929.gitfdbc215.el7.noarch requires ovirt-ansible-hosted-engine-setup >= 1.0.10'] 2019-02-05 11:23:51,390-0500 DEBUG otopi.context context._executeMethod:142 method exception Traceback (most recent call last): File "/tmp/ovirt-6fV8LBWX5i/pythonlib/otopi/context.py", line 132, in _executeMethod method['method']() File "/tmp/ovirt-6fV8LBWX5i/otopi-plugins/otopi/packagers/yumpackager.py", line 248, in _packages self.processTransaction() File "/tmp/ovirt-6fV8LBWX5i/otopi-plugins/otopi/packagers/yumpackager.py", line 262, in processTransaction if self._miniyum.buildTransaction(): File "/tmp/ovirt-6fV8LBWX5i/pythonlib/otopi/miniyum.py", line 920, in buildTransaction raise yum.Errors.YumBaseError(msg) YumBaseError: [u'ovirt-hosted-engine-setup-2.3.5-0.0.master.20190205110929.gitfdbc215.el7.noarch requires ovirt-ansible-hosted-engine-setup >= 1.0.10'] 2019-02-05 11:23:51,391-0500 ERROR otopi.context context._executeMethod:151 Failed to execute stage 'Package installation': [u'ovirt-hosted-engine-setup-2.3.5-0.0.master.20190205110929.gitfdbc215.el7.noarch requires ovirt-ansible-hosted-engine-setup >= 1.0.10'] 2019-02-05 11:23:51,413-0500 DEBUG otopi.plugins.otopi.debug.debug_failure.debug_failure debug_failure._notification:100 tcp connections:
The issue is that on github we already have VERSION="1.0.10" as we can see in https://github.com/oVirt/ovirt-ansible-hosted-engine-setup/blob/master/build...
And this has been bumped before the commit that now is reported as broken.
CI instead is still building the package as 1.0.9 ignoring the commit that bumped the version. Honestly I don't know how I can fix it if the version value is already the desired one in the source code.
I don't see your ovirt-ansible-hosted-engine-setup-1.0.10, only https://plain.resources.ovirt.org/pub/ovirt-master-snapshot/rpm/el7/noarch/o... Not even under "tested": https://plain.resources.ovirt.org/repos/ovirt/tested/master/rpm/el7/noarch/o...
Simone, can you doublecheck that its artifacts have been built and have been accepted by the change queue?
It has been built here once as 1.0.10: https://jenkins.ovirt.org/job/oVirt_ovirt-ansible-hosted-engine-setup_standa...
then on the next commit, CI started building it again as 1.0.9 although in the source code we have 1.0.10 and so this issue.
I don't understand the issue yet (that's not surprising as I do not know what is that "ghpush" job). Which CI job has built the wrong version? can you share its logs? who owns it?
On Wed, 6 Feb 2019 at 11:00, Dan Kenigsberg <danken@redhat.com> wrote:
On Wed, Feb 6, 2019 at 10:54 AM Simone Tiraboschi <stirabos@redhat.com> wrote:
On Wed, Feb 6, 2019 at 9:45 AM Dan Kenigsberg <danken@redhat.com> wrote:
On Wed, Feb 6, 2019 at 10:16 AM Simone Tiraboschi <stirabos@redhat.com>
On Tue, Feb 5, 2019 at 7:07 PM Dafna Ron <dron@redhat.com> wrote:
Hi,
Please note that ovirt-ansible-hosted-engine-setup has a versioning
This is effecting all projects, its been reported to the developers
and should be fixed as soon as possible.
you can view CQ status here:
https://jenkins.ovirt.org/view/Change%20queue%20jobs/job/ovirt-master_change...
It is unfair to refer to an internal pastebin here. It is also not very sensible, as it is quite short.
2019-02-05 11:23:51,390-0500 ERROR otopi.plugins.otopi.packagers.yumpackager yumpackager.error:85 Yum
[u'ovirt-hosted-engine-setup-2.3.5-0.0.master.20190205110929.gitfdbc215.el7.noarch
requires ovirt-ansible-hosted-engine-setup >= 1.0.10'] 2019-02-05 11:23:51,390-0500 DEBUG otopi.context context._executeMethod:142 method exception Traceback (most recent call last): File "/tmp/ovirt-6fV8LBWX5i/pythonlib/otopi/context.py", line 132, in _executeMethod method['method']() File "/tmp/ovirt-6fV8LBWX5i/otopi-plugins/otopi/packagers/yumpackager.py", line 248, in _packages self.processTransaction() File "/tmp/ovirt-6fV8LBWX5i/otopi-plugins/otopi/packagers/yumpackager.py", line 262, in processTransaction if self._miniyum.buildTransaction(): File "/tmp/ovirt-6fV8LBWX5i/pythonlib/otopi/miniyum.py", line 920, in buildTransaction raise yum.Errors.YumBaseError(msg) YumBaseError: [u'ovirt-hosted-engine-setup-2.3.5-0.0.master.20190205110929.gitfdbc215.el7.noarch requires ovirt-ansible-hosted-engine-setup >= 1.0.10'] 2019-02-05 11:23:51,391-0500 ERROR otopi.context context._executeMethod:151 Failed to execute stage 'Package installation': [u'ovirt-hosted-engine-setup-2.3.5-0.0.master.20190205110929.gitfdbc215.el7.noarch requires ovirt-ansible-hosted-engine-setup >= 1.0.10'] 2019-02-05 11:23:51,413-0500 DEBUG otopi.plugins.otopi.debug.debug_failure.debug_failure debug_failure._notification:100 tcp connections:
The issue is that on github we already have VERSION="1.0.10" as we can see in
https://github.com/oVirt/ovirt-ansible-hosted-engine-setup/blob/master/build...
And this has been bumped before the commit that now is reported as
broken.
CI instead is still building the package as 1.0.9 ignoring the commit
wrote: problem with the package and is causing bootstrap to fail for upgrade suite [1] that bumped the version.
Honestly I don't know how I can fix it if the version value is already the desired one in the source code.
I don't see your ovirt-ansible-hosted-engine-setup-1.0.10, only
https://plain.resources.ovirt.org/pub/ovirt-master-snapshot/rpm/el7/noarch/o...
Not even under "tested":
https://plain.resources.ovirt.org/repos/ovirt/tested/master/rpm/el7/noarch/o...
Simone, can you doublecheck that its artifacts have been built and have been accepted by the change queue?
It has been built here once as 1.0.10:
https://jenkins.ovirt.org/job/oVirt_ovirt-ansible-hosted-engine-setup_standa...
then on the next commit, CI started building it again as 1.0.9 although
in the source code we have 1.0.10 and so this issue.
The next build of this job failed because of infra issue. Are you confusing pre-and post-merge builds? *ghpush job runs post merge on merged code *check-pr job runs on PRs change-queue only looks at builds generated by the ghpush job and unless someone intervened manually, the *ghpush job should always handle commits in merge order.
I don't understand the issue yet (that's not surprising as I do not know what is that "ghpush" job). Which CI job has built the wrong version? can you share its logs? who owns it?
-- Barak Korren RHV DevOps team , RHCE, RHCi Red Hat EMEA redhat.com | TRIED. TESTED. TRUSTED. | redhat.com/trusted
On Wed, Feb 6, 2019 at 10:00 AM Dan Kenigsberg <danken@redhat.com> wrote:
On Wed, Feb 6, 2019 at 10:54 AM Simone Tiraboschi <stirabos@redhat.com> wrote:
On Wed, Feb 6, 2019 at 9:45 AM Dan Kenigsberg <danken@redhat.com> wrote:
On Wed, Feb 6, 2019 at 10:16 AM Simone Tiraboschi <stirabos@redhat.com>
On Tue, Feb 5, 2019 at 7:07 PM Dafna Ron <dron@redhat.com> wrote:
Hi,
Please note that ovirt-ansible-hosted-engine-setup has a versioning
This is effecting all projects, its been reported to the developers
and should be fixed as soon as possible.
you can view CQ status here:
https://jenkins.ovirt.org/view/Change%20queue%20jobs/job/ovirt-master_change...
It is unfair to refer to an internal pastebin here. It is also not very sensible, as it is quite short.
2019-02-05 11:23:51,390-0500 ERROR otopi.plugins.otopi.packagers.yumpackager yumpackager.error:85 Yum
[u'ovirt-hosted-engine-setup-2.3.5-0.0.master.20190205110929.gitfdbc215.el7.noarch
requires ovirt-ansible-hosted-engine-setup >= 1.0.10'] 2019-02-05 11:23:51,390-0500 DEBUG otopi.context context._executeMethod:142 method exception Traceback (most recent call last): File "/tmp/ovirt-6fV8LBWX5i/pythonlib/otopi/context.py", line 132, in _executeMethod method['method']() File "/tmp/ovirt-6fV8LBWX5i/otopi-plugins/otopi/packagers/yumpackager.py", line 248, in _packages self.processTransaction() File "/tmp/ovirt-6fV8LBWX5i/otopi-plugins/otopi/packagers/yumpackager.py", line 262, in processTransaction if self._miniyum.buildTransaction(): File "/tmp/ovirt-6fV8LBWX5i/pythonlib/otopi/miniyum.py", line 920, in buildTransaction raise yum.Errors.YumBaseError(msg) YumBaseError: [u'ovirt-hosted-engine-setup-2.3.5-0.0.master.20190205110929.gitfdbc215.el7.noarch requires ovirt-ansible-hosted-engine-setup >= 1.0.10'] 2019-02-05 11:23:51,391-0500 ERROR otopi.context context._executeMethod:151 Failed to execute stage 'Package installation': [u'ovirt-hosted-engine-setup-2.3.5-0.0.master.20190205110929.gitfdbc215.el7.noarch requires ovirt-ansible-hosted-engine-setup >= 1.0.10'] 2019-02-05 11:23:51,413-0500 DEBUG otopi.plugins.otopi.debug.debug_failure.debug_failure debug_failure._notification:100 tcp connections:
The issue is that on github we already have VERSION="1.0.10" as we can see in
https://github.com/oVirt/ovirt-ansible-hosted-engine-setup/blob/master/build...
And this has been bumped before the commit that now is reported as
broken.
CI instead is still building the package as 1.0.9 ignoring the commit
wrote: problem with the package and is causing bootstrap to fail for upgrade suite [1] that bumped the version.
Honestly I don't know how I can fix it if the version value is already the desired one in the source code.
I don't see your ovirt-ansible-hosted-engine-setup-1.0.10, only
https://plain.resources.ovirt.org/pub/ovirt-master-snapshot/rpm/el7/noarch/o...
Not even under "tested":
https://plain.resources.ovirt.org/repos/ovirt/tested/master/rpm/el7/noarch/o...
Simone, can you doublecheck that its artifacts have been built and have been accepted by the change queue?
It has been built here once as 1.0.10:
https://jenkins.ovirt.org/job/oVirt_ovirt-ansible-hosted-engine-setup_standa...
then on the next commit, CI started building it again as 1.0.9 although
in the source code we have 1.0.10 and so this issue.
I don't understand the issue yet (that's not surprising as I do not know what is that "ghpush" job). Which CI job has built the wrong version? can you share its logs? who owns it?
In the git log I see: commit b5a6c1db135d81d75f3330160e7ef4a84c97fd60 (HEAD -> master, upstream/master, origin/master, origin/HEAD, nolog) Author: Simone Tiraboschi <stirabos@redhat.com> Date: Tue Feb 5 10:56:58 2019 +0100 Avoid using no_log when we have to pass back values to otopi commit 96974fad1ee6aee33f8183e49240f8a2a7a617d4 Author: Simone Tiraboschi <stirabos@redhat.com> Date: Thu Jan 31 16:39:58 2019 +0100 use dynamic inclusion to avoid tag inheritance commit 4a9a23fb8e88acba5af4febed43d9e4b02e7a2c5 Author: Simone Tiraboschi <stirabos@redhat.com> Date: Thu Jan 31 15:15:04 2019 +0100 Force facts gathering on partial executions commit 7428b54a5ba8458379b1a27d116f9504bb830e69 Author: Simone Tiraboschi <stirabos@redhat.com> Date: Wed Jan 30 17:01:01 2019 +0100 Use static imports and tags Fixes https://github.com/oVirt/ovirt-ansible-hosted-engine-setup/issues/20 Reuires https://github.com/oVirt/ovirt-ansible-engine-setup/pull/39 Version has been bumped to 1.0.10 on commit 7428b54a5ba8458379b1a27d116f9504bb830e69 since it introduces a backward incompatible change and we need to track it. 7428b54a5ba8458379b1a27d116f9504bb830e69 failed CI tests due to an issue on a different package found yesterday. So 7428b54a5ba8458379b1a27d116f9504bb830e69 got ignored and now CI is building from commit b5a6c1db135d81d75f3330160e7ef4a84c97fd60 (the last one) rebased on something before 7428b54a5ba8458379b1a27d116f9504bb830e69 which is not what we have in git so now, after b5a6c1db135d81d75f3330160e7ef4a84c97fd60 (last commit) the package builds in CI as 1.0.9 although in the code we have 1.0.10 and so the issue.
On Wed, 6 Feb 2019 at 11:15, Simone Tiraboschi <stirabos@redhat.com> wrote:
On Wed, Feb 6, 2019 at 10:00 AM Dan Kenigsberg <danken@redhat.com> wrote:
On Wed, Feb 6, 2019 at 10:54 AM Simone Tiraboschi <stirabos@redhat.com> wrote:
On Wed, Feb 6, 2019 at 9:45 AM Dan Kenigsberg <danken@redhat.com>
On Wed, Feb 6, 2019 at 10:16 AM Simone Tiraboschi <stirabos@redhat.com>
wrote:
On Tue, Feb 5, 2019 at 7:07 PM Dafna Ron <dron@redhat.com> wrote:
Hi,
Please note that ovirt-ansible-hosted-engine-setup has a versioning
wrote: problem with the package and is causing bootstrap to fail for upgrade suite [1]
This is effecting all projects, its been reported to the developers
and should be fixed as soon as possible.
you can view CQ status here:
https://jenkins.ovirt.org/view/Change%20queue%20jobs/job/ovirt-master_change...
It is unfair to refer to an internal pastebin here. It is also not very sensible, as it is quite short.
2019-02-05 11:23:51,390-0500 ERROR otopi.plugins.otopi.packagers.yumpackager yumpackager.error:85 Yum
[u'ovirt-hosted-engine-setup-2.3.5-0.0.master.20190205110929.gitfdbc215.el7.noarch
requires ovirt-ansible-hosted-engine-setup >= 1.0.10'] 2019-02-05 11:23:51,390-0500 DEBUG otopi.context context._executeMethod:142 method exception Traceback (most recent call last): File "/tmp/ovirt-6fV8LBWX5i/pythonlib/otopi/context.py", line 132, in _executeMethod method['method']() File "/tmp/ovirt-6fV8LBWX5i/otopi-plugins/otopi/packagers/yumpackager.py", line 248, in _packages self.processTransaction() File "/tmp/ovirt-6fV8LBWX5i/otopi-plugins/otopi/packagers/yumpackager.py", line 262, in processTransaction if self._miniyum.buildTransaction(): File "/tmp/ovirt-6fV8LBWX5i/pythonlib/otopi/miniyum.py", line 920, in buildTransaction raise yum.Errors.YumBaseError(msg) YumBaseError: [u'ovirt-hosted-engine-setup-2.3.5-0.0.master.20190205110929.gitfdbc215.el7.noarch requires ovirt-ansible-hosted-engine-setup >= 1.0.10'] 2019-02-05 11:23:51,391-0500 ERROR otopi.context context._executeMethod:151 Failed to execute stage 'Package installation': [u'ovirt-hosted-engine-setup-2.3.5-0.0.master.20190205110929.gitfdbc215.el7.noarch requires ovirt-ansible-hosted-engine-setup >= 1.0.10'] 2019-02-05 11:23:51,413-0500 DEBUG otopi.plugins.otopi.debug.debug_failure.debug_failure debug_failure._notification:100 tcp connections:
The issue is that on github we already have VERSION="1.0.10" as we can see in
https://github.com/oVirt/ovirt-ansible-hosted-engine-setup/blob/master/build...
And this has been bumped before the commit that now is reported as
broken.
CI instead is still building the package as 1.0.9 ignoring the
commit that bumped the version.
Honestly I don't know how I can fix it if the version value is already the desired one in the source code.
I don't see your ovirt-ansible-hosted-engine-setup-1.0.10, only
https://plain.resources.ovirt.org/pub/ovirt-master-snapshot/rpm/el7/noarch/o...
Not even under "tested":
https://plain.resources.ovirt.org/repos/ovirt/tested/master/rpm/el7/noarch/o...
Simone, can you doublecheck that its artifacts have been built and have been accepted by the change queue?
It has been built here once as 1.0.10:
https://jenkins.ovirt.org/job/oVirt_ovirt-ansible-hosted-engine-setup_standa...
then on the next commit, CI started building it again as 1.0.9 although
in the source code we have 1.0.10 and so this issue.
I don't understand the issue yet (that's not surprising as I do not know what is that "ghpush" job). Which CI job has built the wrong version? can you share its logs? who owns it?
In the git log I see: commit b5a6c1db135d81d75f3330160e7ef4a84c97fd60 (HEAD -> master, upstream/master, origin/master, origin/HEAD, nolog) Author: Simone Tiraboschi <stirabos@redhat.com> Date: Tue Feb 5 10:56:58 2019 +0100
Avoid using no_log when we have to pass back values to otopi
commit 96974fad1ee6aee33f8183e49240f8a2a7a617d4 Author: Simone Tiraboschi <stirabos@redhat.com> Date: Thu Jan 31 16:39:58 2019 +0100
use dynamic inclusion to avoid tag inheritance
commit 4a9a23fb8e88acba5af4febed43d9e4b02e7a2c5 Author: Simone Tiraboschi <stirabos@redhat.com> Date: Thu Jan 31 15:15:04 2019 +0100
Force facts gathering on partial executions
commit 7428b54a5ba8458379b1a27d116f9504bb830e69 Author: Simone Tiraboschi <stirabos@redhat.com> Date: Wed Jan 30 17:01:01 2019 +0100
Use static imports and tags
Fixes https://github.com/oVirt/ovirt-ansible-hosted-engine-setup/issues/20 Reuires https://github.com/oVirt/ovirt-ansible-engine-setup/pull/39
Version has been bumped to 1.0.10 on commit 7428b54a5ba8458379b1a27d116f9504bb830e69 since it introduces a backward incompatible change and we need to track it.
7428b54a5ba8458379b1a27d116f9504bb830e69 failed CI tests due to an issue on a different package found yesterday.
So 7428b54a5ba8458379b1a27d116f9504bb830e69 got ignored and now CI is building from commit b5a6c1db135d81d75f3330160e7ef4a84c97fd60 (the last one) rebased on something before 7428b54a5ba8458379b1a27d116f9504bb830e69 which is not what we have in git so now, after b5a6c1db135d81d75f3330160e7ef4a84c97fd60 (last commit) the package builds in CI as 1.0.9 although in the code we have 1.0.10 and so the issue.
We never ignore commits, certainly not merged ones... We can fall back to older builds on system test failures and throw away newer build if we suspect they cause the failure, if which case the builds need to be resubmitted, but this logic happens at the build leve not the commit level, there is no commit reordering or dropping anywhere. -- Barak Korren RHV DevOps team , RHCE, RHCi Red Hat EMEA redhat.com | TRIED. TESTED. TRUSTED. | redhat.com/trusted
On Wed, Feb 6, 2019 at 10:23 AM Barak Korren <bkorren@redhat.com> wrote:
On Wed, 6 Feb 2019 at 11:15, Simone Tiraboschi <stirabos@redhat.com> wrote:
On Wed, Feb 6, 2019 at 10:00 AM Dan Kenigsberg <danken@redhat.com> wrote:
On Wed, Feb 6, 2019 at 10:54 AM Simone Tiraboschi <stirabos@redhat.com> wrote:
On Wed, Feb 6, 2019 at 9:45 AM Dan Kenigsberg <danken@redhat.com>
wrote:
On Wed, Feb 6, 2019 at 10:16 AM Simone Tiraboschi <
stirabos@redhat.com> wrote:
On Tue, Feb 5, 2019 at 7:07 PM Dafna Ron <dron@redhat.com> wrote: > > Hi, > > Please note that ovirt-ansible-hosted-engine-setup has a
versioning problem with the package and is causing bootstrap to fail for upgrade suite [1]
> > This is effecting all projects, its been reported to the developers and should be fixed as soon as possible. > > you can view CQ status here: > https://jenkins.ovirt.org/view/Change%20queue%20jobs/job/ovirt-master_change... > > [1] http://pastebin.test.redhat.com/708086
It is unfair to refer to an internal pastebin here. It is also not very sensible, as it is quite short.
2019-02-05 11:23:51,390-0500 ERROR otopi.plugins.otopi.packagers.yumpackager yumpackager.error:85 Yum
[u'ovirt-hosted-engine-setup-2.3.5-0.0.master.20190205110929.gitfdbc215.el7.noarch
requires ovirt-ansible-hosted-engine-setup >= 1.0.10'] 2019-02-05 11:23:51,390-0500 DEBUG otopi.context context._executeMethod:142 method exception Traceback (most recent call last): File "/tmp/ovirt-6fV8LBWX5i/pythonlib/otopi/context.py", line 132, in _executeMethod method['method']() File "/tmp/ovirt-6fV8LBWX5i/otopi-plugins/otopi/packagers/yumpackager.py", line 248, in _packages self.processTransaction() File "/tmp/ovirt-6fV8LBWX5i/otopi-plugins/otopi/packagers/yumpackager.py", line 262, in processTransaction if self._miniyum.buildTransaction(): File "/tmp/ovirt-6fV8LBWX5i/pythonlib/otopi/miniyum.py", line 920, in buildTransaction raise yum.Errors.YumBaseError(msg) YumBaseError: [u'ovirt-hosted-engine-setup-2.3.5-0.0.master.20190205110929.gitfdbc215.el7.noarch requires ovirt-ansible-hosted-engine-setup >= 1.0.10'] 2019-02-05 11:23:51,391-0500 ERROR otopi.context context._executeMethod:151 Failed to execute stage 'Package installation': [u'ovirt-hosted-engine-setup-2.3.5-0.0.master.20190205110929.gitfdbc215.el7.noarch requires ovirt-ansible-hosted-engine-setup >= 1.0.10'] 2019-02-05 11:23:51,413-0500 DEBUG otopi.plugins.otopi.debug.debug_failure.debug_failure debug_failure._notification:100 tcp connections:
>
The issue is that on github we already have VERSION="1.0.10" as we can see in
https://github.com/oVirt/ovirt-ansible-hosted-engine-setup/blob/master/build...
And this has been bumped before the commit that now is reported as
broken.
CI instead is still building the package as 1.0.9 ignoring the
commit that bumped the version.
Honestly I don't know how I can fix it if the version value is already the desired one in the source code.
I don't see your ovirt-ansible-hosted-engine-setup-1.0.10, only
https://plain.resources.ovirt.org/pub/ovirt-master-snapshot/rpm/el7/noarch/o...
Not even under "tested":
https://plain.resources.ovirt.org/repos/ovirt/tested/master/rpm/el7/noarch/o...
Simone, can you doublecheck that its artifacts have been built and have been accepted by the change queue?
It has been built here once as 1.0.10:
https://jenkins.ovirt.org/job/oVirt_ovirt-ansible-hosted-engine-setup_standa...
then on the next commit, CI started building it again as 1.0.9
although in the source code we have 1.0.10 and so this issue.
I don't understand the issue yet (that's not surprising as I do not know what is that "ghpush" job). Which CI job has built the wrong version? can you share its logs? who owns it?
In the git log I see: commit b5a6c1db135d81d75f3330160e7ef4a84c97fd60 (HEAD -> master, upstream/master, origin/master, origin/HEAD, nolog) Author: Simone Tiraboschi <stirabos@redhat.com> Date: Tue Feb 5 10:56:58 2019 +0100
Avoid using no_log when we have to pass back values to otopi
commit 96974fad1ee6aee33f8183e49240f8a2a7a617d4 Author: Simone Tiraboschi <stirabos@redhat.com> Date: Thu Jan 31 16:39:58 2019 +0100
use dynamic inclusion to avoid tag inheritance
commit 4a9a23fb8e88acba5af4febed43d9e4b02e7a2c5 Author: Simone Tiraboschi <stirabos@redhat.com> Date: Thu Jan 31 15:15:04 2019 +0100
Force facts gathering on partial executions
commit 7428b54a5ba8458379b1a27d116f9504bb830e69 Author: Simone Tiraboschi <stirabos@redhat.com> Date: Wed Jan 30 17:01:01 2019 +0100
Use static imports and tags
Fixes https://github.com/oVirt/ovirt-ansible-hosted-engine-setup/issues/20 Reuires https://github.com/oVirt/ovirt-ansible-engine-setup/pull/39
Version has been bumped to 1.0.10 on commit 7428b54a5ba8458379b1a27d116f9504bb830e69 since it introduces a backward incompatible change and we need to track it.
7428b54a5ba8458379b1a27d116f9504bb830e69 failed CI tests due to an issue on a different package found yesterday.
So 7428b54a5ba8458379b1a27d116f9504bb830e69 got ignored and now CI is building from commit b5a6c1db135d81d75f3330160e7ef4a84c97fd60 (the last one) rebased on something before 7428b54a5ba8458379b1a27d116f9504bb830e69 which is not what we have in git so now, after b5a6c1db135d81d75f3330160e7ef4a84c97fd60 (last commit) the package builds in CI as 1.0.9 although in the code we have 1.0.10 and so the issue.
We never ignore commits, certainly not merged ones...
We can fall back to older builds on system test failures and throw away newer build if we suspect they cause the failure, if which case the builds need to be resubmitted, but this logic happens at the build leve not the commit level, there is no commit reordering or dropping anywhere.
I'd double check the CI code for that: on git side, in build.sh at commit b5a6c1db135d81d75f3330160e7ef4a84c97fd60 we have VERSION="1.0.10" as you can see in https://github.com/oVirt/ovirt-ansible-hosted-engine-setup/blob/b5a6c1db135d... but, as you can see in https://jenkins.ovirt.org/job/oVirt_ovirt-ansible-hosted-engine-setup_standa... CI builds it as 1.0.9 ignoring commit 7428b54a5ba8458379b1a27d116f9504bb830e69 that has been merged before b5a6c1db135d81d75f3330160e7ef4a84c97fd60
-- Barak Korren RHV DevOps team , RHCE, RHCi Red Hat EMEA redhat.com | TRIED. TESTED. TRUSTED. | redhat.com/trusted
On Wed, 6 Feb 2019 at 11:34, Simone Tiraboschi <stirabos@redhat.com> wrote:
On Wed, Feb 6, 2019 at 10:23 AM Barak Korren <bkorren@redhat.com> wrote:
On Wed, 6 Feb 2019 at 11:15, Simone Tiraboschi <stirabos@redhat.com> wrote:
On Wed, Feb 6, 2019 at 10:00 AM Dan Kenigsberg <danken@redhat.com> wrote:
On Wed, Feb 6, 2019 at 10:54 AM Simone Tiraboschi <stirabos@redhat.com> wrote:
On Wed, Feb 6, 2019 at 9:45 AM Dan Kenigsberg <danken@redhat.com>
wrote:
On Wed, Feb 6, 2019 at 10:16 AM Simone Tiraboschi <
stirabos@redhat.com> wrote:
> > > > On Tue, Feb 5, 2019 at 7:07 PM Dafna Ron <dron@redhat.com> wrote: >> >> Hi, >> >> Please note that ovirt-ansible-hosted-engine-setup has a versioning problem with the package and is causing bootstrap to fail for upgrade suite [1] >> >> This is effecting all projects, its been reported to the developers and should be fixed as soon as possible. >> >> you can view CQ status here: >> https://jenkins.ovirt.org/view/Change%20queue%20jobs/job/ovirt-master_change... >> >> [1] http://pastebin.test.redhat.com/708086
It is unfair to refer to an internal pastebin here. It is also not very sensible, as it is quite short.
2019-02-05 11:23:51,390-0500 ERROR otopi.plugins.otopi.packagers.yumpackager yumpackager.error:85 Yum
[u'ovirt-hosted-engine-setup-2.3.5-0.0.master.20190205110929.gitfdbc215.el7.noarch
requires ovirt-ansible-hosted-engine-setup >= 1.0.10'] 2019-02-05 11:23:51,390-0500 DEBUG otopi.context context._executeMethod:142 method exception Traceback (most recent call last): File "/tmp/ovirt-6fV8LBWX5i/pythonlib/otopi/context.py", line 132, in _executeMethod method['method']() File "/tmp/ovirt-6fV8LBWX5i/otopi-plugins/otopi/packagers/yumpackager.py", line 248, in _packages self.processTransaction() File "/tmp/ovirt-6fV8LBWX5i/otopi-plugins/otopi/packagers/yumpackager.py", line 262, in processTransaction if self._miniyum.buildTransaction(): File "/tmp/ovirt-6fV8LBWX5i/pythonlib/otopi/miniyum.py", line 920, in buildTransaction raise yum.Errors.YumBaseError(msg) YumBaseError: [u'ovirt-hosted-engine-setup-2.3.5-0.0.master.20190205110929.gitfdbc215.el7.noarch requires ovirt-ansible-hosted-engine-setup >= 1.0.10'] 2019-02-05 11:23:51,391-0500 ERROR otopi.context context._executeMethod:151 Failed to execute stage 'Package installation': [u'ovirt-hosted-engine-setup-2.3.5-0.0.master.20190205110929.gitfdbc215.el7.noarch requires ovirt-ansible-hosted-engine-setup >= 1.0.10'] 2019-02-05 11:23:51,413-0500 DEBUG otopi.plugins.otopi.debug.debug_failure.debug_failure debug_failure._notification:100 tcp connections:
>> > > The issue is that on github we already have > VERSION="1.0.10" > as we can see in > https://github.com/oVirt/ovirt-ansible-hosted-engine-setup/blob/master/build... > > And this has been bumped before the commit that now is reported as broken. > > CI instead is still building the package as 1.0.9 ignoring the commit that bumped the version. > Honestly I don't know how I can fix it if the version value is already the desired one in the source code.
I don't see your ovirt-ansible-hosted-engine-setup-1.0.10, only
https://plain.resources.ovirt.org/pub/ovirt-master-snapshot/rpm/el7/noarch/o...
Not even under "tested":
https://plain.resources.ovirt.org/repos/ovirt/tested/master/rpm/el7/noarch/o...
Simone, can you doublecheck that its artifacts have been built and have been accepted by the change queue?
It has been built here once as 1.0.10:
https://jenkins.ovirt.org/job/oVirt_ovirt-ansible-hosted-engine-setup_standa...
then on the next commit, CI started building it again as 1.0.9
although in the source code we have 1.0.10 and so this issue.
I don't understand the issue yet (that's not surprising as I do not know what is that "ghpush" job). Which CI job has built the wrong version? can you share its logs? who owns it?
In the git log I see: commit b5a6c1db135d81d75f3330160e7ef4a84c97fd60 (HEAD -> master, upstream/master, origin/master, origin/HEAD, nolog) Author: Simone Tiraboschi <stirabos@redhat.com> Date: Tue Feb 5 10:56:58 2019 +0100
Avoid using no_log when we have to pass back values to otopi
commit 96974fad1ee6aee33f8183e49240f8a2a7a617d4 Author: Simone Tiraboschi <stirabos@redhat.com> Date: Thu Jan 31 16:39:58 2019 +0100
use dynamic inclusion to avoid tag inheritance
commit 4a9a23fb8e88acba5af4febed43d9e4b02e7a2c5 Author: Simone Tiraboschi <stirabos@redhat.com> Date: Thu Jan 31 15:15:04 2019 +0100
Force facts gathering on partial executions
commit 7428b54a5ba8458379b1a27d116f9504bb830e69 Author: Simone Tiraboschi <stirabos@redhat.com> Date: Wed Jan 30 17:01:01 2019 +0100
Use static imports and tags
Fixes https://github.com/oVirt/ovirt-ansible-hosted-engine-setup/issues/20 Reuires https://github.com/oVirt/ovirt-ansible-engine-setup/pull/39
Version has been bumped to 1.0.10 on commit 7428b54a5ba8458379b1a27d116f9504bb830e69 since it introduces a backward incompatible change and we need to track it.
7428b54a5ba8458379b1a27d116f9504bb830e69 failed CI tests due to an issue on a different package found yesterday.
So 7428b54a5ba8458379b1a27d116f9504bb830e69 got ignored and now CI is building from commit b5a6c1db135d81d75f3330160e7ef4a84c97fd60 (the last one) rebased on something before 7428b54a5ba8458379b1a27d116f9504bb830e69 which is not what we have in git so now, after b5a6c1db135d81d75f3330160e7ef4a84c97fd60 (last commit) the package builds in CI as 1.0.9 although in the code we have 1.0.10 and so the issue.
We never ignore commits, certainly not merged ones...
We can fall back to older builds on system test failures and throw away newer build if we suspect they cause the failure, if which case the builds need to be resubmitted, but this logic happens at the build leve not the commit level, there is no commit reordering or dropping anywhere.
I'd double check the CI code for that:
on git side, in build.sh at commit b5a6c1db135d81d75f3330160e7ef4a84c97fd60 we have VERSION="1.0.10" as you can see in
https://github.com/oVirt/ovirt-ansible-hosted-engine-setup/blob/b5a6c1db135d...
but, as you can see in
https://jenkins.ovirt.org/job/oVirt_ovirt-ansible-hosted-engine-setup_standa... CI builds it as 1.0.9 ignoring commit 7428b54a5ba8458379b1a27d116f9504bb830e69 that has been merged before b5a6c1db135d81d75f3330160e7ef4a84c97fd60
I'm not sure if build 138 ran before or after 7428b54a5ba8458379b1a27d116f9504bb830e69 was merged. since you merged the PR already, we cannot check what will happen if we rerun the job now. I suspect you're simply looking at the output of a job that stated running before 7428b54a5ba8458379b1a27d116f9504bb830e69 was merged
-- Barak Korren RHV DevOps team , RHCE, RHCi Red Hat EMEA redhat.com | TRIED. TESTED. TRUSTED. | redhat.com/trusted
-- Barak Korren RHV DevOps team , RHCE, RHCi Red Hat EMEA redhat.com | TRIED. TESTED. TRUSTED. | redhat.com/trusted
On Wed, Feb 6, 2019 at 10:44 AM Barak Korren <bkorren@redhat.com> wrote:
On Wed, 6 Feb 2019 at 11:34, Simone Tiraboschi <stirabos@redhat.com> wrote:
On Wed, Feb 6, 2019 at 10:23 AM Barak Korren <bkorren@redhat.com> wrote:
On Wed, 6 Feb 2019 at 11:15, Simone Tiraboschi <stirabos@redhat.com> wrote:
On Wed, Feb 6, 2019 at 10:00 AM Dan Kenigsberg <danken@redhat.com> wrote:
On Wed, Feb 6, 2019 at 10:54 AM Simone Tiraboschi <stirabos@redhat.com> wrote:
On Wed, Feb 6, 2019 at 9:45 AM Dan Kenigsberg <danken@redhat.com>
wrote:
> > On Wed, Feb 6, 2019 at 10:16 AM Simone Tiraboschi < stirabos@redhat.com> wrote: > > > > > > > > On Tue, Feb 5, 2019 at 7:07 PM Dafna Ron <dron@redhat.com> wrote: > >> > >> Hi, > >> > >> Please note that ovirt-ansible-hosted-engine-setup has a versioning problem with the package and is causing bootstrap to fail for upgrade suite [1] > >> > >> This is effecting all projects, its been reported to the developers and should be fixed as soon as possible. > >> > >> you can view CQ status here: > >> https://jenkins.ovirt.org/view/Change%20queue%20jobs/job/ovirt-master_change... > >> > >> [1] http://pastebin.test.redhat.com/708086 > > It is unfair to refer to an internal pastebin here. It is also not > very sensible, as it is quite short. > > 2019-02-05 11:23:51,390-0500 ERROR > otopi.plugins.otopi.packagers.yumpackager yumpackager.error:85 Yum > [u'ovirt-hosted-engine-setup-2.3.5-0.0.master.20190205110929.gitfdbc215.el7.noarch > requires ovirt-ansible-hosted-engine-setup >= 1.0.10'] > 2019-02-05 11:23:51,390-0500 DEBUG otopi.context > context._executeMethod:142 method exception > Traceback (most recent call last): > File "/tmp/ovirt-6fV8LBWX5i/pythonlib/otopi/context.py", line 132, > in _executeMethod > method['method']() > File "/tmp/ovirt-6fV8LBWX5i/otopi-plugins/otopi/packagers/yumpackager.py", > line 248, in _packages > self.processTransaction() > File "/tmp/ovirt-6fV8LBWX5i/otopi-plugins/otopi/packagers/yumpackager.py", > line 262, in processTransaction > if self._miniyum.buildTransaction(): > File "/tmp/ovirt-6fV8LBWX5i/pythonlib/otopi/miniyum.py", line 920, > in buildTransaction > raise yum.Errors.YumBaseError(msg) > YumBaseError: [u'ovirt-hosted-engine-setup-2.3.5-0.0.master.20190205110929.gitfdbc215.el7.noarch > requires ovirt-ansible-hosted-engine-setup >= 1.0.10'] > 2019-02-05 11:23:51,391-0500 ERROR otopi.context > context._executeMethod:151 Failed to execute stage 'Package > installation': [u'ovirt-hosted-engine-setup-2.3.5-0.0.master.20190205110929.gitfdbc215.el7.noarch > requires ovirt-ansible-hosted-engine-setup >= 1.0.10'] > 2019-02-05 11:23:51,413-0500 DEBUG > otopi.plugins.otopi.debug.debug_failure.debug_failure > debug_failure._notification:100 tcp connections: > > >> > > > > The issue is that on github we already have > > VERSION="1.0.10" > > as we can see in > > https://github.com/oVirt/ovirt-ansible-hosted-engine-setup/blob/master/build... > > > > And this has been bumped before the commit that now is reported as broken. > > > > CI instead is still building the package as 1.0.9 ignoring the commit that bumped the version. > > Honestly I don't know how I can fix it if the version value is already the desired one in the source code. > > I don't see your ovirt-ansible-hosted-engine-setup-1.0.10, only > https://plain.resources.ovirt.org/pub/ovirt-master-snapshot/rpm/el7/noarch/o... > Not even under "tested": > https://plain.resources.ovirt.org/repos/ovirt/tested/master/rpm/el7/noarch/o... > > Simone, can you doublecheck that its artifacts have been built and > have been accepted by the change queue?
It has been built here once as 1.0.10:
https://jenkins.ovirt.org/job/oVirt_ovirt-ansible-hosted-engine-setup_standa...
then on the next commit, CI started building it again as 1.0.9
although in the source code we have 1.0.10 and so this issue.
I don't understand the issue yet (that's not surprising as I do not know what is that "ghpush" job). Which CI job has built the wrong version? can you share its logs? who owns it?
In the git log I see: commit b5a6c1db135d81d75f3330160e7ef4a84c97fd60 (HEAD -> master, upstream/master, origin/master, origin/HEAD, nolog) Author: Simone Tiraboschi <stirabos@redhat.com> Date: Tue Feb 5 10:56:58 2019 +0100
Avoid using no_log when we have to pass back values to otopi
commit 96974fad1ee6aee33f8183e49240f8a2a7a617d4 Author: Simone Tiraboschi <stirabos@redhat.com> Date: Thu Jan 31 16:39:58 2019 +0100
use dynamic inclusion to avoid tag inheritance
commit 4a9a23fb8e88acba5af4febed43d9e4b02e7a2c5 Author: Simone Tiraboschi <stirabos@redhat.com> Date: Thu Jan 31 15:15:04 2019 +0100
Force facts gathering on partial executions
commit 7428b54a5ba8458379b1a27d116f9504bb830e69 Author: Simone Tiraboschi <stirabos@redhat.com> Date: Wed Jan 30 17:01:01 2019 +0100
Use static imports and tags
Fixes https://github.com/oVirt/ovirt-ansible-hosted-engine-setup/issues/20 Reuires https://github.com/oVirt/ovirt-ansible-engine-setup/pull/39
Version has been bumped to 1.0.10 on commit 7428b54a5ba8458379b1a27d116f9504bb830e69 since it introduces a backward incompatible change and we need to track it.
7428b54a5ba8458379b1a27d116f9504bb830e69 failed CI tests due to an issue on a different package found yesterday.
So 7428b54a5ba8458379b1a27d116f9504bb830e69 got ignored and now CI is building from commit b5a6c1db135d81d75f3330160e7ef4a84c97fd60 (the last one) rebased on something before 7428b54a5ba8458379b1a27d116f9504bb830e69 which is not what we have in git so now, after b5a6c1db135d81d75f3330160e7ef4a84c97fd60 (last commit) the package builds in CI as 1.0.9 although in the code we have 1.0.10 and so the issue.
We never ignore commits, certainly not merged ones...
We can fall back to older builds on system test failures and throw away newer build if we suspect they cause the failure, if which case the builds need to be resubmitted, but this logic happens at the build leve not the commit level, there is no commit reordering or dropping anywhere.
I'd double check the CI code for that:
on git side, in build.sh at commit b5a6c1db135d81d75f3330160e7ef4a84c97fd60 we have VERSION="1.0.10" as you can see in
https://github.com/oVirt/ovirt-ansible-hosted-engine-setup/blob/b5a6c1db135d...
but, as you can see in
https://jenkins.ovirt.org/job/oVirt_ovirt-ansible-hosted-engine-setup_standa... CI builds it as 1.0.9 ignoring commit 7428b54a5ba8458379b1a27d116f9504bb830e69 that has been merged before b5a6c1db135d81d75f3330160e7ef4a84c97fd60
I'm not sure if build 138 ran before or after 7428b54a5ba8458379b1a27d116f9504bb830e69 was merged. since you merged the PR already, we cannot check what will happen if we rerun the job now.
I suspect you're simply looking at the output of a job that stated running before 7428b54a5ba8458379b1a27d116f9504bb830e69 was merged
On master branch on https://github.com/oVirt/ovirt-ansible-hosted-engine-setup I have: b5a6c1d (HEAD -> master, upstream/master, origin/master, origin/HEAD, nolog) Avoid using no_log when we have to pass back values to otopi 96974fa use dynamic inclusion to avoid tag inheritance 4a9a23f Force facts gathering on partial executions 7428b54 Use static imports and tags 9267118 Fix rhbz1654697 (#108) 5e928a1 build: post ovirt-ansible-hosted-engine-setup-1.0.8 8607d90 (tag: 1.0.8) build: ovirt-ansible-hosted-engine-setup-1.0.8 In the console output of https://jenkins.ovirt.org/job/oVirt_ovirt-ansible-hosted-engine-setup_standa... I see *10:59:42* 2019-02-05 09:59:42,448:INFO:__main__:File 'automation/upstream_sources.yaml' cannot be opened*10:59:42* 2019-02-05 09:59:42,449:INFO:__main__:Executing command: 'git --git-dir=/home/jenkins/workspace/oVirt_ovirt-ansible-hosted-engine-setup_standard-check-pr/ovirt-ansible-hosted-engine-setup/.git --work-tree=/home/jenkins/workspace/oVirt_ovirt-ansible-hosted-engine-setup_standard-check-pr/ovirt-ansible-hosted-engine-setup reset --hard'*10:59:42* 2019-02-05 09:59:42,472:DEBUG:__main__:Git exited with status: 0*10:59:42* 2019-02-05 09:59:42,472:DEBUG:__main__: ---- stderr ----*10:59:42* 2019-02-05 09:59:42,472:DEBUG:__main__: ---- stdout ----*10:59:42* 2019-02-05 09:59:42,472:DEBUG:__main__: HEAD is now at af00824 Merge ceca0a852d94bd5af7226b15d15c9e3ff917cd5c into 9267118f456de46b9059e76b63bc85c18fcab5dd Now, ceca0a852d94bd5af7226b15d15c9e3ff917cd5c is the last commit on my develop branch on https://github.com/tiraboschi/ovirt-ansible-hosted-engine-setup commit ceca0a852d94bd5af7226b15d15c9e3ff917cd5c (origin/nolog) Author: Simone Tiraboschi <stirabos@redhat.com> Date: Tue Feb 5 10:56:58 2019 +0100 Avoid using no_log when we have to pass back values to otopi which corresponds to b5a6c1d on master on https://github.com/oVirt/ovirt-ansible-hosted-engine-setup so "Avoid using no_log when we have to pass back values to otopi" got tested over 9267118 - Fix rhbz1654697 (#108) skipping 96974fa use dynamic inclusion to avoid tag inheritance 4a9a23f Force facts gathering on partial executions 7428b54 Use static imports and tags in the middle. The version bump change is in 7428b54 and so it builds as 1.0.9 and not as 1.0.10 as expected.
-- Barak Korren RHV DevOps team , RHCE, RHCi Red Hat EMEA redhat.com | TRIED. TESTED. TRUSTED. | redhat.com/trusted
-- Barak Korren RHV DevOps team , RHCE, RHCi Red Hat EMEA redhat.com | TRIED. TESTED. TRUSTED. | redhat.com/trusted
On Wed, 6 Feb 2019 at 11:57, Simone Tiraboschi <stirabos@redhat.com> wrote:
On Wed, Feb 6, 2019 at 10:44 AM Barak Korren <bkorren@redhat.com> wrote:
On Wed, 6 Feb 2019 at 11:34, Simone Tiraboschi <stirabos@redhat.com> wrote:
On Wed, Feb 6, 2019 at 10:23 AM Barak Korren <bkorren@redhat.com> wrote:
On Wed, 6 Feb 2019 at 11:15, Simone Tiraboschi <stirabos@redhat.com> wrote:
On Wed, Feb 6, 2019 at 10:00 AM Dan Kenigsberg <danken@redhat.com> wrote:
On Wed, Feb 6, 2019 at 10:54 AM Simone Tiraboschi < stirabos@redhat.com> wrote: > > > > On Wed, Feb 6, 2019 at 9:45 AM Dan Kenigsberg <danken@redhat.com> wrote: >> >> On Wed, Feb 6, 2019 at 10:16 AM Simone Tiraboschi < stirabos@redhat.com> wrote: >> > >> > >> > >> > On Tue, Feb 5, 2019 at 7:07 PM Dafna Ron <dron@redhat.com> wrote: >> >> >> >> Hi, >> >> >> >> Please note that ovirt-ansible-hosted-engine-setup has a versioning problem with the package and is causing bootstrap to fail for upgrade suite [1] >> >> >> >> This is effecting all projects, its been reported to the developers and should be fixed as soon as possible. >> >> >> >> you can view CQ status here: >> >> https://jenkins.ovirt.org/view/Change%20queue%20jobs/job/ovirt-master_change... >> >> >> >> [1] http://pastebin.test.redhat.com/708086 >> >> It is unfair to refer to an internal pastebin here. It is also not >> very sensible, as it is quite short. >> >> 2019-02-05 11:23:51,390-0500 ERROR >> otopi.plugins.otopi.packagers.yumpackager yumpackager.error:85 Yum >> [u'ovirt-hosted-engine-setup-2.3.5-0.0.master.20190205110929.gitfdbc215.el7.noarch >> requires ovirt-ansible-hosted-engine-setup >= 1.0.10'] >> 2019-02-05 11:23:51,390-0500 DEBUG otopi.context >> context._executeMethod:142 method exception >> Traceback (most recent call last): >> File "/tmp/ovirt-6fV8LBWX5i/pythonlib/otopi/context.py", line 132, >> in _executeMethod >> method['method']() >> File "/tmp/ovirt-6fV8LBWX5i/otopi-plugins/otopi/packagers/yumpackager.py", >> line 248, in _packages >> self.processTransaction() >> File "/tmp/ovirt-6fV8LBWX5i/otopi-plugins/otopi/packagers/yumpackager.py", >> line 262, in processTransaction >> if self._miniyum.buildTransaction(): >> File "/tmp/ovirt-6fV8LBWX5i/pythonlib/otopi/miniyum.py", line 920, >> in buildTransaction >> raise yum.Errors.YumBaseError(msg) >> YumBaseError: [u'ovirt-hosted-engine-setup-2.3.5-0.0.master.20190205110929.gitfdbc215.el7.noarch >> requires ovirt-ansible-hosted-engine-setup >= 1.0.10'] >> 2019-02-05 11:23:51,391-0500 ERROR otopi.context >> context._executeMethod:151 Failed to execute stage 'Package >> installation': [u'ovirt-hosted-engine-setup-2.3.5-0.0.master.20190205110929.gitfdbc215.el7.noarch >> requires ovirt-ansible-hosted-engine-setup >= 1.0.10'] >> 2019-02-05 11:23:51,413-0500 DEBUG >> otopi.plugins.otopi.debug.debug_failure.debug_failure >> debug_failure._notification:100 tcp connections: >> >> >> >> > >> > The issue is that on github we already have >> > VERSION="1.0.10" >> > as we can see in >> > https://github.com/oVirt/ovirt-ansible-hosted-engine-setup/blob/master/build... >> > >> > And this has been bumped before the commit that now is reported as broken. >> > >> > CI instead is still building the package as 1.0.9 ignoring the commit that bumped the version. >> > Honestly I don't know how I can fix it if the version value is already the desired one in the source code. >> >> I don't see your ovirt-ansible-hosted-engine-setup-1.0.10, only >> https://plain.resources.ovirt.org/pub/ovirt-master-snapshot/rpm/el7/noarch/o... >> Not even under "tested": >> https://plain.resources.ovirt.org/repos/ovirt/tested/master/rpm/el7/noarch/o... >> >> Simone, can you doublecheck that its artifacts have been built and >> have been accepted by the change queue? > > > It has been built here once as 1.0.10: > https://jenkins.ovirt.org/job/oVirt_ovirt-ansible-hosted-engine-setup_standa... > > then on the next commit, CI started building it again as 1.0.9 although in the source code we have 1.0.10 and so this issue.
I don't understand the issue yet (that's not surprising as I do not know what is that "ghpush" job). Which CI job has built the wrong version? can you share its logs? who owns it?
In the git log I see: commit b5a6c1db135d81d75f3330160e7ef4a84c97fd60 (HEAD -> master, upstream/master, origin/master, origin/HEAD, nolog) Author: Simone Tiraboschi <stirabos@redhat.com> Date: Tue Feb 5 10:56:58 2019 +0100
Avoid using no_log when we have to pass back values to otopi
commit 96974fad1ee6aee33f8183e49240f8a2a7a617d4 Author: Simone Tiraboschi <stirabos@redhat.com> Date: Thu Jan 31 16:39:58 2019 +0100
use dynamic inclusion to avoid tag inheritance
commit 4a9a23fb8e88acba5af4febed43d9e4b02e7a2c5 Author: Simone Tiraboschi <stirabos@redhat.com> Date: Thu Jan 31 15:15:04 2019 +0100
Force facts gathering on partial executions
commit 7428b54a5ba8458379b1a27d116f9504bb830e69 Author: Simone Tiraboschi <stirabos@redhat.com> Date: Wed Jan 30 17:01:01 2019 +0100
Use static imports and tags
Fixes https://github.com/oVirt/ovirt-ansible-hosted-engine-setup/issues/20 Reuires https://github.com/oVirt/ovirt-ansible-engine-setup/pull/39
Version has been bumped to 1.0.10 on commit 7428b54a5ba8458379b1a27d116f9504bb830e69 since it introduces a backward incompatible change and we need to track it.
7428b54a5ba8458379b1a27d116f9504bb830e69 failed CI tests due to an issue on a different package found yesterday.
So 7428b54a5ba8458379b1a27d116f9504bb830e69 got ignored and now CI is building from commit b5a6c1db135d81d75f3330160e7ef4a84c97fd60 (the last one) rebased on something before 7428b54a5ba8458379b1a27d116f9504bb830e69 which is not what we have in git so now, after b5a6c1db135d81d75f3330160e7ef4a84c97fd60 (last commit) the package builds in CI as 1.0.9 although in the code we have 1.0.10 and so the issue.
We never ignore commits, certainly not merged ones...
We can fall back to older builds on system test failures and throw away newer build if we suspect they cause the failure, if which case the builds need to be resubmitted, but this logic happens at the build leve not the commit level, there is no commit reordering or dropping anywhere.
I'd double check the CI code for that:
on git side, in build.sh at commit b5a6c1db135d81d75f3330160e7ef4a84c97fd60 we have VERSION="1.0.10" as you can see in
https://github.com/oVirt/ovirt-ansible-hosted-engine-setup/blob/b5a6c1db135d...
but, as you can see in
https://jenkins.ovirt.org/job/oVirt_ovirt-ansible-hosted-engine-setup_standa... CI builds it as 1.0.9 ignoring commit 7428b54a5ba8458379b1a27d116f9504bb830e69 that has been merged before b5a6c1db135d81d75f3330160e7ef4a84c97fd60
I'm not sure if build 138 ran before or after 7428b54a5ba8458379b1a27d116f9504bb830e69 was merged. since you merged the PR already, we cannot check what will happen if we rerun the job now.
I suspect you're simply looking at the output of a job that stated running before 7428b54a5ba8458379b1a27d116f9504bb830e69 was merged
On master branch on https://github.com/oVirt/ovirt-ansible-hosted-engine-setup I have:
b5a6c1d (HEAD -> master, upstream/master, origin/master, origin/HEAD, nolog) Avoid using no_log when we have to pass back values to otopi 96974fa use dynamic inclusion to avoid tag inheritance 4a9a23f Force facts gathering on partial executions 7428b54 Use static imports and tags 9267118 Fix rhbz1654697 (#108) 5e928a1 build: post ovirt-ansible-hosted-engine-setup-1.0.8 8607d90 (tag: 1.0.8) build: ovirt-ansible-hosted-engine-setup-1.0.8
In the console output of https://jenkins.ovirt.org/job/oVirt_ovirt-ansible-hosted-engine-setup_standa...
I see
*10:59:42* 2019-02-05 09:59:42,448:INFO:__main__:File 'automation/upstream_sources.yaml' cannot be opened*10:59:42* 2019-02-05 09:59:42,449:INFO:__main__:Executing command: 'git --git-dir=/home/jenkins/workspace/oVirt_ovirt-ansible-hosted-engine-setup_standard-check-pr/ovirt-ansible-hosted-engine-setup/.git --work-tree=/home/jenkins/workspace/oVirt_ovirt-ansible-hosted-engine-setup_standard-check-pr/ovirt-ansible-hosted-engine-setup reset --hard'*10:59:42* 2019-02-05 09:59:42,472:DEBUG:__main__:Git exited with status: 0*10:59:42* 2019-02-05 09:59:42,472:DEBUG:__main__: ---- stderr ----*10:59:42* 2019-02-05 09:59:42,472:DEBUG:__main__: ---- stdout ----*10:59:42* 2019-02-05 09:59:42,472:DEBUG:__main__: HEAD is now at af00824 Merge ceca0a852d94bd5af7226b15d15c9e3ff917cd5c into 9267118f456de46b9059e76b63bc85c18fcab5dd
Now, ceca0a852d94bd5af7226b15d15c9e3ff917cd5c is the last commit on my develop branch on https://github.com/tiraboschi/ovirt-ansible-hosted-engine-setup
commit ceca0a852d94bd5af7226b15d15c9e3ff917cd5c (origin/nolog) Author: Simone Tiraboschi <stirabos@redhat.com> Date: Tue Feb 5 10:56:58 2019 +0100
Avoid using no_log when we have to pass back values to otopi
which corresponds to b5a6c1d on master on https://github.com/oVirt/ovirt-ansible-hosted-engine-setup
so "Avoid using no_log when we have to pass back values to otopi" got tested over 9267118 - Fix rhbz1654697 (#108) skipping 96974fa use dynamic inclusion to avoid tag inheritance 4a9a23f Force facts gathering on partial executions 7428b54 Use static imports and tags
in the middle.
The version bump change is in 7428b54 and so it builds as 1.0.9 and not as 1.0.10 as expected.
This is a matter of timing - as I already said - I suspect the build you're looking at happened before b5a6c1d was merged. In any case the build you're seeing ot the `check-pr` job has nothing to do with what gets sent into system testing and checked with OST. The build that gets checked is the one that is generated by the `on-ghpush` job and is always built from merged commits in the branch itself. You can see the last successful build is with the right version: https://jenkins.ovirt.org/job/oVirt_ovirt-ansible-hosted-engine-setup_standa...
-- Barak Korren RHV DevOps team , RHCE, RHCi Red Hat EMEA redhat.com | TRIED. TESTED. TRUSTED. | redhat.com/trusted
-- Barak Korren RHV DevOps team , RHCE, RHCi Red Hat EMEA redhat.com | TRIED. TESTED. TRUSTED. | redhat.com/trusted
-- Barak Korren RHV DevOps team , RHCE, RHCi Red Hat EMEA redhat.com | TRIED. TESTED. TRUSTED. | redhat.com/trusted
On Wed, Feb 6, 2019 at 11:17 AM Barak Korren <bkorren@redhat.com> wrote:
On Wed, 6 Feb 2019 at 11:57, Simone Tiraboschi <stirabos@redhat.com> wrote:
On Wed, Feb 6, 2019 at 10:44 AM Barak Korren <bkorren@redhat.com> wrote:
On Wed, 6 Feb 2019 at 11:34, Simone Tiraboschi <stirabos@redhat.com> wrote:
On Wed, Feb 6, 2019 at 10:23 AM Barak Korren <bkorren@redhat.com> wrote:
On Wed, 6 Feb 2019 at 11:15, Simone Tiraboschi <stirabos@redhat.com> wrote:
On Wed, Feb 6, 2019 at 10:00 AM Dan Kenigsberg <danken@redhat.com> wrote:
> On Wed, Feb 6, 2019 at 10:54 AM Simone Tiraboschi < > stirabos@redhat.com> wrote: > > > > > > > > On Wed, Feb 6, 2019 at 9:45 AM Dan Kenigsberg <danken@redhat.com> > wrote: > >> > >> On Wed, Feb 6, 2019 at 10:16 AM Simone Tiraboschi < > stirabos@redhat.com> wrote: > >> > > >> > > >> > > >> > On Tue, Feb 5, 2019 at 7:07 PM Dafna Ron <dron@redhat.com> > wrote: > >> >> > >> >> Hi, > >> >> > >> >> Please note that ovirt-ansible-hosted-engine-setup has a > versioning problem with the package and is causing bootstrap to fail for > upgrade suite [1] > >> >> > >> >> This is effecting all projects, its been reported to the > developers and should be fixed as soon as possible. > >> >> > >> >> you can view CQ status here: > >> >> > https://jenkins.ovirt.org/view/Change%20queue%20jobs/job/ovirt-master_change... > >> >> > >> >> [1] http://pastebin.test.redhat.com/708086 > >> > >> It is unfair to refer to an internal pastebin here. It is also not > >> very sensible, as it is quite short. > >> > >> 2019-02-05 11:23:51,390-0500 ERROR > >> otopi.plugins.otopi.packagers.yumpackager yumpackager.error:85 Yum > >> > [u'ovirt-hosted-engine-setup-2.3.5-0.0.master.20190205110929.gitfdbc215.el7.noarch > >> requires ovirt-ansible-hosted-engine-setup >= 1.0.10'] > >> 2019-02-05 11:23:51,390-0500 DEBUG otopi.context > >> context._executeMethod:142 method exception > >> Traceback (most recent call last): > >> File "/tmp/ovirt-6fV8LBWX5i/pythonlib/otopi/context.py", line > 132, > >> in _executeMethod > >> method['method']() > >> File > "/tmp/ovirt-6fV8LBWX5i/otopi-plugins/otopi/packagers/yumpackager.py", > >> line 248, in _packages > >> self.processTransaction() > >> File > "/tmp/ovirt-6fV8LBWX5i/otopi-plugins/otopi/packagers/yumpackager.py", > >> line 262, in processTransaction > >> if self._miniyum.buildTransaction(): > >> File "/tmp/ovirt-6fV8LBWX5i/pythonlib/otopi/miniyum.py", line > 920, > >> in buildTransaction > >> raise yum.Errors.YumBaseError(msg) > >> YumBaseError: > [u'ovirt-hosted-engine-setup-2.3.5-0.0.master.20190205110929.gitfdbc215.el7.noarch > >> requires ovirt-ansible-hosted-engine-setup >= 1.0.10'] > >> 2019-02-05 11:23:51,391-0500 ERROR otopi.context > >> context._executeMethod:151 Failed to execute stage 'Package > >> installation': > [u'ovirt-hosted-engine-setup-2.3.5-0.0.master.20190205110929.gitfdbc215.el7.noarch > >> requires ovirt-ansible-hosted-engine-setup >= 1.0.10'] > >> 2019-02-05 11:23:51,413-0500 DEBUG > >> otopi.plugins.otopi.debug.debug_failure.debug_failure > >> debug_failure._notification:100 tcp connections: > >> > >> >> > >> > > >> > The issue is that on github we already have > >> > VERSION="1.0.10" > >> > as we can see in > >> > > https://github.com/oVirt/ovirt-ansible-hosted-engine-setup/blob/master/build... > >> > > >> > And this has been bumped before the commit that now is reported > as broken. > >> > > >> > CI instead is still building the package as 1.0.9 ignoring the > commit that bumped the version. > >> > Honestly I don't know how I can fix it if the version value is > already the desired one in the source code. > >> > >> I don't see your ovirt-ansible-hosted-engine-setup-1.0.10, only > >> > https://plain.resources.ovirt.org/pub/ovirt-master-snapshot/rpm/el7/noarch/o... > >> Not even under "tested": > >> > https://plain.resources.ovirt.org/repos/ovirt/tested/master/rpm/el7/noarch/o... > >> > >> Simone, can you doublecheck that its artifacts have been built and > >> have been accepted by the change queue? > > > > > > It has been built here once as 1.0.10: > > > https://jenkins.ovirt.org/job/oVirt_ovirt-ansible-hosted-engine-setup_standa... > > > > then on the next commit, CI started building it again as 1.0.9 > although in the source code we have 1.0.10 and so this issue. > > I don't understand the issue yet (that's not surprising as I do not > know what is that "ghpush" job). Which CI job has built the wrong > version? can you share its logs? who owns it? >
In the git log I see: commit b5a6c1db135d81d75f3330160e7ef4a84c97fd60 (HEAD -> master, upstream/master, origin/master, origin/HEAD, nolog) Author: Simone Tiraboschi <stirabos@redhat.com> Date: Tue Feb 5 10:56:58 2019 +0100
Avoid using no_log when we have to pass back values to otopi
commit 96974fad1ee6aee33f8183e49240f8a2a7a617d4 Author: Simone Tiraboschi <stirabos@redhat.com> Date: Thu Jan 31 16:39:58 2019 +0100
use dynamic inclusion to avoid tag inheritance
commit 4a9a23fb8e88acba5af4febed43d9e4b02e7a2c5 Author: Simone Tiraboschi <stirabos@redhat.com> Date: Thu Jan 31 15:15:04 2019 +0100
Force facts gathering on partial executions
commit 7428b54a5ba8458379b1a27d116f9504bb830e69 Author: Simone Tiraboschi <stirabos@redhat.com> Date: Wed Jan 30 17:01:01 2019 +0100
Use static imports and tags
Fixes https://github.com/oVirt/ovirt-ansible-hosted-engine-setup/issues/20 Reuires https://github.com/oVirt/ovirt-ansible-engine-setup/pull/39
Version has been bumped to 1.0.10 on commit 7428b54a5ba8458379b1a27d116f9504bb830e69 since it introduces a backward incompatible change and we need to track it.
7428b54a5ba8458379b1a27d116f9504bb830e69 failed CI tests due to an issue on a different package found yesterday.
So 7428b54a5ba8458379b1a27d116f9504bb830e69 got ignored and now CI is building from commit b5a6c1db135d81d75f3330160e7ef4a84c97fd60 (the last one) rebased on something before 7428b54a5ba8458379b1a27d116f9504bb830e69 which is not what we have in git so now, after b5a6c1db135d81d75f3330160e7ef4a84c97fd60 (last commit) the package builds in CI as 1.0.9 although in the code we have 1.0.10 and so the issue.
We never ignore commits, certainly not merged ones...
We can fall back to older builds on system test failures and throw away newer build if we suspect they cause the failure, if which case the builds need to be resubmitted, but this logic happens at the build leve not the commit level, there is no commit reordering or dropping anywhere.
I'd double check the CI code for that:
on git side, in build.sh at commit b5a6c1db135d81d75f3330160e7ef4a84c97fd60 we have VERSION="1.0.10" as you can see in
https://github.com/oVirt/ovirt-ansible-hosted-engine-setup/blob/b5a6c1db135d...
but, as you can see in
https://jenkins.ovirt.org/job/oVirt_ovirt-ansible-hosted-engine-setup_standa... CI builds it as 1.0.9 ignoring commit 7428b54a5ba8458379b1a27d116f9504bb830e69 that has been merged before b5a6c1db135d81d75f3330160e7ef4a84c97fd60
I'm not sure if build 138 ran before or after 7428b54a5ba8458379b1a27d116f9504bb830e69 was merged. since you merged the PR already, we cannot check what will happen if we rerun the job now.
I suspect you're simply looking at the output of a job that stated running before 7428b54a5ba8458379b1a27d116f9504bb830e69 was merged
On master branch on https://github.com/oVirt/ovirt-ansible-hosted-engine-setup I have:
b5a6c1d (HEAD -> master, upstream/master, origin/master, origin/HEAD, nolog) Avoid using no_log when we have to pass back values to otopi 96974fa use dynamic inclusion to avoid tag inheritance 4a9a23f Force facts gathering on partial executions 7428b54 Use static imports and tags 9267118 Fix rhbz1654697 (#108) 5e928a1 build: post ovirt-ansible-hosted-engine-setup-1.0.8 8607d90 (tag: 1.0.8) build: ovirt-ansible-hosted-engine-setup-1.0.8
In the console output of https://jenkins.ovirt.org/job/oVirt_ovirt-ansible-hosted-engine-setup_standa...
I see
*10:59:42* 2019-02-05 09:59:42,448:INFO:__main__:File 'automation/upstream_sources.yaml' cannot be opened*10:59:42* 2019-02-05 09:59:42,449:INFO:__main__:Executing command: 'git --git-dir=/home/jenkins/workspace/oVirt_ovirt-ansible-hosted-engine-setup_standard-check-pr/ovirt-ansible-hosted-engine-setup/.git --work-tree=/home/jenkins/workspace/oVirt_ovirt-ansible-hosted-engine-setup_standard-check-pr/ovirt-ansible-hosted-engine-setup reset --hard'*10:59:42* 2019-02-05 09:59:42,472:DEBUG:__main__:Git exited with status: 0*10:59:42* 2019-02-05 09:59:42,472:DEBUG:__main__: ---- stderr ----*10:59:42* 2019-02-05 09:59:42,472:DEBUG:__main__: ---- stdout ----*10:59:42* 2019-02-05 09:59:42,472:DEBUG:__main__: HEAD is now at af00824 Merge ceca0a852d94bd5af7226b15d15c9e3ff917cd5c into 9267118f456de46b9059e76b63bc85c18fcab5dd
Now, ceca0a852d94bd5af7226b15d15c9e3ff917cd5c is the last commit on my develop branch on https://github.com/tiraboschi/ovirt-ansible-hosted-engine-setup
commit ceca0a852d94bd5af7226b15d15c9e3ff917cd5c (origin/nolog) Author: Simone Tiraboschi <stirabos@redhat.com> Date: Tue Feb 5 10:56:58 2019 +0100
Avoid using no_log when we have to pass back values to otopi
which corresponds to b5a6c1d on master on https://github.com/oVirt/ovirt-ansible-hosted-engine-setup
so "Avoid using no_log when we have to pass back values to otopi" got tested over 9267118 - Fix rhbz1654697 (#108) skipping 96974fa use dynamic inclusion to avoid tag inheritance 4a9a23f Force facts gathering on partial executions 7428b54 Use static imports and tags
in the middle.
The version bump change is in 7428b54 and so it builds as 1.0.9 and not as 1.0.10 as expected.
This is a matter of timing - as I already said - I suspect the build you're looking at happened before b5a6c1d was merged.
In any case the build you're seeing ot the `check-pr` job has nothing to do with what gets sent into system testing and checked with OST. The build that gets checked is the one that is generated by the `on-ghpush` job and is always built from merged commits in the branch itself. You can see the last successful build is with the right version:
https://jenkins.ovirt.org/job/oVirt_ovirt-ansible-hosted-engine-setup_standa...
OK, and there we have ovirt-ansible-hosted-engine-setup-1.0.10-0.1.master.20190205155347.el7.noarch.rpm as expected. Now the issue is why the CI job fails due to the lack of ovirt-ansible-hosted-engine-setup >= 1.0.10 Now I merged another patch to bump the version again to 1.0.11 just to re-trigger the whole flow. Honestly I don't know what else I can do.
-- Barak Korren RHV DevOps team , RHCE, RHCi Red Hat EMEA redhat.com | TRIED. TESTED. TRUSTED. | redhat.com/trusted
-- Barak Korren RHV DevOps team , RHCE, RHCi Red Hat EMEA redhat.com | TRIED. TESTED. TRUSTED. | redhat.com/trusted
-- Barak Korren RHV DevOps team , RHCE, RHCi Red Hat EMEA redhat.com | TRIED. TESTED. TRUSTED. | redhat.com/trusted
בתאריך יום ד׳, 6 בפבר׳ 2019, 12:25, מאת Simone Tiraboschi < stirabos@redhat.com>:
On Wed, Feb 6, 2019 at 11:17 AM Barak Korren <bkorren@redhat.com> wrote:
On Wed, 6 Feb 2019 at 11:57, Simone Tiraboschi <stirabos@redhat.com> wrote:
On Wed, Feb 6, 2019 at 10:44 AM Barak Korren <bkorren@redhat.com> wrote:
On Wed, 6 Feb 2019 at 11:34, Simone Tiraboschi <stirabos@redhat.com> wrote:
On Wed, Feb 6, 2019 at 10:23 AM Barak Korren <bkorren@redhat.com> wrote:
On Wed, 6 Feb 2019 at 11:15, Simone Tiraboschi <stirabos@redhat.com> wrote:
> > > On Wed, Feb 6, 2019 at 10:00 AM Dan Kenigsberg <danken@redhat.com> > wrote: > >> On Wed, Feb 6, 2019 at 10:54 AM Simone Tiraboschi < >> stirabos@redhat.com> wrote: >> > >> > >> > >> > On Wed, Feb 6, 2019 at 9:45 AM Dan Kenigsberg <danken@redhat.com> >> wrote: >> >> >> >> On Wed, Feb 6, 2019 at 10:16 AM Simone Tiraboschi < >> stirabos@redhat.com> wrote: >> >> > >> >> > >> >> > >> >> > On Tue, Feb 5, 2019 at 7:07 PM Dafna Ron <dron@redhat.com> >> wrote: >> >> >> >> >> >> Hi, >> >> >> >> >> >> Please note that ovirt-ansible-hosted-engine-setup has a >> versioning problem with the package and is causing bootstrap to fail for >> upgrade suite [1] >> >> >> >> >> >> This is effecting all projects, its been reported to the >> developers and should be fixed as soon as possible. >> >> >> >> >> >> you can view CQ status here: >> >> >> >> https://jenkins.ovirt.org/view/Change%20queue%20jobs/job/ovirt-master_change... >> >> >> >> >> >> [1] http://pastebin.test.redhat.com/708086 >> >> >> >> It is unfair to refer to an internal pastebin here. It is also >> not >> >> very sensible, as it is quite short. >> >> >> >> 2019-02-05 11:23:51,390-0500 ERROR >> >> otopi.plugins.otopi.packagers.yumpackager yumpackager.error:85 >> Yum >> >> >> [u'ovirt-hosted-engine-setup-2.3.5-0.0.master.20190205110929.gitfdbc215.el7.noarch >> >> requires ovirt-ansible-hosted-engine-setup >= 1.0.10'] >> >> 2019-02-05 11:23:51,390-0500 DEBUG otopi.context >> >> context._executeMethod:142 method exception >> >> Traceback (most recent call last): >> >> File "/tmp/ovirt-6fV8LBWX5i/pythonlib/otopi/context.py", line >> 132, >> >> in _executeMethod >> >> method['method']() >> >> File >> "/tmp/ovirt-6fV8LBWX5i/otopi-plugins/otopi/packagers/yumpackager.py", >> >> line 248, in _packages >> >> self.processTransaction() >> >> File >> "/tmp/ovirt-6fV8LBWX5i/otopi-plugins/otopi/packagers/yumpackager.py", >> >> line 262, in processTransaction >> >> if self._miniyum.buildTransaction(): >> >> File "/tmp/ovirt-6fV8LBWX5i/pythonlib/otopi/miniyum.py", line >> 920, >> >> in buildTransaction >> >> raise yum.Errors.YumBaseError(msg) >> >> YumBaseError: >> [u'ovirt-hosted-engine-setup-2.3.5-0.0.master.20190205110929.gitfdbc215.el7.noarch >> >> requires ovirt-ansible-hosted-engine-setup >= 1.0.10'] >> >> 2019-02-05 11:23:51,391-0500 ERROR otopi.context >> >> context._executeMethod:151 Failed to execute stage 'Package >> >> installation': >> [u'ovirt-hosted-engine-setup-2.3.5-0.0.master.20190205110929.gitfdbc215.el7.noarch >> >> requires ovirt-ansible-hosted-engine-setup >= 1.0.10'] >> >> 2019-02-05 11:23:51,413-0500 DEBUG >> >> otopi.plugins.otopi.debug.debug_failure.debug_failure >> >> debug_failure._notification:100 tcp connections: >> >> >> >> >> >> >> > >> >> > The issue is that on github we already have >> >> > VERSION="1.0.10" >> >> > as we can see in >> >> > >> https://github.com/oVirt/ovirt-ansible-hosted-engine-setup/blob/master/build... >> >> > >> >> > And this has been bumped before the commit that now is >> reported as broken. >> >> > >> >> > CI instead is still building the package as 1.0.9 ignoring the >> commit that bumped the version. >> >> > Honestly I don't know how I can fix it if the version value is >> already the desired one in the source code. >> >> >> >> I don't see your ovirt-ansible-hosted-engine-setup-1.0.10, only >> >> >> https://plain.resources.ovirt.org/pub/ovirt-master-snapshot/rpm/el7/noarch/o... >> >> Not even under "tested": >> >> >> https://plain.resources.ovirt.org/repos/ovirt/tested/master/rpm/el7/noarch/o... >> >> >> >> Simone, can you doublecheck that its artifacts have been built >> and >> >> have been accepted by the change queue? >> > >> > >> > It has been built here once as 1.0.10: >> > >> https://jenkins.ovirt.org/job/oVirt_ovirt-ansible-hosted-engine-setup_standa... >> > >> > then on the next commit, CI started building it again as 1.0.9 >> although in the source code we have 1.0.10 and so this issue. >> >> I don't understand the issue yet (that's not surprising as I do not >> know what is that "ghpush" job). Which CI job has built the wrong >> version? can you share its logs? who owns it? >> > > In the git log I see: > commit b5a6c1db135d81d75f3330160e7ef4a84c97fd60 (HEAD -> master, > upstream/master, origin/master, origin/HEAD, nolog) > Author: Simone Tiraboschi <stirabos@redhat.com> > Date: Tue Feb 5 10:56:58 2019 +0100 > > Avoid using no_log when we have to pass back values to otopi > > commit 96974fad1ee6aee33f8183e49240f8a2a7a617d4 > Author: Simone Tiraboschi <stirabos@redhat.com> > Date: Thu Jan 31 16:39:58 2019 +0100 > > use dynamic inclusion to avoid tag inheritance > > commit 4a9a23fb8e88acba5af4febed43d9e4b02e7a2c5 > Author: Simone Tiraboschi <stirabos@redhat.com> > Date: Thu Jan 31 15:15:04 2019 +0100 > > Force facts gathering on partial executions > > commit 7428b54a5ba8458379b1a27d116f9504bb830e69 > Author: Simone Tiraboschi <stirabos@redhat.com> > Date: Wed Jan 30 17:01:01 2019 +0100 > > Use static imports and tags > > Fixes > https://github.com/oVirt/ovirt-ansible-hosted-engine-setup/issues/20 > Reuires > https://github.com/oVirt/ovirt-ansible-engine-setup/pull/39 > > > > Version has been bumped to 1.0.10 on commit > 7428b54a5ba8458379b1a27d116f9504bb830e69 since it introduces a backward > incompatible change and we need to track it. > > 7428b54a5ba8458379b1a27d116f9504bb830e69 failed CI tests due to an > issue on a different package found yesterday. > > So 7428b54a5ba8458379b1a27d116f9504bb830e69 got ignored and now CI > is building from commit b5a6c1db135d81d75f3330160e7ef4a84c97fd60 (the last > one) rebased on something before 7428b54a5ba8458379b1a27d116f9504bb830e69 > which is not what we have in git so now, after > b5a6c1db135d81d75f3330160e7ef4a84c97fd60 (last commit) the package builds > in CI as 1.0.9 although in the code we have 1.0.10 and so the issue. > > We never ignore commits, certainly not merged ones...
We can fall back to older builds on system test failures and throw away newer build if we suspect they cause the failure, if which case the builds need to be resubmitted, but this logic happens at the build leve not the commit level, there is no commit reordering or dropping anywhere.
I'd double check the CI code for that:
on git side, in build.sh at commit b5a6c1db135d81d75f3330160e7ef4a84c97fd60 we have VERSION="1.0.10" as you can see in
https://github.com/oVirt/ovirt-ansible-hosted-engine-setup/blob/b5a6c1db135d...
but, as you can see in
https://jenkins.ovirt.org/job/oVirt_ovirt-ansible-hosted-engine-setup_standa... CI builds it as 1.0.9 ignoring commit 7428b54a5ba8458379b1a27d116f9504bb830e69 that has been merged before b5a6c1db135d81d75f3330160e7ef4a84c97fd60
I'm not sure if build 138 ran before or after 7428b54a5ba8458379b1a27d116f9504bb830e69 was merged. since you merged the PR already, we cannot check what will happen if we rerun the job now.
I suspect you're simply looking at the output of a job that stated running before 7428b54a5ba8458379b1a27d116f9504bb830e69 was merged
On master branch on https://github.com/oVirt/ovirt-ansible-hosted-engine-setup I have:
b5a6c1d (HEAD -> master, upstream/master, origin/master, origin/HEAD, nolog) Avoid using no_log when we have to pass back values to otopi 96974fa use dynamic inclusion to avoid tag inheritance 4a9a23f Force facts gathering on partial executions 7428b54 Use static imports and tags 9267118 Fix rhbz1654697 (#108) 5e928a1 build: post ovirt-ansible-hosted-engine-setup-1.0.8 8607d90 (tag: 1.0.8) build: ovirt-ansible-hosted-engine-setup-1.0.8
In the console output of https://jenkins.ovirt.org/job/oVirt_ovirt-ansible-hosted-engine-setup_standa...
I see
*10:59:42* 2019-02-05 09:59:42,448:INFO:__main__:File 'automation/upstream_sources.yaml' cannot be opened*10:59:42* 2019-02-05 09:59:42,449:INFO:__main__:Executing command: 'git --git-dir=/home/jenkins/workspace/oVirt_ovirt-ansible-hosted-engine-setup_standard-check-pr/ovirt-ansible-hosted-engine-setup/.git --work-tree=/home/jenkins/workspace/oVirt_ovirt-ansible-hosted-engine-setup_standard-check-pr/ovirt-ansible-hosted-engine-setup reset --hard'*10:59:42* 2019-02-05 09:59:42,472:DEBUG:__main__:Git exited with status: 0*10:59:42* 2019-02-05 09:59:42,472:DEBUG:__main__: ---- stderr ----*10:59:42* 2019-02-05 09:59:42,472:DEBUG:__main__: ---- stdout ----*10:59:42* 2019-02-05 09:59:42,472:DEBUG:__main__: HEAD is now at af00824 Merge ceca0a852d94bd5af7226b15d15c9e3ff917cd5c into 9267118f456de46b9059e76b63bc85c18fcab5dd
Now, ceca0a852d94bd5af7226b15d15c9e3ff917cd5c is the last commit on my develop branch on https://github.com/tiraboschi/ovirt-ansible-hosted-engine-setup
commit ceca0a852d94bd5af7226b15d15c9e3ff917cd5c (origin/nolog) Author: Simone Tiraboschi <stirabos@redhat.com> Date: Tue Feb 5 10:56:58 2019 +0100
Avoid using no_log when we have to pass back values to otopi
which corresponds to b5a6c1d on master on https://github.com/oVirt/ovirt-ansible-hosted-engine-setup
so "Avoid using no_log when we have to pass back values to otopi" got tested over 9267118 - Fix rhbz1654697 (#108) skipping 96974fa use dynamic inclusion to avoid tag inheritance 4a9a23f Force facts gathering on partial executions 7428b54 Use static imports and tags
in the middle.
The version bump change is in 7428b54 and so it builds as 1.0.9 and not as 1.0.10 as expected.
This is a matter of timing - as I already said - I suspect the build you're looking at happened before b5a6c1d was merged.
In any case the build you're seeing ot the `check-pr` job has nothing to do with what gets sent into system testing and checked with OST. The build that gets checked is the one that is generated by the `on-ghpush` job and is always built from merged commits in the branch itself. You can see the last successful build is with the right version:
https://jenkins.ovirt.org/job/oVirt_ovirt-ansible-hosted-engine-setup_standa...
OK, and there we have ovirt-ansible-hosted-engine-setup-1.0.10-0.1.master.20190205155347.el7.noarch.rpm as expected. Now the issue is why the CI job fails due to the lack of ovirt-ansible-hosted-engine-setup >= 1.0.10
Now I merged another patch to bump the version again to 1.0.11 just to re-trigger the whole flow. Honestly I don't know what else I can do.
I don't think solving this was up to you. If the 1.0.11 build was removed from CQ because of an infra issue, Dafna or the infra-owner should have simply re-added it. If it's a matter of repo ordering or whitelisting that causes an older version to be made available during is the run, it needs to be fixed in OST code.
-- Barak Korren RHV DevOps team , RHCE, RHCi Red Hat EMEA redhat.com | TRIED. TESTED. TRUSTED. | redhat.com/trusted
-- Barak Korren RHV DevOps team , RHCE, RHCi Red Hat EMEA redhat.com | TRIED. TESTED. TRUSTED. | redhat.com/trusted
-- Barak Korren RHV DevOps team , RHCE, RHCi Red Hat EMEA redhat.com | TRIED. TESTED. TRUSTED. | redhat.com/trusted
On Wed, Feb 6, 2019 at 11:05 AM Barak Korren <bkorren@redhat.com> wrote:
בתאריך יום ד׳, 6 בפבר׳ 2019, 12:25, מאת Simone Tiraboschi < stirabos@redhat.com>:
On Wed, Feb 6, 2019 at 11:17 AM Barak Korren <bkorren@redhat.com> wrote:
On Wed, 6 Feb 2019 at 11:57, Simone Tiraboschi <stirabos@redhat.com> wrote:
On Wed, Feb 6, 2019 at 10:44 AM Barak Korren <bkorren@redhat.com> wrote:
On Wed, 6 Feb 2019 at 11:34, Simone Tiraboschi <stirabos@redhat.com> wrote:
On Wed, Feb 6, 2019 at 10:23 AM Barak Korren <bkorren@redhat.com> wrote:
> > > On Wed, 6 Feb 2019 at 11:15, Simone Tiraboschi <stirabos@redhat.com> > wrote: > >> >> >> On Wed, Feb 6, 2019 at 10:00 AM Dan Kenigsberg <danken@redhat.com> >> wrote: >> >>> On Wed, Feb 6, 2019 at 10:54 AM Simone Tiraboschi < >>> stirabos@redhat.com> wrote: >>> > >>> > >>> > >>> > On Wed, Feb 6, 2019 at 9:45 AM Dan Kenigsberg <danken@redhat.com> >>> wrote: >>> >> >>> >> On Wed, Feb 6, 2019 at 10:16 AM Simone Tiraboschi < >>> stirabos@redhat.com> wrote: >>> >> > >>> >> > >>> >> > >>> >> > On Tue, Feb 5, 2019 at 7:07 PM Dafna Ron <dron@redhat.com> >>> wrote: >>> >> >> >>> >> >> Hi, >>> >> >> >>> >> >> Please note that ovirt-ansible-hosted-engine-setup has a >>> versioning problem with the package and is causing bootstrap to fail for >>> upgrade suite [1] >>> >> >> >>> >> >> This is effecting all projects, its been reported to the >>> developers and should be fixed as soon as possible. >>> >> >> >>> >> >> you can view CQ status here: >>> >> >> >>> https://jenkins.ovirt.org/view/Change%20queue%20jobs/job/ovirt-master_change... >>> >> >> >>> >> >> [1] http://pastebin.test.redhat.com/708086 >>> >> >>> >> It is unfair to refer to an internal pastebin here. It is also >>> not >>> >> very sensible, as it is quite short. >>> >> >>> >> 2019-02-05 11:23:51,390-0500 ERROR >>> >> otopi.plugins.otopi.packagers.yumpackager yumpackager.error:85 >>> Yum >>> >> >>> [u'ovirt-hosted-engine-setup-2.3.5-0.0.master.20190205110929.gitfdbc215.el7.noarch >>> >> requires ovirt-ansible-hosted-engine-setup >= 1.0.10'] >>> >> 2019-02-05 11:23:51,390-0500 DEBUG otopi.context >>> >> context._executeMethod:142 method exception >>> >> Traceback (most recent call last): >>> >> File "/tmp/ovirt-6fV8LBWX5i/pythonlib/otopi/context.py", line >>> 132, >>> >> in _executeMethod >>> >> method['method']() >>> >> File >>> "/tmp/ovirt-6fV8LBWX5i/otopi-plugins/otopi/packagers/yumpackager.py", >>> >> line 248, in _packages >>> >> self.processTransaction() >>> >> File >>> "/tmp/ovirt-6fV8LBWX5i/otopi-plugins/otopi/packagers/yumpackager.py", >>> >> line 262, in processTransaction >>> >> if self._miniyum.buildTransaction(): >>> >> File "/tmp/ovirt-6fV8LBWX5i/pythonlib/otopi/miniyum.py", line >>> 920, >>> >> in buildTransaction >>> >> raise yum.Errors.YumBaseError(msg) >>> >> YumBaseError: >>> [u'ovirt-hosted-engine-setup-2.3.5-0.0.master.20190205110929.gitfdbc215.el7.noarch >>> >> requires ovirt-ansible-hosted-engine-setup >= 1.0.10'] >>> >> 2019-02-05 11:23:51,391-0500 ERROR otopi.context >>> >> context._executeMethod:151 Failed to execute stage 'Package >>> >> installation': >>> [u'ovirt-hosted-engine-setup-2.3.5-0.0.master.20190205110929.gitfdbc215.el7.noarch >>> >> requires ovirt-ansible-hosted-engine-setup >= 1.0.10'] >>> >> 2019-02-05 11:23:51,413-0500 DEBUG >>> >> otopi.plugins.otopi.debug.debug_failure.debug_failure >>> >> debug_failure._notification:100 tcp connections: >>> >> >>> >> >> >>> >> > >>> >> > The issue is that on github we already have >>> >> > VERSION="1.0.10" >>> >> > as we can see in >>> >> > >>> https://github.com/oVirt/ovirt-ansible-hosted-engine-setup/blob/master/build... >>> >> > >>> >> > And this has been bumped before the commit that now is >>> reported as broken. >>> >> > >>> >> > CI instead is still building the package as 1.0.9 ignoring >>> the commit that bumped the version. >>> >> > Honestly I don't know how I can fix it if the version value >>> is already the desired one in the source code. >>> >> >>> >> I don't see your ovirt-ansible-hosted-engine-setup-1.0.10, only >>> >> >>> https://plain.resources.ovirt.org/pub/ovirt-master-snapshot/rpm/el7/noarch/o... >>> >> Not even under "tested": >>> >> >>> https://plain.resources.ovirt.org/repos/ovirt/tested/master/rpm/el7/noarch/o... >>> >> >>> >> Simone, can you doublecheck that its artifacts have been built >>> and >>> >> have been accepted by the change queue? >>> > >>> > >>> > It has been built here once as 1.0.10: >>> > >>> https://jenkins.ovirt.org/job/oVirt_ovirt-ansible-hosted-engine-setup_standa... >>> > >>> > then on the next commit, CI started building it again as 1.0.9 >>> although in the source code we have 1.0.10 and so this issue. >>> >>> I don't understand the issue yet (that's not surprising as I do not >>> know what is that "ghpush" job). Which CI job has built the wrong >>> version? can you share its logs? who owns it? >>> >> >> In the git log I see: >> commit b5a6c1db135d81d75f3330160e7ef4a84c97fd60 (HEAD -> master, >> upstream/master, origin/master, origin/HEAD, nolog) >> Author: Simone Tiraboschi <stirabos@redhat.com> >> Date: Tue Feb 5 10:56:58 2019 +0100 >> >> Avoid using no_log when we have to pass back values to otopi >> >> commit 96974fad1ee6aee33f8183e49240f8a2a7a617d4 >> Author: Simone Tiraboschi <stirabos@redhat.com> >> Date: Thu Jan 31 16:39:58 2019 +0100 >> >> use dynamic inclusion to avoid tag inheritance >> >> commit 4a9a23fb8e88acba5af4febed43d9e4b02e7a2c5 >> Author: Simone Tiraboschi <stirabos@redhat.com> >> Date: Thu Jan 31 15:15:04 2019 +0100 >> >> Force facts gathering on partial executions >> >> commit 7428b54a5ba8458379b1a27d116f9504bb830e69 >> Author: Simone Tiraboschi <stirabos@redhat.com> >> Date: Wed Jan 30 17:01:01 2019 +0100 >> >> Use static imports and tags >> >> Fixes >> https://github.com/oVirt/ovirt-ansible-hosted-engine-setup/issues/20 >> Reuires >> https://github.com/oVirt/ovirt-ansible-engine-setup/pull/39 >> >> >> >> Version has been bumped to 1.0.10 on commit >> 7428b54a5ba8458379b1a27d116f9504bb830e69 since it introduces a backward >> incompatible change and we need to track it. >> >> 7428b54a5ba8458379b1a27d116f9504bb830e69 failed CI tests due to an >> issue on a different package found yesterday. >> >> So 7428b54a5ba8458379b1a27d116f9504bb830e69 got ignored and now CI >> is building from commit b5a6c1db135d81d75f3330160e7ef4a84c97fd60 (the last >> one) rebased on something before 7428b54a5ba8458379b1a27d116f9504bb830e69 >> which is not what we have in git so now, after >> b5a6c1db135d81d75f3330160e7ef4a84c97fd60 (last commit) the package builds >> in CI as 1.0.9 although in the code we have 1.0.10 and so the issue. >> >> > We never ignore commits, certainly not merged ones... > > We can fall back to older builds on system test failures and throw > away newer build if we suspect they cause the failure, if which case the > builds need to be resubmitted, but this logic happens at the build leve not > the commit level, there is no commit reordering or dropping anywhere. > > I'd double check the CI code for that:
on git side, in build.sh at commit b5a6c1db135d81d75f3330160e7ef4a84c97fd60 we have VERSION="1.0.10" as you can see in
https://github.com/oVirt/ovirt-ansible-hosted-engine-setup/blob/b5a6c1db135d...
but, as you can see in
https://jenkins.ovirt.org/job/oVirt_ovirt-ansible-hosted-engine-setup_standa... CI builds it as 1.0.9 ignoring commit 7428b54a5ba8458379b1a27d116f9504bb830e69 that has been merged before b5a6c1db135d81d75f3330160e7ef4a84c97fd60
I'm not sure if build 138 ran before or after 7428b54a5ba8458379b1a27d116f9504bb830e69 was merged. since you merged the PR already, we cannot check what will happen if we rerun the job now.
I suspect you're simply looking at the output of a job that stated running before 7428b54a5ba8458379b1a27d116f9504bb830e69 was merged
On master branch on https://github.com/oVirt/ovirt-ansible-hosted-engine-setup I have:
b5a6c1d (HEAD -> master, upstream/master, origin/master, origin/HEAD, nolog) Avoid using no_log when we have to pass back values to otopi 96974fa use dynamic inclusion to avoid tag inheritance 4a9a23f Force facts gathering on partial executions 7428b54 Use static imports and tags 9267118 Fix rhbz1654697 (#108) 5e928a1 build: post ovirt-ansible-hosted-engine-setup-1.0.8 8607d90 (tag: 1.0.8) build: ovirt-ansible-hosted-engine-setup-1.0.8
In the console output of https://jenkins.ovirt.org/job/oVirt_ovirt-ansible-hosted-engine-setup_standa...
I see
*10:59:42* 2019-02-05 09:59:42,448:INFO:__main__:File 'automation/upstream_sources.yaml' cannot be opened*10:59:42* 2019-02-05 09:59:42,449:INFO:__main__:Executing command: 'git --git-dir=/home/jenkins/workspace/oVirt_ovirt-ansible-hosted-engine-setup_standard-check-pr/ovirt-ansible-hosted-engine-setup/.git --work-tree=/home/jenkins/workspace/oVirt_ovirt-ansible-hosted-engine-setup_standard-check-pr/ovirt-ansible-hosted-engine-setup reset --hard'*10:59:42* 2019-02-05 09:59:42,472:DEBUG:__main__:Git exited with status: 0*10:59:42* 2019-02-05 09:59:42,472:DEBUG:__main__: ---- stderr ----*10:59:42* 2019-02-05 09:59:42,472:DEBUG:__main__: ---- stdout ----*10:59:42* 2019-02-05 09:59:42,472:DEBUG:__main__: HEAD is now at af00824 Merge ceca0a852d94bd5af7226b15d15c9e3ff917cd5c into 9267118f456de46b9059e76b63bc85c18fcab5dd
Now, ceca0a852d94bd5af7226b15d15c9e3ff917cd5c is the last commit on my develop branch on https://github.com/tiraboschi/ovirt-ansible-hosted-engine-setup
commit ceca0a852d94bd5af7226b15d15c9e3ff917cd5c (origin/nolog) Author: Simone Tiraboschi <stirabos@redhat.com> Date: Tue Feb 5 10:56:58 2019 +0100
Avoid using no_log when we have to pass back values to otopi
which corresponds to b5a6c1d on master on https://github.com/oVirt/ovirt-ansible-hosted-engine-setup
so "Avoid using no_log when we have to pass back values to otopi" got tested over 9267118 - Fix rhbz1654697 (#108) skipping 96974fa use dynamic inclusion to avoid tag inheritance 4a9a23f Force facts gathering on partial executions 7428b54 Use static imports and tags
in the middle.
The version bump change is in 7428b54 and so it builds as 1.0.9 and not as 1.0.10 as expected.
This is a matter of timing - as I already said - I suspect the build you're looking at happened before b5a6c1d was merged.
In any case the build you're seeing ot the `check-pr` job has nothing to do with what gets sent into system testing and checked with OST. The build that gets checked is the one that is generated by the `on-ghpush` job and is always built from merged commits in the branch itself. You can see the last successful build is with the right version:
https://jenkins.ovirt.org/job/oVirt_ovirt-ansible-hosted-engine-setup_standa...
OK, and there we have ovirt-ansible-hosted-engine-setup-1.0.10-0.1.master.20190205155347.el7.noarch.rpm as expected. Now the issue is why the CI job fails due to the lack of ovirt-ansible-hosted-engine-setup >= 1.0.10
Now I merged another patch to bump the version again to 1.0.11 just to re-trigger the whole flow. Honestly I don't know what else I can do.
I don't think solving this was up to you. If the 1.0.11 build was removed from CQ because of an infra issue, Dafna or the infra-owner should have simply re-added it.
I did - twice
If it's a matter of repo ordering or whitelisting that causes an older version to be made available during is the run, it needs to be fixed in OST code.
It was available - that was no the issue. since the issue was in upgrade only it may have been same version with diff hashing on our repos For now we have a different issue which is blocking this one but if upping the version does not help I will delete packages in repo and re-add to CQ.
> -- > Barak Korren > RHV DevOps team , RHCE, RHCi > Red Hat EMEA > redhat.com | TRIED. TESTED. TRUSTED. | redhat.com/trusted >
-- Barak Korren RHV DevOps team , RHCE, RHCi Red Hat EMEA redhat.com | TRIED. TESTED. TRUSTED. | redhat.com/trusted
-- Barak Korren RHV DevOps team , RHCE, RHCi Red Hat EMEA redhat.com | TRIED. TESTED. TRUSTED. | redhat.com/trusted
participants (6)
-
Barak Korren -
Dafna Ron -
Dan Kenigsberg -
Greg Sheremeta -
Nir Soffer -
Simone Tiraboschi