Re: [ovirt-devel] [ OST Failure Report ] [ oVirt Master ] [ upgrade suites failed! ] [ 15/10/17 ]

On Mon, Oct 16, 2017 at 10:21 AM, Yedidyah Bar David <didi(a)redhat.com&gt; wrote: > On Mon, Oct 16, 2017 at 9:28 AM, Daniel Belenky <dbelenky(a)redhat.com&gt; > wrote: > >> can someone address this issue? every patch to *ovirt-engine* that is >> based on top of this patch is failing OST and* won't deploy to the >> tested repo*. >> >> On Sun, Oct 15, 2017 at 9:33 AM, Daniel Belenky <dbelenky(a)redhat.com&gt; >> wrote: >> >>> Hi all, >>> The following tests are failing both of the upgrade suites in OST >>> (upgrade_from_release and upgrade_from_prevrelease). >>> >>> *Link to console:* ovirt-master_change-queue-tester/3146/console >>> <http://jenkins.ovirt.org/job/ovirt-master_change-queue-tester/3146/consol... >>> *Link to test logs:* >>> - upgrade-from-release-suit-master-el7 >>> <http://jenkins.ovirt.org/job/ovirt-master_change-queue-tester/3146/artifa... >>> - upgrade-from-prevrelease-suit-master-el7 >>> <http://jenkins.ovirt.org/job/ovirt-master_change-queue-tester/3146/artifa... >>> *Suspected patch:* https://gerrit.ovirt.org/#/c/82615/5 >>> *Please note that every patch that is based on top of the patch above >>> was not deployed to the tested repo.* >>> >>> *Error snippet from engine setup log:* >>> >> > Please add a direct link next time, if possible. This is it: > > http://jenkins.ovirt.org/job/ovirt-master_change-queue-teste > r/3146/artifact/exported-artifacts/upgrade-from-release- > suit-master-el7/test_logs/upgrade-from-release-suite- > master/post-001_upgrade_engine.py/lago-upgrade-from-release- > suite-master-engine/_var_log/ovirt-engine/setup/ovirt- > engine-setup-20171013222617-73f0df.log > > And a bit above the snippet below, there is: > > 2017-10-13 22:26:24,274-0400 DEBUG otopi.plugins.ovirt_engine_set > up.ovirt_engine.upgrade.asynctasks plugin.execute:926 execute-output: > ('/usr/share/ovirt-engine/setup/dbutils/taskcleaner.sh', '-l', > '/var/log/ovirt-engine/setup/ovirt-engine-setup-20171013222617-73f0df.log', > '-u', 'engine', '-s', 'localhost', '-p', '5432', '-d', 'engine', '-q', > '-r', '-Z') stderr: > > /usr/share/ovirt-engine/bin/generate-pgpass.sh: line 3: /usr/share/ovirt-engine/setup/dbutils/engine-prolog.sh: No such file or directory > > > 2017-10-13 22:26:24,274-0400 DEBUG otopi.context context._executeMethod:143 method exception >>> Traceback (most recent call last): >>> File "/usr/lib/python2.7/site-packages/otopi/context.py", line 133, in _executeMethod >>> method['method']() >>> File "/usr/share/ovirt-engine/setup/bin/../plugins/ovirt-engine-setup/ovirt-engine/upgrade/asynctasks.py", line 470, in _validateZombies >>> self._clearZombies() >>> File "/usr/share/ovirt-engine/setup/bin/../plugins/ovirt-engine-setup/ovirt-engine/upgrade/asynctasks.py", line 135, in _clearZombies >>> 'Failed to clear zombie commands. ' >>> RuntimeError: Failed to clear zombie commands. Please access support in attempt to resolve the problem >>> 2017-10-13 22:26:24,275-0400 ERROR otopi.context context._executeMethod:152 Failed to execute stage 'Setup validation': Failed to clear zombie commands. Please access support in attempt to resolve the problem >>> >>> > With [1], taskcleaner.sh sources generate-pgpass.sh . > > generate-pgpass.sh is in ovirt-engine-tools, which in upgrade flows, is > not > yet upgraded (at the point of above failure). > > generate-pgpass.sh in 4.1 used to source engine-prolog.sh , using a path > relative to "$0". In master it does not, but we now upgrade and it does. > > This, in principle, is the core of the bug: > > A file, such as generate-pgpass.sh, that's supposed to be sourced > from some other files, should not by itself source other files > that are relative to "$0", because it can't know what "$0" is - it's > the path of the script sourcing it, not of itself. > > It seems like luckily we were not affected by this in 4.1, because > all of the files that sourced generate-pgpass.sh were together with > it in the same directory. But with [1], taskcleaner does too now, > and is in a different directory. > > Not sure what's the best solution: > > - revert [1] (and introduce it later on, in 4.3) > > - patch 4.1's generate-pgpass.sh and require the fixed 4.1 version > in 4.2 setup > > - Somehow trick everything to work together? Not sure. Seems like > you can't set $0. > > [1] https://gerrit.ovirt.org/82511 > The same bug exists with unlock_entity: https://gerrit.ovirt.org/82615 So we should probably revert both.

> > > Regards, > > >> -- >>> >>> DANIEL BELENKY >>> >>> RHV DEVOPS >>> >>> EMEA VIRTUALIZATION R&D >>> <https://red.ht/sig> >>> >> >> >> >> -- >> >> DANIEL BELENKY >> >> RHV DEVOPS >> >> EMEA VIRTUALIZATION R&D >> <https://red.ht/sig> >> >> _______________________________________________ >> Devel mailing list >> Devel(a)ovirt.org >> http://lists.ovirt.org/mailman/listinfo/devel >> > > > > -- > Didi > -- Didi _______________________________________________ Devel mailing list Devel(a)ovirt.org http://lists.ovirt.org/mailman/listinfo/devel