[ovirt-devel] [ OST Failure Report ] [ oVirt Master ] [ upgrade suites failed! ] [ 15/10/17 ]

Martin Perina mperina at redhat.com
Mon Oct 16 07:44:51 UTC 2017 

On Mon, Oct 16, 2017 at 9:38 AM, Yedidyah Bar David <didi at redhat.com> wrote:

> On Mon, Oct 16, 2017 at 10:34 AM, Miroslava Voglova <mvoglova at redhat.com>
> wrote:
>
>> Fix on review https://gerrit.ovirt.org/#/c/82799/
>>
>
> That's indeed a related patch, but not sure how it solves current failure.
>

​Let's copy generate-pgpass.sh to packaging/setup/dbutils a​nd source it
from this location for taskcleaner.sh and unlock_entity.sh


>
>>
>> On Mon, Oct 16, 2017 at 9:32 AM, Yaniv Kaul <ykaul at redhat.com> wrote:
>>
>>>
>>>
>>> On Mon, Oct 16, 2017 at 10:24 AM, Yedidyah Bar David <didi at redhat.com>
>>> wrote:
>>>
>>>> On Mon, Oct 16, 2017 at 10:21 AM, Yedidyah Bar David <didi at redhat.com>
>>>> wrote:
>>>>
>>>>> On Mon, Oct 16, 2017 at 9:28 AM, Daniel Belenky <dbelenky at redhat.com>
>>>>> wrote:
>>>>>
>>>>>> can someone address this issue? every patch to *ovirt-engine* that
>>>>>> is based on top of this patch is failing OST and* won't deploy to
>>>>>> the tested repo*.
>>>>>>
>>>>>> On Sun, Oct 15, 2017 at 9:33 AM, Daniel Belenky <dbelenky at redhat.com>
>>>>>> wrote:
>>>>>>
>>>>>>> Hi all,
>>>>>>> The following tests are failing both of the upgrade suites in OST
>>>>>>> (upgrade_from_release and upgrade_from_prevrelease).
>>>>>>>
>>>>>>> *Link to console:* ovirt-master_change-queue-tester/3146/console
>>>>>>> <http://jenkins.ovirt.org/job/ovirt-master_change-queue-tester/3146/console>
>>>>>>> *Link to test logs:*
>>>>>>>     - upgrade-from-release-suit-master-el7
>>>>>>> <http://jenkins.ovirt.org/job/ovirt-master_change-queue-tester/3146/artifact/exported-artifacts/upgrade-from-release-suit-master-el7>
>>>>>>>     - upgrade-from-prevrelease-suit-master-el7
>>>>>>> <http://jenkins.ovirt.org/job/ovirt-master_change-queue-tester/3146/artifact/exported-artifacts/upgrade-from-prevrelease-suit-master-el7>
>>>>>>> *Suspected patch:* https://gerrit.ovirt.org/#/c/82615/5
>>>>>>> *Please note that every patch that is based on top of the patch
>>>>>>> above was not deployed to the tested repo.*
>>>>>>>
>>>>>>> *Error snippet from engine setup log:*
>>>>>>>
>>>>>>
>>>>> Please add a direct link next time, if possible. This is it:
>>>>>
>>>>> http://jenkins.ovirt.org/job/ovirt-master_change-queue-teste
>>>>> r/3146/artifact/exported-artifacts/upgrade-from-release-suit
>>>>> -master-el7/test_logs/upgrade-from-release-suite-master/post
>>>>> -001_upgrade_engine.py/lago-upgrade-from-release-suite-maste
>>>>> r-engine/_var_log/ovirt-engine/setup/ovirt-engine-setup-
>>>>> 20171013222617-73f0df.log
>>>>>
>>>>> And a bit above the snippet below, there is:
>>>>>
>>>>> 2017-10-13 22:26:24,274-0400 DEBUG otopi.plugins.ovirt_engine_set
>>>>> up.ovirt_engine.upgrade.asynctasks plugin.execute:926 execute-output:
>>>>> ('/usr/share/ovirt-engine/setup/dbutils/taskcleaner.sh', '-l',
>>>>> '/var/log/ovirt-engine/setup/ovirt-engine-setup-20171013222617-73f0df.log',
>>>>> '-u', 'engine', '-s', 'localhost', '-p', '5432', '-d', 'engine', '-q',
>>>>> '-r', '-Z') stderr:
>>>>>
>>>>> /usr/share/ovirt-engine/bin/generate-pgpass.sh: line 3: /usr/share/ovirt-engine/setup/dbutils/engine-prolog.sh: No such file or directory
>>>>>
>>>>>
>>>>> 2017-10-13 22:26:24,274-0400 DEBUG otopi.context context._executeMethod:143 method exception
>>>>>>> Traceback (most recent call last):
>>>>>>>   File "/usr/lib/python2.7/site-packages/otopi/context.py", line 133, in _executeMethod
>>>>>>>     method['method']()
>>>>>>>   File "/usr/share/ovirt-engine/setup/bin/../plugins/ovirt-engine-setup/ovirt-engine/upgrade/asynctasks.py", line 470, in _validateZombies
>>>>>>>     self._clearZombies()
>>>>>>>   File "/usr/share/ovirt-engine/setup/bin/../plugins/ovirt-engine-setup/ovirt-engine/upgrade/asynctasks.py", line 135, in _clearZombies
>>>>>>>     'Failed to clear zombie commands. '
>>>>>>> RuntimeError: Failed to clear zombie commands. Please access support in attempt to resolve the problem
>>>>>>> 2017-10-13 22:26:24,275-0400 ERROR otopi.context context._executeMethod:152 Failed to execute stage 'Setup validation': Failed to clear zombie commands. Please access support in attempt to resolve the problem
>>>>>>>
>>>>>>>
>>>>> With [1], taskcleaner.sh sources generate-pgpass.sh .
>>>>>
>>>>> generate-pgpass.sh is in ovirt-engine-tools, which in upgrade flows,
>>>>> is not
>>>>> yet upgraded (at the point of above failure).
>>>>>
>>>>> generate-pgpass.sh in 4.1 used to source engine-prolog.sh , using a
>>>>> path
>>>>> relative to "$0". In master it does not, but we now upgrade and it
>>>>> does.
>>>>>
>>>>> This, in principle, is the core of the bug:
>>>>>
>>>>> A file, such as generate-pgpass.sh, that's supposed to be sourced
>>>>> from some other files, should not by itself source other files
>>>>> that are relative to "$0", because it can't know what "$0" is - it's
>>>>> the path of the script sourcing it, not of itself.
>>>>>
>>>>> It seems like luckily we were not affected by this in 4.1, because
>>>>> all of the files that sourced generate-pgpass.sh were together with
>>>>> it in the same directory. But with [1], taskcleaner does too now,
>>>>> and is in a different directory.
>>>>>
>>>>> Not sure what's the best solution:
>>>>>
>>>>> - revert [1] (and introduce it later on, in 4.3)
>>>>>
>>>>> - patch 4.1's generate-pgpass.sh and require the fixed 4.1 version
>>>>> in 4.2 setup
>>>>>
>>>>
> E.g.:
>
> https://gerrit.ovirt.org/82800
>
>
>>
>>>>> - Somehow trick everything to work together? Not sure. Seems like
>>>>> you can't set $0.
>>>>>
>>>>> [1] https://gerrit.ovirt.org/82511
>>>>>
>>>>
>>>> The same bug exists with unlock_entity:
>>>>
>>>> https://gerrit.ovirt.org/82615
>>>>
>>>> So we should probably revert both.
>>>>
>>>
>>> I tend to agree.
>>> Doesn't mean we cannot fix this for 4.2, but let's revert for the
>>> meantime.
>>> Y.
>>>
>>>
>>>>
>>>>
>>>>>
>>>>>
>>>>> Regards,
>>>>>
>>>>>
>>>>>> --
>>>>>>>
>>>>>>> DANIEL BELENKY
>>>>>>>
>>>>>>> RHV DEVOPS
>>>>>>>
>>>>>>> EMEA VIRTUALIZATION R&D
>>>>>>> <https://red.ht/sig>
>>>>>>>
>>>>>>
>>>>>>
>>>>>>
>>>>>> --
>>>>>>
>>>>>> DANIEL BELENKY
>>>>>>
>>>>>> RHV DEVOPS
>>>>>>
>>>>>> EMEA VIRTUALIZATION R&D
>>>>>> <https://red.ht/sig>
>>>>>>
>>>>>> _______________________________________________
>>>>>> Devel mailing list
>>>>>> Devel at ovirt.org
>>>>>> http://lists.ovirt.org/mailman/listinfo/devel
>>>>>>
>>>>>
>>>>>
>>>>>
>>>>> --
>>>>> Didi
>>>>>
>>>>
>>>>
>>>>
>>>> --
>>>> Didi
>>>>
>>>> _______________________________________________
>>>> Devel mailing list
>>>> Devel at ovirt.org
>>>> http://lists.ovirt.org/mailman/listinfo/devel
>>>>
>>>
>>>
>>> _______________________________________________
>>> Devel mailing list
>>> Devel at ovirt.org
>>> http://lists.ovirt.org/mailman/listinfo/devel
>>>
>>
>>
>
>
> --
> Didi
>
> _______________________________________________
> Devel mailing list
> Devel at ovirt.org
> http://lists.ovirt.org/mailman/listinfo/devel
>
-------------- next part --------------
An HTML attachment was scrubbed...
URL: <http://lists.ovirt.org/pipermail/devel/attachments/20171016/66e8820a/attachment.html>

More information about the Devel mailing list