Hi all, The following tests are failing both of the upgrade suites in OST (upgrade_from_release and upgrade_from_prevrelease). *Link to console:* ovirt-master_change-queue-tester/3146/console <http://jenkins.ovirt.org/job/ovirt-master_change-queue-tester/3146/consol... *Link to test logs:* - upgrade-from-release-suit-master-el7 <http://jenkins.ovirt.org/job/ovirt-master_change-queue-tester/3146/artifa... - upgrade-from-prevrelease-suit-master-el7 <http://jenkins.ovirt.org/job/ovirt-master_change-queue-tester/3146/artifa... *Suspected patch:* https://gerrit.ovirt.org/#/c/82615/5 *Please note that every patch that is based on top of the patch above was not deployed to the tested repo.* *Error snippet from engine setup log:* 2017-10-13 22:26:24,274-0400 DEBUG otopi.context context._executeMethod:143 method exception Traceback (most recent call last): File "/usr/lib/python2.7/site-packages/otopi/context.py", line 133, in _executeMethod method['method']() File "/usr/share/ovirt-engine/setup/bin/../plugins/ovirt-engine-setup/ovirt-engine/upgrade/asynctasks.py", line 470, in _validateZombies self._clearZombies() File "/usr/share/ovirt-engine/setup/bin/../plugins/ovirt-engine-setup/ovirt-engine/upgrade/asynctasks.py", line 135, in _clearZombies 'Failed to clear zombie commands. ' RuntimeError: Failed to clear zombie commands. Please access support in attempt to resolve the problem 2017-10-13 22:26:24,275-0400 ERROR otopi.context context._executeMethod:152 Failed to execute stage 'Setup validation': Failed to clear zombie commands. Please access support in attempt to resolve the problem -- DANIEL BELENKY RHV DEVOPS EMEA VIRTUALIZATION R&D <https://red.ht/sig>

On Mon, Oct 16, 2017 at 9:28 AM, Daniel Belenky <dbelenky(a)redhat.com&gt; wrote: > can someone address this issue? every patch to *ovirt-engine* that is > based on top of this patch is failing OST and* won't deploy to the > tested repo*. > > On Sun, Oct 15, 2017 at 9:33 AM, Daniel Belenky <dbelenky(a)redhat.com&gt; > wrote: > >> Hi all, >> The following tests are failing both of the upgrade suites in OST >> (upgrade_from_release and upgrade_from_prevrelease). >> >> *Link to console:* ovirt-master_change-queue-tester/3146/console >> <http://jenkins.ovirt.org/job/ovirt-master_change-queue-tester/3146/consol... >> *Link to test logs:* >> - upgrade-from-release-suit-master-el7 >> <http://jenkins.ovirt.org/job/ovirt-master_change-queue-tester/3146/artifa... >> - upgrade-from-prevrelease-suit-master-el7 >> <http://jenkins.ovirt.org/job/ovirt-master_change-queue-tester/3146/artifa... >> *Suspected patch:* https://gerrit.ovirt.org/#/c/82615/5 >> *Please note that every patch that is based on top of the patch above >> was not deployed to the tested repo.* >> >> *Error snippet from engine setup log:* >> > Please add a direct link next time, if possible. This is it: http://jenkins.ovirt.org/job/ovirt-master_change-queue- tester/3146/artifact/exported-artifacts/upgrade-from- release-suit-master-el7/test_logs/upgrade-from-release- suite-master/post-001_upgrade_engine.py/lago-upgrade-from- release-suite-master-engine/_var_log/ovirt-engine/setup/ ovirt-engine-setup-20171013222617-73f0df.log And a bit above the snippet below, there is: 2017-10-13 22:26:24,274-0400 DEBUG otopi.plugins.ovirt_engine_ setup.ovirt_engine.upgrade.asynctasks plugin.execute:926 execute-output: ('/usr/share/ovirt-engine/setup/dbutils/taskcleaner.sh', '-l', '/var/log/ovirt-engine/setup/ovirt-engine-setup-20171013222617-73f0df.log', '-u', 'engine', '-s', 'localhost', '-p', '5432', '-d', 'engine', '-q', '-r', '-Z') stderr: /usr/share/ovirt-engine/bin/generate-pgpass.sh: line 3: /usr/share/ovirt-engine/setup/dbutils/engine-prolog.sh: No such file or directory 2017-10-13 22:26:24,274-0400 DEBUG otopi.context context._executeMethod:143 method exception >> Traceback (most recent call last): >> File "/usr/lib/python2.7/site-packages/otopi/context.py", line 133, in _executeMethod >> method['method']() >> File "/usr/share/ovirt-engine/setup/bin/../plugins/ovirt-engine-setup/ovirt-engine/upgrade/asynctasks.py", line 470, in _validateZombies >> self._clearZombies() >> File "/usr/share/ovirt-engine/setup/bin/../plugins/ovirt-engine-setup/ovirt-engine/upgrade/asynctasks.py", line 135, in _clearZombies >> 'Failed to clear zombie commands. ' >> RuntimeError: Failed to clear zombie commands. Please access support in attempt to resolve the problem >> 2017-10-13 22:26:24,275-0400 ERROR otopi.context context._executeMethod:152 Failed to execute stage 'Setup validation': Failed to clear zombie commands. Please access support in attempt to resolve the problem >> >> With [1], taskcleaner.sh sources generate-pgpass.sh . generate-pgpass.sh is in ovirt-engine-tools, which in upgrade flows, is not yet upgraded (at the point of above failure). generate-pgpass.sh in 4.1 used to source engine-prolog.sh , using a path relative to "$0". In master it does not, but we now upgrade and it does. This, in principle, is the core of the bug: A file, such as generate-pgpass.sh, that's supposed to be sourced from some other files, should not by itself source other files that are relative to "$0", because it can't know what "$0" is - it's the path of the script sourcing it, not of itself. It seems like luckily we were not affected by this in 4.1, because all of the files that sourced generate-pgpass.sh were together with it in the same directory. But with [1], taskcleaner does too now, and is in a different directory. Not sure what's the best solution: - revert [1] (and introduce it later on, in 4.3) - patch 4.1's generate-pgpass.sh and require the fixed 4.1 version in 4.2 setup - Somehow trick everything to work together? Not sure. Seems like you can't set $0. [1] https://gerrit.ovirt.org/82511

Regards, > -- >> >> DANIEL BELENKY >> >> RHV DEVOPS >> >> EMEA VIRTUALIZATION R&D >> <https://red.ht/sig> >> > > > > -- > > DANIEL BELENKY > > RHV DEVOPS > > EMEA VIRTUALIZATION R&D > <https://red.ht/sig> > > _______________________________________________ > Devel mailing list > Devel(a)ovirt.org > http://lists.ovirt.org/mailman/listinfo/devel > -- Didi

On Mon, Oct 16, 2017 at 10:24 AM, Yedidyah Bar David <didi(a)redhat.com&gt; wrote: > On Mon, Oct 16, 2017 at 10:21 AM, Yedidyah Bar David <didi(a)redhat.com&gt; > wrote: > >> On Mon, Oct 16, 2017 at 9:28 AM, Daniel Belenky <dbelenky(a)redhat.com&gt; >> wrote: >> >>> can someone address this issue? every patch to *ovirt-engine* that is >>> based on top of this patch is failing OST and* won't deploy to the >>> tested repo*. >>> >>> On Sun, Oct 15, 2017 at 9:33 AM, Daniel Belenky <dbelenky(a)redhat.com&gt; >>> wrote: >>> >>>> Hi all, >>>> The following tests are failing both of the upgrade suites in OST >>>> (upgrade_from_release and upgrade_from_prevrelease). >>>> >>>> *Link to console:* ovirt-master_change-queue-tester/3146/console >>>> <http://jenkins.ovirt.org/job/ovirt-master_change-queue-tester/3146/consol... >>>> *Link to test logs:* >>>> - upgrade-from-release-suit-master-el7 >>>> <http://jenkins.ovirt.org/job/ovirt-master_change-queue-tester/3146/artifa... >>>> - upgrade-from-prevrelease-suit-master-el7 >>>> <http://jenkins.ovirt.org/job/ovirt-master_change-queue-tester/3146/artifa... >>>> *Suspected patch:* https://gerrit.ovirt.org/#/c/82615/5 >>>> *Please note that every patch that is based on top of the patch above >>>> was not deployed to the tested repo.* >>>> >>>> *Error snippet from engine setup log:* >>>> >>> >> Please add a direct link next time, if possible. This is it: >> >> http://jenkins.ovirt.org/job/ovirt-master_change-queue-teste >> r/3146/artifact/exported-artifacts/upgrade-from-release-suit >> -master-el7/test_logs/upgrade-from-release-suite-master/ >> post-001_upgrade_engine.py/lago-upgrade-from-release-suit >> e-master-engine/_var_log/ovirt-engine/setup/ovirt-engine- >> setup-20171013222617-73f0df.log >> >> And a bit above the snippet below, there is: >> >> 2017-10-13 22:26:24,274-0400 DEBUG otopi.plugins.ovirt_engine_set >> up.ovirt_engine.upgrade.asynctasks plugin.execute:926 execute-output: >> ('/usr/share/ovirt-engine/setup/dbutils/taskcleaner.sh', '-l', >> '/var/log/ovirt-engine/setup/ovirt-engine-setup-20171013222617-73f0df.log', >> '-u', 'engine', '-s', 'localhost', '-p', '5432', '-d', 'engine', '-q', >> '-r', '-Z') stderr: >> >> /usr/share/ovirt-engine/bin/generate-pgpass.sh: line 3: /usr/share/ovirt-engine/setup/dbutils/engine-prolog.sh: No such file or directory >> >> >> 2017-10-13 22:26:24,274-0400 DEBUG otopi.context context._executeMethod:143 method exception >>>> Traceback (most recent call last): >>>> File "/usr/lib/python2.7/site-packages/otopi/context.py", line 133, in _executeMethod >>>> method['method']() >>>> File "/usr/share/ovirt-engine/setup/bin/../plugins/ovirt-engine-setup/ovirt-engine/upgrade/asynctasks.py", line 470, in _validateZombies >>>> self._clearZombies() >>>> File "/usr/share/ovirt-engine/setup/bin/../plugins/ovirt-engine-setup/ovirt-engine/upgrade/asynctasks.py", line 135, in _clearZombies >>>> 'Failed to clear zombie commands. ' >>>> RuntimeError: Failed to clear zombie commands. Please access support in attempt to resolve the problem >>>> 2017-10-13 22:26:24,275-0400 ERROR otopi.context context._executeMethod:152 Failed to execute stage 'Setup validation': Failed to clear zombie commands. Please access support in attempt to resolve the problem >>>> >>>> >> With [1], taskcleaner.sh sources generate-pgpass.sh . >> >> generate-pgpass.sh is in ovirt-engine-tools, which in upgrade flows, is >> not >> yet upgraded (at the point of above failure). >> >> generate-pgpass.sh in 4.1 used to source engine-prolog.sh , using a path >> relative to "$0". In master it does not, but we now upgrade and it does. >> >> This, in principle, is the core of the bug: >> >> A file, such as generate-pgpass.sh, that's supposed to be sourced >> from some other files, should not by itself source other files >> that are relative to "$0", because it can't know what "$0" is - it's >> the path of the script sourcing it, not of itself. >> >> It seems like luckily we were not affected by this in 4.1, because >> all of the files that sourced generate-pgpass.sh were together with >> it in the same directory. But with [1], taskcleaner does too now, >> and is in a different directory. >> >> Not sure what's the best solution: >> >> - revert [1] (and introduce it later on, in 4.3) >> >> - patch 4.1's generate-pgpass.sh and require the fixed 4.1 version >> in 4.2 setup >> >> - Somehow trick everything to work together? Not sure. Seems like >> you can't set $0. >> >> [1] https://gerrit.ovirt.org/82511 >> > > The same bug exists with unlock_entity: > > https://gerrit.ovirt.org/82615 > > So we should probably revert both. > I tend to agree. Doesn't mean we cannot fix this for 4.2, but let's revert for the meantime. Y. > > >> >> >> Regards, >> >> >>> -- >>>> >>>> DANIEL BELENKY >>>> >>>> RHV DEVOPS >>>> >>>> EMEA VIRTUALIZATION R&D >>>> <https://red.ht/sig> >>>> >>> >>> >>> >>> -- >>> >>> DANIEL BELENKY >>> >>> RHV DEVOPS >>> >>> EMEA VIRTUALIZATION R&D >>> <https://red.ht/sig> >>> >>> _______________________________________________ >>> Devel mailing list >>> Devel(a)ovirt.org >>> http://lists.ovirt.org/mailman/listinfo/devel >>> >> >> >> >> -- >> Didi >> > > > > -- > Didi > > _______________________________________________ > Devel mailing list > Devel(a)ovirt.org > http://lists.ovirt.org/mailman/listinfo/devel >

Fix on review https://gerrit.ovirt.org/#/c/82799/

On Mon, Oct 16, 2017 at 9:32 AM, Yaniv Kaul <ykaul(a)redhat.com&gt; wrote: > > > On Mon, Oct 16, 2017 at 10:24 AM, Yedidyah Bar David <didi(a)redhat.com&gt; > wrote: > >> On Mon, Oct 16, 2017 at 10:21 AM, Yedidyah Bar David <didi(a)redhat.com&gt; >> wrote: >> >>> On Mon, Oct 16, 2017 at 9:28 AM, Daniel Belenky <dbelenky(a)redhat.com&gt; >>> wrote: >>> >>>> can someone address this issue? every patch to *ovirt-engine* that is >>>> based on top of this patch is failing OST and* won't deploy to the >>>> tested repo*. >>>> >>>> On Sun, Oct 15, 2017 at 9:33 AM, Daniel Belenky <dbelenky(a)redhat.com&gt; >>>> wrote: >>>> >>>>> Hi all, >>>>> The following tests are failing both of the upgrade suites in OST >>>>> (upgrade_from_release and upgrade_from_prevrelease). >>>>> >>>>> *Link to console:* ovirt-master_change-queue-tester/3146/console >>>>> <http://jenkins.ovirt.org/job/ovirt-master_change-queue-tester/3146/consol... >>>>> *Link to test logs:* >>>>> - upgrade-from-release-suit-master-el7 >>>>> <http://jenkins.ovirt.org/job/ovirt-master_change-queue-tester/3146/artifa... >>>>> - upgrade-from-prevrelease-suit-master-el7 >>>>> <http://jenkins.ovirt.org/job/ovirt-master_change-queue-tester/3146/artifa... >>>>> *Suspected patch:* https://gerrit.ovirt.org/#/c/82615/5 >>>>> *Please note that every patch that is based on top of the patch above >>>>> was not deployed to the tested repo.* >>>>> >>>>> *Error snippet from engine setup log:* >>>>> >>>> >>> Please add a direct link next time, if possible. This is it: >>> >>> http://jenkins.ovirt.org/job/ovirt-master_change-queue-teste >>> r/3146/artifact/exported-artifacts/upgrade-from-release-suit >>> -master-el7/test_logs/upgrade-from-release-suite-master/post >>> -001_upgrade_engine.py/lago-upgrade-from-release-suite- >>> master-engine/_var_log/ovirt-engine/setup/ovirt-engine-setu >>> p-20171013222617-73f0df.log >>> >>> And a bit above the snippet below, there is: >>> >>> 2017-10-13 22:26:24,274-0400 DEBUG otopi.plugins.ovirt_engine_set >>> up.ovirt_engine.upgrade.asynctasks plugin.execute:926 execute-output: >>> ('/usr/share/ovirt-engine/setup/dbutils/taskcleaner.sh', '-l', >>> '/var/log/ovirt-engine/setup/ovirt-engine-setup-20171013222617-73f0df.log', >>> '-u', 'engine', '-s', 'localhost', '-p', '5432', '-d', 'engine', '-q', >>> '-r', '-Z') stderr: >>> >>> /usr/share/ovirt-engine/bin/generate-pgpass.sh: line 3: /usr/share/ovirt-engine/setup/dbutils/engine-prolog.sh: No such file or directory >>> >>> >>> 2017-10-13 22:26:24,274-0400 DEBUG otopi.context context._executeMethod:143 method exception >>>>> Traceback (most recent call last): >>>>> File "/usr/lib/python2.7/site-packages/otopi/context.py", line 133, in _executeMethod >>>>> method['method']() >>>>> File "/usr/share/ovirt-engine/setup/bin/../plugins/ovirt-engine-setup/ovirt-engine/upgrade/asynctasks.py", line 470, in _validateZombies >>>>> self._clearZombies() >>>>> File "/usr/share/ovirt-engine/setup/bin/../plugins/ovirt-engine-setup/ovirt-engine/upgrade/asynctasks.py", line 135, in _clearZombies >>>>> 'Failed to clear zombie commands. ' >>>>> RuntimeError: Failed to clear zombie commands. Please access support in attempt to resolve the problem >>>>> 2017-10-13 22:26:24,275-0400 ERROR otopi.context context._executeMethod:152 Failed to execute stage 'Setup validation': Failed to clear zombie commands. Please access support in attempt to resolve the problem >>>>> >>>>> >>> With [1], taskcleaner.sh sources generate-pgpass.sh . >>> >>> generate-pgpass.sh is in ovirt-engine-tools, which in upgrade flows, is >>> not >>> yet upgraded (at the point of above failure). >>> >>> generate-pgpass.sh in 4.1 used to source engine-prolog.sh , using a path >>> relative to "$0". In master it does not, but we now upgrade and it does. >>> >>> This, in principle, is the core of the bug: >>> >>> A file, such as generate-pgpass.sh, that's supposed to be sourced >>> from some other files, should not by itself source other files >>> that are relative to "$0", because it can't know what "$0" is - it's >>> the path of the script sourcing it, not of itself. >>> >>> It seems like luckily we were not affected by this in 4.1, because >>> all of the files that sourced generate-pgpass.sh were together with >>> it in the same directory. But with [1], taskcleaner does too now, >>> and is in a different directory. >>> >>> Not sure what's the best solution: >>> >>> - revert [1] (and introduce it later on, in 4.3) >>> >>> - patch 4.1's generate-pgpass.sh and require the fixed 4.1 version >>> in 4.2 setup >>> >>

>>> - Somehow trick everything to work together? Not sure. Seems like >>> you can't set $0. >>> >>> [1] https://gerrit.ovirt.org/82511 >>> >> >> The same bug exists with unlock_entity: >> >> https://gerrit.ovirt.org/82615 >> >> So we should probably revert both. >> > > I tend to agree. > Doesn't mean we cannot fix this for 4.2, but let's revert for the > meantime. > Y. > > >> >> >>> >>> >>> Regards, >>> >>> >>>> -- >>>>> >>>>> DANIEL BELENKY >>>>> >>>>> RHV DEVOPS >>>>> >>>>> EMEA VIRTUALIZATION R&D >>>>> <https://red.ht/sig> >>>>> >>>> >>>> >>>> >>>> -- >>>> >>>> DANIEL BELENKY >>>> >>>> RHV DEVOPS >>>> >>>> EMEA VIRTUALIZATION R&D >>>> <https://red.ht/sig> >>>> >>>> _______________________________________________ >>>> Devel mailing list >>>> Devel(a)ovirt.org >>>> http://lists.ovirt.org/mailman/listinfo/devel >>>> >>> >>> >>> >>> -- >>> Didi >>> >> >> >> >> -- >> Didi >> >> _______________________________________________ >> Devel mailing list >> Devel(a)ovirt.org >> http://lists.ovirt.org/mailman/listinfo/devel >> > > > _______________________________________________ > Devel mailing list > Devel(a)ovirt.org > http://lists.ovirt.org/mailman/listinfo/devel >

On Mon, Oct 16, 2017 at 9:38 AM, Yedidyah Bar David <didi(a)redhat.com&gt; wrote: > On Mon, Oct 16, 2017 at 10:34 AM, Miroslava Voglova <mvoglova(a)redhat.com&gt; > wrote: > >> Fix on review https://gerrit.ovirt.org/#/c/82799/ >> > > That's indeed a related patch, but not sure how it solves current failure. > ​Let's copy generate-pgpass.sh to packaging/setup/dbutils a​nd source it from this location for taskcleaner.sh and unlock_entity.sh

> >> >> On Mon, Oct 16, 2017 at 9:32 AM, Yaniv Kaul <ykaul(a)redhat.com&gt; wrote: >> >>> >>> >>> On Mon, Oct 16, 2017 at 10:24 AM, Yedidyah Bar David <didi(a)redhat.com&gt; >>> wrote: >>> >>>> On Mon, Oct 16, 2017 at 10:21 AM, Yedidyah Bar David <didi(a)redhat.com&gt; >>>> wrote: >>>> >>>>> On Mon, Oct 16, 2017 at 9:28 AM, Daniel Belenky <dbelenky(a)redhat.com&gt; >>>>> wrote: >>>>> >>>>>> can someone address this issue? every patch to *ovirt-engine* that >>>>>> is based on top of this patch is failing OST and* won't deploy to >>>>>> the tested repo*. >>>>>> >>>>>> On Sun, Oct 15, 2017 at 9:33 AM, Daniel Belenky <dbelenky(a)redhat.com >>>>>> > wrote: >>>>>> >>>>>>> Hi all, >>>>>>> The following tests are failing both of the upgrade suites in OST >>>>>>> (upgrade_from_release and upgrade_from_prevrelease). >>>>>>> >>>>>>> *Link to console:* ovirt-master_change-queue-tester/3146/console >>>>>>> <http://jenkins.ovirt.org/job/ovirt-master_change-queue-tester/3146/consol... >>>>>>> *Link to test logs:* >>>>>>> - upgrade-from-release-suit-master-el7 >>>>>>> <http://jenkins.ovirt.org/job/ovirt-master_change-queue-tester/3146/artifa... >>>>>>> - upgrade-from-prevrelease-suit-master-el7 >>>>>>> <http://jenkins.ovirt.org/job/ovirt-master_change-queue-tester/3146/artifa... >>>>>>> *Suspected patch:* https://gerrit.ovirt.org/#/c/82615/5 >>>>>>> *Please note that every patch that is based on top of the patch >>>>>>> above was not deployed to the tested repo.* >>>>>>> >>>>>>> *Error snippet from engine setup log:* >>>>>>> >>>>>> >>>>> Please add a direct link next time, if possible. This is it: >>>>> >>>>> http://jenkins.ovirt.org/job/ovirt-master_change-queue-teste >>>>> r/3146/artifact/exported-artifacts/upgrade-from-release-suit >>>>> -master-el7/test_logs/upgrade-from-release-suite-master/post >>>>> -001_upgrade_engine.py/lago-upgrade-from-release-suite-maste >>>>> r-engine/_var_log/ovirt-engine/setup/ovirt-engine-setup-2017 >>>>> 1013222617-73f0df.log >>>>> >>>>> And a bit above the snippet below, there is: >>>>> >>>>> 2017-10-13 22:26:24,274-0400 DEBUG otopi.plugins.ovirt_engine_set >>>>> up.ovirt_engine.upgrade.asynctasks plugin.execute:926 >>>>> execute-output: ('/usr/share/ovirt-engine/setup/dbutils/taskcleaner.sh', >>>>> '-l', '/var/log/ovirt-engine/setup/ovirt-engine-setup-20171013222617-73f0df.log', >>>>> '-u', 'engine', '-s', 'localhost', '-p', '5432', '-d', 'engine', '-q', >>>>> '-r', '-Z') stderr: >>>>> >>>>> /usr/share/ovirt-engine/bin/generate-pgpass.sh: line 3: /usr/share/ovirt-engine/setup/dbutils/engine-prolog.sh: No such file or directory >>>>> >>>>> >>>>> 2017-10-13 22:26:24,274-0400 DEBUG otopi.context context._executeMethod:143 method exception >>>>>>> Traceback (most recent call last): >>>>>>> File "/usr/lib/python2.7/site-packages/otopi/context.py", line 133, in _executeMethod >>>>>>> method['method']() >>>>>>> File "/usr/share/ovirt-engine/setup/bin/../plugins/ovirt-engine-setup/ovirt-engine/upgrade/asynctasks.py", line 470, in _validateZombies >>>>>>> self._clearZombies() >>>>>>> File "/usr/share/ovirt-engine/setup/bin/../plugins/ovirt-engine-setup/ovirt-engine/upgrade/asynctasks.py", line 135, in _clearZombies >>>>>>> 'Failed to clear zombie commands. ' >>>>>>> RuntimeError: Failed to clear zombie commands. Please access support in attempt to resolve the problem >>>>>>> 2017-10-13 22:26:24,275-0400 ERROR otopi.context context._executeMethod:152 Failed to execute stage 'Setup validation': Failed to clear zombie commands. Please access support in attempt to resolve the problem >>>>>>> >>>>>>> >>>>> With [1], taskcleaner.sh sources generate-pgpass.sh . >>>>> >>>>> generate-pgpass.sh is in ovirt-engine-tools, which in upgrade flows, >>>>> is not >>>>> yet upgraded (at the point of above failure). >>>>> >>>>> generate-pgpass.sh in 4.1 used to source engine-prolog.sh , using a >>>>> path >>>>> relative to "$0". In master it does not, but we now upgrade and it >>>>> does. >>>>> >>>>> This, in principle, is the core of the bug: >>>>> >>>>> A file, such as generate-pgpass.sh, that's supposed to be sourced >>>>> from some other files, should not by itself source other files >>>>> that are relative to "$0", because it can't know what "$0" is - it's >>>>> the path of the script sourcing it, not of itself. >>>>> >>>>> It seems like luckily we were not affected by this in 4.1, because >>>>> all of the files that sourced generate-pgpass.sh were together with >>>>> it in the same directory. But with [1], taskcleaner does too now, >>>>> and is in a different directory. >>>>> >>>>> Not sure what's the best solution: >>>>> >>>>> - revert [1] (and introduce it later on, in 4.3) >>>>> >>>>> - patch 4.1's generate-pgpass.sh and require the fixed 4.1 version >>>>> in 4.2 setup >>>>> >>>> > E.g.: > > https://gerrit.ovirt.org/82800 > > >> >>>>> - Somehow trick everything to work together? Not sure. Seems like >>>>> you can't set $0. >>>>> >>>>> [1] https://gerrit.ovirt.org/82511 >>>>> >>>> >>>> The same bug exists with unlock_entity: >>>> >>>> https://gerrit.ovirt.org/82615 >>>> >>>> So we should probably revert both. >>>> >>> >>> I tend to agree. >>> Doesn't mean we cannot fix this for 4.2, but let's revert for the >>> meantime. >>> Y. >>> >>> >>>> >>>> >>>>> >>>>> >>>>> Regards, >>>>> >>>>> >>>>>> -- >>>>>>> >>>>>>> DANIEL BELENKY >>>>>>> >>>>>>> RHV DEVOPS >>>>>>> >>>>>>> EMEA VIRTUALIZATION R&D >>>>>>> <https://red.ht/sig> >>>>>>> >>>>>> >>>>>> >>>>>> >>>>>> -- >>>>>> >>>>>> DANIEL BELENKY >>>>>> >>>>>> RHV DEVOPS >>>>>> >>>>>> EMEA VIRTUALIZATION R&D >>>>>> <https://red.ht/sig> >>>>>> >>>>>> _______________________________________________ >>>>>> Devel mailing list >>>>>> Devel(a)ovirt.org >>>>>> http://lists.ovirt.org/mailman/listinfo/devel >>>>>> >>>>> >>>>> >>>>> >>>>> -- >>>>> Didi >>>>> >>>> >>>> >>>> >>>> -- >>>> Didi >>>> >>>> _______________________________________________ >>>> Devel mailing list >>>> Devel(a)ovirt.org >>>> http://lists.ovirt.org/mailman/listinfo/devel >>>> >>> >>> >>> _______________________________________________ >>> Devel mailing list >>> Devel(a)ovirt.org >>> http://lists.ovirt.org/mailman/listinfo/devel >>> >> >> > > > -- > Didi > > _______________________________________________ > Devel mailing list > Devel(a)ovirt.org > http://lists.ovirt.org/mailman/listinfo/devel >

On Mon, Oct 16, 2017 at 11:01 AM, Miroslava Voglova <mvoglova(a)redhat.com&gt; wrote: > On Mon, Oct 16, 2017 at 9:44 AM, Martin Perina <mperina(a)redhat.com&gt; > wrote: > >> >> >> On Mon, Oct 16, 2017 at 9:38 AM, Yedidyah Bar David <didi(a)redhat.com&gt; >> wrote: >> >>> On Mon, Oct 16, 2017 at 10:34 AM, Miroslava Voglova < >>> mvoglova(a)redhat.com&gt; wrote: >>> >>>> Fix on review https://gerrit.ovirt.org/#/c/82799/ >>>> >>> >>> That's indeed a related patch, but not sure how it solves current >>> failure. >>> >> >> ​Let's copy generate-pgpass.sh to packaging/setup/dbutils a​nd source it >> from this location for taskcleaner.sh and unlock_entity.sh >> > > > After offline discussion merging https://gerrit.ovirt.org/82800 and then > https://gerrit.ovirt.org/#/c/82799/ will fix the issue. Both patches are > needed. > So this means: Merge https://gerrit.ovirt.org/82800 . We might want to open a real 4.1 bug for this.

Build 4.1.7 (or 4.1.8?) with it .

Patch 4.2 engine-setup to require tools-4.1.7.

Adding Sandro. > > >> >> >>> >>>> >>>> On Mon, Oct 16, 2017 at 9:32 AM, Yaniv Kaul <ykaul(a)redhat.com&gt; wrote: >>>> >>>>> >>>>> >>>>> On Mon, Oct 16, 2017 at 10:24 AM, Yedidyah Bar David <didi(a)redhat.com >>>>> > wrote: >>>>> >>>>>> On Mon, Oct 16, 2017 at 10:21 AM, Yedidyah Bar David < >>>>>> didi(a)redhat.com&gt; wrote: >>>>>> >>>>>>> On Mon, Oct 16, 2017 at 9:28 AM, Daniel Belenky < >>>>>>> dbelenky(a)redhat.com&gt; wrote: >>>>>>> >>>>>>>> can someone address this issue? every patch to *ovirt-engine* that >>>>>>>> is based on top of this patch is failing OST and* won't deploy to >>>>>>>> the tested repo*. >>>>>>>> >>>>>>>> On Sun, Oct 15, 2017 at 9:33 AM, Daniel Belenky < >>>>>>>> dbelenky(a)redhat.com&gt; wrote: >>>>>>>> >>>>>>>>> Hi all, >>>>>>>>> The following tests are failing both of the upgrade suites in OST >>>>>>>>> (upgrade_from_release and upgrade_from_prevrelease). >>>>>>>>> >>>>>>>>> *Link to console:* ovirt-master_change-queue-tester/3146/console >>>>>>>>> <http://jenkins.ovirt.org/job/ovirt-master_change-queue-tester/3146/consol... >>>>>>>>> *Link to test logs:* >>>>>>>>> - upgrade-from-release-suit-master-el7 >>>>>>>>> <http://jenkins.ovirt.org/job/ovirt-master_change-queue-tester/3146/artifa... >>>>>>>>> - upgrade-from-prevrelease-suit-master-el7 >>>>>>>>> <http://jenkins.ovirt.org/job/ovirt-master_change-queue-tester/3146/artifa... >>>>>>>>> *Suspected patch:* https://gerrit.ovirt.org/#/c/82615/5 >>>>>>>>> *Please note that every patch that is based on top of the patch >>>>>>>>> above was not deployed to the tested repo.* >>>>>>>>> >>>>>>>>> *Error snippet from engine setup log:* >>>>>>>>> >>>>>>>> >>>>>>> Please add a direct link next time, if possible. This is it: >>>>>>> >>>>>>> http://jenkins.ovirt.org/job/ovirt-master_change-queue-teste >>>>>>> r/3146/artifact/exported-artifacts/upgrade-from-release-suit >>>>>>> -master-el7/test_logs/upgrade-from-release-suite-master/post >>>>>>> -001_upgrade_engine.py/lago-upgrade-from-release-suite-maste >>>>>>> r-engine/_var_log/ovirt-engine/setup/ovirt-engine-setup-2017 >>>>>>> 1013222617-73f0df.log >>>>>>> >>>>>>> And a bit above the snippet below, there is: >>>>>>> >>>>>>> 2017-10-13 22:26:24,274-0400 DEBUG otopi.plugins.ovirt_engine_set >>>>>>> up.ovirt_engine.upgrade.asynctasks plugin.execute:926 >>>>>>> execute-output: ('/usr/share/ovirt-engine/setup/dbutils/taskcleaner.sh', >>>>>>> '-l', '/var/log/ovirt-engine/setup/ovirt-engine-setup-20171013222617-73f0df.log', >>>>>>> '-u', 'engine', '-s', 'localhost', '-p', '5432', '-d', 'engine', '-q', >>>>>>> '-r', '-Z') stderr: >>>>>>> >>>>>>> /usr/share/ovirt-engine/bin/generate-pgpass.sh: line 3: /usr/share/ovirt-engine/setup/dbutils/engine-prolog.sh: No such file or directory >>>>>>> >>>>>>> >>>>>>> 2017-10-13 22:26:24,274-0400 DEBUG otopi.context context._executeMethod:143 method exception >>>>>>>>> Traceback (most recent call last): >>>>>>>>> File "/usr/lib/python2.7/site-packages/otopi/context.py", line 133, in _executeMethod >>>>>>>>> method['method']() >>>>>>>>> File "/usr/share/ovirt-engine/setup/bin/../plugins/ovirt-engine-setup/ovirt-engine/upgrade/asynctasks.py", line 470, in _validateZombies >>>>>>>>> self._clearZombies() >>>>>>>>> File "/usr/share/ovirt-engine/setup/bin/../plugins/ovirt-engine-setup/ovirt-engine/upgrade/asynctasks.py", line 135, in _clearZombies >>>>>>>>> 'Failed to clear zombie commands. ' >>>>>>>>> RuntimeError: Failed to clear zombie commands. Please access support in attempt to resolve the problem >>>>>>>>> 2017-10-13 22:26:24,275-0400 ERROR otopi.context context._executeMethod:152 Failed to execute stage 'Setup validation': Failed to clear zombie commands. Please access support in attempt to resolve the problem >>>>>>>>> >>>>>>>>> >>>>>>> With [1], taskcleaner.sh sources generate-pgpass.sh . >>>>>>> >>>>>>> generate-pgpass.sh is in ovirt-engine-tools, which in upgrade >>>>>>> flows, is not >>>>>>> yet upgraded (at the point of above failure). >>>>>>> >>>>>>> generate-pgpass.sh in 4.1 used to source engine-prolog.sh , using a >>>>>>> path >>>>>>> relative to "$0". In master it does not, but we now upgrade and it >>>>>>> does. >>>>>>> >>>>>>> This, in principle, is the core of the bug: >>>>>>> >>>>>>> A file, such as generate-pgpass.sh, that's supposed to be sourced >>>>>>> from some other files, should not by itself source other files >>>>>>> that are relative to "$0", because it can't know what "$0" is - it's >>>>>>> the path of the script sourcing it, not of itself. >>>>>>> >>>>>>> It seems like luckily we were not affected by this in 4.1, because >>>>>>> all of the files that sourced generate-pgpass.sh were together with >>>>>>> it in the same directory. But with [1], taskcleaner does too now, >>>>>>> and is in a different directory. >>>>>>> >>>>>>> Not sure what's the best solution: >>>>>>> >>>>>>> - revert [1] (and introduce it later on, in 4.3) >>>>>>> >>>>>>> - patch 4.1's generate-pgpass.sh and require the fixed 4.1 version >>>>>>> in 4.2 setup >>>>>>> >>>>>> >>> E.g.: >>> >>> https://gerrit.ovirt.org/82800 >>> >>> >>>> >>>>>>> - Somehow trick everything to work together? Not sure. Seems like >>>>>>> you can't set $0. >>>>>>> >>>>>>> [1] https://gerrit.ovirt.org/82511 >>>>>>> >>>>>> >>>>>> The same bug exists with unlock_entity: >>>>>> >>>>>> https://gerrit.ovirt.org/82615 >>>>>> >>>>>> So we should probably revert both. >>>>>> >>>>> >>>>> I tend to agree. >>>>> Doesn't mean we cannot fix this for 4.2, but let's revert for the >>>>> meantime. >>>>> Y. >>>>> >>>>> >>>>>> >>>>>> >>>>>>> >>>>>>> >>>>>>> Regards, >>>>>>> >>>>>>> >>>>>>>> -- >>>>>>>>> >>>>>>>>> DANIEL BELENKY >>>>>>>>> >>>>>>>>> RHV DEVOPS >>>>>>>>> >>>>>>>>> EMEA VIRTUALIZATION R&D >>>>>>>>> <https://red.ht/sig> >>>>>>>>> >>>>>>>> >>>>>>>> >>>>>>>> >>>>>>>> -- >>>>>>>> >>>>>>>> DANIEL BELENKY >>>>>>>> >>>>>>>> RHV DEVOPS >>>>>>>> >>>>>>>> EMEA VIRTUALIZATION R&D >>>>>>>> <https://red.ht/sig> >>>>>>>> >>>>>>>> _______________________________________________ >>>>>>>> Devel mailing list >>>>>>>> Devel(a)ovirt.org >>>>>>>> http://lists.ovirt.org/mailman/listinfo/devel >>>>>>>> >>>>>>> >>>>>>> >>>>>>> >>>>>>> -- >>>>>>> Didi >>>>>>> >>>>>> >>>>>> >>>>>> >>>>>> -- >>>>>> Didi >>>>>> >>>>>> _______________________________________________ >>>>>> Devel mailing list >>>>>> Devel(a)ovirt.org >>>>>> http://lists.ovirt.org/mailman/listinfo/devel >>>>>> >>>>> >>>>> >>>>> _______________________________________________ >>>>> Devel mailing list >>>>> Devel(a)ovirt.org >>>>> http://lists.ovirt.org/mailman/listinfo/devel >>>>> >>>> >>>> >>> >>> >>> -- >>> Didi >>> >>> _______________________________________________ >>> Devel mailing list >>> Devel(a)ovirt.org >>> http://lists.ovirt.org/mailman/listinfo/devel >>> >> >> > -- Didi

Hey, I see that both of the patches that were supposed to fix the issue ( https://gerrit.ovirt.org/82800 and https://gerrit.ovirt.org/#/c/82799/) were merged, but the issue remains.

On Mon, Oct 16, 2017 at 11:45 AM, Sandro Bonazzola <sbonazzo(a)redhat.com&gt; wrote: > > > 2017-10-16 10:30 GMT+02:00 Yedidyah Bar David <didi(a)redhat.com&gt;: > >> On Mon, Oct 16, 2017 at 11:01 AM, Miroslava Voglova <mvoglova(a)redhat.com >> > wrote: >> >>> On Mon, Oct 16, 2017 at 9:44 AM, Martin Perina <mperina(a)redhat.com&gt; >>> wrote: >>> >>>> >>>> >>>> On Mon, Oct 16, 2017 at 9:38 AM, Yedidyah Bar David <didi(a)redhat.com&gt; >>>> wrote: >>>> >>>>> On Mon, Oct 16, 2017 at 10:34 AM, Miroslava Voglova < >>>>> mvoglova(a)redhat.com&gt; wrote: >>>>> >>>>>> Fix on review https://gerrit.ovirt.org/#/c/82799/ >>>>>> >>>>> >>>>> That's indeed a related patch, but not sure how it solves current >>>>> failure. >>>>> >>>> >>>> ​Let's copy generate-pgpass.sh to packaging/setup/dbutils a​nd source >>>> it from this location for taskcleaner.sh and unlock_entity.sh >>>> >>> >>> >>> After offline discussion merging https://gerrit.ovirt.org/82800 and >>> then https://gerrit.ovirt.org/#/c/82799/ will fix the issue. Both >>> patches are needed. >>> >> >> So this means: >> >> Merge https://gerrit.ovirt.org/82800 . We might want to open a >> real 4.1 bug for this. >> > > Agreed, let's open a real 4.1.7 bug to track this and get proper > verification by QE. > > >> >> Build 4.1.7 (or 4.1.8?) with it . >> > > Let's point to 4.1.7 > > >> >> Patch 4.2 engine-setup to require tools-4.1.7. >> > > well, tools >= 4.1.7 :-) > > > > >> >> Adding Sandro. >> >> >>> >>> >>>> >>>> >>>>> >>>>>> >>>>>> On Mon, Oct 16, 2017 at 9:32 AM, Yaniv Kaul <ykaul(a)redhat.com&gt; >>>>>> wrote: >>>>>> >>>>>>> >>>>>>> >>>>>>> On Mon, Oct 16, 2017 at 10:24 AM, Yedidyah Bar David < >>>>>>> didi(a)redhat.com&gt; wrote: >>>>>>> >>>>>>>> On Mon, Oct 16, 2017 at 10:21 AM, Yedidyah Bar David < >>>>>>>> didi(a)redhat.com&gt; wrote: >>>>>>>> >>>>>>>>> On Mon, Oct 16, 2017 at 9:28 AM, Daniel Belenky < >>>>>>>>> dbelenky(a)redhat.com&gt; wrote: >>>>>>>>> >>>>>>>>>> can someone address this issue? every patch to *ovirt-engine* that >>>>>>>>>> is based on top of this patch is failing OST and* won't deploy >>>>>>>>>> to the tested repo*. >>>>>>>>>> >>>>>>>>>> On Sun, Oct 15, 2017 at 9:33 AM, Daniel Belenky < >>>>>>>>>> dbelenky(a)redhat.com&gt; wrote: >>>>>>>>>> >>>>>>>>>>> Hi all, >>>>>>>>>>> The following tests are failing both of the upgrade suites in >>>>>>>>>>> OST (upgrade_from_release and upgrade_from_prevrelease). >>>>>>>>>>> >>>>>>>>>>> *Link to console:* ovirt-master_change-q >>>>>>>>>>> ueue-tester/3146/console >>>>>>>>>>> <http://jenkins.ovirt.org/job/ovirt-master_change-queue-tester/3146/consol... >>>>>>>>>>> *Link to test logs:* >>>>>>>>>>> - upgrade-from-release-suit-master-el7 >>>>>>>>>>> <http://jenkins.ovirt.org/job/ovirt-master_change-queue-tester/3146/artifa... >>>>>>>>>>> - upgrade-from-prevrelease-suit-master-el7 >>>>>>>>>>> <http://jenkins.ovirt.org/job/ovirt-master_change-queue-tester/3146/artifa... >>>>>>>>>>> *Suspected patch:* https://gerrit.ovirt.org/#/c/82615/5 >>>>>>>>>>> *Please note that every patch that is based on top of the patch >>>>>>>>>>> above was not deployed to the tested repo.* >>>>>>>>>>> >>>>>>>>>>> *Error snippet from engine setup log:* >>>>>>>>>>> >>>>>>>>>> >>>>>>>>> Please add a direct link next time, if possible. This is it: >>>>>>>>> >>>>>>>>> http://jenkins.ovirt.org/job/ovirt-master_change-queue-teste >>>>>>>>> r/3146/artifact/exported-artifacts/upgrade-from-release-suit >>>>>>>>> -master-el7/test_logs/upgrade-from-release-suite-master/post >>>>>>>>> -001_upgrade_engine.py/lago-upgrade-from-release-suite-maste >>>>>>>>> r-engine/_var_log/ovirt-engine/setup/ovirt-engine-setup-2017 >>>>>>>>> 1013222617-73f0df.log >>>>>>>>> >>>>>>>>> And a bit above the snippet below, there is: >>>>>>>>> >>>>>>>>> 2017-10-13 22:26:24,274-0400 DEBUG otopi.plugins.ovirt_engine_set >>>>>>>>> up.ovirt_engine.upgrade.asynctasks plugin.execute:926 >>>>>>>>> execute-output: ('/usr/share/ovirt-engine/setup/dbutils/taskcleaner.sh', >>>>>>>>> '-l', '/var/log/ovirt-engine/setup/o >>>>>>>>> virt-engine-setup-20171013222617-73f0df.log', '-u', 'engine', >>>>>>>>> '-s', 'localhost', '-p', '5432', '-d', 'engine', '-q', '-r', '-Z') stderr: >>>>>>>>> >>>>>>>>> /usr/share/ovirt-engine/bin/generate-pgpass.sh: line 3: /usr/share/ovirt-engine/setup/dbutils/engine-prolog.sh: No such file or directory >>>>>>>>> >>>>>>>>> >>>>>>>>> 2017-10-13 22:26:24,274-0400 DEBUG otopi.context context._executeMethod:143 method exception >>>>>>>>>>> Traceback (most recent call last): >>>>>>>>>>> File "/usr/lib/python2.7/site-packages/otopi/context.py", line 133, in _executeMethod >>>>>>>>>>> method['method']() >>>>>>>>>>> File "/usr/share/ovirt-engine/setup/bin/../plugins/ovirt-engine-setup/ovirt-engine/upgrade/asynctasks.py", line 470, in _validateZombies >>>>>>>>>>> self._clearZombies() >>>>>>>>>>> File "/usr/share/ovirt-engine/setup/bin/../plugins/ovirt-engine-setup/ovirt-engine/upgrade/asynctasks.py", line 135, in _clearZombies >>>>>>>>>>> 'Failed to clear zombie commands. ' >>>>>>>>>>> RuntimeError: Failed to clear zombie commands. Please access support in attempt to resolve the problem >>>>>>>>>>> 2017-10-13 22:26:24,275-0400 ERROR otopi.context context._executeMethod:152 Failed to execute stage 'Setup validation': Failed to clear zombie commands. Please access support in attempt to resolve the problem >>>>>>>>>>> >>>>>>>>>>> >>>>>>>>> With [1], taskcleaner.sh sources generate-pgpass.sh . >>>>>>>>> >>>>>>>>> generate-pgpass.sh is in ovirt-engine-tools, which in upgrade >>>>>>>>> flows, is not >>>>>>>>> yet upgraded (at the point of above failure). >>>>>>>>> >>>>>>>>> generate-pgpass.sh in 4.1 used to source engine-prolog.sh , using >>>>>>>>> a path >>>>>>>>> relative to "$0". In master it does not, but we now upgrade and >>>>>>>>> it does. >>>>>>>>> >>>>>>>>> This, in principle, is the core of the bug: >>>>>>>>> >>>>>>>>> A file, such as generate-pgpass.sh, that's supposed to be sourced >>>>>>>>> from some other files, should not by itself source other files >>>>>>>>> that are relative to "$0", because it can't know what "$0" is - >>>>>>>>> it's >>>>>>>>> the path of the script sourcing it, not of itself. >>>>>>>>> >>>>>>>>> It seems like luckily we were not affected by this in 4.1, because >>>>>>>>> all of the files that sourced generate-pgpass.sh were together >>>>>>>>> with >>>>>>>>> it in the same directory. But with [1], taskcleaner does too now, >>>>>>>>> and is in a different directory. >>>>>>>>> >>>>>>>>> Not sure what's the best solution: >>>>>>>>> >>>>>>>>> - revert [1] (and introduce it later on, in 4.3) >>>>>>>>> >>>>>>>>> - patch 4.1's generate-pgpass.sh and require the fixed 4.1 version >>>>>>>>> in 4.2 setup >>>>>>>>> >>>>>>>> >>>>> E.g.: >>>>> >>>>> https://gerrit.ovirt.org/82800 >>>>> >>>>> >>>>>> >>>>>>>>> - Somehow trick everything to work together? Not sure. Seems like >>>>>>>>> you can't set $0. >>>>>>>>> >>>>>>>>> [1] https://gerrit.ovirt.org/82511 >>>>>>>>> >>>>>>>> >>>>>>>> The same bug exists with unlock_entity: >>>>>>>> >>>>>>>> https://gerrit.ovirt.org/82615 >>>>>>>> >>>>>>>> So we should probably revert both. >>>>>>>> >>>>>>> >>>>>>> I tend to agree. >>>>>>> Doesn't mean we cannot fix this for 4.2, but let's revert for the >>>>>>> meantime. >>>>>>> Y. >>>>>>> >>>>>>> >>>>>>>> >>>>>>>> >>>>>>>>> >>>>>>>>> >>>>>>>>> Regards, >>>>>>>>> >>>>>>>>> >>>>>>>>>> -- >>>>>>>>>>> >>>>>>>>>>> DANIEL BELENKY >>>>>>>>>>> >>>>>>>>>>> RHV DEVOPS >>>>>>>>>>> >>>>>>>>>>> EMEA VIRTUALIZATION R&D >>>>>>>>>>> <https://red.ht/sig> >>>>>>>>>>> >>>>>>>>>> >>>>>>>>>> >>>>>>>>>> >>>>>>>>>> -- >>>>>>>>>> >>>>>>>>>> DANIEL BELENKY >>>>>>>>>> >>>>>>>>>> RHV DEVOPS >>>>>>>>>> >>>>>>>>>> EMEA VIRTUALIZATION R&D >>>>>>>>>> <https://red.ht/sig> >>>>>>>>>> >>>>>>>>>> _______________________________________________ >>>>>>>>>> Devel mailing list >>>>>>>>>> Devel(a)ovirt.org >>>>>>>>>> http://lists.ovirt.org/mailman/listinfo/devel >>>>>>>>>> >>>>>>>>> >>>>>>>>> >>>>>>>>> >>>>>>>>> -- >>>>>>>>> Didi >>>>>>>>> >>>>>>>> >>>>>>>> >>>>>>>> >>>>>>>> -- >>>>>>>> Didi >>>>>>>> >>>>>>>> _______________________________________________ >>>>>>>> Devel mailing list >>>>>>>> Devel(a)ovirt.org >>>>>>>> http://lists.ovirt.org/mailman/listinfo/devel >>>>>>>> >>>>>>> >>>>>>> >>>>>>> _______________________________________________ >>>>>>> Devel mailing list >>>>>>> Devel(a)ovirt.org >>>>>>> http://lists.ovirt.org/mailman/listinfo/devel >>>>>>> >>>>>> >>>>>> >>>>> >>>>> >>>>> -- >>>>> Didi >>>>> >>>>> _______________________________________________ >>>>> Devel mailing list >>>>> Devel(a)ovirt.org >>>>> http://lists.ovirt.org/mailman/listinfo/devel >>>>> >>>> >>>> >>> >> >> >> -- >> Didi >> > > > > -- > > SANDRO BONAZZOLA > > ASSOCIATE MANAGER, SOFTWARE ENGINEERING, EMEA ENG VIRTUALIZATION R&D > > Red Hat EMEA <https://www.redhat.com/> > <https://red.ht/sig> > TRIED. TESTED. TRUSTED. <https://redhat.com/trusted> > <http://www.teraplan.it/redhat-osd-2017/> > -- DANIEL BELENKY RHV DEVOPS EMEA VIRTUALIZATION R&D <https://red.ht/sig>