[ovirt-users] Upgrading engine 3.5.5-1.el6 to latest patches issue (and resolved)

[Yedidyah Bar David]

On Wed, Dec 2, 2015 at 4:27 PM, Scott Worthington
<scott.c.worthington at gmail.com> wrote:
> Hello,
>
> This is just an informational e-mail for others who are updating their 3.5
> systems on CentOS 6.7 who may experience the same issue that I experienced.
>
> I am upgrading a stand alone 3.5.5-1.el6 engine on CentOS 6.7 (no High
> Availability cruft) to the latest patches just recently released.
>
> The DWH and report engine are also installed on this same stand alone system.
>
> I did a 'yum update' and the following was downloaded & updated:
>
> Dec 02 08:26:26 Updated: postgresql-libs-8.4.20-4.el6_7.x86_64
> Dec 02 08:26:27 Updated: ovirt-engine-lib-3.5.6.2-1.el6.noarch
> Dec 02 08:26:28 Updated: ovirt-engine-setup-base-3.5.6.2-1.el6.noarch
> Dec 02 08:26:29 Updated:
> ovirt-engine-setup-plugin-ovirt-engine-common-3.5.6.2-1.el6.noarch
> Dec 02 08:26:30 Updated:
> ovirt-engine-setup-plugin-websocket-proxy-3.5.6.2-1.el6.noarch
> Dec 02 08:26:31 Updated:
> ovirt-engine-setup-plugin-ovirt-engine-3.5.6.2-1.el6.noarch
> Dec 02 08:26:33 Updated: postgresql-8.4.20-4.el6_7.x86_64
> Dec 02 08:26:35 Updated: postgresql-server-8.4.20-4.el6_7.x86_64
> Dec 02 08:26:35 Updated: ovirt-engine-setup-3.5.6.2-1.el6.noarch
> Dec 02 08:26:36 Updated: ovirt-engine-websocket-proxy-3.5.6.2-1.el6.noarch
> Dec 02 08:26:36 Updated: ovirt-engine-extensions-api-impl-3.5.6.2-1.el6.noarch
> Dec 02 08:26:37 Updated: ovirt-engine-sdk-python-3.5.6.0-1.el6.noarch
> Dec 02 08:26:41 Updated: ovirt-guest-tools-iso-3.5-8.fc22.noarch
> Dec 02 08:26:50 Updated: 1:java-1.6.0-openjdk-1.6.0.37-1.13.9.4.el6_7.x86_64
>
> Here is where I ran into an issue..
>
> Running an 'engine-setup' to load the latest patches caused this:
>
> [ INFO  ] Cleaning async tasks and compensations
> [ INFO  ] Checking the Engine database consistency
> [ INFO  ] Stage: Transaction setup
> [ INFO  ] Stopping dwh service
> [ INFO  ] Stopping reports service
> [ INFO  ] Stopping engine service
> [ INFO  ] Stopping ovirt-fence-kdump-listener service
> [ INFO  ] Stopping websocket-proxy service
> [ ERROR ] dwhd is currently running. Its hostname is
> tpa-ovirt-engine-01.allianceoneinc.com. Please stop it before running Setup.
> [ ERROR ] Failed to execute stage 'Transaction setup': dwhd is currently running
> [ INFO  ] Yum Performing yum transaction rollback
> [ INFO  ] Stage: Clean up
>           Log file is located at
> /var/log/ovirt-engine/setup/ovirt-engine-setup-20151202083052-q8131e.log
> [ INFO  ] Generating answer file
> '/var/lib/ovirt-engine/setup/answers/20151202083138-setup.conf'
> [ INFO  ] Stage: Pre-termination
> [ INFO  ] Stage: Termination
> [ ERROR ] Execution of setup failed
>
> I did a "ps auxwww | grep ovirt" and "ps auxwww | grep dhw", but came up with
> nothing.  In fact, there was nothing obviously 'ovirt' related running.
>
> To get the "engine-setup" to work, I thought that perhaps something was stuck
> running.
>
> So I did...
>
> service ovirt-engine stop
> service ovirt-engine-dwhd stop
> service ovirt-engine-reportsd stop
> service ovirt-engine-websocket-proxy stop
>
> Followed by...
>
> engine-setup
>
> ...and still got the error that the DWH was still running.
>
> Not to waste time, I figured that a reboot might clear and/or reset out some
> process that I am not aware of.
>
> I did not want anything ovirt engine related starting up, so I examined what
> was running on start up:
>
> chkconfig --list | grep ovirt | grep on
>
> Which returned (sorry for wraps, it's thunderbird):
>
> ovirt-engine    0:off   1:off   2:on    3:on    4:on    5:on    6:off
> ovirt-engine-dwhd       0:off   1:off   2:on    3:on    4:on    5:on    6:off
> ovirt-engine-reportsd   0:off   1:off   2:on    3:on    4:on    5:on    6:off
> ovirt-fence-kdump-listener      0:off   1:off   2:on    3:on    4:on    5:on
>   6:off
> ovirt-websocket-proxy   0:off   1:off   2:on    3:on    4:on    5:on    6:off
>
> I turned these off on boot:
>
> chkconfig ovirt-engine off
> chkconfig ovirt-engine-dwhd off
> chkconfig ovirt-engine-reportsd off
> chkconfig ovirt-fence-kdump-listener off
> chkconfig ovirt-websocket-proxy off
>
> Rebooted the server, logged back in, and 'engine-setup' ran flawlessly.
>
> After the upgrade, I turned the services back on for the next reboot:
>
> chkconfig ovirt-engine on
> chkconfig ovirt-engine-dwhd on
> chkconfig ovirt-engine-reportsd on
> chkconfig ovirt-fence-kdump-listener on
> chkconfig ovirt-websocket-proxy on
>
> Hope that helps someone else.

This is a known issue [1]. See also [2].

Sorry you had to go through this.

Actually not sure how what you did above solved this. It's solvable by
restarting dwhd (comment 1 in [1]) or by manually updating the db [2].
Perhaps you did restart dwhd at some point? Other than that, can't think
of anything that will reset this flag.

Thanks for the report, anyway!

[1] https://bugzilla.redhat.com/show_bug.cgi?id=1286441
[2] http://www.ovirt.org/Ovirt-engine-backup#DWH_up_during_backup

Best,
-- 
Didi