[ovirt-users] Ovirt-3.4.1: How to: Upgrading Hosted Engine Cluster

[Doron Fediuck]

----- Original Message -----
> From: "Daniel Helgenberger" <daniel.helgenberger at m-box.de>
> To: users at ovirt.org
> Sent: Friday, May 9, 2014 6:45:36 PM
> Subject: [ovirt-users] Ovirt-3.4.1: How to: Upgrading Hosted Engine Cluster
> 
> Hello,
> 
> failing to find a procedure how to actually upgrade a HA cluster, I did
> the following witch turned out to be working pretty well.
> 
> I am somewhat new to oVirt and was amazed how well actually; I did not
> need to shutdown a single VM (well, one because of mem usage; many of my
> running VMs have fancy stuff like iscsi and FC luns via a Quantum
> Stornext HA Cluster):
> 
>      1. Set cluster to global maintance
>      2. Login to ovit engine and to the upgrade according to the release
>         nodes.
>      3. After the upgrade is finished and the engine running, set the
>         first Node in local maintenance.
>      4. Login the first node and yum update (with the removal of
>         ovirt-release as mentioned in release notes).* I rebooted the
>         node because of the kernel update.
>      5. Return to oVirt and reinstall the Node from GUI, it will be set
>         to operational automatically**
>      6. Repeat steps 3-6 for the rest of the Nodes.
>      7. Remove global maintenance.
>      8. Update the last Node.***
> 
> * I first tried to do this with re-install from GUI. This failed; so I
> used the yum - update method to update all relevant services
> 
> ** I do not know if this was necessary. I did this because the
> hosted-engine --deploy does the same thing when adding a host.
> 
> *** I found this to be necessary because I had all my Nodes in local
> maintenance and could not migrate the Hosted engine from the last node
> any more. The host activation in oVirt did not remove the local
> maintenance set prior to the update (witch it should, IMHO). It might be
> desirable to have a hosted-engine command option to remove local
> maintenance for that reason.
> 
> 
> 
> --
> 
> Daniel Helgenberger
> m box bewegtbild GmbH
> 
> P: +49/30/2408781-22
> F: +49/30/2408781-10
> 
> ACKERSTR. 19
> D-10115 BERLIN
> 
> 
> www.m-box.de  www.monkeymen.tv
> 
> Geschäftsführer: Martin Retschitzegger / Michaela Göllner
> Handeslregister: Amtsgericht Charlottenburg / HRB 112767
> 

Hi Martin,
Thanks for sharing!

A few notes on your nice 8 steps ;)

- There are 2 maintenance modes to cover host maintenance (local) and
  VM maintenance (global). Global maintenance disarms all HA hosts in
  the cluster, so use it with caution as there's no fail over in this
  mode.

- Initially these were available only as a command line[1]. Since 3.4.0,
  this was integrated into the UI, so all you need to do is move a host
  to maintenance in order to achieve local maintenance, and activate it
  to remove the maintenance mode. For global maintenance, right click
  the engine VM, and you will see enable/disable ha-maintenance for
  the global mode.

- No need to re-install nodes. All you need to do is activate it.

- Basically a standard procedure should include:
  * Move host to maintenance, login and update the host, activate the host.
  * Follow the above for all other HA hosts
  * Set the engine VM to ha maintenance (global), login to the vm and
    upgrade it, unset vm's ha-maintenance.

Appreciate your feedback.
Also, were you aware of [1] or did you look for info elsewhere? I'd like
to know what we can do to improve documentation.

Thanks again,
Doron

[1] http://www.ovirt.org/Hosted_Engine_Howto#Maintaining_the_setup