On Wed, Oct 5, 2016 at 10:30 AM, <aleksey.maksimov@it-kb.ru> wrote:

Well.
Then, in the case of conditions:

1) the vm is not available anymore due to storage corruption
2) an empty shared storage is available
3) engine backup exists
4) all VMs still running on the hosts in the cluster

The recovery plan will be like this (as I understand it):

1) On all the hosts (if they are still available):

# service ovirt-ha-broker stop
# service ovirt-ha-agent stop
# chkconfig --del ovirt-ha-broker
# chkconfig --del ovirt-ha-agent

2) On first host (if the original host is not available anymore, provision a new host from scratch and proceed on this new host):

2.1) # hosted-engine --deploy

◾use same fqdn you had previously in the HE VM.
◾point to the new shared storage
◾provide the same admin password you used in previous setup
◾install the OS on the vm

I'd suggest to use the engine appliance also for this.

You can just say No when it asks about automatically running engine-setup.

◾confirm it has been installed

on Hosted Engine VM:

a) Install the ovirt-engine rpms on the vm but don't run engine-setup:
  # yum install http://resources.ovirt.org/pub/yum-repo/ovirt-release40.rpm
  # yum install epel-release
  # yum install ovirt-engine
b) Restore the backup:
  # engine-backup --mode=restore --file=file_name --log=log_file_name --provision-db --provision-dwh-db --restore-permissions

In order to let the engine auto-import the new hosted-engine storage domain, you have to remove the old one.

The same for the engine VM. Unfortunately you cannot do that from the engine since they are somehow protected to avoid unintentional damages.

The easiest way is to remove them from the DB before running engine-setup.

I'm working on a helper utility to make it easiser:

https://gerrit.ovirt.org/#/c/64966/

I think I'll integrate it with engine-backup to simply do it with an additional CLI flag.

c) Run "engine-setup"

2.2) Open Administration Portal and remove the all old hosts used for Hosted Engine

Right, we can also integrate this step in the HE cleaning helper.

2.3) Confirm that the engine has been installed (Return to the host and continue the hosted-engine deployment script by selecting option 1) and then finish the deploy.

2.4) In Administration Portal activate new host

3) On all additional hosts run "hosted-engine --deploy".

I strongly suggest to deploy them from the engine and not from CLI.

CLI deploy support for additional HE host is deprecated an it will be removed in 4.1.

Right?