[ovirt-users] Recovering hosted-engine

Mon Jul 3 14:25:18 UTC 2017

It depends on how 'toasted' the engine is.

It is possible to mount the disk, if for example it's a kernel upgrade
that's gone wrong and stopping the machine form booting

On 3 July 2017 at 15:23, Yedidyah Bar David <didi at redhat.com> wrote:

> On Mon, Jul 3, 2017 at 4:40 PM, Andrew Dent <adent at ctcroydon.com.au>
> wrote:
> > Hi Didi
> >
> > Fair enough.
> > If I'm in this situation.....
> > I have 3 hosts with 6 production VMs.
> > The hosted-engine VM is completely toast and not recoverable.
>
> Meaning? It does not even boot? If so, then I am afraid that you
> need to follow the linked procedure, including deploying hosted-engine
> again on new (or fully cleaned) storage.
>
> Are all 3 hosts also hosted-engine hosts? And you want to keep them
> this way? If so, you'll have to reinstall them.
>
> I think, didn't try, that after you deploy a new hosted-engine host,
> and restore the engine, the new engine will see your existing hosts,
> but hosted-engine-ha will not work.
>
> If you want to try and keep your VMs running, I guess your best bet
> is to try to add a new non-hosted-engine host (one or more, as needed),
> migrate your VMs to it (them), then remove the hosted-engine hosts and
> deploy them again as described there.
>
> > However I have a backup of the hosted-engine database (do I need anything
> > else).
>
> Only database? Or a backup taken by engine-backup? You also need various
> configuration files. To see the full list, in case you didn't use
> engine-backup but can restore individual files, check BACKUP_PATHS inside
> the script. But this is definitely not supported - you should first try
> verifying by restoring on a test VM and see what happens.
>
> >
> > Is it possible to build a new VM, import the backup of the previous
> > hosted-engine database and reconnect the storage domains and VMs in their
> > running state without any VMs experiencing an outage?
>
> As I wrote above, I think it is, but hosted-engine-ha will not work.
>
> If at all possible, I strongly suggest to try to simulate this in a
> test env. Can be done using nested-kvm VMs for hosts, if you do not have
> enough real hardware.
>
> >
> > The URL
> > http://www.ovirt.org/documentation/self-hosted/
> chap-Backing_up_and_Restoring_an_EL-Based_Self-Hosted_Environment/
> > looks to be longer now.
> > I'll review, test and see if this will give me what I'm looking for.
> >
> > The broken link still seems to be broken.
> > When I click the link, the browser ends up at this
> > http://www.ovirt.org/documentation/self-hosted/
> chap-Backing_up_and_Restoring_an_EL-Based_Self-Hosted_
> Environment/chap-Installing_Additional_Hosts_to_a_Self-Hosted_Environment#
> documentation%20self%20hosted%20chap%20Backing%20up%20and%
> 20Restoring%20an%20EL%20Based%20Self%20Hosted%20Environment%
> 20chap%20Installing%20Additional%20Hosts%20to%20a%
> 20Self%20Hosted%20Environment
> >
> > But I suspect it should be
> > http://www.ovirt.org/documentation/self-hosted/
> chap-Installing_Additional_Hosts_to_a_Self-Hosted_Environment/
>
> In my previous mail I just wrote I sent a pull request, not that
> the site is updated already. I updated it since and it now seems
> ok to me.
>
> Best,
>
> >
> > Kind regards
> >
> > Andrew
> >
> > ------ Original Message ------
> > From: "Yedidyah Bar David" <didi at redhat.com>
> > To: "Andrew Dent" <adent at ctcroydon.com.au>
> > Cc: "users" <users at ovirt.org>
> > Sent: 3/07/2017 11:12:05 PM
> > Subject: Re: [ovirt-users] Recovering hosted-engine
> >
> >> On Mon, Jul 3, 2017 at 3:46 PM, Andrew Dent <adent at ctcroydon.com.au>
> >> wrote:
> >>>
> >>>  Has anyone successfully completed a hosted-engine recovery on a
> multiple
> >>>  host setup with production VMs?
> >>
> >>
> >> I'd like to clarify that "recovery" can span a large spectrum of
> >> flows, from a trivial "I did some change to the engine database
> >> that broke stuff and I want to restore a backup I took prior to
> >> this change" to a full system restoration including purchasing
> >> and deploying new (perhaps different) hosts/network/storage
> >> hardware, including many other flows in between.
> >>
> >> So when you plan for recovery, you should define very well what
> >> flows you plan to handle, and how you handle each.
> >>
> >> The linked procedure correctly says it's "providing an example".
> >>
> >>>
> >>>  Kind regards
> >>>
> >>>
> >>>  Andrew
> >>>
> >>>
> >>>
> >>>  ------ Original Message ------
> >>>  From: "Andrew Dent" <adent at ctcroydon.com.au>
> >>>  To: "users" <users at ovirt.org>
> >>>  Sent: 2/07/2017 2:22:16 PM
> >>>  Subject: [ovirt-users] Recovering hosted-engine
> >>>
> >>>  Hi
> >>>
> >>>  A couple of questions about hosted-engine recovery.
> >>>  Part way through this URL, in the section "Workflow for Restoring the
> >>>  Self-Hosted Engine Environment"
> >>>
> >>> http://www.ovirt.org/documentation/self-hosted/
> chap-Backing_up_and_Restoring_an_EL-Based_Self-Hosted_Environment/
> >>>  it looks like once the hosted-engine is recovered on Host 1, the VMs
> on
> >>> Host
> >>>  2 and 3 will be running, but not accessible to the recovered Hosted
> >>> Engine.
> >>>  Is that correct?
> >>
> >>
> >> I am pretty certain that the procedure assumed that all hosts need
> >> restoration,
> >> not that some are still up-and-running.
> >>
> >>>  If so, how to you remove host 2 and host 3 from the environment, then
> >>> add
> >>>  back in again while keeping the VMs running?
> >>
> >>
> >> That's a good question.
> >>
> >> Please try to describe the exact flow you have in mind. What's broken
> and
> >> needs restoration, and how do you plan to do that?
> >>
> >>>
> >>>  Host 2 and Host 3 are not recoverable in their current state. These
> >>> hosts
> >>>  need to be removed from the environment, and then added again to the
> >>>  environment using the hosted-engine deployment script. For more
> >>> information
> >>>  on these actions, see the Removing Non-Operational Hosts from a
> Restored
> >>>  Self-Hosted Engine Environment section below and Chapter 7: Installing
> >>>  Additional Hosts to a Self-Hosted Environment.
> >>>
> >>>  BTW: The link referring to chapter 7 is broken.
> >>
> >>
> >> You are right. The link in the bottom of the page seems working.
> >> Now pushed [1] to fix. Thanks for the report!
> >>
> >> [1]
> >>
> >> Best,
> >> --
> >> Didi
> >
> >
>
>
>
> --
> Didi
> _______________________________________________
> Users mailing list
> Users at ovirt.org
> http://lists.ovirt.org/mailman/listinfo/users
>
-------------- next part --------------
An HTML attachment was scrubbed...
URL: <http://lists.ovirt.org/pipermail/users/attachments/20170703/d7e8b0c2/attachment-0001.html>