On Mon, Jul 3, 2017 at 3:46 PM, Andrew Dent <adent(a)ctcroydon.com.au&gt; wrote: > Has anyone successfully completed a hosted-engine recovery on a >multiple > host setup with production VMs? I'd like to clarify that "recovery" can span a large spectrum of flows, from a trivial "I did some change to the engine database that broke stuff and I want to restore a backup I took prior to this change" to a full system restoration including purchasing and deploying new (perhaps different) hosts/network/storage hardware, including many other flows in between. So when you plan for recovery, you should define very well what flows you plan to handle, and how you handle each. The linked procedure correctly says it's "providing an example". > > Kind regards > > > Andrew > > > > ------ Original Message ------ > From: "Andrew Dent" <adent(a)ctcroydon.com.au&gt; > To: "users" <users(a)ovirt.org&gt; > Sent: 2/07/2017 2:22:16 PM > Subject: [ovirt-users] Recovering hosted-engine > > Hi > > A couple of questions about hosted-engine recovery. > Part way through this URL, in the section "Workflow for Restoring the > Self-Hosted Engine Environment" > >http://www.ovirt.org/documentation/self-hosted/chap-Backing_up_and_Restoring_an_EL-Based_Self-Hosted_Environment/ > it looks like once the hosted-engine is recovered on Host 1, the VMs >on Host > 2 and 3 will be running, but not accessible to the recovered Hosted >Engine. > Is that correct? I am pretty certain that the procedure assumed that all hosts need restoration, not that some are still up-and-running. > If so, how to you remove host 2 and host 3 from the environment, then >add > back in again while keeping the VMs running? That's a good question. Please try to describe the exact flow you have in mind. What's broken and needs restoration, and how do you plan to do that? > > Host 2 and Host 3 are not recoverable in their current state. These >hosts > need to be removed from the environment, and then added again to the > environment using the hosted-engine deployment script. For more >information > on these actions, see the Removing Non-Operational Hosts from a >Restored > Self-Hosted Engine Environment section below and Chapter 7: >Installing > Additional Hosts to a Self-Hosted Environment. > > BTW: The link referring to chapter 7 is broken. You are right. The link in the bottom of the page seems working. Now pushed [1] to fix. Thanks for the report! [1] Best, -- Didi

On Mon, Jul 3, 2017 at 4:40 PM, Andrew Dent <adent(a)ctcroydon.com.au&gt; wrote: > Hi Didi > > Fair enough. > If I'm in this situation..... > I have 3 hosts with 6 production VMs. > The hosted-engine VM is completely toast and not recoverable. Meaning? It does not even boot? If so, then I am afraid that you need to follow the linked procedure, including deploying hosted-engine again on new (or fully cleaned) storage. Are all 3 hosts also hosted-engine hosts? And you want to keep them this way? If so, you'll have to reinstall them. I think, didn't try, that after you deploy a new hosted-engine host, and restore the engine, the new engine will see your existing hosts, but hosted-engine-ha will not work. If you want to try and keep your VMs running, I guess your best bet is to try to add a new non-hosted-engine host (one or more, as needed), migrate your VMs to it (them), then remove the hosted-engine hosts and deploy them again as described there. > However I have a backup of the hosted-engine database (do I need anything > else). Only database? Or a backup taken by engine-backup? You also need various configuration files. To see the full list, in case you didn't use engine-backup but can restore individual files, check BACKUP_PATHS inside the script. But this is definitely not supported - you should first try verifying by restoring on a test VM and see what happens. > > Is it possible to build a new VM, import the backup of the previous > hosted-engine database and reconnect the storage domains and VMs in their > running state without any VMs experiencing an outage? As I wrote above, I think it is, but hosted-engine-ha will not work. If at all possible, I strongly suggest to try to simulate this in a test env. Can be done using nested-kvm VMs for hosts, if you do not have enough real hardware. > > The URL > http://www.ovirt.org/documentation/self-hosted/ chap-Backing_up_and_Restoring_an_EL-Based_Self-Hosted_Environment/ > looks to be longer now. > I'll review, test and see if this will give me what I'm looking for. > > The broken link still seems to be broken. > When I click the link, the browser ends up at this > http://www.ovirt.org/documentation/self-hosted/ chap-Backing_up_and_Restoring_an_EL-Based_Self-Hosted_ Environment/chap-Installing_Additional_Hosts_to_a_Self-Hosted_Environment# documentation%20self%20hosted%20chap%20Backing%20up%20and% 20Restoring%20an%20EL%20Based%20Self%20Hosted%20Environment% 20chap%20Installing%20Additional%20Hosts%20to%20a% 20Self%20Hosted%20Environment > > But I suspect it should be > http://www.ovirt.org/documentation/self-hosted/ chap-Installing_Additional_Hosts_to_a_Self-Hosted_Environment/ In my previous mail I just wrote I sent a pull request, not that the site is updated already. I updated it since and it now seems ok to me. Best, > > Kind regards > > Andrew > > ------ Original Message ------ > From: "Yedidyah Bar David" <didi(a)redhat.com&gt; > To: "Andrew Dent" <adent(a)ctcroydon.com.au&gt; > Cc: "users" <users(a)ovirt.org&gt; > Sent: 3/07/2017 11:12:05 PM > Subject: Re: [ovirt-users] Recovering hosted-engine > >> On Mon, Jul 3, 2017 at 3:46 PM, Andrew Dent <adent(a)ctcroydon.com.au&gt; >> wrote: >>> >>> Has anyone successfully completed a hosted-engine recovery on a multiple >>> host setup with production VMs? >> >> >> I'd like to clarify that "recovery" can span a large spectrum of >> flows, from a trivial "I did some change to the engine database >> that broke stuff and I want to restore a backup I took prior to >> this change" to a full system restoration including purchasing >> and deploying new (perhaps different) hosts/network/storage >> hardware, including many other flows in between. >> >> So when you plan for recovery, you should define very well what >> flows you plan to handle, and how you handle each. >> >> The linked procedure correctly says it's "providing an example". >> >>> >>> Kind regards >>> >>> >>> Andrew >>> >>> >>> >>> ------ Original Message ------ >>> From: "Andrew Dent" <adent(a)ctcroydon.com.au&gt; >>> To: "users" <users(a)ovirt.org&gt; >>> Sent: 2/07/2017 2:22:16 PM >>> Subject: [ovirt-users] Recovering hosted-engine >>> >>> Hi >>> >>> A couple of questions about hosted-engine recovery. >>> Part way through this URL, in the section "Workflow for Restoring the >>> Self-Hosted Engine Environment" >>> >>> http://www.ovirt.org/documentation/self-hosted/ chap-Backing_up_and_Restoring_an_EL-Based_Self-Hosted_Environment/ >>> it looks like once the hosted-engine is recovered on Host 1, the VMs on >>> Host >>> 2 and 3 will be running, but not accessible to the recovered Hosted >>> Engine. >>> Is that correct? >> >> >> I am pretty certain that the procedure assumed that all hosts need >> restoration, >> not that some are still up-and-running. >> >>> If so, how to you remove host 2 and host 3 from the environment, then >>> add >>> back in again while keeping the VMs running? >> >> >> That's a good question. >> >> Please try to describe the exact flow you have in mind. What's broken and >> needs restoration, and how do you plan to do that? >> >>> >>> Host 2 and Host 3 are not recoverable in their current state. These >>> hosts >>> need to be removed from the environment, and then added again to the >>> environment using the hosted-engine deployment script. For more >>> information >>> on these actions, see the Removing Non-Operational Hosts from a Restored >>> Self-Hosted Engine Environment section below and Chapter 7: Installing >>> Additional Hosts to a Self-Hosted Environment. >>> >>> BTW: The link referring to chapter 7 is broken. >> >> >> You are right. The link in the bottom of the page seems working. >> Now pushed [1] to fix. Thanks for the report! >> >> [1] >> >> Best, >> -- >> Didi > > -- Didi _______________________________________________ Users mailing list Users(a)ovirt.org http://lists.ovirt.org/mailman/listinfo/users