On Wed, Feb 8, 2017 at 4:18 PM, Doug Ingham <dougti@gmail.com> wrote:

Hi Dan,

On 8 February 2017 at 18:10, Dan Yasny <dyasny@gmail.com> wrote:

On Wed, Feb 8, 2017 at 4:07 PM, Doug Ingham <dougti@gmail.com> wrote:
Hi Guys,
My Hosted-Engine has failed & it looks like the easiest solution will be to install a new one. Now before I try to re-add the old hosts (still running the guest VMs) & import the storage domain into the new engine, in case things don't go to plan, I want to make sure I'm able to bring up the guests on the hosts manually.

The problem is vdsClient is giving me an "Unexpected exception", without much more info as to why it's failing.

Any idea?

[root@v0 ~]# vdsClient -s 0 list table | grep georep
9d1c3fef-498e-4c20-b124-01364d4d45a8 30455 georep-proxy Down

[root@v0 ~]# vdsClient -s 0 continue 9d1c3fef-498e-4c20-b124-01364d4d45a8
Unexpected exception

/var/log/vdsm/vdsm.log
periodic/1063::WARNING::2017-02-08 17:57:52,532::periodic::276::virt.periodic.VmDispatcher::(__call__) could not run <class 'vdsm.virt.periodic.DriveWatermarkMonitor'> on ['65c9807c-7216-40b3-927c-5fd93bbd42ba', u'9d1c3fef-498e-4c20-b124-01364d4d45a8']

continue meane un-pause, not "start from a stopped state"

I search the manual for start/init/resume syntax, and "continue" was the closest thing I found.

I don't have a 4.x vdsm handy, but on 3.6 the verb is "create". With a LOT of params of course.

now having said that, if you expect the VMs not to be able to start after you rebuild the engine and the VMs exist on the hosts, I'd collect a virsh -r dumpxml VMNAME for each - that way you have the disks in use, and all the VM configuration in a file, and with some minor LVM manipulation you'll be able to start the VM via virsh

My main concern is that I might have to halt the VMs or VDSM services for some reason when trying to migrate to the new engine. I just want to make sure that no matter what happens, I can still get the VMs back online.

The way oVirt works is very much tied into the engine DB. When you click "start" on a VM, the engine will query the DB, pull out the VM details (CPU config, disks, RAM etc), pick a suitable host, enable the hosts' access to the disks on the storage domain, generate the libvirt domxml for the VM (the file you'd get from virsh dumpxml) and start the VM according to the generated XML. With vdsClient and without the engine DB you'll be missing all those details the database provides, while my way, with the XML, they are all already in place, populated by the engine when it was still alive.

I'm still getting myself acquainted with virsh/vdsClient. Could you provide any insight into what I'd have to do to restart the guests manually?

see above. But seriously, above all, I'd recommend you backup the engine (it comes with a utility) often and well. I do it via cron every hour in production, keeping a rotation of hourly and daily backups, just in case. It doesn't take much space or resources, but it's more than just best practice - that database is the summary of the entire setup.

Thanks,
--
Doug