[ovirt-users] how to prepare for an engine re-install

[Yedidyah Bar David]

----- Original Message -----
> From: "Robert Story" <rstory at tislabs.com>
> To: Users at ovirt.org
> Sent: Wednesday, February 18, 2015 5:12:40 PM
> Subject: [ovirt-users] how to prepare for an engine re-install
> 
> I'm running ovirt (engine 3.4.0-1.el6) on a dedicated engine machine with a
> degraded raid array. This has caused the root filesystem to go read-only,
> and the engine crashed. I can restart the engine, but after doing so, the
> web page just loads a blank page. All nodes and VMs seem to be fine, I just
> can't manage them via oVirt.
> 
> Because I'm offsite, my recovery option is likely limited to a reinstall of
> the OS and ovirt.  I'm hoping for some pointers on what I can do in advance
> to minimize the pain of this transition.
> 
> - Am I going to be able to recover without having to shut down VMs,
>   exporting and importing?

Perhaps, depends on your plan

> 
> - what directories should I back up?

If at all possible, you should try engine-backup. I guess it will fail if
the fs is read-only, because postgres will not start and engine-backup
uses it.

You might get by with copying the stuff engine-backup does (search for
BACKUP_PATHS in its start), and also a cold copy of the database - all
of /var/lib/pgsql/data .

Try copying the db to another machine and 'service postgresql start'.
If it works, and you can connect to the db, full recovery will probably
work.

> 
> - when I reinstall, should I upgrade and install 3.5.1, or first install
>   3.4, get it working, and then think about upgrading?

I'd separate them. Reinstall with exact same version if possible - not sure
you can find 3.4.0 on the yum repo, perhaps you can find somewhere in your
cache the version you use, or something like that - then upgrade when all is
quiet.

> 
> - Is it possible/recommended/crazy to migrate to a hosted engine instead
>   of a dedicated machine?

In principle should be possible.

> 
> So I want to prepare for the worst before I try a reboot.

If it's currently read-only, why not simply copy all of it somewhere? Then
if a reboot kills it, you still have the copy to try and recover from.

> If I'm lucky, the
> reboot will at least come up with a writable filesystem, and I won't need to
> re-install. If it doesn't come up, I want to be as ready as possible for
> the reinstall.
> 
> Any hints and/or tips appreciated..

If at all possible, practice on a test system before trying live :-)

See [1] for an overview of a migration to hosted-engine. Most of it is relevant
to any migration, if engine-backup works for you. Otherwise it's even simpler -
just copy files and hope that postgresql will manage to work with the db.

[1] http://www.ovirt.org/Migrate_to_Hosted_Engine

Good luck and best regards,
-- 
Didi