-----Ursprüngliche Nachricht----- Von: Yedidyah Bar David [mailto:didi@redhat.com] Gesendet: Mittwoch, 28. März 2018 10:06 An: Sven Achtelik Cc: users@ovirt.org Betreff: Re: [ovirt-users] Recovering oVirt-Engine with a backup before upgrading to 4.2
On Tue, Mar 27, 2018 at 9:14 PM, Sven Achtelik <Sven.Achtelik@eps.aero> wrote:
Hi All,
I’m still facing issues with my HE engine. Here are the steps that I took to end up in this situation:
- Update Engine from 4.1.7 to 4.1.9
o That worked as expected
- Automatic Backup of Engine DB in the night
- Upgraded Engine from 4.1.9 to 4.2.1
o That worked fine
- Noticed Issues with the HA support for HE
o Cause was not having the latest ovirt-ha agent/broker version on hosts
- After updating the first host with the latest packages for the Agent/Broker engine was started twice
o As a result the Engine VM Disk was corrupted and there is no Backup of the Disk
o There is also no Backup of the Engine DB with version 4.2
- VM disk was repaired with fsck.ext4, but DB is corrupt
o Can’t restore the Engine DB because the Backup DB from Engine V 4.1
- Rolled back all changes on Engine VM to 4.1.9 and imported Backup
o Checked for HA VMs to set as disabled and started the Engine
- Login is fine but the Engine is having trouble picking up and information from the Hosts
o No information on running VMs or hosts status
- Final Situation
o 2 Hosts have VMs still running and I can’t stop those
o I still have the image of my corrupted Engine VM (v4.2)
Since there were no major changes after upgrading from 4.1 to 4.2, would it be possible to manually restore the 4.1 DB to the 4.2 Engine VM to this up and running again or are there modifications made to the DB on upgrading that are relevant for this ?
engine-backup requires restoring to the same version used to take the backup, with a single exception - on 4.0, it can restore 3.6.
It's very easy to patch it to allow also 4.1->4.2, search inside it for "VALID_BACKUP_RESTORE_PAIRS". However, I do not think anyone ever tested this, so no idea might break. In 3.6->4.0 days, we did have to fix a few other things, notably apache httpd and iptables->firewalld:
https://bugzilla.redhat.com/show_bug.cgi?id=1318580
All my work on rolling back to 4.1.9 with the DB restore failed as the Engine is not capable of picking up information from the hosts.
No idea why, but not sure it's related to your restore flow.
Lessons learned is to always make a copy/snapshot of the engine VM disk before upgrading anything.
If it's a hosted-engine, this isn't supported - see my reply on the list ~ 1 hour ago...
What are my options on getting back to a working environment ? Any help or hint is greatly appreciated.
Restore again with either methods - what you tried, or patching engine- backup and restore directly into 4.2 - and if the engine fails to talk to the hosts, try to debug/fix this.
If you suspect corruption more severe that just the db, you can install a fresh engine machine from scratch and restore to it. If it's a hosted-engine, you'll need to deploy hosted-engine from scratch, check docs about hosted-engine backup/restore.
I read through those documents and it seems that I would need an extra Host/Hardware which I don't have. https://ovirt.org/documentation/self-hosted/chap-Backing_up_and_Restoring_an... So how would I be able to get a new setup working when I would like to use the Engine-VM-Image ? At this point it sounds like I would have to manually reinstall the machine that is left over and running. I'm lost at this point.
Best regards, -- Didi