[ovirt-users] upgrade hosted-engine os ( not hosts )

[Yedidyah Bar David]

On Thu, Jun 30, 2016 at 9:47 AM, Paul Groeneweg | Pazion <paul at pazion.nl> wrote:
> Ok that sounds good.
>
> And if I deploy a new hosted-engine on a fresh host I need to do a storage
> domain import after restore?

Please describe exact flow you plan.

In principle you can use a flow similar to the one described in the "migrate
to hosted-engine" doc you mentioned before. Run engine-backup backup on the
existing engine, deploy new hosted-engine on a new host and new storage,
then restore the backup and engine-setup.

You'll keep all your existing VMs and hosts/storage/etc.

The main issue will be about HE shared storage, from the POV of the HA daemons
on the existing HE hosts. They'll look at the old storage configured, and we
do not supply a tool to change that. So in principle, you can try
something like:

1. Move to global maintenance
2. backup
3. deploy hosted-engine on a new host and storage, import the backup in the
engine vm and engine-setup.
4. Migrate all running VMs from one of the old HE hosts to the new HE host.
5. Move this old host to maint and remove it.
6. Clean this old host somehow (simplest is OS reinstall) and deploy it again
as an additional HE host of the new hosted-engine setup (that is, when asked
for hosted storage, supply the new storage).
7. Repeat steps 4-6 for all the remaining old hosts.

This is obviously somewhat more complex than just setting up a completely new
engine (no backup/restore) and only import the VMs, but in principle might
allow no downtime at all for the other VMs.

> Or is my config including master storage domain restored and I can use the
> old storage domain with need to import?

Your config is not restored unless you restore it manually.
Unless you use the tool when it's ready, and it will do that for you :-)

Best,

>
>
>
> Op do 30 jun. 2016 om 08:41 schreef Yedidyah Bar David <didi at redhat.com>:
>>
>> On Thu, Jun 30, 2016 at 9:34 AM, Paul Groeneweg | Pazion <paul at pazion.nl>
>> wrote:
>> > Hi Yedidyah,
>> >
>> > Thank you for the comprehensive answers.
>> >
>> > I think I go for a complete reinstall ( read also OS upgrade tool is not
>> > adviced on 6.6 or higher as there might be newer packages as on 7 ). No
>> > doubting to re-use current VM or setup from scratch ( fresh host with
>> > new
>> > hosted-engine and existing storage domein ).
>> >
>> > You explain the steps ( 1 to 6 ), but then don't talk about storage
>> > domain
>> > import.
>> > Does it mean, when I reinstall the hosted-engine in the current he VM
>> > and
>> > restore an engine-backup ( step 5 ) I am able to start vm from Host and
>> > it
>> > is still connected to the master storage ( so no need for storage
>> > import) ?
>>
>> Indeed - if you didn't touch the storage, the restored engine should
>> already
>> knows all it needs to know.
>>
>> Best,
>>
>> >
>> > Best Regards,
>> > Paul Groeneweg
>> >
>> >
>> > Op do 30 jun. 2016 om 08:00 schreef Yedidyah Bar David
>> > <didi at redhat.com>:
>> >>
>> >> On Wed, Jun 29, 2016 at 10:07 PM, Paul Groeneweg | Pazion
>> >> <paul at pazion.nl> wrote:
>> >> >
>> >> > I am looking for a way to get my hosted-engine running on el7 so I
>> >> > can
>> >> > upgrade to oVirt 4.0. Currently my hosts already run el7, but my
>> >> > hosted-engine is still el6.
>> >> >
>> >> > I read
>> >> >
>> >> >
>> >> > https://www.ovirt.org/documentation/how-to/hosted-engine-host-OS-upgrade/
>> >> > but this is only about the hosts.
>> >> >
>> >> > I read https://www.ovirt.org/documentation/how-to/hosted-engine/, but
>> >> > it
>> >> > only mentions upgrade of the hosted-engine software, not the OS.
>> >> >
>> >> > I understood I can do a fresh hosted-engine install, and then import
>> >> > my
>> >> > storage domain to the new hosted engine, but:
>> >> >
>> >> > - Do I need to restore my hosted engine database? ( like described
>> >> > here:
>> >> >
>> >> >
>> >> > http://www.ovirt.org/develop/developer-guide/engine/migrate-to-hosted-engine/
>> >> > )
>> >>
>> >> You might not have to, if you only care about the imported VMs from
>> >> your
>> >> storage. This will not keep other configuration, such as
>> >> users/roles/permissions
>> >> etc.
>> >>
>> >> > - Can I directly install hosted-engine 4.0 and then import the
>> >> > storage
>> >> > domain? Or should I install same hosted-engine version?
>> >>
>> >> AFAIK 4.0 engine can import 3.6 storage domains without problem.
>> >>
>> >> > - Do I first need another master storage domain or can I directly
>> >> > import
>> >> > my
>> >> > old master storage domain?
>> >>
>> >> No idea. Even if you do, you can create a small empty one and later
>> >> remove
>> >> it.
>> >>
>> >> > - When importing the storage domain what is the risk it fails ( I
>> >> > have
>> >> > backups, but it would cost a day to restore all )
>> >>
>> >> No idea, but IIRC we got many successful reports and at most few
>> >> failures
>> >> for this.
>> >>
>> >> > - How long would import take? few minutes or hours? ( I want to keep
>> >> > down
>> >> > time as low as possible ).
>> >>
>> >> Again no idea. Perhaps do some test?
>> >>
>> >> >
>> >> > Another option would be upgrade the OS ( with redhat-upgrade-tool )
>> >> > or
>> >> > is
>> >> > this a path for disaster?
>> >>
>> >> Didn't work for us well, so we decided to not support it. If you decide
>> >> to
>> >> try,
>> >> make sure you test carefully beforehand. From ovirt's POV:
>> >> 1. You'll need to handle postgresql upgrade.
>> >> 2. Right after OS upgrade, you'll still have (I think) el6 packages
>> >> of the engine. It will hopefully be in a good-enough state for upgrade
>> >> to 4.0, but we didn't test this.
>> >> 3. Specifically, if upgrade fails, rollback will most likely not work,
>> >> so you'll have to manually handle this - take a full vm backup and make
>> >> sure you can restore it.
>> >>
>> >> >
>> >> > I hope someone can tell me how I can smoothly upgrade my
>> >> > hosted-engine
>> >> > up to
>> >> > el7 and run oVirt 4.
>> >>
>> >> We are working on a tool/wizard to help with this process. It used to
>> >> work,
>> >> but at some point it was decided that one of the actions it does is
>> >> risky
>> >> and was blocked, thus the tool is broken currently.
>> >>
>> >> You can invoke the tool by running: 'hosted-engine
>> >> --upgrade-appliance'.
>> >> As noted above, this is currently broken.
>> >>
>> >> There are several open bugs about it, e.g.:
>> >>
>> >> https://bugzilla.redhat.com/show_bug.cgi?id=1319457
>> >> https://bugzilla.redhat.com/show_bug.cgi?id=1343425
>> >> https://bugzilla.redhat.com/show_bug.cgi?id=1343593 (closed, this is
>> >> what broke the tool)
>> >>
>> >> Basically, you can manually do what the tool is supposed to do:
>> >> 1. Make sure state is clean and stable (no running/pending storage
>> >> actions,
>> >> no VMs in the middle of migration etc), all clusters are compat level
>> >> 3.6,
>> >> etc.
>> >> 2. Move to global maintenance
>> >> 3. backup the engine using engine-backup and keep the backup elsewhere
>> >> 4. Reinstall engine vm with el7 and 4.0 engine (the tool will use the
>> >> engine
>> >> appliance, you might too but not sure how exactly).
>> >> 5. Restore the backup and run engine-setup.
>> >> 6. If all looks ok, leave global maintenance.
>> >>
>> >> If you manually keep a full backup of the engine vm before step 4,
>> >> you might be able to restore this backup if there are problems.
>> >> Doing this in the provided tool is currently the main blocking issue
>> >> for it. Hopefully will be provided in 4.0.1.
>> >>
>> >> Best,
>> >> --
>> >> Didi
>>
>>
>>
>> --
>> Didi



-- 
Didi