[ovirt-users] upgrade hosted-engine os ( not hosts )

[Yedidyah Bar David]

On Thu, Jun 30, 2016 at 9:46 PM, Paul Groeneweg | Pazion <paul at pazion.nl> wrote:
> Things are getting more clear to me, thanks!
> And this whole upgrade is something I should prepare well and not try to do
> in a few hours at night :)
>
> Resuming, there are multiple solutions:
>
> 1. upgrade el6 to el7
> no go => I need to upgrade postgresql and redhat warns about upgrading 6.6
> to 7 because of newer packages
> 2. install a new HE on new Host new storage domain
> a lot of work  => I have to turn off VMs, import storage domain into new
> setup and reinstall hosts.
> 3. install a new HE on new Host existing storage domain
> possibly not without issues => I have to manually update references to the
> hosted storage or better wait till there is an upgrade tool.
> 4. (re)install HE in current VMS container.
> => I can keep my Hosts and reference to the storage domain.
>
> So I guess my best bet is to go for option 4.
>
> The flow I am planning to follow would be:
>
> 1. Move to global maintenance ( keep VMs running )
> 2. backup ( I have also a complete disk install of old hosted-engine )
> 3. reboot with different conf, to boot from cd
> 4. acces console and run install + import the backup in the engine vm
> ? 5. run engine-setup to configure new install with engine restore data
> 6. reboot hosted-engine
> 7. Remove from global maintenance
> 8. launch web gui and I should be able to manage all still running VMs
> again.
>
> Would above be working or am I missing something and I  doubt step 5. Is
> this needed?

Yes.

In a normal backup/restore flow, it's less important, although officially
it's mandatory. But in above flow, it's not just backup/restore, but also
upgrade from 3.6 to 4.0.

In principle you can also do all of the above with 3.6, at least inside
the engine, verify that all seems good, then upgrade to 4.0 later. See
also:

https://bugzilla.redhat.com/1332463

But I do not remember reports from people trying it.

One very important thing you did not mention: Try this on a test system,
including rollback to your backup, after you finished everything and
(supposedly) found out something broken. Better safe than sorry :-)

Best,

>
> Best Regards,
> Paul
>
>
>
> Op do 30 jun. 2016 om 14:44 schreef Simone Tiraboschi <stirabos at redhat.com>:
>>
>> On Thu, Jun 30, 2016 at 8:34 AM, Paul Groeneweg | Pazion <paul at pazion.nl>
>> wrote:
>> > Hi Yedidyah,
>> >
>> > Thank you for the comprehensive answers.
>> >
>> > I think I go for a complete reinstall ( read also OS upgrade tool is not
>> > adviced on 6.6 or higher as there might be newer packages as on 7 ). No
>> > doubting to re-use current VM or setup from scratch ( fresh host with
>> > new
>> > hosted-engine and existing storage domein ).
>>
>> If you are planning to redeploy hosted-engien from scratch and
>> restoring on the new engine DB a backup of the previous one, please
>> carefully consider this:
>> https://bugzilla.redhat.com/show_bug.cgi?id=1240466#c21
>>
>> So, if you choose that path you'll have also to manually remove any
>> reference to the previous hosted-engine from the restored DB.
>> I'd strongly suggest you to wait for the upgrade tool to be fixed
>> since manually doing this upgrade can be really error prone.
>>
>> > You explain the steps ( 1 to 6 ), but then don't talk about storage
>> > domain
>> > import.
>> > Does it mean, when I reinstall the hosted-engine in the current he VM
>> > and
>> > restore an engine-backup ( step 5 ) I am able to start vm from Host and
>> > it
>> > is still connected to the master storage ( so no need for storage
>> > import) ?
>> >
>> > Best Regards,
>> > Paul Groeneweg
>> >
>> >
>> > Op do 30 jun. 2016 om 08:00 schreef Yedidyah Bar David
>> > <didi at redhat.com>:
>> >>
>> >> On Wed, Jun 29, 2016 at 10:07 PM, Paul Groeneweg | Pazion
>> >> <paul at pazion.nl> wrote:
>> >> >
>> >> > I am looking for a way to get my hosted-engine running on el7 so I
>> >> > can
>> >> > upgrade to oVirt 4.0. Currently my hosts already run el7, but my
>> >> > hosted-engine is still el6.
>> >> >
>> >> > I read
>> >> >
>> >> >
>> >> > https://www.ovirt.org/documentation/how-to/hosted-engine-host-OS-upgrade/
>> >> > but this is only about the hosts.
>> >> >
>> >> > I read https://www.ovirt.org/documentation/how-to/hosted-engine/, but
>> >> > it
>> >> > only mentions upgrade of the hosted-engine software, not the OS.
>> >> >
>> >> > I understood I can do a fresh hosted-engine install, and then import
>> >> > my
>> >> > storage domain to the new hosted engine, but:
>> >> >
>> >> > - Do I need to restore my hosted engine database? ( like described
>> >> > here:
>> >> >
>> >> >
>> >> > http://www.ovirt.org/develop/developer-guide/engine/migrate-to-hosted-engine/
>> >> > )
>> >>
>> >> You might not have to, if you only care about the imported VMs from
>> >> your
>> >> storage. This will not keep other configuration, such as
>> >> users/roles/permissions
>> >> etc.
>> >>
>> >> > - Can I directly install hosted-engine 4.0 and then import the
>> >> > storage
>> >> > domain? Or should I install same hosted-engine version?
>> >>
>> >> AFAIK 4.0 engine can import 3.6 storage domains without problem.
>> >>
>> >> > - Do I first need another master storage domain or can I directly
>> >> > import
>> >> > my
>> >> > old master storage domain?
>> >>
>> >> No idea. Even if you do, you can create a small empty one and later
>> >> remove
>> >> it.
>> >>
>> >> > - When importing the storage domain what is the risk it fails ( I
>> >> > have
>> >> > backups, but it would cost a day to restore all )
>> >>
>> >> No idea, but IIRC we got many successful reports and at most few
>> >> failures
>> >> for this.
>> >>
>> >> > - How long would import take? few minutes or hours? ( I want to keep
>> >> > down
>> >> > time as low as possible ).
>> >>
>> >> Again no idea. Perhaps do some test?
>> >>
>> >> >
>> >> > Another option would be upgrade the OS ( with redhat-upgrade-tool )
>> >> > or
>> >> > is
>> >> > this a path for disaster?
>> >>
>> >> Didn't work for us well, so we decided to not support it. If you decide
>> >> to
>> >> try,
>> >> make sure you test carefully beforehand. From ovirt's POV:
>> >> 1. You'll need to handle postgresql upgrade.
>> >> 2. Right after OS upgrade, you'll still have (I think) el6 packages
>> >> of the engine. It will hopefully be in a good-enough state for upgrade
>> >> to 4.0, but we didn't test this.
>> >> 3. Specifically, if upgrade fails, rollback will most likely not work,
>> >> so you'll have to manually handle this - take a full vm backup and make
>> >> sure you can restore it.
>> >>
>> >> >
>> >> > I hope someone can tell me how I can smoothly upgrade my
>> >> > hosted-engine
>> >> > up to
>> >> > el7 and run oVirt 4.
>> >>
>> >> We are working on a tool/wizard to help with this process. It used to
>> >> work,
>> >> but at some point it was decided that one of the actions it does is
>> >> risky
>> >> and was blocked, thus the tool is broken currently.
>> >>
>> >> You can invoke the tool by running: 'hosted-engine
>> >> --upgrade-appliance'.
>> >> As noted above, this is currently broken.
>> >>
>> >> There are several open bugs about it, e.g.:
>> >>
>> >> https://bugzilla.redhat.com/show_bug.cgi?id=1319457
>> >> https://bugzilla.redhat.com/show_bug.cgi?id=1343425
>> >> https://bugzilla.redhat.com/show_bug.cgi?id=1343593 (closed, this is
>> >> what broke the tool)
>> >>
>> >> Basically, you can manually do what the tool is supposed to do:
>> >> 1. Make sure state is clean and stable (no running/pending storage
>> >> actions,
>> >> no VMs in the middle of migration etc), all clusters are compat level
>> >> 3.6,
>> >> etc.
>> >> 2. Move to global maintenance
>> >> 3. backup the engine using engine-backup and keep the backup elsewhere
>> >> 4. Reinstall engine vm with el7 and 4.0 engine (the tool will use the
>> >> engine
>> >> appliance, you might too but not sure how exactly).
>> >> 5. Restore the backup and run engine-setup.
>> >> 6. If all looks ok, leave global maintenance.
>> >>
>> >> If you manually keep a full backup of the engine vm before step 4,
>> >> you might be able to restore this backup if there are problems.
>> >> Doing this in the provided tool is currently the main blocking issue
>> >> for it. Hopefully will be provided in 4.0.1.
>> >>
>> >> Best,
>> >> --
>> >> Didi
>> >
>> >
>> > _______________________________________________
>> > Users mailing list
>> > Users at ovirt.org
>> > http://lists.ovirt.org/mailman/listinfo/users
>> >



-- 
Didi