2:24 a.m.
Hi,
On Sun, Sep 5, 2021 at 9:19 AM <erdosi.peter(a)kifu.gov.hu> wrote:
Hy!
I've just finished the process in the subject, and want to share a few things with
others, and also want to ask a question.
Thanks :-)
Long story "short" (it's my IT infra@home, I've oVirt in bigger setups
at work): I had 3 machines before: two desktop, and one Dell R710 server. One of the
desktop running FreeNAS with an SSD RAID1, the other desktop and my Dell machine was my
two hypervisors. Luckily, i've just got two HP DL380 G6, which become my hypervisors,
and my Dell machine became the TrueNAS storage (redundant PSU, more core, more RAM, more
disk :) ).
When I started the procedure, I used the latest oVirt 4.3, but it was the time to upgrade
the version, and also want to migrage all my Data (with the self hosted engine too) to the
TrueNAS Dell machine (but... I've only have the two SSD on my old FreeNAS, so it has
to be moved)
After I replaced the hypervisors, and migrated all my VM data to the new storage to new
disks (iSCSI->iSCSI, live and cold storage migration) it was the time, to shut off the
FreeNAS, and start the HE redeploy.
The main steps, what I take:
- Undeployed the hosted-engine from the host with ID: 2 (the ID came from hosted-engine
--vm-status command, the machine name is: Jupiter)
- Moved all my VMs to this host, only the HE remained on the machine with ID: 1 (name:
Saturn)
- Removed the remained stuff with: "hosted-engine --clean-metadata --host-id=2
--force-clean", so the Saturn was the only machine capable of running the HE
- Set Global maintenance mode=true
- Stop the ovirt-engine in the old HE machine
- Create a backup with "engine-backup --mode=backup --file=engine.backup
--log=engine-backup.log" and copy it to an another machine (my desktop actually)
- "hosted-engine --vm-shutdown"
- Saturn shutdown, 4.4.6 installer in (it was written to DVD before, don't want to
create another) Saturn complete reinstall.
- FreeNAS shutdown, SSD moved to TrueNAS, RAID 1/zVol create, iSCSI export...
- After the base network was created, and the backup was moved back to the new Saturn, I
started the new host deploy with: "hosted-engine --deploy
--restore-from-file=engine.backup"
Catch 1 and 2: If you do this, you must know the old DC and the Cluster name!
Are you sure you _must_? I think it will create new DC/cluster for you
if you type new names. Obviously, this might not be what you want.
So probably what you mean is something like:
If I restore from backup, I think the tool should check the DC/cluster
names in the backup, and let me select among them, perhaps even try to
intelligently guess the one that should be default (or just pick one
at random).
I agree this makes sense, you are welcome to file an RFE bug.
Not sure it will be prioritized - patches are welcome!
write it down somewhere! or you need to get out the PSQL dbs from the
backup, and extract from it... (like i had to)
Would also be useful if you detail your commands as a workaround, if
you file such a bug.
The process goes almost flawless, but at some point, I've got a
deployment error (code: 519) which tells me, the Saturn have missing networks, and I can
connect to https://saturn.mope.local:6900/ovirt-engine
Catch 3: this URL not worked, since it was not the HE's URL, and I cannot login
because the FQDN checking...
The deploy process temporarily configures the engine to allow using
the host's FQDN to connect to the engine, and forwards port 6900 for
you.
There might have been other complications I do not understand that
prevented this from working well.
What happened when you tried?
If you think this is a real bug, and can reliably reproduce it, please
file one in bugzilla. Thanks!
This may need some further improvement ;) After some thinking,
finally I've made a socks proxy with SSH (1080 forwarded to Saturn) and I was able to
login to the locally running HE and make the network changes, which was required to
continue the deploy process... Also (since the old FreeNAS box was on my desk, the HE and
the two hypervisor was unable to connect to my old SSD iSCSI, so I have to remove it...
I understand this was somewhat annoying, but you have to realize that
there are many different backup/restore/upgrade/migration scenarios,
and it's not easy to decide what's best to do.
In your specific case, if you are confident that you'll not need to be
able to abort in the middle and revert to the old engine+storage, you
could have removed it from the engine before taking the backup.
(cannot put on maintenance, but I was able to delete it from
Storage->Domains tab) After this, Saturn came up, got the "green arrow", so I
removed the lock file which the deploy give me, and the deploy continued...
After this, I selected the new SSD RAID1 on my Dell iSCSI box, and the deploy was finally
able to copy the HE to the iSCSI, so far so good :)
Finally, I've got my new 4.4.6 setup, with Self hosted HE on my new TrueNAS@Dell.
(and all my VMs running on Jupiter at this time, without any errors)
The next step was to live migrate off Jupiter to Saturn, DVD in, Jupiter remove from the
Cluster, and reinstall to 4.4.6, and readd Jupiter (this time, with HE deployed again)
After I've put back Jupiter, and made the required initial setup with the network
(VLANs pulled to the bonds, iSCSI IPs set, etc) the cluster was up and running.
The next step was to upgrade the hypervisors to the latest image with rolling update, it
was worked as before, so the time come to move the cluster and DC compatilbility level
from 4.3 -> 4.6... This forced me to reboot all my VMs, as it written when I made this
change, but this worked too. At last: I've restarted the HE with hosted-engine
--vm-shutdown / --vm-start, because it has some "!" at the BIOS version...
And this is my question actually: after the restart, the BIOS version of the HE machine
remained the old one, and still have the "!" which states: "The
chipset/firmware type doen not match the cluster chipset/firmware type"
Doen anyone know, how can the HE BIOS updated after compatibility leve increase?
Sorry, not sure. Perhaps you can try to edit the VM from the web admin
ui. Adding Arik.
Generally speaking:
1. The VM is created fresh, unrelated to anything in your backup, IMO.
2. So this is perhaps a result of starting from 4.4.6. To prevent
this, you could have upgraded the host to latest version *before*
starting the deploy/restore. But I might be wrong.
Sorry, if my mail should be goes to the users list, but because of "Catch 3" I
think is's better here.
Generally speaking it should have been on the users list - most bugs
are reported there, not here. But that's ok :-). This list is more for
developers. E.g. If you decide to write a patch for one of the above
issues, you might want to discuss it here if needed.
Also I want to write this down, maybe someone find usefull, and this procedure can be
used in the Docs too, if you want!
I think most of what you wrote is already covered by docs, no?
You might want to also check RHV docs (right now they tend to be more
up-to-date than oVirt), and also open docs bugs in bugzilla. If you
think there is something concrete missing, please file a docs bug (and
if it's missing in oVirt and exists in RHV, that's not a reason to not
file the bug - from oVirt's POV, the bug exists!).
Thanks and best regards,
--
Didi