First, it's a scenario the oVirt team seem to actually embrace themselves.
I believe I have seen that type of deployment mentioned either in one of the forward
looking blog posts or Red Hat summit videos.
And then it simply makes a lot of sense to have those crucial management VMs that control
the swarm of resources in a cloud run under a fault-tolerant VM management environment
such as oVirt.
As for nested virtualization, I can only provide some negative experience with oVirt.
I've done nested virtualization with VMware, running ESX on VMWare Workstation and
then some VMs underneath that ESX successfully.
When I tried to do the same with oVirt, running an oVirt hyperconverged cluster as VMs on
VMware workstation, I got reasonably far, but then every VM I tried to launch nested would
just stop right at boot.
It first happened with the hosted-engine, as that tries to start as a VM, but it happened
more obviously when I tried to migrate VMs (just a plain Fedora 30 in this case) from a
physical compute node running oVirt nodeOS to a virtualized compute node, running the same
current oVirt nodeOS image on VMware Workstation (Windows 2016 host, if that matters).
The machine would start the live-migration properly and then just freeze and stay frozen
until I migrated it back or indeed to any other non-virtualized node.
I haven't as yet tried to do that with nested KVM, which I believe might actually
work: But it's on my things to test, so I'll report back, once I get around it.
Actually I am wondering, a) how deep this nesting would be allowed to go and b) if the
'leaf' nodes at the last nesting level should actually have nesting support
activated in their kernel parameters...
Any insight here would be well appreciated.