I'm running oVirt + Gluster in HCI config and had similar questions as you when
building it out.
- single point of failure in this router (not really - just in case
oVirt is badly broken and I need to access internal vlans to recover it)
There is no SPOF if you're doing 3x HCI nodes. I regularly put 1 of my 3 Nodes into
Maintenance or shutdown Gluster and have had no SPOFs. Are you only doing a single Node?
If so, the point of failure is ... that 1 node :)
* have this router as virtual appliance inside oVirt (something like
pfSense for example)
I'm running pfSense in hardware still (a Netgate ARM device). There's plenty of
opinions on Reddit, StackOverflow, etc. about running any router in VM. There's
several steps you'd need to take when I looked into it, and if you setup pfSense's
interfaces as virtio / vhost I'd imagine you'd bump into limitations b/c those
para devices weren't intended to do things like hardware offload, advanced routing,
etc.; so you may have to setup PCI passthru / SR-IOV to get all of pfSense's routing
capabilities. So I'm keeping pfSense in hardware ... though I've thought of
creating a backup pfSense instance in VM encase of hardware disaster to keep my Internet
up in "limp mode" ... but creating a cellular Hotspot is my current backup plan
:)
Install all hosts and HE with public addresses
Why? The HE is a manager to the cluster and sits on the management network (ovirtmgmt),
so giving it public IPs would be adding a security risk to the setup. I keep my HE
accessible only via local VLAN and that's how most folks lock it down. Are you
thinking the HE or HCI includes a load balancer? Eitherway, oVirt doesn't, but
putting a load balancer in front of VM's and giving it your public IP would make more
sense for exposing things to the Internet ... but I'm assuming too much and don't
know what your cluster will be running.