I'm looking through quite a few bug reports and mailing list threads, but want to make
sure I'm not missing some recent development. It appears that doing iSCSI with two
separate, non-routed subnets is still not possible with 4.4.x. I have the dead-standard
iSCSI setup with two separate switches, separate interfaces on hosts and storage, and
separate subnets that have no gateway and are completely unreachable except from directly
attached interfaces.
The hosted-engine comes up with multiple paths and everything is perfect, but that's
because Ansible/hosted-engine deploy script have configured things correctly. Once you
need to import or add new storage domains, it's not possible to do so in a way that
gets both paths connected *and* persists across host reboots. Following the docs to create
'iSCSI Multipath" bonds (plural, you can _not_ create a single bond that includes
both interfaces and hope things route correctly... oVirt will try to connect from the
interface for storage network A to the target on storage network B, which can't happen
since they are not routed (and should not be). So, there's nothing in the docs about
how you can accomplish multipathing, but there are a few mailing list messages that say
"just create two separate "iSCSI Multipath" bonds in the datacenter, one
for each of your two interfaces. You can do this, and you'll get hopeful that things
might work now. You can do discovery and it succeeds, because no
more trying to connect to unreachable targets. However, and big caveat, there's no
way to tell this new/imported domain, "Oh, use this other interface as well, so you
have redundant paths". Once the domain is attached and activated, you have a single
path. You can then manage the domain, do a discovery, see a path that isn't connected
yet, and log into it as well. Now you have two paths, is everything right with the
world?!? Nope, it's impossible to persist that connection, it will be gone on next
reboot and you'll always have to manually visit each host, do discovery, and login.
Nothing in the UI allows you to "Save" that second connection in a way that it
will be used again. Clicking "OK" does not, and going back to the "iSCSI
Multipath" area of the Data Center you can't edit each of the bonds and make sure
each logical network has every possible target checked, because these targets you've
manually logged into are never listed in that area of the UI.
So I really, really hope I'm wrong because I'd like to move past this snag and
onto the next one (which is that bond interfaces in 4.4.x will not allow you to attach
additional networks... works great in 4.3, appears broken in 4.4.x). But, no sense chasing
that yet if iSCSI multipath isn't possible, which is looking likely.
Has anyone had success, running iSCSI in by far the most common setup out there, but also
in a way oVirt really doesn't want to let you? This is driving me nuts, I've paved
and rebuilt these hosts dozens of times now, trying different methods in the hopes of
getting multipath that persists.