[ovirt-users] Re: SHE with multipath on multiple iSCSI targets

$ curl -k -s -u 'admin@internal:supersecret' -H 'Accept: application/json' ' https://engine.example.com/ovirt-engine/api/storagedomains' | jq '[ .["storage_domain"?]|.[0:2]|.[]| {name:.name,storage: .storage | {type:.type, lun: [ .volume_group.logical_units.logical_unit[] | {portal: .portal, target: .target } ]} } ]' [ { "name": "hosted_storage", "storage": { "type": "iscsi", "lun": [ { "portal": "10.201.204.10:3260,0", "target": "iqn.2002-03.com.compellent:5000d31004235a43" } ] } }, { "name": "ovirt-data", "storage": { "type": "iscsi", "lun": [ { "portal": "10.201.205.20:3260,0", "target": "iqn.2002-03.com.compellent:5000d3100423524b" }, { "portal": "10.201.205.20:3260,0", "target": "iqn.2002-03.com.compellent:5000d3100423524a" }, { "portal": "10.201.205.10:3260,0", "target": "iqn.2002-03.com.compellent:5000d31004235a4b" }, { "portal": "10.201.205.10:3260,0", "target": "iqn.2002-03.com.compellent:5000d31004235a4a" }, { "portal": "10.201.204.20:3260,0", "target": "iqn.2002-03.com.compellent:5000d31004235246" }, { "portal": "10.201.204.20:3260,0", "target": "iqn.2002-03.com.compellent:5000d31004235245" }, { "portal": "10.201.204.10:3260,0", "target": "iqn.2002-03.com.compellent:5000d31004235a46" }, { "portal": "10.201.204.10:3260,0", "target": "iqn.2002-03.com.compellent:5000d31004235a45" } ] } } ]

Dear sirs, we're trying to install an SHE oVirt 4.3.6 using an iSCSI SAN that provides us multiple targets (8) over 4 different portals. Following 5.1 of "RHV installation using Cockpit" we set up iscsi and multipath ahead with a 90GB LUN to be used for the engine VM. Unfortunately, even if the node OS sees correctly the LUN, e.g.: # multipath -ll > 36000d31004235a000000000000000018 dm-34 COMPELNT,Compellent Vol > size=90G features='1 queue_if_no_path' hwhandler='0' wp=rw > `-+- policy='service-time 0' prio=1 status=active > |- 16:0:0:3 sdu 65:64 active ready running > |- 24:0:0:3 sdt 65:48 active ready running > |- 17:0:0:3 sds 65:32 active ready running > |- 25:0:0:3 sdv 65:80 active ready running > |- 15:0:0:3 sdx 65:112 active ready running > |- 29:0:0:3 sdw 65:96 active ready running > |- 28:0:0:3 sdy 65:128 active ready running > |- 20:0:0:3 sdz 65:144 active ready running > `- 21:0:0:3 sdaa 65:160 active ready running # lsscsi -ist | grep 36000d31004235a000000000000000018 > [15:0:0:3] disk iqn.2002-03.com.compellent:5000d31004235a43,t,0x0 > /dev/sdx 36000d31004235a000000000000000018 96.6GB > [16:0:0:3] disk iqn.2002-03.com.compellent:5000d31004235a43,t,0x0 > /dev/sdu 36000d31004235a000000000000000018 96.6GB > [17:0:0:3] disk iqn.2002-03.com.compellent:5000d31004235a44,t,0x0 > /dev/sds 36000d31004235a000000000000000018 96.6GB > [20:0:0:3] disk iqn.2002-03.com.compellent:5000d31004235243,t,0x0 > /dev/sdz 36000d31004235a000000000000000018 96.6GB > [21:0:0:3] disk iqn.2002-03.com.compellent:5000d31004235244,t,0x0 > /dev/sdaa 36000d31004235a000000000000000018 96.6GB > [24:0:0:3] disk iqn.2002-03.com.compellent:5000d31004235a48,t,0x0 > /dev/sdt 36000d31004235a000000000000000018 96.6GB > [25:0:0:3] disk iqn.2002-03.com.compellent:5000d31004235a49,t,0x0 > /dev/sdv 36000d31004235a000000000000000018 96.6GB > [28:0:0:3] disk iqn.2002-03.com.compellent:5000d31004235248,t,0x0 > /dev/sdy 36000d31004235a000000000000000018 96.6GB > [29:0:0:3] disk iqn.2002-03.com.compellent:5000d31004235249,t,0x0 > /dev/sdw 36000d31004235a000000000000000018 96.6GB > # iscsiadm -m session | grep -E '[a2]4[3489]' | sort -k 3.8 > tcp: [1] 10.201.204.10:3260,0 > iqn.2002-03.com.compellent:5000d31004235a43 (non-flash) > tcp: [2] 10.201.204.10:3260,0 > iqn.2002-03.com.compellent:5000d31004235a43 (non-flash) > tcp: [3] 10.201.204.10:3260,0 > iqn.2002-03.com.compellent:5000d31004235a44 (non-flash) > tcp: [6] 10.201.204.20:3260,0 > iqn.2002-03.com.compellent:5000d31004235243 (non-flash) > tcp: [7] 10.201.204.20:3260,0 > iqn.2002-03.com.compellent:5000d31004235244 (non-flash) > tcp: [10] 10.201.205.10:3260,0 > iqn.2002-03.com.compellent:5000d31004235a48 (non-flash) > tcp: [11] 10.201.205.10:3260,0 > iqn.2002-03.com.compellent:5000d31004235a49 (non-flash) > tcp: [14] 10.201.205.20:3260,0 > iqn.2002-03.com.compellent:5000d31004235248 (non-flash) > tcp: [15] 10.201.205.20:3260,0 > iqn.2002-03.com.compellent:5000d31004235249 (non-flash) > during the engine deploy neither via cockpit nor with "hosted-engine --deploy" it was able to see any LUN. Having a lot of portals (4) and each having 4 targets (two of which being independents path to the LUN), we chose a portal to discover, it proposed us 4 targets, and we had to choose a target to log in, but in the end, no LUN was proposed. After a few attempts, we found that, if we logged out from any targets (and clean up the open-iscsi database), after the login in an appropriate target, the installer proposed us the LUN, and we successfully deployed the SHE. But unfortunately, such an installation doesn't track effectively all the 8 paths over 8 different targets, but just the one chosen during the installation (this is different, for example, with respect to other LUNs for data domains that we attached afterwards). Thus we miss the HA we aimed to for the engine data domain. I said this, because when I ask for /ovirt-engine/api/storagedomains, whilst a second LUN has 8 children with different targets below storage_domain/${another data domain}/storage/volume_group/logical_units/logical_unit/, for the hosted_storage I just have one. My questions are: a) is there a way to add the other targets/paths consistently (even through the REST interface)? b) is there another installation, or recovery, procedure to get the engine data domain in high availability on our iSCSI topology? Thank you so much for your suggestions, they are really appreciated, Best regards, Francesco Castellano