SHE with multipath on multiple iSCSI targets
Dear sirs, we're trying to install an SHE oVirt 4.3.6 using an iSCSI SAN that provides us multiple targets (8) over 4 different portals. Following 5.1 of "RHV installation using Cockpit" we set up iscsi and multipath ahead with a 90GB LUN to be used for the engine VM. Unfortunately, even if the node OS sees correctly the LUN, e.g.: # multipath -ll
36000d31004235a000000000000000018 dm-34 COMPELNT,Compellent Vol size=90G features='1 queue_if_no_path' hwhandler='0' wp=rw `-+- policy='service-time 0' prio=1 status=active |- 16:0:0:3 sdu 65:64 active ready running |- 24:0:0:3 sdt 65:48 active ready running |- 17:0:0:3 sds 65:32 active ready running |- 25:0:0:3 sdv 65:80 active ready running |- 15:0:0:3 sdx 65:112 active ready running |- 29:0:0:3 sdw 65:96 active ready running |- 28:0:0:3 sdy 65:128 active ready running |- 20:0:0:3 sdz 65:144 active ready running `- 21:0:0:3 sdaa 65:160 active ready running
# lsscsi -ist | grep 36000d31004235a000000000000000018
[15:0:0:3] disk iqn.2002-03.com.compellent:5000d31004235a43,t,0x0 /dev/sdx 36000d31004235a000000000000000018 96.6GB [16:0:0:3] disk iqn.2002-03.com.compellent:5000d31004235a43,t,0x0 /dev/sdu 36000d31004235a000000000000000018 96.6GB [17:0:0:3] disk iqn.2002-03.com.compellent:5000d31004235a44,t,0x0 /dev/sds 36000d31004235a000000000000000018 96.6GB [20:0:0:3] disk iqn.2002-03.com.compellent:5000d31004235243,t,0x0 /dev/sdz 36000d31004235a000000000000000018 96.6GB [21:0:0:3] disk iqn.2002-03.com.compellent:5000d31004235244,t,0x0 /dev/sdaa 36000d31004235a000000000000000018 96.6GB [24:0:0:3] disk iqn.2002-03.com.compellent:5000d31004235a48,t,0x0 /dev/sdt 36000d31004235a000000000000000018 96.6GB [25:0:0:3] disk iqn.2002-03.com.compellent:5000d31004235a49,t,0x0 /dev/sdv 36000d31004235a000000000000000018 96.6GB [28:0:0:3] disk iqn.2002-03.com.compellent:5000d31004235248,t,0x0 /dev/sdy 36000d31004235a000000000000000018 96.6GB [29:0:0:3] disk iqn.2002-03.com.compellent:5000d31004235249,t,0x0 /dev/sdw 36000d31004235a000000000000000018 96.6GB
# iscsiadm -m session | grep -E '[a2]4[3489]' | sort -k 3.8
tcp: [1] 10.201.204.10:3260,0 iqn.2002-03.com.compellent:5000d31004235a43 (non-flash) tcp: [2] 10.201.204.10:3260,0 iqn.2002-03.com.compellent:5000d31004235a43 (non-flash) tcp: [3] 10.201.204.10:3260,0 iqn.2002-03.com.compellent:5000d31004235a44 (non-flash) tcp: [6] 10.201.204.20:3260,0 iqn.2002-03.com.compellent:5000d31004235243 (non-flash) tcp: [7] 10.201.204.20:3260,0 iqn.2002-03.com.compellent:5000d31004235244 (non-flash) tcp: [10] 10.201.205.10:3260,0 iqn.2002-03.com.compellent:5000d31004235a48 (non-flash) tcp: [11] 10.201.205.10:3260,0 iqn.2002-03.com.compellent:5000d31004235a49 (non-flash) tcp: [14] 10.201.205.20:3260,0 iqn.2002-03.com.compellent:5000d31004235248 (non-flash) tcp: [15] 10.201.205.20:3260,0 iqn.2002-03.com.compellent:5000d31004235249 (non-flash)
during the engine deploy neither via cockpit nor with "hosted-engine --deploy" it was able to see any LUN. Having a lot of portals (4) and each having 4 targets (two of which being independents path to the LUN), we chose a portal to discover, it proposed us 4 targets, and we had to choose a target to log in, but in the end, no LUN was proposed. After a few attempts, we found that, if we logged out from any targets (and clean up the open-iscsi database), after the login in an appropriate target, the installer proposed us the LUN, and we successfully deployed the SHE. But unfortunately, such an installation doesn't track effectively all the 8 paths over 8 different targets, but just the one chosen during the installation (this is different, for example, with respect to other LUNs for data domains that we attached afterwards). Thus we miss the HA we aimed to for the engine data domain. I said this, because when I ask for /ovirt-engine/api/storagedomains, whilst a second LUN has 8 children with different targets below storage_domain/${another data domain}/storage/volume_group/logical_units/logical_unit/, for the hosted_storage I just have one. My questions are: a) is there a way to add the other targets/paths consistently (even through the REST interface)? b) is there another installation, or recovery, procedure to get the engine data domain in high availability on our iSCSI topology? Thank you so much for your suggestions, they are really appreciated, Best regards, Francesco Castellano
Dear ones, I beg your pardon for coming back on the same issue: let me rephrase more succinctly. A view from the REST API about our data domain is:
$ curl -k -s -u 'admin@internal:supersecret' -H 'Accept: application/json' ' https://engine.example.com/ovirt-engine/api/storagedomains' | jq '[ .["storage_domain"?]|.[0:2]|.[]| {name:.name,storage: .storage | {type:.type, lun: [ .volume_group.logical_units.logical_unit[] | {portal: .portal, target: .target } ]} } ]' [ { "name": "hosted_storage", "storage": { "type": "iscsi", "lun": [ { "portal": "10.201.204.10:3260,0", "target": "iqn.2002-03.com.compellent:5000d31004235a43" } ] } }, { "name": "ovirt-data", "storage": { "type": "iscsi", "lun": [ { "portal": "10.201.205.20:3260,0", "target": "iqn.2002-03.com.compellent:5000d3100423524b" }, { "portal": "10.201.205.20:3260,0", "target": "iqn.2002-03.com.compellent:5000d3100423524a" }, { "portal": "10.201.205.10:3260,0", "target": "iqn.2002-03.com.compellent:5000d31004235a4b" }, { "portal": "10.201.205.10:3260,0", "target": "iqn.2002-03.com.compellent:5000d31004235a4a" }, { "portal": "10.201.204.20:3260,0", "target": "iqn.2002-03.com.compellent:5000d31004235246" }, { "portal": "10.201.204.20:3260,0", "target": "iqn.2002-03.com.compellent:5000d31004235245" }, { "portal": "10.201.204.10:3260,0", "target": "iqn.2002-03.com.compellent:5000d31004235a46" }, { "portal": "10.201.204.10:3260,0", "target": "iqn.2002-03.com.compellent:5000d31004235a45" } ] } } ]
Is there a way to add to the "hosted_storage" all the available path on different iSCSI portals and different iSCSI targets, as I did for the "ovirt-data" storage domain? Thank you very much for your suggestions. Best regards, Francesco Castellano On Wed, Nov 6, 2019 at 2:53 PM Francesco Castellano < francesco.castellano@gmail.com> wrote:
Dear sirs,
we're trying to install an SHE oVirt 4.3.6 using an iSCSI SAN that provides us multiple targets (8) over 4 different portals. Following 5.1 of "RHV installation using Cockpit" we set up iscsi and multipath ahead with a 90GB LUN to be used for the engine VM. Unfortunately, even if the node OS sees correctly the LUN, e.g.:
# multipath -ll
36000d31004235a000000000000000018 dm-34 COMPELNT,Compellent Vol size=90G features='1 queue_if_no_path' hwhandler='0' wp=rw `-+- policy='service-time 0' prio=1 status=active |- 16:0:0:3 sdu 65:64 active ready running |- 24:0:0:3 sdt 65:48 active ready running |- 17:0:0:3 sds 65:32 active ready running |- 25:0:0:3 sdv 65:80 active ready running |- 15:0:0:3 sdx 65:112 active ready running |- 29:0:0:3 sdw 65:96 active ready running |- 28:0:0:3 sdy 65:128 active ready running |- 20:0:0:3 sdz 65:144 active ready running `- 21:0:0:3 sdaa 65:160 active ready running
# lsscsi -ist | grep 36000d31004235a000000000000000018
[15:0:0:3] disk iqn.2002-03.com.compellent:5000d31004235a43,t,0x0 /dev/sdx 36000d31004235a000000000000000018 96.6GB [16:0:0:3] disk iqn.2002-03.com.compellent:5000d31004235a43,t,0x0 /dev/sdu 36000d31004235a000000000000000018 96.6GB [17:0:0:3] disk iqn.2002-03.com.compellent:5000d31004235a44,t,0x0 /dev/sds 36000d31004235a000000000000000018 96.6GB [20:0:0:3] disk iqn.2002-03.com.compellent:5000d31004235243,t,0x0 /dev/sdz 36000d31004235a000000000000000018 96.6GB [21:0:0:3] disk iqn.2002-03.com.compellent:5000d31004235244,t,0x0 /dev/sdaa 36000d31004235a000000000000000018 96.6GB [24:0:0:3] disk iqn.2002-03.com.compellent:5000d31004235a48,t,0x0 /dev/sdt 36000d31004235a000000000000000018 96.6GB [25:0:0:3] disk iqn.2002-03.com.compellent:5000d31004235a49,t,0x0 /dev/sdv 36000d31004235a000000000000000018 96.6GB [28:0:0:3] disk iqn.2002-03.com.compellent:5000d31004235248,t,0x0 /dev/sdy 36000d31004235a000000000000000018 96.6GB [29:0:0:3] disk iqn.2002-03.com.compellent:5000d31004235249,t,0x0 /dev/sdw 36000d31004235a000000000000000018 96.6GB
# iscsiadm -m session | grep -E '[a2]4[3489]' | sort -k 3.8
tcp: [1] 10.201.204.10:3260,0 iqn.2002-03.com.compellent:5000d31004235a43 (non-flash) tcp: [2] 10.201.204.10:3260,0 iqn.2002-03.com.compellent:5000d31004235a43 (non-flash) tcp: [3] 10.201.204.10:3260,0 iqn.2002-03.com.compellent:5000d31004235a44 (non-flash) tcp: [6] 10.201.204.20:3260,0 iqn.2002-03.com.compellent:5000d31004235243 (non-flash) tcp: [7] 10.201.204.20:3260,0 iqn.2002-03.com.compellent:5000d31004235244 (non-flash) tcp: [10] 10.201.205.10:3260,0 iqn.2002-03.com.compellent:5000d31004235a48 (non-flash) tcp: [11] 10.201.205.10:3260,0 iqn.2002-03.com.compellent:5000d31004235a49 (non-flash) tcp: [14] 10.201.205.20:3260,0 iqn.2002-03.com.compellent:5000d31004235248 (non-flash) tcp: [15] 10.201.205.20:3260,0 iqn.2002-03.com.compellent:5000d31004235249 (non-flash)
during the engine deploy neither via cockpit nor with "hosted-engine --deploy" it was able to see any LUN.
Having a lot of portals (4) and each having 4 targets (two of which being independents path to the LUN), we chose a portal to discover, it proposed us 4 targets, and we had to choose a target to log in, but in the end, no LUN was proposed.
After a few attempts, we found that, if we logged out from any targets (and clean up the open-iscsi database), after the login in an appropriate target, the installer proposed us the LUN, and we successfully deployed the SHE.
But unfortunately, such an installation doesn't track effectively all the 8 paths over 8 different targets, but just the one chosen during the installation (this is different, for example, with respect to other LUNs for data domains that we attached afterwards). Thus we miss the HA we aimed to for the engine data domain.
I said this, because when I ask for /ovirt-engine/api/storagedomains, whilst a second LUN has 8 children with different targets below storage_domain/${another data domain}/storage/volume_group/logical_units/logical_unit/, for the hosted_storage I just have one.
My questions are:
a) is there a way to add the other targets/paths consistently (even through the REST interface)? b) is there another installation, or recovery, procedure to get the engine data domain in high availability on our iSCSI topology?
Thank you so much for your suggestions, they are really appreciated,
Best regards, Francesco Castellano
Good day Francesco, If I understand you correctly you can try the following via GUI: 1) Go to "Storage" menu. 2) Go to "Domain" menu. 3) Press "New Domain" button. The "New Domain" window will open. 4) Choose "iSCSI" from the "Storage Type" dropdown list. 5) At the "Discover Targets" section: 5a) For each path: 5a1) Enter path at the "Address" field. 5a2) Press "Discover" button. 5b) Press "Login All" button. Hope it helps. Pavel On Mon, Nov 11, 2019 at 11:48 AM Francesco Castellano < francesco.castellano@gmail.com> wrote:
Dear ones,
I beg your pardon for coming back on the same issue: let me rephrase more succinctly.
A view from the REST API about our data domain is:
$ curl -k -s -u 'admin@internal:supersecret' -H 'Accept: application/json' ' https://engine.example.com/ovirt-engine/api/storagedomains' | jq '[ .["storage_domain"?]|.[0:2]|.[]| {name:.name,storage: .storage | {type:.type, lun: [ .volume_group.logical_units.logical_unit[] | {portal: .portal, target: .target } ]} } ]' [ { "name": "hosted_storage", "storage": { "type": "iscsi", "lun": [ { "portal": "10.201.204.10:3260,0", "target": "iqn.2002-03.com.compellent:5000d31004235a43" } ] } }, { "name": "ovirt-data", "storage": { "type": "iscsi", "lun": [ { "portal": "10.201.205.20:3260,0", "target": "iqn.2002-03.com.compellent:5000d3100423524b" }, { "portal": "10.201.205.20:3260,0", "target": "iqn.2002-03.com.compellent:5000d3100423524a" }, { "portal": "10.201.205.10:3260,0", "target": "iqn.2002-03.com.compellent:5000d31004235a4b" }, { "portal": "10.201.205.10:3260,0", "target": "iqn.2002-03.com.compellent:5000d31004235a4a" }, { "portal": "10.201.204.20:3260,0", "target": "iqn.2002-03.com.compellent:5000d31004235246" }, { "portal": "10.201.204.20:3260,0", "target": "iqn.2002-03.com.compellent:5000d31004235245" }, { "portal": "10.201.204.10:3260,0", "target": "iqn.2002-03.com.compellent:5000d31004235a46" }, { "portal": "10.201.204.10:3260,0", "target": "iqn.2002-03.com.compellent:5000d31004235a45" } ] } } ]
Is there a way to add to the "hosted_storage" all the available path on different iSCSI portals and different iSCSI targets, as I did for the "ovirt-data" storage domain?
Thank you very much for your suggestions.
Best regards, Francesco Castellano
On Wed, Nov 6, 2019 at 2:53 PM Francesco Castellano < francesco.castellano@gmail.com> wrote:
Dear sirs,
we're trying to install an SHE oVirt 4.3.6 using an iSCSI SAN that provides us multiple targets (8) over 4 different portals. Following 5.1 of "RHV installation using Cockpit" we set up iscsi and multipath ahead with a 90GB LUN to be used for the engine VM. Unfortunately, even if the node OS sees correctly the LUN, e.g.:
# multipath -ll
36000d31004235a000000000000000018 dm-34 COMPELNT,Compellent Vol size=90G features='1 queue_if_no_path' hwhandler='0' wp=rw `-+- policy='service-time 0' prio=1 status=active |- 16:0:0:3 sdu 65:64 active ready running |- 24:0:0:3 sdt 65:48 active ready running |- 17:0:0:3 sds 65:32 active ready running |- 25:0:0:3 sdv 65:80 active ready running |- 15:0:0:3 sdx 65:112 active ready running |- 29:0:0:3 sdw 65:96 active ready running |- 28:0:0:3 sdy 65:128 active ready running |- 20:0:0:3 sdz 65:144 active ready running `- 21:0:0:3 sdaa 65:160 active ready running
# lsscsi -ist | grep 36000d31004235a000000000000000018
[15:0:0:3] disk iqn.2002-03.com.compellent:5000d31004235a43,t,0x0 /dev/sdx 36000d31004235a000000000000000018 96.6GB [16:0:0:3] disk iqn.2002-03.com.compellent:5000d31004235a43,t,0x0 /dev/sdu 36000d31004235a000000000000000018 96.6GB [17:0:0:3] disk iqn.2002-03.com.compellent:5000d31004235a44,t,0x0 /dev/sds 36000d31004235a000000000000000018 96.6GB [20:0:0:3] disk iqn.2002-03.com.compellent:5000d31004235243,t,0x0 /dev/sdz 36000d31004235a000000000000000018 96.6GB [21:0:0:3] disk iqn.2002-03.com.compellent:5000d31004235244,t,0x0 /dev/sdaa 36000d31004235a000000000000000018 96.6GB [24:0:0:3] disk iqn.2002-03.com.compellent:5000d31004235a48,t,0x0 /dev/sdt 36000d31004235a000000000000000018 96.6GB [25:0:0:3] disk iqn.2002-03.com.compellent:5000d31004235a49,t,0x0 /dev/sdv 36000d31004235a000000000000000018 96.6GB [28:0:0:3] disk iqn.2002-03.com.compellent:5000d31004235248,t,0x0 /dev/sdy 36000d31004235a000000000000000018 96.6GB [29:0:0:3] disk iqn.2002-03.com.compellent:5000d31004235249,t,0x0 /dev/sdw 36000d31004235a000000000000000018 96.6GB
# iscsiadm -m session | grep -E '[a2]4[3489]' | sort -k 3.8
tcp: [1] 10.201.204.10:3260,0 iqn.2002-03.com.compellent:5000d31004235a43 (non-flash) tcp: [2] 10.201.204.10:3260,0 iqn.2002-03.com.compellent:5000d31004235a43 (non-flash) tcp: [3] 10.201.204.10:3260,0 iqn.2002-03.com.compellent:5000d31004235a44 (non-flash) tcp: [6] 10.201.204.20:3260,0 iqn.2002-03.com.compellent:5000d31004235243 (non-flash) tcp: [7] 10.201.204.20:3260,0 iqn.2002-03.com.compellent:5000d31004235244 (non-flash) tcp: [10] 10.201.205.10:3260,0 iqn.2002-03.com.compellent:5000d31004235a48 (non-flash) tcp: [11] 10.201.205.10:3260,0 iqn.2002-03.com.compellent:5000d31004235a49 (non-flash) tcp: [14] 10.201.205.20:3260,0 iqn.2002-03.com.compellent:5000d31004235248 (non-flash) tcp: [15] 10.201.205.20:3260,0 iqn.2002-03.com.compellent:5000d31004235249 (non-flash)
during the engine deploy neither via cockpit nor with "hosted-engine --deploy" it was able to see any LUN.
Having a lot of portals (4) and each having 4 targets (two of which being independents path to the LUN), we chose a portal to discover, it proposed us 4 targets, and we had to choose a target to log in, but in the end, no LUN was proposed.
After a few attempts, we found that, if we logged out from any targets (and clean up the open-iscsi database), after the login in an appropriate target, the installer proposed us the LUN, and we successfully deployed the SHE.
But unfortunately, such an installation doesn't track effectively all the 8 paths over 8 different targets, but just the one chosen during the installation (this is different, for example, with respect to other LUNs for data domains that we attached afterwards). Thus we miss the HA we aimed to for the engine data domain.
I said this, because when I ask for /ovirt-engine/api/storagedomains, whilst a second LUN has 8 children with different targets below storage_domain/${another data domain}/storage/volume_group/logical_units/logical_unit/, for the hosted_storage I just have one.
My questions are:
a) is there a way to add the other targets/paths consistently (even through the REST interface)? b) is there another installation, or recovery, procedure to get the engine data domain in high availability on our iSCSI topology?
Thank you so much for your suggestions, they are really appreciated,
Best regards, Francesco Castellano
_______________________________________________ Users mailing list -- users@ovirt.org To unsubscribe send an email to users-leave@ovirt.org Privacy Statement: https://www.ovirt.org/site/privacy-policy/ oVirt Code of Conduct: https://www.ovirt.org/community/about/community-guidelines/ List Archives: https://lists.ovirt.org/archives/list/users@ovirt.org/message/QGFAU53JDJIKOT...
Hi Pavel, thanks. This happens to be a prefect way to add a storage domain with all the available paths; but unfortunately, I wanted to add the missing paths for the hosted storage data domain, the data domain hosting the self-hosted engine vm; and I'm not sure how to "change" the configs for hosted-storage domain without the GUI provided by the engine itself. Kind regards, Francesco On Mon, Nov 11, 2019 at 6:10 PM Pavel Bar <pbar@redhat.com> wrote:
Good day Francesco, If I understand you correctly you can try the following via GUI:
1) Go to "Storage" menu. 2) Go to "Domain" menu. 3) Press "New Domain" button. The "New Domain" window will open. 4) Choose "iSCSI" from the "Storage Type" dropdown list. 5) At the "Discover Targets" section: 5a) For each path: 5a1) Enter path at the "Address" field. 5a2) Press "Discover" button. 5b) Press "Login All" button.
Hope it helps.
Pavel
On Mon, Nov 11, 2019 at 11:48 AM Francesco Castellano < francesco.castellano@gmail.com> wrote:
Dear ones,
I beg your pardon for coming back on the same issue: let me rephrase more succinctly.
A view from the REST API about our data domain is:
$ curl -k -s -u 'admin@internal:supersecret' -H 'Accept: application/json' ' https://engine.example.com/ovirt-engine/api/storagedomains' | jq '[ .["storage_domain"?]|.[0:2]|.[]| {name:.name,storage: .storage | {type:.type, lun: [ .volume_group.logical_units.logical_unit[] | {portal: .portal, target: .target } ]} } ]' [ { "name": "hosted_storage", "storage": { "type": "iscsi", "lun": [ { "portal": "10.201.204.10:3260,0", "target": "iqn.2002-03.com.compellent:5000d31004235a43" } ] } }, { "name": "ovirt-data", "storage": { "type": "iscsi", "lun": [ { "portal": "10.201.205.20:3260,0", "target": "iqn.2002-03.com.compellent:5000d3100423524b" }, { "portal": "10.201.205.20:3260,0", "target": "iqn.2002-03.com.compellent:5000d3100423524a" }, { "portal": "10.201.205.10:3260,0", "target": "iqn.2002-03.com.compellent:5000d31004235a4b" }, { "portal": "10.201.205.10:3260,0", "target": "iqn.2002-03.com.compellent:5000d31004235a4a" }, { "portal": "10.201.204.20:3260,0", "target": "iqn.2002-03.com.compellent:5000d31004235246" }, { "portal": "10.201.204.20:3260,0", "target": "iqn.2002-03.com.compellent:5000d31004235245" }, { "portal": "10.201.204.10:3260,0", "target": "iqn.2002-03.com.compellent:5000d31004235a46" }, { "portal": "10.201.204.10:3260,0", "target": "iqn.2002-03.com.compellent:5000d31004235a45" } ] } } ]
Is there a way to add to the "hosted_storage" all the available path on different iSCSI portals and different iSCSI targets, as I did for the "ovirt-data" storage domain?
Thank you very much for your suggestions.
Best regards, Francesco Castellano
On Wed, Nov 6, 2019 at 2:53 PM Francesco Castellano < francesco.castellano@gmail.com> wrote:
Dear sirs,
we're trying to install an SHE oVirt 4.3.6 using an iSCSI SAN that provides us multiple targets (8) over 4 different portals. Following 5.1 of "RHV installation using Cockpit" we set up iscsi and multipath ahead with a 90GB LUN to be used for the engine VM. Unfortunately, even if the node OS sees correctly the LUN, e.g.:
# multipath -ll
36000d31004235a000000000000000018 dm-34 COMPELNT,Compellent Vol size=90G features='1 queue_if_no_path' hwhandler='0' wp=rw `-+- policy='service-time 0' prio=1 status=active |- 16:0:0:3 sdu 65:64 active ready running |- 24:0:0:3 sdt 65:48 active ready running |- 17:0:0:3 sds 65:32 active ready running |- 25:0:0:3 sdv 65:80 active ready running |- 15:0:0:3 sdx 65:112 active ready running |- 29:0:0:3 sdw 65:96 active ready running |- 28:0:0:3 sdy 65:128 active ready running |- 20:0:0:3 sdz 65:144 active ready running `- 21:0:0:3 sdaa 65:160 active ready running
# lsscsi -ist | grep 36000d31004235a000000000000000018
[15:0:0:3] disk iqn.2002-03.com.compellent:5000d31004235a43,t,0x0 /dev/sdx 36000d31004235a000000000000000018 96.6GB [16:0:0:3] disk iqn.2002-03.com.compellent:5000d31004235a43,t,0x0 /dev/sdu 36000d31004235a000000000000000018 96.6GB [17:0:0:3] disk iqn.2002-03.com.compellent:5000d31004235a44,t,0x0 /dev/sds 36000d31004235a000000000000000018 96.6GB [20:0:0:3] disk iqn.2002-03.com.compellent:5000d31004235243,t,0x0 /dev/sdz 36000d31004235a000000000000000018 96.6GB [21:0:0:3] disk iqn.2002-03.com.compellent:5000d31004235244,t,0x0 /dev/sdaa 36000d31004235a000000000000000018 96.6GB [24:0:0:3] disk iqn.2002-03.com.compellent:5000d31004235a48,t,0x0 /dev/sdt 36000d31004235a000000000000000018 96.6GB [25:0:0:3] disk iqn.2002-03.com.compellent:5000d31004235a49,t,0x0 /dev/sdv 36000d31004235a000000000000000018 96.6GB [28:0:0:3] disk iqn.2002-03.com.compellent:5000d31004235248,t,0x0 /dev/sdy 36000d31004235a000000000000000018 96.6GB [29:0:0:3] disk iqn.2002-03.com.compellent:5000d31004235249,t,0x0 /dev/sdw 36000d31004235a000000000000000018 96.6GB
# iscsiadm -m session | grep -E '[a2]4[3489]' | sort -k 3.8
tcp: [1] 10.201.204.10:3260,0 iqn.2002-03.com.compellent:5000d31004235a43 (non-flash) tcp: [2] 10.201.204.10:3260,0 iqn.2002-03.com.compellent:5000d31004235a43 (non-flash) tcp: [3] 10.201.204.10:3260,0 iqn.2002-03.com.compellent:5000d31004235a44 (non-flash) tcp: [6] 10.201.204.20:3260,0 iqn.2002-03.com.compellent:5000d31004235243 (non-flash) tcp: [7] 10.201.204.20:3260,0 iqn.2002-03.com.compellent:5000d31004235244 (non-flash) tcp: [10] 10.201.205.10:3260,0 iqn.2002-03.com.compellent:5000d31004235a48 (non-flash) tcp: [11] 10.201.205.10:3260,0 iqn.2002-03.com.compellent:5000d31004235a49 (non-flash) tcp: [14] 10.201.205.20:3260,0 iqn.2002-03.com.compellent:5000d31004235248 (non-flash) tcp: [15] 10.201.205.20:3260,0 iqn.2002-03.com.compellent:5000d31004235249 (non-flash)
during the engine deploy neither via cockpit nor with "hosted-engine --deploy" it was able to see any LUN.
Having a lot of portals (4) and each having 4 targets (two of which being independents path to the LUN), we chose a portal to discover, it proposed us 4 targets, and we had to choose a target to log in, but in the end, no LUN was proposed.
After a few attempts, we found that, if we logged out from any targets (and clean up the open-iscsi database), after the login in an appropriate target, the installer proposed us the LUN, and we successfully deployed the SHE.
But unfortunately, such an installation doesn't track effectively all the 8 paths over 8 different targets, but just the one chosen during the installation (this is different, for example, with respect to other LUNs for data domains that we attached afterwards). Thus we miss the HA we aimed to for the engine data domain.
I said this, because when I ask for /ovirt-engine/api/storagedomains, whilst a second LUN has 8 children with different targets below storage_domain/${another data domain}/storage/volume_group/logical_units/logical_unit/, for the hosted_storage I just have one.
My questions are:
a) is there a way to add the other targets/paths consistently (even through the REST interface)? b) is there another installation, or recovery, procedure to get the engine data domain in high availability on our iSCSI topology?
Thank you so much for your suggestions, they are really appreciated,
Best regards, Francesco Castellano
_______________________________________________ Users mailing list -- users@ovirt.org To unsubscribe send an email to users-leave@ovirt.org Privacy Statement: https://www.ovirt.org/site/privacy-policy/ oVirt Code of Conduct: https://www.ovirt.org/community/about/community-guidelines/ List Archives: https://lists.ovirt.org/archives/list/users@ovirt.org/message/QGFAU53JDJIKOT...
I have the same issue and haven't been able to find a solution. It seems that the initial LUN is hard coded into /etc/ovirt-hosted-engine/hosted-engine.conf and there's no way to add additional paths to it. ---- On Mon, 11 Nov 2019 17:42:17 +0000 Francesco Castellano <francesco.castellano@gmail.com> wrote ---- Hi Pavel, thanks. This happens to be a prefect way to add a storage domain with all the available paths; but unfortunately, I wanted to add the missing paths for the hosted storage data domain, the data domain hosting the self-hosted engine vm; and I'm not sure how to "change" the configs for hosted-storage domain without the GUI provided by the engine itself. Kind regards, Francesco On Mon, Nov 11, 2019 at 6:10 PM Pavel Bar <mailto:pbar@redhat.com> wrote: Good day Francesco, If I understand you correctly you can try the following via GUI: 1) Go to "Storage" menu. 2) Go to "Domain" menu. 3) Press "New Domain" button. The "New Domain" window will open. 4) Choose "iSCSI" from the "Storage Type" dropdown list. 5) At the "Discover Targets" section: 5a) For each path: 5a1) Enter path at the "Address" field. 5a2) Press "Discover" button. 5b) Press "Login All" button. Hope it helps. Pavel On Mon, Nov 11, 2019 at 11:48 AM Francesco Castellano <mailto:francesco.castellano@gmail.com> wrote: Dear ones, I beg your pardon for coming back on the same issue: let me rephrase more succinctly. A view from the REST API about our data domain is: $ curl -k -s -u 'admin@internal:supersecret' -H 'Accept: application/json' 'https://engine.example.com/ovirt-engine/api/storagedomains' | jq '[ .["storage_domain"?]|.[0:2]|.[]| {name:.name,storage: .storage | {type:.type, lun: [ .volume_group.logical_units.logical_unit[] | {portal: .portal, target: .target } ]} } ]' [ { "name": "hosted_storage", "storage": { "type": "iscsi", "lun": [ { "portal": "http://10.201.204.10:3260,0", "target": "iqn.2002-03.com.compellent:5000d31004235a43" } ] } }, { "name": "ovirt-data", "storage": { "type": "iscsi", "lun": [ { "portal": "http://10.201.205.20:3260,0", "target": "iqn.2002-03.com.compellent:5000d3100423524b" }, { "portal": "http://10.201.205.20:3260,0", "target": "iqn.2002-03.com.compellent:5000d3100423524a" }, { "portal": "http://10.201.205.10:3260,0", "target": "iqn.2002-03.com.compellent:5000d31004235a4b" }, { "portal": "http://10.201.205.10:3260,0", "target": "iqn.2002-03.com.compellent:5000d31004235a4a" }, { "portal": "http://10.201.204.20:3260,0", "target": "iqn.2002-03.com.compellent:5000d31004235246" }, { "portal": "http://10.201.204.20:3260,0", "target": "iqn.2002-03.com.compellent:5000d31004235245" }, { "portal": "http://10.201.204.10:3260,0", "target": "iqn.2002-03.com.compellent:5000d31004235a46" }, { "portal": "http://10.201.204.10:3260,0", "target": "iqn.2002-03.com.compellent:5000d31004235a45" } ] } } ] Is there a way to add to the "hosted_storage" all the available path on different iSCSI portals and different iSCSI targets, as I did for the "ovirt-data" storage domain? Thank you very much for your suggestions. Best regards, Francesco Castellano On Wed, Nov 6, 2019 at 2:53 PM Francesco Castellano <mailto:francesco.castellano@gmail.com> wrote: Dear sirs, we're trying to install an SHE oVirt 4.3.6 using an iSCSI SAN that provides us multiple targets (8) over 4 different portals. Following 5.1 of "RHV installation using Cockpit" we set up iscsi and multipath ahead with a 90GB LUN to be used for the engine VM. Unfortunately, even if the node OS sees correctly the LUN, e.g.: # multipath -ll 36000d31004235a000000000000000018 dm-34 COMPELNT,Compellent Vol size=90G features='1 queue_if_no_path' hwhandler='0' wp=rw `-+- policy='service-time 0' prio=1 status=active |- 16:0:0:3 sdu 65:64 active ready running |- 24:0:0:3 sdt 65:48 active ready running |- 17:0:0:3 sds 65:32 active ready running |- 25:0:0:3 sdv 65:80 active ready running |- 15:0:0:3 sdx 65:112 active ready running |- 29:0:0:3 sdw 65:96 active ready running |- 28:0:0:3 sdy 65:128 active ready running |- 20:0:0:3 sdz 65:144 active ready running `- 21:0:0:3 sdaa 65:160 active ready running # lsscsi -ist | grep 36000d31004235a000000000000000018 [15:0:0:3] disk iqn.2002-03.com.compellent:5000d31004235a43,t,0x0 /dev/sdx 36000d31004235a000000000000000018 96.6GB [16:0:0:3] disk iqn.2002-03.com.compellent:5000d31004235a43,t,0x0 /dev/sdu 36000d31004235a000000000000000018 96.6GB [17:0:0:3] disk iqn.2002-03.com.compellent:5000d31004235a44,t,0x0 /dev/sds 36000d31004235a000000000000000018 96.6GB [20:0:0:3] disk iqn.2002-03.com.compellent:5000d31004235243,t,0x0 /dev/sdz 36000d31004235a000000000000000018 96.6GB [21:0:0:3] disk iqn.2002-03.com.compellent:5000d31004235244,t,0x0 /dev/sdaa 36000d31004235a000000000000000018 96.6GB [24:0:0:3] disk iqn.2002-03.com.compellent:5000d31004235a48,t,0x0 /dev/sdt 36000d31004235a000000000000000018 96.6GB [25:0:0:3] disk iqn.2002-03.com.compellent:5000d31004235a49,t,0x0 /dev/sdv 36000d31004235a000000000000000018 96.6GB [28:0:0:3] disk iqn.2002-03.com.compellent:5000d31004235248,t,0x0 /dev/sdy 36000d31004235a000000000000000018 96.6GB [29:0:0:3] disk iqn.2002-03.com.compellent:5000d31004235249,t,0x0 /dev/sdw 36000d31004235a000000000000000018 96.6GB # iscsiadm -m session | grep -E '[a2]4[3489]' | sort -k 3.8 tcp: [1] http://10.201.204.10:3260,0 iqn.2002-03.com.compellent:5000d31004235a43 (non-flash) tcp: [2] http://10.201.204.10:3260,0 iqn.2002-03.com.compellent:5000d31004235a43 (non-flash) tcp: [3] http://10.201.204.10:3260,0 iqn.2002-03.com.compellent:5000d31004235a44 (non-flash) tcp: [6] http://10.201.204.20:3260,0 iqn.2002-03.com.compellent:5000d31004235243 (non-flash) tcp: [7] http://10.201.204.20:3260,0 iqn.2002-03.com.compellent:5000d31004235244 (non-flash) tcp: [10] http://10.201.205.10:3260,0 iqn.2002-03.com.compellent:5000d31004235a48 (non-flash) tcp: [11] http://10.201.205.10:3260,0 iqn.2002-03.com.compellent:5000d31004235a49 (non-flash) tcp: [14] http://10.201.205.20:3260,0 iqn.2002-03.com.compellent:5000d31004235248 (non-flash) tcp: [15] http://10.201.205.20:3260,0 iqn.2002-03.com.compellent:5000d31004235249 (non-flash) during the engine deploy neither via cockpit nor with "hosted-engine --deploy" it was able to see any LUN. Having a lot of portals (4) and each having 4 targets (two of which being independents path to the LUN), we chose a portal to discover, it proposed us 4 targets, and we had to choose a target to log in, but in the end, no LUN was proposed. After a few attempts, we found that, if we logged out from any targets (and clean up the open-iscsi database), after the login in an appropriate target, the installer proposed us the LUN, and we successfully deployed the SHE. But unfortunately, such an installation doesn't track effectively all the 8 paths over 8 different targets, but just the one chosen during the installation (this is different, for example, with respect to other LUNs for data domains that we attached afterwards). Thus we miss the HA we aimed to for the engine data domain. I said this, because when I ask for /ovirt-engine/api/storagedomains, whilst a second LUN has 8 children with different targets below storage_domain/${another data domain}/storage/volume_group/logical_units/logical_unit/, for the hosted_storage I just have one. My questions are: a) is there a way to add the other targets/paths consistently (even through the REST interface)? b) is there another installation, or recovery, procedure to get the engine data domain in high availability on our iSCSI topology? Thank you so much for your suggestions, they are really appreciated, Best regards, Francesco Castellano _______________________________________________ Users mailing list -- mailto:users@ovirt.org To unsubscribe send an email to mailto:users-leave@ovirt.org Privacy Statement: https://www.ovirt.org/site/privacy-policy/ oVirt Code of Conduct: https://www.ovirt.org/community/about/community-guidelines/ List Archives: https://lists.ovirt.org/archives/list/users@ovirt.org/message/QGFAU53JDJIKOT... _______________________________________________ Users mailing list -- mailto:users@ovirt.org To unsubscribe send an email to mailto:users-leave@ovirt.org Privacy Statement: https://www.ovirt.org/site/privacy-policy/ oVirt Code of Conduct: https://www.ovirt.org/community/about/community-guidelines/ List Archives: https://lists.ovirt.org/archives/list/users@ovirt.org/message/QZZ6A3R4ZYYLFG...
Adding Sandro, Sandro, do you have any ideas? On Tue, Nov 12, 2019 at 1:03 PM Alan G <alan+ovirt@griff.me.uk> wrote:
I have the same issue and haven't been able to find a solution. It seems that the initial LUN is hard coded into /etc/ovirt-hosted-engine/hosted-engine.conf and there's no way to add additional paths to it.
---- On Mon, 11 Nov 2019 17:42:17 +0000 *Francesco Castellano <francesco.castellano@gmail.com <francesco.castellano@gmail.com>>* wrote ----
Hi Pavel,
thanks. This happens to be a prefect way to add a storage domain with all the available paths; but unfortunately, I wanted to add the missing paths for the hosted storage data domain, the data domain hosting the self-hosted engine vm; and I'm not sure how to "change" the configs for hosted-storage domain without the GUI provided by the engine itself.
Kind regards, Francesco
On Mon, Nov 11, 2019 at 6:10 PM Pavel Bar <pbar@redhat.com> wrote:
Good day Francesco, If I understand you correctly you can try the following via GUI:
1) Go to "Storage" menu. 2) Go to "Domain" menu. 3) Press "New Domain" button. The "New Domain" window will open. 4) Choose "iSCSI" from the "Storage Type" dropdown list. 5) At the "Discover Targets" section: 5a) For each path: 5a1) Enter path at the "Address" field. 5a2) Press "Discover" button. 5b) Press "Login All" button.
Hope it helps.
Pavel
On Mon, Nov 11, 2019 at 11:48 AM Francesco Castellano < francesco.castellano@gmail.com> wrote:
Dear ones,
I beg your pardon for coming back on the same issue: let me rephrase more succinctly.
A view from the REST API about our data domain is:
$ curl -k -s -u 'admin@internal:supersecret' -H 'Accept: application/json' ' https://engine.example.com/ovirt-engine/api/storagedomains' | jq '[ .["storage_domain"?]|.[0:2]|.[]| {name:.name,storage: .storage | {type:.type, lun: [ .volume_group.logical_units.logical_unit[] | {portal: .portal, target: .target } ]} } ]' [ { "name": "hosted_storage", "storage": { "type": "iscsi", "lun": [ { "portal": "10.201.204.10:3260,0", "target": "iqn.2002-03.com.compellent:5000d31004235a43" } ] } }, { "name": "ovirt-data", "storage": { "type": "iscsi", "lun": [ { "portal": "10.201.205.20:3260,0", "target": "iqn.2002-03.com.compellent:5000d3100423524b" }, { "portal": "10.201.205.20:3260,0", "target": "iqn.2002-03.com.compellent:5000d3100423524a" }, { "portal": "10.201.205.10:3260,0", "target": "iqn.2002-03.com.compellent:5000d31004235a4b" }, { "portal": "10.201.205.10:3260,0", "target": "iqn.2002-03.com.compellent:5000d31004235a4a" }, { "portal": "10.201.204.20:3260,0", "target": "iqn.2002-03.com.compellent:5000d31004235246" }, { "portal": "10.201.204.20:3260,0", "target": "iqn.2002-03.com.compellent:5000d31004235245" }, { "portal": "10.201.204.10:3260,0", "target": "iqn.2002-03.com.compellent:5000d31004235a46" }, { "portal": "10.201.204.10:3260,0", "target": "iqn.2002-03.com.compellent:5000d31004235a45" } ] } } ]
Is there a way to add to the "hosted_storage" all the available path on different iSCSI portals and different iSCSI targets, as I did for the "ovirt-data" storage domain?
Thank you very much for your suggestions.
Best regards, Francesco Castellano
On Wed, Nov 6, 2019 at 2:53 PM Francesco Castellano < francesco.castellano@gmail.com> wrote:
Dear sirs,
we're trying to install an SHE oVirt 4.3.6 using an iSCSI SAN that provides us multiple targets (8) over 4 different portals. Following 5.1 of "RHV installation using Cockpit" we set up iscsi and multipath ahead with a 90GB LUN to be used for the engine VM. Unfortunately, even if the node OS sees correctly the LUN, e.g.:
# multipath -ll 36000d31004235a000000000000000018 dm-34 COMPELNT,Compellent Vol size=90G features='1 queue_if_no_path' hwhandler='0' wp=rw `-+- policy='service-time 0' prio=1 status=active |- 16:0:0:3 sdu 65:64 active ready running |- 24:0:0:3 sdt 65:48 active ready running |- 17:0:0:3 sds 65:32 active ready running |- 25:0:0:3 sdv 65:80 active ready running |- 15:0:0:3 sdx 65:112 active ready running |- 29:0:0:3 sdw 65:96 active ready running |- 28:0:0:3 sdy 65:128 active ready running |- 20:0:0:3 sdz 65:144 active ready running `- 21:0:0:3 sdaa 65:160 active ready running
# lsscsi -ist | grep 36000d31004235a000000000000000018 [15:0:0:3] disk iqn.2002-03.com.compellent:5000d31004235a43,t,0x0 /dev/sdx 36000d31004235a000000000000000018 96.6GB [16:0:0:3] disk iqn.2002-03.com.compellent:5000d31004235a43,t,0x0 /dev/sdu 36000d31004235a000000000000000018 96.6GB [17:0:0:3] disk iqn.2002-03.com.compellent:5000d31004235a44,t,0x0 /dev/sds 36000d31004235a000000000000000018 96.6GB [20:0:0:3] disk iqn.2002-03.com.compellent:5000d31004235243,t,0x0 /dev/sdz 36000d31004235a000000000000000018 96.6GB [21:0:0:3] disk iqn.2002-03.com.compellent:5000d31004235244,t,0x0 /dev/sdaa 36000d31004235a000000000000000018 96.6GB [24:0:0:3] disk iqn.2002-03.com.compellent:5000d31004235a48,t,0x0 /dev/sdt 36000d31004235a000000000000000018 96.6GB [25:0:0:3] disk iqn.2002-03.com.compellent:5000d31004235a49,t,0x0 /dev/sdv 36000d31004235a000000000000000018 96.6GB [28:0:0:3] disk iqn.2002-03.com.compellent:5000d31004235248,t,0x0 /dev/sdy 36000d31004235a000000000000000018 96.6GB [29:0:0:3] disk iqn.2002-03.com.compellent:5000d31004235249,t,0x0 /dev/sdw 36000d31004235a000000000000000018 96.6GB
# iscsiadm -m session | grep -E '[a2]4[3489]' | sort -k 3.8 tcp: [1] 10.201.204.10:3260,0 iqn.2002-03.com.compellent:5000d31004235a43 (non-flash) tcp: [2] 10.201.204.10:3260,0 iqn.2002-03.com.compellent:5000d31004235a43 (non-flash) tcp: [3] 10.201.204.10:3260,0 iqn.2002-03.com.compellent:5000d31004235a44 (non-flash) tcp: [6] 10.201.204.20:3260,0 iqn.2002-03.com.compellent:5000d31004235243 (non-flash) tcp: [7] 10.201.204.20:3260,0 iqn.2002-03.com.compellent:5000d31004235244 (non-flash) tcp: [10] 10.201.205.10:3260,0 iqn.2002-03.com.compellent:5000d31004235a48 (non-flash) tcp: [11] 10.201.205.10:3260,0 iqn.2002-03.com.compellent:5000d31004235a49 (non-flash) tcp: [14] 10.201.205.20:3260,0 iqn.2002-03.com.compellent:5000d31004235248 (non-flash) tcp: [15] 10.201.205.20:3260,0 iqn.2002-03.com.compellent:5000d31004235249 (non-flash)
during the engine deploy neither via cockpit nor with "hosted-engine --deploy" it was able to see any LUN.
Having a lot of portals (4) and each having 4 targets (two of which being independents path to the LUN), we chose a portal to discover, it proposed us 4 targets, and we had to choose a target to log in, but in the end, no LUN was proposed.
After a few attempts, we found that, if we logged out from any targets (and clean up the open-iscsi database), after the login in an appropriate target, the installer proposed us the LUN, and we successfully deployed the SHE.
But unfortunately, such an installation doesn't track effectively all the 8 paths over 8 different targets, but just the one chosen during the installation (this is different, for example, with respect to other LUNs for data domains that we attached afterwards). Thus we miss the HA we aimed to for the engine data domain.
I said this, because when I ask for /ovirt-engine/api/storagedomains, whilst a second LUN has 8 children with different targets below storage_domain/${another data domain}/storage/volume_group/logical_units/logical_unit/, for the hosted_storage I just have one.
My questions are:
a) is there a way to add the other targets/paths consistently (even through the REST interface)? b) is there another installation, or recovery, procedure to get the engine data domain in high availability on our iSCSI topology?
Thank you so much for your suggestions, they are really appreciated,
Best regards, Francesco Castellano
_______________________________________________ Users mailing list -- users@ovirt.org To unsubscribe send an email to users-leave@ovirt.org Privacy Statement: https://www.ovirt.org/site/privacy-policy/ oVirt Code of Conduct: https://www.ovirt.org/community/about/community-guidelines/ List Archives: https://lists.ovirt.org/archives/list/users@ovirt.org/message/QGFAU53JDJIKOT...
_______________________________________________ Users mailing list -- users@ovirt.org To unsubscribe send an email to users-leave@ovirt.org Privacy Statement: https://www.ovirt.org/site/privacy-policy/ oVirt Code of Conduct: https://www.ovirt.org/community/about/community-guidelines/ List Archives: https://lists.ovirt.org/archives/list/users@ovirt.org/message/QZZ6A3R4ZYYLFG...
_______________________________________________ Users mailing list -- users@ovirt.org To unsubscribe send an email to users-leave@ovirt.org Privacy Statement: https://www.ovirt.org/site/privacy-policy/ oVirt Code of Conduct: https://www.ovirt.org/community/about/community-guidelines/ List Archives: https://lists.ovirt.org/archives/list/users@ovirt.org/message/UJP3ZVC4K3OLB6...
participants (3)
-
Alan G -
Francesco Castellano -
Pavel Bar