Deploy Hosted Engine with PowerFlex (formerly ScaleIO) storage
Hi This is the first time I have posted a question on here. I am attempting to deploy a Self Hosted Engine with RHV 4.4.8, RHEL 8.4 using a PowerFlex (ScaleIO) volume. I have done this several times in the past with oVirt 4.3.x and 4.4.? but for some reason on this occasion I am hitting a problem. I think I might know what it is but I don't know how to fix it. The deployment appears to go fine up until the point where the question is asked about which storage device I want the HE copied to, I select Fibre Channel but unfortunately this fails and does not find any suitable devices - I know that PowerFlex is not technically FC but the devices are recognised by multipathd and have always worked previously (I was instrumental in getting the udev rules changed in the PowerFlex 3.6 version to ensure the devices could be recognised by multipathd, whereas previously it had required a udev rules edit). The device I am trying to use is this: # multipath -v2 -l 1abc8aad74a6630f-c4c045df00000000 dm-2 ##,## size=120G features='1 queue_if_no_path' hwhandler='0' wp=rw `-+- policy='service-time 0' prio=0 status=active `- #:#:#:# scinia 252:0 active undef running I can see in the log that the device is being partially discovered but it seems that a 'vendor_id' is wanted and this is something that udev does not report for a PowerFlex device. 2021-11-02 07:07:10,057+0000 INFO otopi.ovirt_hosted_engine_setup.ansible_utils ansible_utils._process_output:112 TASK [ovirt.ovirt.hosted_engine_setup : Get Fibre Channel LUNs] 2021-11-02 07:07:10,960+0000 INFO otopi.ovirt_hosted_engine_setup.ansible_utils ansible_utils._process_output:112 ok: [localhost] 2021-11-02 07:07:11,261+0000 DEBUG otopi.ovirt_hosted_engine_setup.ansible_utils ansible_utils._process_output:106 PLAY RECAP [localhost] : ok: 5 changed: 0 unreachable: 0 skipped: 0 failed: 0 2021-11-02 07:07:11,362+0000 DEBUG otopi.ovirt_hosted_engine_setup.ansible_utils ansible_utils.run:223 ansible-playbook rc: 0 2021-11-02 07:07:11,362+0000 DEBUG otopi.ovirt_hosted_engine_setup.ansible_utils ansible_utils.run:230 ansible-playbook stdout: 2021-11-02 07:07:11,362+0000 DEBUG otopi.ovirt_hosted_engine_setup.ansible_utils ansible_utils.run:233 ansible-playbook stderr: 2021-11-02 07:07:11,362+0000 DEBUG otopi.plugins.gr_he_ansiblesetup.core.storage_domain storage_domain._query_fc_lunid:379 {'otopi_fc_devices': {'changed': False, 'ovirt_host_storages': [{'href': '/ovirt-engine/api/hosts/80308d4a-ef91-40fd-8ee e-60c8314b7e13/storage/19eadf533875920f-46da4fac00000000', 'id': '19eadf533875920f-46da4fac00000000', 'host': {'href': '/ovirt-engine/api/hosts/80308d4a-ef91-40fd-8eee-60c8314b7e13', 'id': '80308d4a-ef91-40fd-8eee-60c8314b7e13'}, 'logical_unit s': [{'discard_max_size': 1048576, 'discard_zeroes_data': False, 'id': '19eadf533875920f-46da4fac00000000', 'lun_mapping': 0, 'paths': 1, 'size': 111669149696, 'status': 'free'}], 'type': 'fcp'}], 'invocation': {'module_args': {'host': 'pfxhci 14.hci.spoc', 'fcp': {'lun_id': -1}, 'fetch_nested': False, 'nested_attributes': [], 'follows': [], 'iscsi': None}}, '_ansible_no_log': False}, 'ansible-playbook_rc': 0} 2021-11-02 07:07:11,362+0000 DEBUG otopi.plugins.gr_he_ansiblesetup.core.storage_domain storage_domain._select_lun:398 [{'href': '/ovirt-engine/api/hosts/80308d4a-ef91-40fd-8eee-60c8314b7e13/storage/19eadf533875920f-46da4fac00000000', 'id': '1 9eadf533875920f-46da4fac00000000', 'host': {'href': '/ovirt-engine/api/hosts/80308d4a-ef91-40fd-8eee-60c8314b7e13', 'id': '80308d4a-ef91-40fd-8eee-60c8314b7e13'}, 'logical_units': [{'discard_max_size': 1048576, 'discard_zeroes_data': False, 'i d': '19eadf533875920f-46da4fac00000000', 'lun_mapping': 0, 'paths': 1, 'size': 111669149696, 'status': 'free'}], 'type': 'fcp'}] 2021-11-02 07:07:11,362+0000 DEBUG otopi.plugins.gr_he_ansiblesetup.core.storage_domain storage_domain._select_lun:407 [{'href': '/ovirt-engine/api/hosts/80308d4a-ef91-40fd-8eee-60c8314b7e13/storage/19eadf533875920f-46da4fac00000000', 'id': '1 9eadf533875920f-46da4fac00000000', 'host': {'href': '/ovirt-engine/api/hosts/80308d4a-ef91-40fd-8eee-60c8314b7e13', 'id': '80308d4a-ef91-40fd-8eee-60c8314b7e13'}, 'logical_units': [{'discard_max_size': 1048576, 'discard_zeroes_data': False, 'i d': '19eadf533875920f-46da4fac00000000', 'lun_mapping': 0, 'paths': 1, 'size': 111669149696, 'status': 'free'}], 'type': 'fcp'}] 2021-11-02 07:07:11,362+0000 DEBUG otopi.context context._executeMethod:145 method exception Traceback (most recent call last): File "/usr/lib/python3.6/site-packages/otopi/context.py", line 132, in _executeMethod method['method']() File "/usr/share/ovirt-hosted-engine-setup/scripts/../plugins/gr-he-ansiblesetup/core/storage_domain.py", line 738, in _closeup lun = self._query_fc_lunid() File "/usr/share/ovirt-hosted-engine-setup/scripts/../plugins/gr-he-ansiblesetup/core/storage_domain.py", line 395, in _query_fc_lunid return self._select_lun(available_luns) File "/usr/share/ovirt-hosted-engine-setup/scripts/../plugins/gr-he-ansiblesetup/core/storage_domain.py", line 418, in _select_lun 'vendorID': entry['logical_units'][0]['vendor_id'], KeyError: 'vendor_id' 2021-11-02 07:07:11,364+0000 ERROR otopi.context context._executeMethod:154 Failed to execute stage 'Closing up': 'vendor_id' 2021-11-02 07:07:11,364+0000 DEBUG otopi.context context.dumpEnvironment:765 ENVIRONMENT DUMP - BEGIN 2021-11-02 07:07:11,364+0000 DEBUG otopi.context context.dumpEnvironment:775 ENV BASE/error=bool:'True' 2021-11-02 07:07:11,365+0000 DEBUG otopi.context context.dumpEnvironment:775 ENV BASE/exceptionInfo=list:'[(<class 'KeyError'>, KeyError('vendor_id',), <traceback object at 0x7ff0f4e6da88>)]' 2021-11-02 07:07:11,365+0000 DEBUG otopi.context context.dumpEnvironment:775 ENV QUESTION/1/OVEHOSTED_STORAGE_DOMAIN_TYPE=str:'fc' Is there any way I can edit something to prevent this vendor_id being required? As I said, this has worked previously so I can only assume something has been added that now checks this vendor_id Many thanks in advance
On Thu, Nov 4, 2021 at 6:32 AM <twseagulls.kmj@gmail.com> wrote:
Hi
This is the first time I have posted a question on here.
Welcome :-).
I am attempting to deploy a Self Hosted Engine with RHV 4.4.8, RHEL 8.4 using a PowerFlex (ScaleIO) volume. I have done this several times in the past with oVirt 4.3.x and 4.4.? but for some reason on this occasion I am hitting a problem. I think I might know what it is but I don't know how to fix it. The deployment appears to go fine up until the point where the question is asked about which storage device I want the HE copied to, I select Fibre Channel but unfortunately this fails and does not find any suitable devices - I know that PowerFlex is not technically FC but the devices are recognised by multipathd and have always worked previously (I was instrumental in getting the udev rules changed in the PowerFlex 3.6 version to ensure the devices could be recognised by multipathd, whereas previously it had required a udev rules edit).
The device I am trying to use is this: # multipath -v2 -l 1abc8aad74a6630f-c4c045df00000000 dm-2 ##,## size=120G features='1 queue_if_no_path' hwhandler='0' wp=rw `-+- policy='service-time 0' prio=0 status=active `- #:#:#:# scinia 252:0 active undef running
I can see in the log that the device is being partially discovered but it seems that a 'vendor_id' is wanted and this is something that udev does not report for a PowerFlex device.
2021-11-02 07:07:10,057+0000 INFO otopi.ovirt_hosted_engine_setup.ansible_utils ansible_utils._process_output:112 TASK [ovirt.ovirt.hosted_engine_setup : Get Fibre Channel LUNs] 2021-11-02 07:07:10,960+0000 INFO otopi.ovirt_hosted_engine_setup.ansible_utils ansible_utils._process_output:112 ok: [localhost] 2021-11-02 07:07:11,261+0000 DEBUG otopi.ovirt_hosted_engine_setup.ansible_utils ansible_utils._process_output:106 PLAY RECAP [localhost] : ok: 5 changed: 0 unreachable: 0 skipped: 0 failed: 0 2021-11-02 07:07:11,362+0000 DEBUG otopi.ovirt_hosted_engine_setup.ansible_utils ansible_utils.run:223 ansible-playbook rc: 0 2021-11-02 07:07:11,362+0000 DEBUG otopi.ovirt_hosted_engine_setup.ansible_utils ansible_utils.run:230 ansible-playbook stdout: 2021-11-02 07:07:11,362+0000 DEBUG otopi.ovirt_hosted_engine_setup.ansible_utils ansible_utils.run:233 ansible-playbook stderr: 2021-11-02 07:07:11,362+0000 DEBUG otopi.plugins.gr_he_ansiblesetup.core.storage_domain storage_domain._query_fc_lunid:379 {'otopi_fc_devices': {'changed': False, 'ovirt_host_storages': [{'href': '/ovirt-engine/api/hosts/80308d4a-ef91-40fd-8ee e-60c8314b7e13/storage/19eadf533875920f-46da4fac00000000', 'id': '19eadf533875920f-46da4fac00000000', 'host': {'href': '/ovirt-engine/api/hosts/80308d4a-ef91-40fd-8eee-60c8314b7e13', 'id': '80308d4a-ef91-40fd-8eee-60c8314b7e13'}, 'logical_unit s': [{'discard_max_size': 1048576, 'discard_zeroes_data': False, 'id': '19eadf533875920f-46da4fac00000000', 'lun_mapping': 0, 'paths': 1, 'size': 111669149696, 'status': 'free'}], 'type': 'fcp'}], 'invocation': {'module_args': {'host': 'pfxhci 14.hci.spoc', 'fcp': {'lun_id': -1}, 'fetch_nested': False, 'nested_attributes': [], 'follows': [], 'iscsi': None}}, '_ansible_no_log': False}, 'ansible-playbook_rc': 0} 2021-11-02 07:07:11,362+0000 DEBUG otopi.plugins.gr_he_ansiblesetup.core.storage_domain storage_domain._select_lun:398 [{'href': '/ovirt-engine/api/hosts/80308d4a-ef91-40fd-8eee-60c8314b7e13/storage/19eadf533875920f-46da4fac00000000', 'id': '1 9eadf533875920f-46da4fac00000000', 'host': {'href': '/ovirt-engine/api/hosts/80308d4a-ef91-40fd-8eee-60c8314b7e13', 'id': '80308d4a-ef91-40fd-8eee-60c8314b7e13'}, 'logical_units': [{'discard_max_size': 1048576, 'discard_zeroes_data': False, 'i d': '19eadf533875920f-46da4fac00000000', 'lun_mapping': 0, 'paths': 1, 'size': 111669149696, 'status': 'free'}], 'type': 'fcp'}] 2021-11-02 07:07:11,362+0000 DEBUG otopi.plugins.gr_he_ansiblesetup.core.storage_domain storage_domain._select_lun:407 [{'href': '/ovirt-engine/api/hosts/80308d4a-ef91-40fd-8eee-60c8314b7e13/storage/19eadf533875920f-46da4fac00000000', 'id': '1 9eadf533875920f-46da4fac00000000', 'host': {'href': '/ovirt-engine/api/hosts/80308d4a-ef91-40fd-8eee-60c8314b7e13', 'id': '80308d4a-ef91-40fd-8eee-60c8314b7e13'}, 'logical_units': [{'discard_max_size': 1048576, 'discard_zeroes_data': False, 'i d': '19eadf533875920f-46da4fac00000000', 'lun_mapping': 0, 'paths': 1, 'size': 111669149696, 'status': 'free'}], 'type': 'fcp'}] 2021-11-02 07:07:11,362+0000 DEBUG otopi.context context._executeMethod:145 method exception Traceback (most recent call last): File "/usr/lib/python3.6/site-packages/otopi/context.py", line 132, in _executeMethod method['method']() File "/usr/share/ovirt-hosted-engine-setup/scripts/../plugins/gr-he-ansiblesetup/core/storage_domain.py", line 738, in _closeup lun = self._query_fc_lunid() File "/usr/share/ovirt-hosted-engine-setup/scripts/../plugins/gr-he-ansiblesetup/core/storage_domain.py", line 395, in _query_fc_lunid return self._select_lun(available_luns) File "/usr/share/ovirt-hosted-engine-setup/scripts/../plugins/gr-he-ansiblesetup/core/storage_domain.py", line 418, in _select_lun 'vendorID': entry['logical_units'][0]['vendor_id'], KeyError: 'vendor_id' 2021-11-02 07:07:11,364+0000 ERROR otopi.context context._executeMethod:154 Failed to execute stage 'Closing up': 'vendor_id' 2021-11-02 07:07:11,364+0000 DEBUG otopi.context context.dumpEnvironment:765 ENVIRONMENT DUMP - BEGIN 2021-11-02 07:07:11,364+0000 DEBUG otopi.context context.dumpEnvironment:775 ENV BASE/error=bool:'True' 2021-11-02 07:07:11,365+0000 DEBUG otopi.context context.dumpEnvironment:775 ENV BASE/exceptionInfo=list:'[(<class 'KeyError'>, KeyError('vendor_id',), <traceback object at 0x7ff0f4e6da88>)]' 2021-11-02 07:07:11,365+0000 DEBUG otopi.context context.dumpEnvironment:775 ENV QUESTION/1/OVEHOSTED_STORAGE_DOMAIN_TYPE=str:'fc'
Is there any way I can edit something to prevent this vendor_id being required?
It should be rather simple to patch the deploy code not fail on missing vendor_id, it's used only for reporting. But it might not be required, for now. You can try instead something like: hosted-engine --deploy --otopi-environment=OVEHOSTED_STORAGE/LunID=str:19eadf533875920f-46da4fac00000000 I didn't try this, not sure it would be enough. If it's not, you might check the generated answer file from an old successful run and copy and edit some more relevant lines (those with OVEHOSTED_STORAGE), put in an answer file, and pass it with --config-append=answerfile.
As I said, this has worked previously so I can only assume something has been added that now checks this vendor_id
No, the code checking vendor_id didn't change for at least 3 years. Do you still have setup logs of successful deployments with this storage (or as similar as possible)? Can you check them? Search for otopi_fc_devices? It sounds either as some change on the storage side (does not report vendor id anymore) or on the engine/sdk/ansible-module/whatever (not passing it anymore). Nikolai, did you test FC HE deploy? Can you check logs? Thanks and best regards, -- Didi
Hi Many thanks for your response and the suggestion. I think my memory may be playing tricks with me, I definitely deployed a few 4.3 systems with self hosted engines and I have one 4.4 environment with a self hosted engine but I cannot remember how long ago it was deployed, the other 4.4 deployments may well have had the engine residing on a dedicated host or VM - too many deployments to remember them all clearly :-) Anyway, I did manage to fix (work around?) the problem, almost before you send your response. I worked out that the productID and vendorID are merely used for reporting the details of the discovered FC LUN, by removing any reference to them in /usr/share/ovirt-hosted-engine-setup/plugins/gr-he-ansiblesetup/core/storage_domain.py the deployment completed successfully. As far as I know, the reporting from the storage has not changed, running multipath -v2 -l never showed a vendorID or productID (assuming it is the multipath command that is used to gather this info). Thanks for the help!! I will be back I am sure. If anyone has questions on PowerFlex/ScaleIO, don't be afraid to ask, it works a treat with oVirt/RHV/Oracle Linux KVM :-)
participants (3)
-
Kevin Jones -
twseagulls.kmj＠gmail.com
-
Yedidyah Bar David