- Users - oVirt List Archives

"Add NFS storage domain", "status": "FAILED",
by info＠worldhostess.com 10 Sep '20

10 Sep '20

I am new to oVirt and this is my first installation https://ovirt.org/documentation/installing_ovirt_as_a_self-hosted_engine_us ing_the_cockpit_web_interface/ Everything seems fine, but then when I tried step 5, "Hosted Engine Deployment" it keep giving me this error. This information might be too much, but I am not familiar to know what is important. 2020-09-08 06:53:58,991+0200 DEBUG var changed: host "localhost" var "datacenter_name" type "<class 'ansible.utils.unsafe_proxy.AnsibleUnsafeText'>" value: ""Default"" 2020-09-08 06:53:58,991+0200 INFO ansible ok {'status': 'OK', 'ansible_type': 'task', 'ansible_playbook': '/usr/share/ovirt-hosted-engine-setup/ansible/trigger_role.yml', 'ansible_host': 'localhost', 'ansible_task': 'Fetch Datacenter name', 'task_duration': 0} 2020-09-08 06:53:58,991+0200 DEBUG ansible on_any args <ansible.executor.task_result.TaskResult object at 0x7f825fb425c0> kwargs 2020-09-08 06:53:59,242+0200 INFO ansible task start {'status': 'OK', 'ansible_type': 'task', 'ansible_playbook': '/usr/share/ovirt-hosted-engine-setup/ansible/trigger_role.yml', 'ansible_task': 'ovirt.hosted_engine_setup : Add NFS storage domain'} 2020-09-08 06:53:59,242+0200 DEBUG ansible on_any args TASK: ovirt.hosted_engine_setup : Add NFS storage domain kwargs is_conditional:False 2020-09-08 06:53:59,242+0200 DEBUG ansible on_any args localhostTASK: ovirt.hosted_engine_setup : Add NFS storage domain kwargs 2020-09-08 06:56:04,962+0200 DEBUG var changed: host "localhost" var "ansible_play_hosts" type "<class 'list'>" value: "[]" 2020-09-08 06:56:04,963+0200 DEBUG var changed: host "localhost" var "ansible_play_batch" type "<class 'list'>" value: "[]" 2020-09-08 06:56:04,963+0200 DEBUG var changed: host "localhost" var "play_hosts" type "<class 'list'>" value: "[]" 2020-09-08 06:56:04,963+0200 ERROR ansible failed { "ansible_host": "localhost", "ansible_playbook": "/usr/share/ovirt-hosted-engine-setup/ansible/trigger_role.yml", "ansible_result": { "_ansible_no_log": false, "changed": false, "exception": "Traceback (most recent call last):\n File \"/tmp/ansible_ovirt_storage_domain_payload_ir_v98cz/ansible_ovirt_storage_d omain_payload.zip/ansible/modules/cloud/ovirt/ovirt_storage_domain.py\", line 792, in main\n File \"/tmp/ansible_ovirt_storage_domain_payload_ir_v98cz/ansible_ovirt_storage_d omain_payload.zip/ansible/module_utils/ovirt.py\", line 623, in create\n **kwargs\n File \"/usr/lib64/python3.6/site-packages/ovirtsdk4/services.py\", line 26097, in add\n return self._internal_add(storage_domain, headers, query, wait)\n File \"/usr/lib64/python3.6/site-packages/ovirtsdk4/service.py\", line 232, in _internal_add\n return future.wait() if wait else future\n File \"/usr/lib64/python3.6/site-packages/ovirtsdk4/service.py\", line 55, in wait\n return self._code(response)\n File \"/usr/lib64/python3.6/site-packages/ovirtsdk4/service.py\", line 229, in callback\n self._check_fault(response)\n File \"/usr/lib64/python3.6/site-packages/ovirtsdk4/service.py\", line 132, in _check_fault\n self._raise_error(response, body)\n File \"/usr/lib64/python3.6/site-packages/ovirtsdk4/service.py\", line 118, in _raise_error\n raise error\novirtsdk4.Error: Fault reason is \"Operation Failed\". Fault detail is \"[Problem while trying to mount target]\". HTTP response code is 400.\n", "invocation": { "module_args": { "backup": null, "comment": null, "critical_space_action_blocker": null, "data_center": "Default", "description": null, "destroy": null, "discard_after_delete": null, "domain_function": "data", "fcp": null, "fetch_nested": false, "format": null, "glusterfs": null, "host": "host.worldhostess.co.za", "id": null, "iscsi": null, "localfs": null, "managed_block_storage": null, "name": "hosted_storage", "nested_attributes": [], "nfs": { "address": "storage01.worldhostess.co.za", "mount_options": null, "path": "/data/nfs", "version": "auto" }, "poll_interval": 3, "posixfs": null, "state": "unattached", "timeout": 180, "wait": true, "warning_low_space": null, "wipe_after_delete": null } }, "msg": "Fault reason is \"Operation Failed\". Fault detail is \"[Problem while trying to mount target]\". HTTP response code is 400." }, "ansible_task": "Add NFS storage domain", "ansible_type": "task", "status": "FAILED", "task_duration": 125 } 2020-09-08 06:56:04,963+0200 DEBUG ansible on_any args <ansible.executor.task_result.TaskResult object at 0x7f825f9c3a58> kwargs ignore_errors:None 2020-09-08 06:56:04,964+0200 INFO ansible stats { "ansible_playbook": "/usr/share/ovirt-hosted-engine-setup/ansible/trigger_role.yml", "ansible_playbook_duration": "02:16 Minutes", "ansible_result": "type: <class 'dict'>\nstr: {'localhost': {'ok': 14, 'failures': 1, 'unreachable': 0, 'changed': 0, 'skipped': 2, 'rescued': 0, 'ignored': 0}}", "ansible_type": "finish", "status": "FAILED" } 2020-09-08 06:56:04,964+0200 INFO SUMMARY: Duration Task Name -------- -------- [ < 1 sec ] Execute just a specific set of steps [ 00:01 ] Force facts gathering [ < 1 sec ] Check local VM dir stat [ < 1 sec ] Obtain SSO token using username/password credentials [ < 1 sec ] Fetch host facts [ < 1 sec ] Fetch cluster ID [ < 1 sec ] Fetch cluster facts [ < 1 sec ] Fetch Datacenter facts [ < 1 sec ] Fetch Datacenter ID [ < 1 sec ] Fetch Datacenter name [ FAILED ] Add NFS storage domain 2020-09-08 06:56:04,965+0200 DEBUG ansible on_any args <ansible.executor.stats.AggregateStats object at 0x7f826252ef60> kwargs Yours Sincerely, Dr Henni Appelgryn PhD DBA MBA AEP MDP WorldHostess Skype: genie_za WeChat: HenniAppelgryn LinkedIn: https://www.linkedin.com/in/henni-appelgryn/ Hosting with a difference

4 7

Failed to create a new VM in ovirt due to "The host did not satisfy internal filter Memory" error
by KISHOR K 10 Sep '20

10 Sep '20

Hi All, I'm new to Ovirt and not having a perfect experience with Ovirt yet. I ran into a strange issue today when I tried to create a new VM with 32GB in ovirt. VM creation basically due to an error, pasted below. Cannot run VM. There is no host that satisfies current scheduling constraints. See below for details:, The host host-01 did not satisfy internal filter Memory., The host host-01 did not satisfy internal filter Memory. After some troubleshooting, I found that there was enough available memory (around 120 GB free ) in the host to host new VM and there were around 10 VMs already running on this host. But, I noticed that ovirt is actually scheduling and creating VM based on "Max free Memory for scheduling new VMs" value, which seems to be set/updated based on total allocated memory for all VMs running on the host and it's not really the then consumed memory by running VMs. Can anyone help to explain if it is some kind of bug in ovirt or it expected behavior? If it is expected behavior, is there any possibility to change it to make sure VM is created based on actual free memory? Thanks a lot in advance for your support. /Kishore /Kishore

4 8

Diskless oVirt Nodes
by harryo.dk＠gmail.com 10 Sep '20

10 Sep '20

Hi, why does my ovirt hypervisor node crash when i pull the hypervisor boot drive? The following thread claims that the oVirt node hypervisor should lodad into memory, so boot disk failure shouldn't have any impact on a running node. https://lists.ovirt.org/pipermail/users/2014-December/029771.html Only memory failure should have a impact, and if RAM fails there is a possibility that the hypervisor OS would have crashed regardless. If you load the hypervisor OS into memory there will be moved/combined a point of failure (the boot disk) with another(the RAM). RAM=memory

3 6

Hyperconverged set up with Gluster and 6 Nodes
by Holger Petrick 09 Sep '20

09 Sep '20

Dear All, in all documentation to set up oVirt in a Hyperconverged system with Gluster says that it must have 3 Enterprise nodes. In our test environment we set up successfully with oVirt wizard an oVirt cluster with gluster + Arbiter and hosted engine. Is it also possible to build a cluster with 6 nodes? The wizard gives only the option to add 3 nodes. Is this kind of design supported? My idea is, first to set up the Gluster, then install the oVirt nodes and deploy the engine. Would this work? Any documentation avaiable? Thanks Holger

3 5

Hosted engine install failure: ipv6.gateway: gateway cannot be set if there are no addresses configured
by Sverker Abrahamsson 08 Sep '20

08 Sep '20

Hi, I'm attempting to install hosted engine but getting this failure: [ ERROR ] fatal: [localhost]: FAILED! => {"changed": false, "msg": "The host has been set in non_operational status, deployment errors: code 505: Host h1-mgmt.limetransit.com installation failed. Failed to configure management network on the host., code 1120: Failed to configure management network on host h1-mgmt.limetransit.com due to setup networks failure., code 9000: Failed to verify Power Management configuration for Host h1-mgmt.limetransit.com., code 10802: VDSM h1-mgmt.limetransit.com command HostSetupNetworksVDS failed: Internal JSON-RPC error: {'reason': 'Unexpected failure of libnm when running the mainloop: run execution'}, fix accordingly and re-deploy."} Looking in vdsm.log I find this which I believe is the root cause: MainProcess|jsonrpc/0::DEBUG::2020-09-02 16:38:25,897::supervdsm_server::93::SuperVdsm.ServerCallback::(wrapper) call setupNetworks with ({'ovirtmgmt': {'vlan': '4000', 'netmask': '255.255.255.0', 'ipv6autoconf': False, 'nic': 'enp4s0', 'bridged': 'true', 'ipaddr': '172.27.1.1', 'defaultRoute': True, 'dhcpv6': False, 'STP': 'no', 'mtu': 1500, 'switch': 'legacy'}}, {}, {'connectivityTimeout': 120, 'commitOnSuccess': True, 'connectivityCh eck': 'true'}) {} MainProcess|jsonrpc/0::INFO::2020-09-02 16:38:25,897::api::220::root::(setupNetworks) Setting up network according to configuration: networks:{'ovirtmgmt': {'vlan': '4000', 'netmask': '255.255.255.0', 'ipv6autoconf': Fal se, 'nic': 'enp4s0', 'bridged': 'true', 'ipaddr': '172.27.1.1', 'defaultRoute': True, 'dhcpv6': False, 'STP': 'no', 'mtu': 1500, 'switch': 'legacy'}}, bondings:{}, options:{'connectivityTimeout': 120, 'commitOnSuccess': True, 'connectivityCheck': 'true'} MainProcess|jsonrpc/0::DEBUG::2020-09-02 16:38:25,902::cmdutils::130::root::(exec_cmd) /sbin/tc qdisc show (cwd None) MainProcess|jsonrpc/0::DEBUG::2020-09-02 16:38:25,906::cmdutils::138::root::(exec_cmd) SUCCESS: <err> = b''; <rc> = 0 MainProcess|jsonrpc/0::DEBUG::2020-09-02 16:38:25,945::vsctl::74::root::(commit) Executing commands: /usr/bin/ovs-vsctl --timeout=5 --oneline --format=json -- list Bridge -- list Port -- list Interface MainProcess|jsonrpc/0::DEBUG::2020-09-02 16:38:25,945::cmdutils::130::root::(exec_cmd) /usr/bin/ovs-vsctl --timeout=5 --oneline --format=json -- list Bridge -- list Port -- list Interface (cwd None) MainProcess|jsonrpc/0::DEBUG::2020-09-02 16:38:25,952::cmdutils::138::root::(exec_cmd) SUCCESS: <err> = b''; <rc> = 0 MainProcess|jsonrpc/0::INFO::2020-09-02 16:38:25,957::netconfpersistence::58::root::(setNetwork) Adding network ovirtmgmt({'vlan': 4000, 'netmask': '255.255.255.0', 'ipv6autoconf': False, 'nic': 'enp4s0', 'bridged': True , 'ipaddr': '172.27.1.1', 'defaultRoute': True, 'dhcpv6': False, 'mtu': 1500, 'switch': 'legacy', 'stp': False, 'bootproto': 'none', 'nameservers': ['213.133.98.98']}) MainProcess|jsonrpc/0::DEBUG::2020-09-02 16:38:25,958::commands::153::common.commands::(start) /usr/bin/taskset --cpu-list 0-7 /usr/libexec/vdsm/hooks/before_network_setup/50_fcoe (cwd None) MainProcess|jsonrpc/0::INFO::2020-09-02 16:38:26,154::hooks::122::root::(_runHooksDir) /usr/libexec/vdsm/hooks/before_network_setup/50_fcoe: rc=0 err=b'' MainProcess|jsonrpc/0::INFO::2020-09-02 16:38:26,155::configurator::195::root::(_setup_nmstate) Processing setup through nmstate MainProcess|jsonrpc/0::INFO::2020-09-02 16:38:26,175::configurator::197::root::(_setup_nmstate) Desired state: {'interfaces': [{'name': 'enp4s0', 'state': 'up', 'mtu': 1500}, {'vlan': {'id': 4000, 'base-iface': 'enp4s0'} , 'name': 'enp4s0.4000', 'type': 'vlan', 'state': 'up', 'mtu': 1500, 'ipv4': {'enabled': False}, 'ipv6': {'enabled': False}}, {'name': 'ovirtmgmt', 'type': 'linux-bridge', 'state': 'up', 'mtu': 1500, 'bridge': {'port': [ {'name': 'enp4s0.4000'}], 'options': {'stp': {'enabled': False}}}, 'ipv4': {'enabled': True, 'address': [{'ip': '172.27.1.1', 'prefix-length': 24}], 'dhcp': False}, 'ipv6': {'enabled': False}}], 'dns-resolver': {'config' : {'server': ['213.133.98.98']}}} MainProcess|jsonrpc/0::DEBUG::2020-09-02 16:38:26,217::checkpoint::121::root::(create) Checkpoint /org/freedesktop/NetworkManager/Checkpoint/1 created for all devices: 60 MainProcess|jsonrpc/0::DEBUG::2020-09-02 16:38:26,218::netapplier::239::root::(_add_interfaces) Adding new interfaces: ['ovirtmgmt'] MainProcess|jsonrpc/0::DEBUG::2020-09-02 16:38:26,221::netapplier::251::root::(_edit_interfaces) Editing interfaces: ['enp4s0.4000', 'enp4s0'] MainProcess|jsonrpc/0::DEBUG::2020-09-02 16:38:26,224::nmclient::136::root::(execute_next_action) Executing NM action: func=add_connection_async MainProcess|jsonrpc/0::DEBUG::2020-09-02 16:38:26,231::connection::329::root::(_add_connection_callback) Connection adding succeeded: dev=ovirtmgmt MainProcess|jsonrpc/0::DEBUG::2020-09-02 16:38:26,232::nmclient::136::root::(execute_next_action) Executing NM action: func=commit_changes_async MainProcess|jsonrpc/0::ERROR::2020-09-02 16:38:26,233::nmclient::200::root::(quit) NM main-loop aborted: Connection update failed: error=nm-connection-error-quark: ipv6.gateway: gateway cannot be set if there are no addresses configured (7), dev=enp4s0/<enum NM_DEVICE_STATE_ACTIVATED of type NM.DeviceState> MainProcess|jsonrpc/0::DEBUG::2020-09-02 16:38:26,246::checkpoint::164::root::(rollback) Checkpoint /org/freedesktop/NetworkManager/Checkpoint/1 rollback executed: dbus.Dictionary({dbus.String('/org/freedesktop/NetworkManager/Devices/6'): dbus.UInt32(0), dbus.String('/org/freedesktop/NetworkManager/Devices/7'): dbus.UInt32(0), dbus.String('/org/freedesktop/NetworkManager/Devices/8'): dbus.UInt32(0), dbus.String('/org/freedesktop/NetworkManager/Devices/9'): dbus.UInt32(0), dbus.String('/org/freedesktop/NetworkManager/Devices/3'): dbus.UInt32(0), dbus.String('/org/freedesktop/NetworkManager/Devices/2'): dbus.UInt32(0), dbus.String('/org/freedesktop/NetworkManager/Devices/4'): dbus.UInt32(0), dbus.String('/org/freedesktop/NetworkManager/Devices/1'): dbus.UInt32(0), dbus.String('/org/freedesktop/NetworkManager/Devices/5'): dbus.UInt32(0)}, signature=dbus.Signature('su')) MainProcess|jsonrpc/0::ERROR::2020-09-02 16:38:31,251::supervdsm_server::97::SuperVdsm.ServerCallback::(wrapper) Error in setupNetworks Traceback (most recent call last): File "/usr/lib/python3.6/site-packages/vdsm/supervdsm_server.py", line 95, in wrapper res = func(*args, **kwargs) File "/usr/lib/python3.6/site-packages/vdsm/network/api.py", line 241, in setupNetworks _setup_networks(networks, bondings, options, net_info) File "/usr/lib/python3.6/site-packages/vdsm/network/api.py", line 266, in _setup_networks networks, bondings, options, net_info, in_rollback File "/usr/lib/python3.6/site-packages/vdsm/network/netswitch/configurator.py", line 154, in setup _setup_nmstate(networks, bondings, options, in_rollback) File "/usr/lib/python3.6/site-packages/vdsm/network/netswitch/configurator.py", line 199, in _setup_nmstate nmstate.setup(desired_state, verify_change=not in_rollback) File "/usr/lib/python3.6/site-packages/vdsm/network/nmstate.py", line 63, in setup state_apply(desired_state, verify_change=verify_change) File "/usr/lib/python3.6/site-packages/libnmstate/deprecation.py", line 40, in wrapper return func(*args, **kwargs) File "/usr/lib/python3.6/site-packages/libnmstate/nm/nmclient.py", line 96, in wrapped ret = func(*args, **kwargs) File "/usr/lib/python3.6/site-packages/libnmstate/netapplier.py", line 73, in apply state.State(desired_state), verify_change, commit, rollback_timeout File "/usr/lib/python3.6/site-packages/libnmstate/netapplier.py", line 163, in _apply_ifaces_state con_profiles=ifaces_add_configs + ifaces_edit_configs, File "/usr/lib64/python3.6/contextlib.py", line 88, in __exit__ next(self.gen) File "/usr/lib/python3.6/site-packages/libnmstate/netapplier.py", line 232, in _setup_providers mainloop.run(timeout=MAINLOOP_TIMEOUT) File "/usr/lib/python3.6/site-packages/libnmstate/nm/nmclient.py", line 177, in run f"Unexpected failure of libnm when running the mainloop: {err}" libnmstate.error.NmstateLibnmError: Unexpected failure of libnm when running the mainloop: run execution My host have a main network interface and a vlan interface on the same physical interface which is an internal management network which should be used as management network. The vlan interface doesn't have any ipv6 interface, but nmclient fails on setting ipv6 gateway. This seems like a bug? /Sverker _______________________________________________ discuss mailing list discuss(a)openvswitch.org https://mail.openvswitch.org/mailman/listinfo/ovs-discuss

4 21

iSCSI multipath issue
by Nick Kas 08 Sep '20

08 Sep '20

Hello evryone, setup ovirt 4.4.1 on CentOS 8.2 as an experiment, and I am trying to get an iSCSI domain working but have issues. The little experimental cluster has 3 hosts. There is an ovirtmgmt network on the default vlan, and two iSCSI network (172.27.0/1.X) with vlans 20/21. ovirtmgmt has all the functions (Data, display, migration etc), and the iSCSI networks nothing yet, and they are not set as required. The SAN device is already serving a few iSCSI volumes to a vmware cluster, so I know things are fine on this end. It has two controllers, and four NICs per controller so a total of 8 NICs, half of the NICS per controller on 172.27.0.X and half on 172.27.1.X. When I create the iSCSI domain, I login to only one of the targets, and add the Volume, all is good and I can use the disc fine. However when I login to more than one of the targets, then I start having issues with the Volume. Even when I enabled multipath in the cluster, and I created a single multipath by selecting both of the 172.27.0/1.X networks, and all the targets, the end result was the same. The hosts have difficulty accessing the volume, they may even swing between 'non-operational' and 'up' if I transfer data to the volume. When I ssh to the hosts and i check things in the command line I also get inconsistent results between hosts, and blocks that appear with lsblk when I first setup iSCSI have dissapeared after I try to actively use the volume. I am new to iSCSI so I am not sure how to debug this. I am not sure if my multipath configuration is correct or not. The documentation on this part was not very detailed. I also tried to remove the domain, and try to experiment with mounting the iSCSI volume from the command line, but I cannot even discover the target from the command line, which is very bizarre. The command iscsiadm --mode discovery --target sendtargets --portal 172.27.0.55 --discover returns the message 'iscsiadm: cannot make connection to 172.27.0.55: No route to host'. Yet through ovirt, and if I select only one target, everything work fine! Any suggestions on how to start debugging this would really be appreciated.

2 3

New Installation OVIRT NODE 4.4.1
by henni＠africadragon.com 08 Sep '20

08 Sep '20

I am on step 5, but I am not sure what I am doing wrong. I get the following error. Any help will be appreciated. I have a subdomain, but I am not sure if it is needed. I am not sure if I have to add things in /etc/hosts [ INFO ] ok: [localhost] [ INFO ] TASK [ovirt.hosted_engine_setup : Obtain SSO token using username/password credentials] [ INFO ] ok: [localhost] [ INFO ] TASK [ovirt.hosted_engine_setup : Fetch host facts] [ INFO ] ok: [localhost] [ INFO ] TASK [ovirt.hosted_engine_setup : Fetch cluster ID] [ INFO ] ok: [localhost] [ INFO ] TASK [ovirt.hosted_engine_setup : Fetch cluster facts] [ INFO ] ok: [localhost] [ INFO ] TASK [ovirt.hosted_engine_setup : Fetch Datacenter facts] [ INFO ] ok: [localhost] [ INFO ] TASK [ovirt.hosted_engine_setup : Fetch Datacenter ID] [ INFO ] ok: [localhost] [ INFO ] TASK [ovirt.hosted_engine_setup : Fetch Datacenter name] [ INFO ] ok: [localhost] [ INFO ] TASK [ovirt.hosted_engine_setup : Add NFS storage domain] [ ERROR ] ovirtsdk4.Error: Fault reason is "Operation Failed". Fault detail is "[Problem while trying to mount target]". HTTP response code is 400. [ ERROR ] fatal: [localhost]: FAILED! => {"changed": false, "msg": "Fault reason is \"Operation Failed\". Fault detail is \"[Problem while trying to mount target]\". HTTP response code is 400."}

2 1

Engine certificate and HTTPS : broken on recent Firefox
by Michaël Couren 08 Sep '20

08 Sep '20

Hi, Is there a "verified"* way to add a valid certificate to an oVirt 4.3 engine ? (or suppress HTTP Strict Transport Security) It is no more possible to add a security exception (on latest Firefox 80.0.1 Linux) and I don't want to use 2 browsers (like sanp-chromium just for oVirt...) *"verified" : I mean we have already tried official doc (many months ago) but with no success Thanks -- Cordialement / Best regards, Michaël Couren, ABES, Montpellier, France.

2 1

Using Ansible to automate failed Drive replacement
by kushagra2agarwal＠gmail.com 07 Sep '20

07 Sep '20

Do we have any inbuild ansible modules/tasks which can help to automate replacing failed drive/volume in oVirt hypervisor

3 5

Re: KeyCloak Integration
by Artur Socha 07 Sep '20

07 Sep '20

Hi Anton, Just to let you know. I investigated this issue. If you want to use keycloak in version >=10 you would need to define all additional scopes as 'optional client scopes' in your client configuration. In my case, on my test environment, I only had to add 'ovirt-ext=auth:sequence-priority=~' but in your case you may need all listed in error_description: *{"error_description":"Cannot authenticate user Invalid scopes: ovirt-app-api ovirt-ext=revoke:revoke-all ovirt-ext=token-info:authz-search ovirt-ext=token-info:public-authz-search ovirt-ext=token-info:validate ovirt-ext=token:password-access.","error":"access_denied"}* This configuration change is required because it has been changed/fixed how 'unknown' scopes are handled in keycloak. Now keycloak must always be aware of all scopes and previously unknown ones were simply ignored. Here is BZ with details: https://bugzilla.redhat.com/show_bug.cgi?id=1849569 best, Artur On Tue, Jun 23, 2020 at 5:03 PM Artur Socha <asocha(a)redhat.com> wrote: > On Tue, 2020-06-23 at 14:41 +0000, Anton Louw wrote: > > > > Hi Artur, > > > > Apologies for the late response. So we have downgraded the version of > KeyCloak, and all seems to be working 100% again, I can obtain a token, and > do API calls. > > Hi Anton, > I'm glad it works now. This keycloak version (9.0.x) will stay for some > time the recommended & supported choice for oVirt because it is part of > 'Red Hat SSO' just like oVirt is part of 'Red Hat Virtualization'. > Artur > > > > Thank you very much for all the help > > > > *From:* Artur Socha <asocha(a)redhat.com> > *Sent:* 22 June 2020 16:52 > *To:* Anton Louw <Anton.Louw(a)voxtelecom.co.za>; users(a)ovirt.org > *Cc:* Stephen Hutchinson <Stephen.Hutchinson(a)voxtelecom.co.za> > *Subject:* Re: [ovirt-users] KeyCloak Integration > > > > On Mon, 2020-06-22 at 15:14 +0200, Artur Socha wrote: > > Anton, > > I managed to re-create the issue on my local environment. > > Previously I tested it against Keycloak 8.0.1 with users loaded from LDAP. > Currently I have users/groups created via Keycloak management panel. I need > to investigate it further which of the two changes is the root cause (it > works fine with the old setup) > > > > One more update: it seems the issue is keycloak version related. Trying to > figure out what was changed and how it affected engine sso integration. > > > > Latest keycloak version I tested and verified that works is 9.0.3. Perhaps > it could be possible for you to use it until we fully support 10.0.x ? > > Artur > > > > *Anton Louw* > *Cloud Engineer: Storage and Virtualization* at *Vox* > ------------------------------ > *T:* 087 805 0000 | *D:* 087 805 1572 > *M:* N/A > *E:* anton.louw(a)voxtelecom.co.za > *A:* Rutherford Estate, 1 Scott Street, Waverley, Johannesburg > www.vox.co.za > > [image: F] <https://www.facebook.com/voxtelecomZA> > [image: T] <https://www.twitter.com/voxtelecom> > [image: I] <https://www.instagram.com/voxtelecomza/> > [image: L] <https://www.linkedin.com/company/voxtelecom> > [image: Y] <https://www.youtube.com/user/VoxTelecom> > > Artur > > > > On Mon, 2020-06-22 at 11:05 +0000, Anton Louw wrote: > > > > Hi Artur, > > > > Great, thanks a lot! 😊 > > > > > > *Anton Louw* > > *Cloud Engineer: Storage and Virtualization* at *Vox* > ------------------------------ > > *T:* 087 805 0000 | *D:* 087 805 1572 > *M:* N/A > *E:* anton.louw(a)voxtelecom.co.za > *A:* Rutherford Estate, 1 Scott Street, Waverley, Johannesburg > www.vox.co.za > > > > [image: F] <https://www.facebook.com/voxtelecomZA> > > > > [image: T] <https://www.twitter.com/voxtelecom> > > > > [image: I] <https://www.instagram.com/voxtelecomza> > > > > [image: L] <https://www.linkedin.com/company/voxtelecom> > > > > [image: Y] <https://www.youtube.com/user/VoxTelecom> > > > > > > *From:* Artur Socha <asocha(a)redhat.com> > *Sent:* 22 June 2020 11:23 > *To:* Anton Louw <Anton.Louw(a)voxtelecom.co.za>; users(a)ovirt.org > *Cc:* Stephen Hutchinson <Stephen.Hutchinson(a)voxtelecom.co.za> > *Subject:* Re: [ovirt-users] KeyCloak Integration > > > > Hi Anton, > > Thanks for the specs. I have create BZ issue for tracking: > > https://bugzilla.redhat.com/show_bug.cgi?id=1849569 > > Feel free to add comments/change it when needed. > > > > Artur > > > > On Fri, 2020-06-19 at 10:57 +0000, Anton Louw wrote: > > > > Hi Artur, > > > > Please see below: > > > > ovirt-engine.noarch 4.3.10.4-1.el7 @ovirt-4.3 > > ovirt-engine-extension-aaa-misc.noarch 1.0.4-1.el7 @ovirt-4.3 > > mod_auth_openidc.x86_64 1.8.8-5.el7 @base > > > > [root@virt ~]# cat /etc/*elease > > CentOS Linux release 7.7.1908 (Core) > > NAME="CentOS Linux" > > VERSION="7 (Core)" > > ID="centos" > > ID_LIKE="rhel fedora" > > VERSION_ID="7" > > PRETTY_NAME="CentOS Linux 7 (Core)" > > ANSI_COLOR="0;31" > > CPE_NAME="cpe:/o:centos:centos:7" > > HOME_URL="https://www.centos.org/" > > BUG_REPORT_URL="https://bugs.centos.org/" > > > > CENTOS_MANTISBT_PROJECT="CentOS-7" > > CENTOS_MANTISBT_PROJECT_VERSION="7" > > REDHAT_SUPPORT_PRODUCT="centos" > > REDHAT_SUPPORT_PRODUCT_VERSION="7" > > > > CentOS Linux release 7.7.1908 (Core) > > CentOS Linux release 7.7.1908 (Core) > > > > KeyCloak – > > > > Server Version > > 10.0.1 > > > > Thanks a lot for your help Artur. Please let me know if you need anything > else. > > > > *From:* Artur Socha <asocha(a)redhat.com> > *Sent:* 19 June 2020 12:39 > *To:* Anton Louw <Anton.Louw(a)voxtelecom.co.za>; users(a)ovirt.org > *Cc:* Stephen Hutchinson <Stephen.Hutchinson(a)voxtelecom.co.za> > *Subject:* Re: [ovirt-users] KeyCloak Integration > > > > On Fri, 2020-06-19 at 10:21 +0000, Anton Louw wrote: > > > > Yes I didn’t get to the OVN part yet, as I first wanted to test the if the > token can be obtained. > > > > This is the first time we are testing KeyCloak in any environment, so we > have never been able to obtain a token for API access. > > > > Please post the exact versions of: > > - ovirt-engine* : > > yum list --installed | grep ovirt-engine > > yum list --intalled | grep ovirt-engine-extension-aaa-misc > > yum list --installed | grep mod_auth_openidc > > - keycloak > > - OS > > cat /etc/*elease > > > > I'll submit a bug ... which, most likely, I will assign to myself anyway :) > > > > Artur > > > > > > *Anton Louw* > > *Cloud Engineer: Storage and Virtualization* at *Vox* > ------------------------------ > > *T:* 087 805 0000 | *D:* 087 805 1572 > *M:* N/A > *E:* anton.louw(a)voxtelecom.co.za > *A:* Rutherford Estate, 1 Scott Street, Waverley, Johannesburg > www.vox.co.za > > > > [image: F] <https://www.facebook.com/voxtelecomZA> > > > > [image: T] <https://www.twitter.com/voxtelecom> > > > > [image: I] <https://www.instagram.com/voxtelecomza> > > > > [image: L] <https://www.linkedin.com/company/voxtelecom> > > > > [image: Y] <https://www.youtube.com/user/VoxTelecom> > > > > > > Thanks > > > > *From:* Artur Socha <asocha(a)redhat.com> > *Sent:* 19 June 2020 12:16 > *To:* Anton Louw <Anton.Louw(a)voxtelecom.co.za>; users(a)ovirt.org > *Cc:* Stephen Hutchinson <Stephen.Hutchinson(a)voxtelecom.co.za> > *Subject:* Re: [ovirt-users] KeyCloak Integration > > > > On Fri, 2020-06-19 at 10:03 +0000, Anton Louw wrote: > > > > Hi Artur, > > > > Sure, please see below output: > > > > [root@virt ~]# curl -vvv -H "Accept:application/json" ' > https://virt.example.co.za/ovirt-engine/sso/oauth/token?grant_type=password… > ' > > * About to connect() to virt.example.co.za port 443 (#0) > > * Trying 127.0.0.1... > > * Connected to virt.example.co.za (127.0.0.1) port 443 (#0) > > * Initializing NSS with certpath: sql:/etc/pki/nssdb > > * CAfile: /etc/pki/tls/certs/ca-bundle.crt > > CApath: none > > * SSL connection using TLS_ECDHE_RSA_WITH_AES_256_GCM_SHA384 > > * Server certificate: > > * subject: CN=*.example.co.za,OU=Domain Control Validated > > * start date: Sep 25 07:46:12 2019 GMT > > * expire date: Oct 02 07:39:01 2020 GMT > > * common name: *example.co.za > > * issuer: CN=Starfield Secure Certificate Authority - G2,OU= > http://certs.starfieldtech.com/repository/,O="Starfield Technologies, > Inc.",L=Scottsdale,ST=Arizona,C=US > > > GET > /ovirt-engine/sso/oauth/token?grant_type=password&username=myuser&password=mypass&scope=ovirt-app-api > HTTP/1.1 > > > User-Agent: curl/7.29.0 > > > Host: virt.example.co.za > > > Accept:application/json > > > > > < HTTP/1.1 400 Bad Request > > < Date: Fri, 19 Jun 2020 09:52:11 GMT > > < Server: Apache/2.4.6 (CentOS) OpenSSL/1.0.2k-fips > > < Set-Cookie: locale=en_US; path=/; secure; HttpOnly; Max-Age=2147483647; > Expires=Wed, 07-Jul-2088 13:06:18 GMT > > < X-XSS-PROTECTION: 1; MODE=BLOCK > > < X-CONTENT-TYPE-OPTIONS: NOSNIFF > > < X-FRAME-OPTIONS: SAMEORIGIN > > < Content-Type: application/json > > < Content-Length: 233 > > < Connection: close > > < > > * Closing connection 0 > > {"error_code":"access_denied","error":"Cannot authenticate user Invalid > scopes: ovirt-app-api ovirt-ext=token-info:authz-search > ovirt-ext=token-info:public-authz-search ovirt-ext=token-info:validate > ovirt-ext=token:password-access."} > > > > 1) Test connection using python script (from the blog post ) using sdk. I > suspect it will not work either. > > Testing from Python gives me the same error as well. > > > > 2) I saw some errors in the log on revoking token. Please go to keycloak > admin panel, and under users kill all its active sessions. Then, please > without logging in to engine admin UI, use that curl to obtain token. > > Tested this again, but still getting the below: > > {"error_code":"access_denied","error":"Cannot authenticate user Invalid > scopes: ovirt-app-api ovirt-ext=token-info:authz-search > ovirt-ext=token-info:public-authz-search ovirt-ext=token-info:validate > ovirt-ext=token:password-access."} > > > > Thanks for these test ... unfortunately nothing helped > > > > > > 3) Does it work without OVN integration enabled? > > Can you explain a bit more? How can I disable OVN integration to test this? > > > > I had in mind reverting OVN vs Keycloak integration done according to > "Configuring OVN" chapter in > https://blogs.ovirt.org/2019/01/federate-ovirt-engine-authentication-to-ope… > > Unless, of course, you skipped it. > > > > Most likely you found a bug. Have you ever been able to obtain token for > api access with keycloak integration (even with you previous environments)? > > I am now trying to understand what happened and how to reproduce it before > submitting the bug into http://bugzilla.redhat.com > > > > > > *Anton Louw* > > *Cloud Engineer: Storage and Virtualization* at *Vox* > ------------------------------ > > *T:* 087 805 0000 | *D:* 087 805 1572 > *M:* N/A > *E:* anton.louw(a)voxtelecom.co.za > *A:* Rutherford Estate, 1 Scott Street, Waverley, Johannesburg > www.vox.co.za > > > > [image: F] <https://www.facebook.com/voxtelecomZA> > > > > [image: T] <https://www.twitter.com/voxtelecom> > > > > [image: I] <https://www.instagram.com/voxtelecomza> > > > > [image: L] <https://www.linkedin.com/company/voxtelecom> > > > > [image: Y] <https://www.youtube.com/user/VoxTelecom> > > > > > > > > Thanks > > > > > > *Anton Louw* > > *Cloud Engineer: Storage and Virtualization* at *Vox* > ------------------------------ > > *T:* 087 805 0000 | *D:* 087 805 1572 > *M:* N/A > *E:* anton.louw(a)voxtelecom.co.za > *A:* Rutherford Estate, 1 Scott Street, Waverley, Johannesburg > www.vox.co.za > > > > [image: F] <https://www.facebook.com/voxtelecomZA> > > > > [image: T] <https://www.twitter.com/voxtelecom> > > > > [image: I] <https://www.instagram.com/voxtelecomza> > > > > [image: L] <https://www.linkedin.com/company/voxtelecom> > > > > [image: Y] <https://www.youtube.com/user/VoxTelecom> > > > > > > *From:* Artur Socha <asocha(a)redhat.com> > *Sent:* 19 June 2020 11:40 > *To:* Anton Louw <Anton.Louw(a)voxtelecom.co.za>; users(a)ovirt.org > *Cc:* Stephen Hutchinson <Stephen.Hutchinson(a)voxtelecom.co.za> > *Subject:* Re: [ovirt-users] KeyCloak Integration > > > > On Fri, 2020-06-19 at 08:34 +0000, Anton Louw wrote: > > > > Hi Artur, > > > > Thank you for the quick response. > > > > I have actually tried creating another user, but I still get the same > error. I have attached the output of curl -vvv as well as the logs the > engine and keycloak logs. > > > > This `curl -vvv ...` is actually is incorrect because it is missing -H > before 'Accept' header. However, previous attempts that led to this error > seemed to be fine. Could you just re-send output of the correct curl? > > > > There are few things we can test to try to narrow down the root cause: > > > > 1) Test connection using python script (from the blog post ) using sdk. I > suspect it will not work either. > > > > 2) I saw some errors in the log on revoking token. Please go to keycloak > admin panel, and under users kill all its active sessions. Then, please > without logging in to engine admin UI, use that curl to obtain token. > > > > 3) Does it work without OVN integration enabled? > > > > Artur > > > > > > > > Thank you > > > > > > *Anton Louw* > > *Cloud Engineer: Storage and Virtualization* at *Vox* > ------------------------------ > > *T:* 087 805 0000 | *D:* 087 805 1572 > *M:* N/A > *E:* anton.louw(a)voxtelecom.co.za > *A:* Rutherford Estate, 1 Scott Street, Waverley, Johannesburg > www.vox.co.za > > > > [image: F] <https://www.facebook.com/voxtelecomZA> > > > > [image: T] <https://www.twitter.com/voxtelecom> > > > > [image: I] <https://www.instagram.com/voxtelecomza> > > > > [image: L] <https://www.linkedin.com/company/voxtelecom> > > > > [image: Y] <https://www.youtube.com/user/VoxTelecom> > > > > > > *From:* Artur Socha <asocha(a)redhat.com> > *Sent:* 19 June 2020 10:23 > *To:* Anton Louw <Anton.Louw(a)voxtelecom.co.za>; users(a)ovirt.org > *Subject:* Re: [ovirt-users] KeyCloak Integration > > > > O > > n Fri, 2020-06-19 at 07:35 +0000, Anton Louw via Users wrote: > > > > Hi Everybody, > > > > Hi Anton, > > > > So I have implemented KeyCloak into our oVirt environment, which works, up > until a point. So WebUI access works, but when calling the API, using: > > *curl -k -H "Accept: application/json" '* > *https://virt.example.co.za/ovirt-engine/sso/oauth/token?grant_type=password&username=admin@openidchttp&password=mypass&scope=ovirt-app-api* > <https://virt.example.co.za/ovirt-engine/sso/oauth/token?grant_type=password…> > *'* > > > > I get the below error: > > > > *{"error_description":"Cannot authenticate user Invalid scopes: > ovirt-app-api ovirt-ext=revoke:revoke-all ovirt-ext=token-info:authz-search > ovirt-ext=token-info:public-authz-search ovirt-ext=token-info:validate > ovirt-ext=token:password-access.","error":"access_denied"}* > > > > If my configs are removed, and I use “admin@internal” for my username, > then it works. > > > > I followed the below article step by step, and I double checked that all > the scopes are added into KeyCloak (ovirt-app-api and ovirt-app-admin) > > > > > https://blogs.ovirt.org/2019/01/federate-ovirt-engine-authentication-to-ope… > > > > Anybody have any ideas? > > > > It is my blind shot but could create & check another user? > > > > One more thing to check please use curl -vvv to check if there are any > redirects along the way. > > I will check keycloak settings on my setup - perhaps there is something > non-obvious that could have been missed. > > > > Any chance to get a bit more logs from engine.log and even from keycloak? > Perhaps there is something there that could help. > > > > Artur > > > > > > Thank you > > > > *Anton Louw* > > *Cloud Engineer: Storage and Virtualization* at *Vox* > ------------------------------ > > *T:* 087 805 0000 | *D:* 087 805 1572 > *M:* N/A > *E:* anton.louw(a)voxtelecom.co.za > *A:* Rutherford Estate, 1 Scott Street, Waverley, Johannesburg > www.vox.co.za > > > > [image: F] <https://www.facebook.com/voxtelecomZA> > > > > [image: T] <https://www.twitter.com/voxtelecom> > > > > [image: I] <https://www.instagram.com/voxtelecomza> > > > > [image: L] <https://www.linkedin.com/company/voxtelecom> > > > > [image: Y] <https://www.youtube.com/user/VoxTelecom> > > > > > > [image: #VoxBrand] > <https://www.vox.co.za/fibre/fibre-to-the-home/?prod=HOME> > > > *Disclaimer* > > The contents of this email are confidential to the sender and the intended > recipient. Unless the contents are clearly and entirely of a personal > nature, they are subject to copyright in favour of the holding company of > the Vox group of companies. Any recipient who receives this email in error > should immediately report the error to the sender and permanently delete > this email from all storage devices. > > This email has been scanned for viruses and malware, and may have been > automatically archived by *Mimecast Ltd*, an innovator in Software as a > Service (SaaS) for business. Providing a *safer* and *more useful* place > for your human generated data. Specializing in; Security, archiving and > compliance. To find out more Click Here > <https://www.voxtelecom.co.za/security/mimecast/?prod=Enterprise>. > > > > _______________________________________________ > > Users mailing list -- > > * <users(a)ovirt.org>* > > *users(a)ovirt.org <users(a)ovirt.org>* > > > > > > > > > > > > > > > > To unsubscribe send an email to > > * <users-leave(a)ovirt.org>* > > *users-leave(a)ovirt.org <users-leave(a)ovirt.org>* > > > > > > > > > > > > > > > > Privacy Statement: > > * <https://www.ovirt.org/privacy-policy.html>* > > *https://www.ovirt.org/privacy-policy.html > <https://www.ovirt.org/privacy-policy.html>* > > > > > > > > > > > > > > > > oVirt Code of Conduct: > > * <https://www.ovirt.org/community/about/community-guidelines/>* > > *https://www.ovirt.org/community/about/community-guidelines/ > <https://www.ovirt.org/community/about/community-guidelines/>* > > > > > > > > > > > > > > > > List Archives: > > * > <https://lists.ovirt.org/archives/list/users@ovirt.org/message/CC54IPZLYJYE2…>* > > *https://lists.ovirt.org/archives/list/users@ovirt.org/message/CC54IPZLYJYE2B3NP4LT4TN4CJX4C7BU/ > <https://lists.ovirt.org/archives/list/users@ovirt.org/message/CC54IPZLYJYE2…>* > > > > > > > > > > > > > > > > > > > > > > > > > > > -- Artur Socha Senior Software Engineer, RHV Red Hat

1 0