[ovirt-users] Re: KeyCloak Integration

Hi Artur, Apologies for the late response. So we have downgraded the version of KeyCloak, and all seems to be working 100% again, I can obtain a token, and do API calls. Thank you very much for all the help From: Artur Socha <asocha(a)redhat.com&gt; Sent: 22 June 2020 16:52 To: Anton Louw <Anton.Louw(a)voxtelecom.co.za>; users(a)ovirt.org Cc: Stephen Hutchinson <Stephen.Hutchinson(a)voxtelecom.co.za&gt; Subject: Re: [ovirt-users] KeyCloak Integration On Mon, 2020-06-22 at 15:14 +0200, Artur Socha wrote: Anton, I managed to re-create the issue on my local environment. Previously I tested it against Keycloak 8.0.1 with users loaded from LDAP. Currently I have users/groups created via Keycloak management panel. I need to investigate it further which of the two changes is the root cause (it works fine with the old setup) One more update: it seems the issue is keycloak version related. Trying to figure out what was changed and how it affected engine sso integration. Latest keycloak version I tested and verified that works is 9.0.3. Perhaps it could be possible for you to use it until we fully support 10.0.x ? Artur Artur On Mon, 2020-06-22 at 11:05 +0000, Anton Louw wrote: Hi Artur, Great, thanks a lot! 😊 Anton Louw Cloud Engineer: Storage and Virtualization at Vox ________________________________ T: 087 805 0000 | D: 087 805 1572 M: N/A E: anton.louw@voxtelecom.co.za<mailto:anton.louw@voxtelecom.co.za> A: Rutherford Estate, 1 Scott Street, Waverley, Johannesburg www.vox.co.za<http://www.vox.co.za> [F]<https://www.facebook.com/voxtelecomZA> [T]<https://www.twitter.com/voxtelecom> [I]<https://www.instagram.com/voxtelecomza> [L]<https://www.linkedin.com/company/voxtelecom> [Y]<https://www.youtube.com/user/VoxTelecom> From: Artur Socha <asocha@redhat.com<mailto:asocha@redhat.com>> Sent: 22 June 2020 11:23 To: Anton Louw <Anton.Louw@voxtelecom.co.za<mailto:Anton.Louw@voxtelecom.co.za>>; users@ovirt.org<mailto:users@ovirt.org> Cc: Stephen Hutchinson <Stephen.Hutchinson@voxtelecom.co.za<mailto:Stephen.Hutchinson@voxtelecom.co.za>> Subject: Re: [ovirt-users] KeyCloak Integration Hi Anton, Thanks for the specs. I have create BZ issue for tracking: https://bugzilla.redhat.com/show_bug.cgi?id=1849569<https://bugzilla.r... Feel free to add comments/change it when needed. Artur On Fri, 2020-06-19 at 10:57 +0000, Anton Louw wrote: Hi Artur, Please see below: ovirt-engine.noarch 4.3.10.4-1.el7 @ovirt-4.3 ovirt-engine-extension-aaa-misc.noarch 1.0.4-1.el7 @ovirt-4.3 mod_auth_openidc.x86_64 1.8.8-5.el7 @base [root@virt ~]# cat /etc/*elease CentOS Linux release 7.7.1908 (Core) NAME="CentOS Linux" VERSION="7 (Core)" ID="centos" ID_LIKE="rhel fedora" VERSION_ID="7" PRETTY_NAME="CentOS Linux 7 (Core)" ANSI_COLOR="0;31" CPE_NAME="cpe:/o:centos:centos:7" HOME_URL="https://www.centos.org/<https://www.centos.org/>" BUG_REPORT_URL="https://bugs.centos.org/<https://bugs.centos.org/... CENTOS_MANTISBT_PROJECT="CentOS-7" CENTOS_MANTISBT_PROJECT_VERSION="7" REDHAT_SUPPORT_PRODUCT="centos" REDHAT_SUPPORT_PRODUCT_VERSION="7" CentOS Linux release 7.7.1908 (Core) CentOS Linux release 7.7.1908 (Core) KeyCloak – Server Version 10.0.1 Thanks a lot for your help Artur. Please let me know if you need anything else. From: Artur Socha <asocha@redhat.com<mailto:asocha@redhat.com>> Sent: 19 June 2020 12:39 To: Anton Louw <Anton.Louw@voxtelecom.co.za<mailto:Anton.Louw@voxtelecom.co.za>>; users@ovirt.org<mailto:users@ovirt.org> Cc: Stephen Hutchinson <Stephen.Hutchinson@voxtelecom.co.za<mailto:Stephen.Hutchinson@voxtelecom.co.za>> Subject: Re: [ovirt-users] KeyCloak Integration On Fri, 2020-06-19 at 10:21 +0000, Anton Louw wrote: Yes I didn’t get to the OVN part yet, as I first wanted to test the if the token can be obtained. This is the first time we are testing KeyCloak in any environment, so we have never been able to obtain a token for API access. Please post the exact versions of: - ovirt-engine* : yum list --installed | grep ovirt-engine yum list --intalled | grep ovirt-engine-extension-aaa-misc yum list --installed | grep mod_auth_openidc - keycloak - OS cat /etc/*elease I'll submit a bug ... which, most likely, I will assign to myself anyway :) Artur Anton Louw Cloud Engineer: Storage and Virtualization at Vox ________________________________ T: 087 805 0000 | D: 087 805 1572 M: N/A E: anton.louw@voxtelecom.co.za<mailto:anton.louw@voxtelecom.co.za> A: Rutherford Estate, 1 Scott Street, Waverley, Johannesburg www.vox.co.za<http://www.vox.co.za> [F]<https://www.facebook.com/voxtelecomZA> [T]<https://www.twitter.com/voxtelecom> [I]<https://www.instagram.com/voxtelecomza> [L]<https://www.linkedin.com/company/voxtelecom> [Y]<https://www.youtube.com/user/VoxTelecom> Thanks From: Artur Socha <asocha@redhat.com<mailto:asocha@redhat.com>> Sent: 19 June 2020 12:16 To: Anton Louw <Anton.Louw@voxtelecom.co.za<mailto:Anton.Louw@voxtelecom.co.za>>; users@ovirt.org<mailto:users@ovirt.org> Cc: Stephen Hutchinson <Stephen.Hutchinson@voxtelecom.co.za<mailto:Stephen.Hutchinson@voxtelecom.co.za>> Subject: Re: [ovirt-users] KeyCloak Integration On Fri, 2020-06-19 at 10:03 +0000, Anton Louw wrote: Hi Artur, Sure, please see below output: [root@virt ~]# curl -vvv -H "Accept:application/json" 'https://virt.example.co.za/ovirt-engine/sso/oauth/token?grant_type=password&username=myuser&password=mypass&scope=ovirt-app-api<https://virt.example.co.za/ovirt-engine/sso/oauth/token?grant_type=password&username=myuser&password=mypass&scope=ovirt-app-api>' * About to connect() to virt.example.co.za<http://virt.example.co.za> port 443 (#0) * Trying 127.0.0.1<http://127.0.0.1>... * Connected to virt.example.co.za<http://virt.example.co.za> (127.0.0.1<http://127.0.0.1>) port 443 (#0) * Initializing NSS with certpath: sql:/etc/pki/nssdb * CAfile: /etc/pki/tls/certs/ca-bundle.crt CApath: none * SSL connection using TLS_ECDHE_RSA_WITH_AES_256_GCM_SHA384 * Server certificate: * subject: CN=*.example.co.za,OU=Domain Control Validated * start date: Sep 25 07:46:12 2019 GMT * expire date: Oct 02 07:39:01 2020 GMT * common name: *example.co.za<http://example.co.za> * issuer: CN=Starfield Secure Certificate Authority - G2,OU=http://certs.starfieldtech.com/repository/,O=<http://certs.starf... Technologies, Inc.",L=Scottsdale,ST=Arizona,C=US < HTTP/1.1 400 Bad Request < Date: Fri, 19 Jun 2020 09:52:11 GMT < Server: Apache/2.4.6 (CentOS) OpenSSL/1.0.2k-fips < Set-Cookie: locale=en_US; path=/; secure; HttpOnly; Max-Age=2147483647; Expires=Wed, 07-Jul-2088 13:06:18 GMT < X-XSS-PROTECTION: 1; MODE=BLOCK < X-CONTENT-TYPE-OPTIONS: NOSNIFF < X-FRAME-OPTIONS: SAMEORIGIN < Content-Type: application/json < Content-Length: 233 < Connection: close < * Closing connection 0 {"error_code":"access_denied","error":"Cannot authenticate user Invalid scopes: ovirt-app-api ovirt-ext=token-info:authz-search ovirt-ext=token-info:public-authz-search ovirt-ext=token-info:validate ovirt-ext=token:password-access."} 1) Test connection using python script (from the blog post ) using sdk. I suspect it will not work either. Testing from Python gives me the same error as well. 2) I saw some errors in the log on revoking token. Please go to keycloak admin panel, and under users kill all its active sessions. Then, please without logging in to engine admin UI, use that curl to obtain token. Tested this again, but still getting the below: {"error_code":"access_denied","error":"Cannot authenticate user Invalid scopes: ovirt-app-api ovirt-ext=token-info:authz-search ovirt-ext=token-info:public-authz-search ovirt-ext=token-info:validate ovirt-ext=token:password-access."} Thanks for these test ... unfortunately nothing helped 3) Does it work without OVN integration enabled? Can you explain a bit more? How can I disable OVN integration to test this? I had in mind reverting OVN vs Keycloak integration done according to "Configuring OVN" chapter in https://blogs.ovirt.org/2019/01/federate-ovirt-engine-authentication-to-o... Unless, of course, you skipped it. Most likely you found a bug. Have you ever been able to obtain token for api access with keycloak integration (even with you previous environments)? I am now trying to understand what happened and how to reproduce it before submitting the bug into http://bugzilla.redhat.com<http://bugzilla.redhat.com> Anton Louw Cloud Engineer: Storage and Virtualization at Vox ________________________________ T: 087 805 0000 | D: 087 805 1572 M: N/A E: anton.louw@voxtelecom.co.za<mailto:anton.louw@voxtelecom.co.za> A: Rutherford Estate, 1 Scott Street, Waverley, Johannesburg www.vox.co.za<http://www.vox.co.za> [F]<https://www.facebook.com/voxtelecomZA> [T]<https://www.twitter.com/voxtelecom> [I]<https://www.instagram.com/voxtelecomza> [L]<https://www.linkedin.com/company/voxtelecom> [Y]<https://www.youtube.com/user/VoxTelecom> Thanks Anton Louw Cloud Engineer: Storage and Virtualization at Vox ________________________________ T: 087 805 0000 | D: 087 805 1572 M: N/A E: anton.louw@voxtelecom.co.za<mailto:anton.louw@voxtelecom.co.za> A: Rutherford Estate, 1 Scott Street, Waverley, Johannesburg www.vox.co.za<http://www.vox.co.za> [F]<https://www.facebook.com/voxtelecomZA> [T]<https://www.twitter.com/voxtelecom> [I]<https://www.instagram.com/voxtelecomza> [L]<https://www.linkedin.com/company/voxtelecom> [Y]<https://www.youtube.com/user/VoxTelecom> Anton Louw Cloud Engineer: Storage and Virtualization ______________________________________ D: 087 805 1572 | M: N/A A: Rutherford Estate, 1 Scott Street, Waverley, Johannesburg anton.louw(a)voxtelecom.co.za www.vox.co.za From: Artur Socha <asocha@redhat.com<mailto:asocha@redhat.com>> Sent: 19 June 2020 11:40 To: Anton Louw <Anton.Louw@voxtelecom.co.za<mailto:Anton.Louw@voxtelecom.co.za>>; users@ovirt.org<mailto:users@ovirt.org> Cc: Stephen Hutchinson <Stephen.Hutchinson@voxtelecom.co.za<mailto:Stephen.Hutchinson@voxtelecom.co.za>> Subject: Re: [ovirt-users] KeyCloak Integration On Fri, 2020-06-19 at 08:34 +0000, Anton Louw wrote: Hi Artur, Thank you for the quick response. I have actually tried creating another user, but I still get the same error. I have attached the output of curl -vvv as well as the logs the engine and keycloak logs. This `curl -vvv ...` is actually is incorrect because it is missing -H before 'Accept' header. However, previous attempts that led to this error seemed to be fine. Could you just re-send output of the correct curl? There are few things we can test to try to narrow down the root cause: 1) Test connection using python script (from the blog post ) using sdk. I suspect it will not work either. 2) I saw some errors in the log on revoking token. Please go to keycloak admin panel, and under users kill all its active sessions. Then, please without logging in to engine admin UI, use that curl to obtain token. 3) Does it work without OVN integration enabled? Artur Thank you Anton Louw Cloud Engineer: Storage and Virtualization at Vox ________________________________ T: 087 805 0000 | D: 087 805 1572 M: N/A E: anton.louw@voxtelecom.co.za<mailto:anton.louw@voxtelecom.co.za> A: Rutherford Estate, 1 Scott Street, Waverley, Johannesburg www.vox.co.za<http://www.vox.co.za> [F]<https://www.facebook.com/voxtelecomZA> [T]<https://www.twitter.com/voxtelecom> [I]<https://www.instagram.com/voxtelecomza> [L]<https://www.linkedin.com/company/voxtelecom> [Y]<https://www.youtube.com/user/VoxTelecom> From: Artur Socha <asocha@redhat.com<mailto:asocha@redhat.com>> Sent: 19 June 2020 10:23 To: Anton Louw <Anton.Louw@voxtelecom.co.za<mailto:Anton.Louw@voxtelecom.co.za>>; users@ovirt.org<mailto:users@ovirt.org> Subject: Re: [ovirt-users] KeyCloak Integration O n Fri, 2020-06-19 at 07:35 +0000, Anton Louw via Users wrote: Hi Everybody, Hi Anton, So I have implemented KeyCloak into our oVirt environment, which works, up until a point. So WebUI access works, but when calling the API, using: curl -k -H "Accept: application/json" 'https://virt.example.co.za/ovirt-engine/sso/oauth/token?grant_type=password&username=admin@openidchttp&password=mypass&scope=ovirt-app-api<https://virt.example.co.za/ovirt-engine/sso/oauth/token?grant_type=password&username=admin@openidchttp&password=mypass&scope=ovirt-app-api>' I get the below error: {"error_description":"Cannot authenticate user Invalid scopes: ovirt-app-api ovirt-ext=revoke:revoke-all ovirt-ext=token-info:authz-search ovirt-ext=token-info:public-authz-search ovirt-ext=token-info:validate ovirt-ext=token:password-access.","error":"access_denied"} If my configs are removed, and I use “admin@internal” for my username, then it works. I followed the below article step by step, and I double checked that all the scopes are added into KeyCloak (ovirt-app-api and ovirt-app-admin) https://blogs.ovirt.org/2019/01/federate-ovirt-engine-authentication-to-o... Anybody have any ideas? It is my blind shot but could create & check another user? One more thing to check please use curl -vvv to check if there are any redirects along the way. I will check keycloak settings on my setup - perhaps there is something non-obvious that could have been missed. Any chance to get a bit more logs from engine.log and even from keycloak? Perhaps there is something there that could help. Artur Thank you Anton Louw Cloud Engineer: Storage and Virtualization at Vox ________________________________ T: 087 805 0000 | D: 087 805 1572 M: N/A E: anton.louw@voxtelecom.co.za<mailto:anton.louw@voxtelecom.co.za> A: Rutherford Estate, 1 Scott Street, Waverley, Johannesburg www.vox.co.za<http://www.vox.co.za> [F]<https://www.facebook.com/voxtelecomZA> [T]<https://www.twitter.com/voxtelecom> [I]<https://www.instagram.com/voxtelecomza> [L]<https://www.linkedin.com/company/voxtelecom> [Y]<https://www.youtube.com/user/VoxTelecom> [#VoxBrand]<https://www.vox.co.za/fibre/fibre-to-the-home/?prod=HOME> Disclaimer The contents of this email are confidential to the sender and the intended recipient. Unless the contents are clearly and entirely of a personal nature, they are subject to copyright in favour of the holding company of the Vox group of companies. Any recipient who receives this email in error should immediately report the error to the sender and permanently delete this email from all storage devices. This email has been scanned for viruses and malware, and may have been automatically archived by Mimecast Ltd, an innovator in Software as a Service (SaaS) for business. Providing a safer and more useful place for your human generated data. Specializing in; Security, archiving and compliance. To find out more Click Here<https://www.voxtelecom.co.za/security/mimecast/?prod=Enterprise>. _______________________________________________ Users mailing list -- <mailto:users@ovirt.org> users@ovirt.org<mailto:users@ovirt.org> To unsubscribe send an email to <mailto:users-leave@ovirt.org> users-leave@ovirt.org<mailto:users-leave@ovirt.org> Privacy Statement: <https://www.ovirt.org/privacy-policy.html> https://www.ovirt.org/privacy-policy.html<https://www.ovirt.org/privac... oVirt Code of Conduct: <https://www.ovirt.org/community/about/community-guidelines/> https://www.ovirt.org/community/about/community-guidelines/<https://ww... List Archives: <https://lists.ovirt.org/archives/list/users@ovirt.org/message/CC54IPZLYJY... https://lists.ovirt.org/archives/list/users@ovirt.org/message/CC54IPZLYJY...