Failed to upload ISO oVirt 4.4.0 - imageio unable to verify certificate
I have changed the Engine SSL certificate with our own certificates (using FreeIPA), i have followed the procedure described in the documents carefully and it is indeed working well. I haven now tried to upload an ISO and it fails, using "test connection" says the connection to the imageio service is ok, but when the upload starts it fails. I can see those errors in the ovirt-imageio daemon log file. 2020-07-10 06:00:48,386 ERROR (Thread-11) [http] Server error Traceback (most recent call last): File "/usr/lib64/python3.6/site-packages/ovirt_imageio/_internal/http.py", line 699, in __call__ self.dispatch(req, resp) File "/usr/lib64/python3.6/site-packages/ovirt_imageio/_internal/http.py", line 744, in dispatch return method(req, resp, *match.groups()) File "/usr/lib64/python3.6/site-packages/ovirt_imageio/_internal/cors.py", line 84, in wrapper return func(self, req, resp, *args) File "/usr/lib64/python3.6/site-packages/ovirt_imageio/_internal/images.py", line 66, in put backends.get(req, ticket, self.config), File "/usr/lib64/python3.6/site-packages/ovirt_imageio/_internal/backends/__init__.py", line 53, in get cafile=config.tls.ca_file) File "/usr/lib64/python3.6/site-packages/ovirt_imageio/_internal/backends/http.py", line 48, in open secure=options.get("secure", True)) File "/usr/lib64/python3.6/site-packages/ovirt_imageio/_internal/backends/http.py", line 63, in __init__ options = self._options() File "/usr/lib64/python3.6/site-packages/ovirt_imageio/_internal/backends/http.py", line 364, in _options self._con.request("OPTIONS", self.url.path) File "/usr/lib64/python3.6/http/client.py", line 1254, in request self._send_request(method, url, body, headers, encode_chunked) File "/usr/lib64/python3.6/http/client.py", line 1300, in _send_request self.endheaders(body, encode_chunked=encode_chunked) File "/usr/lib64/python3.6/http/client.py", line 1249, in endheaders self._send_output(message_body, encode_chunked=encode_chunked) File "/usr/lib64/python3.6/http/client.py", line 1036, in _send_output self.send(msg) File "/usr/lib64/python3.6/http/client.py", line 974, in send self.connect() File "/usr/lib64/python3.6/http/client.py", line 1422, in connect server_hostname=server_hostname) File "/usr/lib64/python3.6/ssl.py", line 365, in wrap_socket _context=self, _session=session) File "/usr/lib64/python3.6/ssl.py", line 776, in __init__ self.do_handshake() File "/usr/lib64/python3.6/ssl.py", line 1036, in do_handshake self._sslobj.do_handshake() File "/usr/lib64/python3.6/ssl.py", line 648, in do_handshake self._sslobj.do_handshake() ssl.SSLError: [SSL: CERTIFICATE_VERIFY_FAILED] certificate verify failed (_ssl.c:897) Looking at old posts here, i have also tried: keytool -importcert -trustcacerts -keystore /etc/pki/ovirt-engine/.truststore -storepass mypass -file /etc/pki/ovirt-engine/apache-ca.pem And restarted both the ovirt-imageio service and the ovirt-engine service, it didn't help.
There is a lot of misinformation, I don't have the ovirt-imageio-proxy service and only ovirt-imageio, can i assume this is the same? I have tried to follow every workaround i have found (including many bugs that were closed) but none helped me to solve it. I can confirm that this worked well in 4.3.10.
Replying to myself again, i managed to "solve" this. in /etc/ovirt-imageio/conf.d/50-engine.conf it uses the key_file and cert_file of the apache by default. For the CA cert it is indeed using the apache-ca.pem as expected (?), it seems to use the same CA when trying to reach the VDSM imageio daemon running on each host for obvious reasons those are two different CA, the apache-ca.pem is used by the Engine "frontend". Changing the ca_file to /etc/pki/ovirt-engine/ca.pem and restart the imageio daemon on the ovirt-engine solved this issue. The information here: http://ovirt.github.io/ovirt-imageio/overview.html is misleading.
+Yedidyah Bar David <didi@redhat.com> , +Nir Soffer <nsoffer@redhat.com> , +Eyal Shenitzky <eshenitz@redhat.com> can you please have a look here? Il giorno ven 10 lug 2020 alle ore 09:56 Erez Zarum <erezz@nanosek.com> ha scritto:
Replying to myself again, i managed to "solve" this. in /etc/ovirt-imageio/conf.d/50-engine.conf it uses the key_file and cert_file of the apache by default. For the CA cert it is indeed using the apache-ca.pem as expected (?), it seems to use the same CA when trying to reach the VDSM imageio daemon running on each host for obvious reasons those are two different CA, the apache-ca.pem is used by the Engine "frontend". Changing the ca_file to /etc/pki/ovirt-engine/ca.pem and restart the imageio daemon on the ovirt-engine solved this issue. The information here: http://ovirt.github.io/ovirt-imageio/overview.html is misleading. _______________________________________________ Users mailing list -- users@ovirt.org To unsubscribe send an email to users-leave@ovirt.org Privacy Statement: https://www.ovirt.org/privacy-policy.html oVirt Code of Conduct: https://www.ovirt.org/community/about/community-guidelines/ List Archives: https://lists.ovirt.org/archives/list/users@ovirt.org/message/FUV7B43YSWAG6E...
-- Sandro Bonazzola MANAGER, SOFTWARE ENGINEERING, EMEA R&D RHV Red Hat EMEA <https://www.redhat.com/> sbonazzo@redhat.com <https://www.redhat.com/> *Red Hat respects your work life balance. Therefore there is no need to answer this email out of your office hours. <https://mojo.redhat.com/docs/DOC-1199578>*
On Fri, Jul 10, 2020 at 10:56 AM Erez Zarum <erezz@nanosek.com> wrote:
Replying to myself again, i managed to "solve" this.
You actually solved it, no quotes required :-)
in /etc/ovirt-imageio/conf.d/50-engine.conf it uses the key_file and cert_file of the apache by default. For the CA cert it is indeed using the apache-ca.pem as expected (?), it seems to use the same CA when trying to reach the VDSM imageio daemon running on each host for obvious reasons those are two different CA, the apache-ca.pem is used by the Engine "frontend". Changing the ca_file to /etc/pki/ovirt-engine/ca.pem and restart the imageio daemon on the ovirt-engine solved this issue.
Right, you need to change the ovirt-imgeio configuration to replace the CA. But note that you should not touch: /etc/ovirt-imageio/conf.d/50-engine.conf This file is owned by engine and your changes will be dropped silently on the next upgrade. You need to add your own configuration file, maybe: /etc/ovirt-imageio/99-local.conf Where you can override what you need: [tls] ca_file = ... This is documented in the top of 50-vdsm.conf: # Configuration overrides for vdsm. # # WARNING: This file owned by vdsm. If you modify this file your changes will # be overwritten in the next vdsm upgrade. # # To change the configuration create a new drop-in file with a higher prefix, # so your setting will override vdsm and builtin configuration: # # $ cat /etc/ovirt-imageio/conf.d/99-locl.conf # [logger_root] # level = DEBUG # # This example overrides ovirt-imageio service log level to DEBUG. But the documentation is missing on engine side. Please file engine bug for this.
The information here: http://ovirt.github.io/ovirt-imageio/overview.html is misleading.
Please file ovirt-imageio Documentation bug for this. Nir
On Mon, Jul 13, 2020 at 5:46 PM Nir Soffer <nsoffer@redhat.com> wrote:
On Fri, Jul 10, 2020 at 10:56 AM Erez Zarum <erezz@nanosek.com> wrote:
Replying to myself again, i managed to "solve" this.
You actually solved it, no quotes required :-)
in /etc/ovirt-imageio/conf.d/50-engine.conf it uses the key_file and cert_file of the apache by default. For the CA cert it is indeed using the apache-ca.pem as expected (?), it seems to use the same CA when trying to reach the VDSM imageio daemon running on each host for obvious reasons those are two different CA, the apache-ca.pem is used by the Engine "frontend". Changing the ca_file to /etc/pki/ovirt-engine/ca.pem and restart the imageio daemon on the ovirt-engine solved this issue.
Right, you need to change the ovirt-imgeio configuration to replace the CA.
But note that you should not touch:
/etc/ovirt-imageio/conf.d/50-engine.conf
This file is owned by engine and your changes will be dropped silently on the next upgrade.
You need to add your own configuration file, maybe:
/etc/ovirt-imageio/99-local.conf
Where you can override what you need:
[tls] ca_file = ...
This is documented in the top of 50-vdsm.conf:
# Configuration overrides for vdsm. # # WARNING: This file owned by vdsm. If you modify this file your changes will # be overwritten in the next vdsm upgrade. # # To change the configuration create a new drop-in file with a higher prefix, # so your setting will override vdsm and builtin configuration: # # $ cat /etc/ovirt-imageio/conf.d/99-locl.conf # [logger_root] # level = DEBUG # # This example overrides ovirt-imageio service log level to DEBUG.
But the documentation is missing on engine side. Please file engine bug for this.
Sorry, this is already documented also on engine side: # Configuration overrides for ovirt-engine. # # WARNING: This file owned by ovirt-engine. If you modify this file your # changes will be overwritten in the next ovirt-engine upgrade. # # To change the configuration create a new drop-in file with higher prefix, # so your setting will override ovirt-engine configuration: # # $ cat /etc/ovirt-imageio/conf.d/99-locl.conf # [tls] # ca_file = # # This example overrides ca_file to be empty string. This can be useful if # the host certificates are signed by a trusted CA. There are some typos but it is very clear. Typos fixed here: https://gerrit.ovirt.org/c/110265/ https://gerrit.ovirt.org/c/110266/
The information here: http://ovirt.github.io/ovirt-imageio/overview.html is misleading.
Please file ovirt-imageio Documentation bug for this.
Nir
participants (3)
-
Erez Zarum -
Nir Soffer -
Sandro Bonazzola