9:51 a.m.
I installed an SSL cert from a public CA (Let's Encrypt) on my engine,
following this:
https://access.redhat.com/documentation/en-us/red_hat_virtualization/4.2/...
That gets the regular web UI working, but I can't upload an ISO. I
assume that I need to do something with the imageio-proxy service on the
engine, but not sure what... I tried replacing imageio-proxy.cer and
imageio-proxy.key.nopass, but that didn't work.
I'm trying to avoid ever needing to install a special CA cert in
browsers.
--
Chris Adams <cma(a)cmadams.net>
11:07 a.m.
Did you try running engine-setup ?
Regards,
Paul S.
________________________________________
From: Chris Adams <cma(a)cmadams.net>
Sent: 29 January 2019 15:51
To: users(a)ovirt.org
Subject: [ovirt-users] Use public-signed SSL certs?
I installed an SSL cert from a public CA (Let's Encrypt) on my engine,
following this:
https://access.redhat.com/documentation/en-us/red_hat_virtualization/4.2/...
That gets the regular web UI working, but I can't upload an ISO. I
assume that I need to do something with the imageio-proxy service on the
engine, but not sure what... I tried replacing imageio-proxy.cer and
imageio-proxy.key.nopass, but that didn't work.
I'm trying to avoid ever needing to install a special CA cert in
browsers.
--
Chris Adams <cma(a)cmadams.net>
_______________________________________________
Users mailing list -- users(a)ovirt.org
To unsubscribe send an email to users-leave(a)ovirt.org
Privacy Statement: https://www.ovirt.org/site/privacy-policy/
oVirt Code of Conduct: https://www.ovirt.org/community/about/community-guidelines/
List Archives:
https://lists.ovirt.org/archives/list/users@ovirt.org/message/NZBGRCKW6WA...
To view the terms under which this email is distributed, please go to:-
http://leedsbeckett.ac.uk/disclaimer/email/
12:30 p.m.
I had not. Can that be run non-interactively to do whatever is needed?
I'm using a Let's Encrypt cert, which needs to have a 100% automated
deployment.
Once upon a time, Staniforth, Paul <P.Staniforth(a)leedsbeckett.ac.uk> said:
Did you try running engine-setup ?
Regards,
Paul S.
________________________________________
From: Chris Adams <cma(a)cmadams.net>
Sent: 29 January 2019 15:51
To: users(a)ovirt.org
Subject: [ovirt-users] Use public-signed SSL certs?
I installed an SSL cert from a public CA (Let's Encrypt) on my engine,
following this:
https://access.redhat.com/documentation/en-us/red_hat_virtualization/4.2/...
That gets the regular web UI working, but I can't upload an ISO. I
assume that I need to do something with the imageio-proxy service on the
engine, but not sure what... I tried replacing imageio-proxy.cer and
imageio-proxy.key.nopass, but that didn't work.
I'm trying to avoid ever needing to install a special CA cert in
browsers.
--
Chris Adams <cma(a)cmadams.net>
_______________________________________________
Users mailing list -- users(a)ovirt.org
To unsubscribe send an email to users-leave(a)ovirt.org
Privacy Statement: https://www.ovirt.org/site/privacy-policy/
oVirt Code of Conduct: https://www.ovirt.org/community/about/community-guidelines/
List Archives:
https://lists.ovirt.org/archives/list/users@ovirt.org/message/NZBGRCKW6WA...
To view the terms under which this email is distributed, please go to:-
http://leedsbeckett.ac.uk/disclaimer/email/
--
Chris Adams <cma(a)cmadams.net>
1:29 p.m.
On 1/29/19 1:30 PM, Chris Adams wrote:
Can that be run non-interactively to do whatever is needed?
I'm using a Let's Encrypt cert, which needs to have a 100% automated
deployment.
Yes, I believe so. Look at the whole biz with the "answers" file and
the --config-append=file option. You should already have a generated
answers file laying around from when you ran engine-setup before. See
/var/lib/ovirt-engine/setup/answers IIRC.
1:47 p.m.
Once upon a time, John Florian <jflorian(a)doubledog.org> said:
On 1/29/19 1:30 PM, Chris Adams wrote:
>Can that be run non-interactively to do whatever is needed?
>I'm using a Let's Encrypt cert, which needs to have a 100% automated
>deployment.
Yes, I believe so. Look at the whole biz with the "answers" file
and the --config-append=file option. You should already have a
generated answers file laying around from when you ran engine-setup
before. See /var/lib/ovirt-engine/setup/answers IIRC.
Hmm, that won't work - it looks like you can't run engine-setup on a
hosted engine unless you first set hosted-engine HA to global
maintenance.
Is running engine-setup necessary to install/update certificates, or
maybe is there a simpler way?
--
Chris Adams <cma(a)cmadams.net>
2:13 p.m.
On 1/29/19 2:47 PM, Chris Adams wrote:
Once upon a time, John Florian <jflorian(a)doubledog.org> said:
> On 1/29/19 1:30 PM, Chris Adams wrote:
>> Can that be run non-interactively to do whatever is needed?
>> I'm using a Let's Encrypt cert, which needs to have a 100% automated
>> deployment.
> Yes, I believe so. Look at the whole biz with the "answers" file
> and the --config-append=file option. You should already have a
> generated answers file laying around from when you ran engine-setup
> before. See /var/lib/ovirt-engine/setup/answers IIRC.
Hmm, that won't work - it looks like you can't run engine-setup on a
hosted engine unless you first set hosted-engine HA to global
maintenance.
Is running engine-setup necessary to install/update certificates, or
maybe is there a simpler way?
I'm quite certain you can do it w/o engine-setup if you hit all the
right file locations.
5:35 p.m.
On 1/29/19 3:13 PM, John Florian wrote:
On 1/29/19 2:47 PM, Chris Adams wrote:
> Once upon a time, John Florian <jflorian(a)doubledog.org> said:
>> On 1/29/19 1:30 PM, Chris Adams wrote:
>>> Can that be run non-interactively to do whatever is needed?
>>> I'm using a Let's Encrypt cert, which needs to have a 100% automated
>>> deployment.
>> Yes, I believe so. Look at the whole biz with the "answers" file
>> and the --config-append=file option. You should already have a
>> generated answers file laying around from when you ran engine-setup
>> before. See /var/lib/ovirt-engine/setup/answers IIRC.
> Hmm, that won't work - it looks like you can't run engine-setup on a
> hosted engine unless you first set hosted-engine HA to global
> maintenance.
>
> Is running engine-setup necessary to install/update certificates, or
> maybe is there a simpler way?
I'm quite certain you can do it w/o engine-setup if you hit all the
right file locations.
Just to follow up on this Chris, I have my puppet drop my CA cert in
/etc/pki/ca-trust/source/anchors/, my self-signed cert
in/etc/pki/ovirt-engine/certs/ and my key in
/etc/pki/ovirt-engine/keys. I also manage
/etc/ovirt-engine/engine.conf.d/99-custom-truststore.conf to have:
ENGINE_HTTPS_PKI_TRUST_STORE="/etc/pki/java/cacerts"
ENGINE_HTTPS_PKI_TRUST_STORE_PASSWORD=""
I believe this gives me everything you seek.
--
John Florian
6:48 p.m.
Once upon a time, John Florian <jflorian(a)doubledog.org> said:
Just to follow up on this Chris, I have my puppet drop my CA cert in
/etc/pki/ca-trust/source/anchors/, my self-signed cert
in/etc/pki/ovirt-engine/certs/ and my key in
/etc/pki/ovirt-engine/keys. I also manage
/etc/ovirt-engine/engine.conf.d/99-custom-truststore.conf to have:
ENGINE_HTTPS_PKI_TRUST_STORE="/etc/pki/java/cacerts"
ENGINE_HTTPS_PKI_TRUST_STORE_PASSWORD=""
I believe this gives me everything you seek.
That works to get the core engine UI using a new cert (that and a little
more are in the Red Hat URL in my original message). It doesn't handle
the imageio-proxy however.
--
Chris Adams <cma(a)cmadams.net>
1:45 a.m.
On Tue, Jan 29, 2019 at 6:05 PM Chris Adams <cma(a)cmadams.net> wrote:
I installed an SSL cert from a public CA (Let's Encrypt) on my engine,
following this:
https://access.redhat.com/documentation/en-us/red_hat_virtualization/4.2/...
That gets the regular web UI working, but I can't upload an ISO. I
assume that I need to do something with the imageio-proxy service on the
engine, but not sure what... I tried replacing imageio-proxy.cer and
imageio-proxy.key.nopass, but that didn't work.
Did you restart the imageio-proxy?
What didn't work? What happened?
I'm trying to avoid ever needing to install a special CA cert in
browsers.
Makes sense.
This is known bug:
https://bugzilla.redhat.com/show_bug.cgi?id=1637809
Before opening it, we had a bug about fixing the documentation you
point at:
https://bugzilla.redhat.com/show_bug.cgi?id=1385617
As mentioned there, what you tried to do should have worked.
Best regards,
--
Didi
8:17 a.m.
Once upon a time, Yedidyah Bar David <didi(a)redhat.com> said:
On Tue, Jan 29, 2019 at 6:05 PM Chris Adams <cma(a)cmadams.net>
wrote:
> I installed an SSL cert from a public CA (Let's Encrypt) on my engine,
> following this:
>
>
https://access.redhat.com/documentation/en-us/red_hat_virtualization/4.2/...
>
> That gets the regular web UI working, but I can't upload an ISO. I
> assume that I need to do something with the imageio-proxy service on the
> engine, but not sure what... I tried replacing imageio-proxy.cer and
> imageio-proxy.key.nopass, but that didn't work.
Did you restart the imageio-proxy?
What didn't work? What happened?
I did restart the service. When I then try to upload an ISO image, I
get "Paused by System" and this in engine.log:
########################################################################
2019-01-30 08:12:15,871-06 ERROR
[org.ovirt.engine.core.bll.storage.disk.image.TransferDiskImageCommand]
(EE-ManagedThreadFactory-engineScheduled-Thread-52) [0052c7ad-38d7-429d-be3a-eb0e496d5ee8]
Failed to add image ticket to ovirt-imageio-proxy: javax.net.ssl.SSLHandshakeException:
sun.security.validator.ValidatorException: PKIX path building failed:
sun.security.provider.certpath.SunCertPathBuilderException: unable to find valid
certification path to requested target
at sun.security.ssl.Alerts.getSSLException(Alerts.java:192) [jsse.jar:1.8.0_191]
at sun.security.ssl.SSLSocketImpl.fatal(SSLSocketImpl.java:1946) [jsse.jar:1.8.0_191]
at sun.security.ssl.Handshaker.fatalSE(Handshaker.java:316) [jsse.jar:1.8.0_191]
at sun.security.ssl.Handshaker.fatalSE(Handshaker.java:310) [jsse.jar:1.8.0_191]
at sun.security.ssl.ClientHandshaker.serverCertificate(ClientHandshaker.java:1639)
[jsse.jar:1.8.0_191]
at sun.security.ssl.ClientHandshaker.processMessage(ClientHandshaker.java:223)
[jsse.jar:1.8.0_191]
at sun.security.ssl.Handshaker.processLoop(Handshaker.java:1037) [jsse.jar:1.8.0_191]
at sun.security.ssl.Handshaker.process_record(Handshaker.java:965) [jsse.jar:1.8.0_191]
at sun.security.ssl.SSLSocketImpl.readRecord(SSLSocketImpl.java:1064)
[jsse.jar:1.8.0_191]
at sun.security.ssl.SSLSocketImpl.performInitialHandshake(SSLSocketImpl.java:1367)
[jsse.jar:1.8.0_191]
at sun.security.ssl.SSLSocketImpl.startHandshake(SSLSocketImpl.java:1395)
[jsse.jar:1.8.0_191]
at sun.security.ssl.SSLSocketImpl.startHandshake(SSLSocketImpl.java:1379)
[jsse.jar:1.8.0_191]
at sun.net.www.protocol.https.HttpsClient.afterConnect(HttpsClient.java:559)
[rt.jar:1.8.0_191]
at
sun.net.www.protocol.https.AbstractDelegateHttpsURLConnection.connect(AbstractDelegateHttpsURLConnection.java:185)
[rt.jar:1.8.0_191]
at
sun.net.www.protocol.http.HttpURLConnection.getOutputStream0(HttpURLConnection.java:1334)
[rt.jar:1.8.0_191]
at
sun.net.www.protocol.http.HttpURLConnection.getOutputStream(HttpURLConnection.java:1309)
[rt.jar:1.8.0_191]
at
sun.net.www.protocol.https.HttpsURLConnectionImpl.getOutputStream(HttpsURLConnectionImpl.java:259)
[rt.jar:1.8.0_191]
at
org.ovirt.engine.core.bll.storage.disk.image.TransferImageCommand.addImageTicketToProxy(TransferImageCommand.java:654)
[bll.jar:]
at
org.ovirt.engine.core.bll.storage.disk.image.TransferImageCommand.startImageTransferSession(TransferImageCommand.java:579)
[bll.jar:]
at
org.ovirt.engine.core.bll.storage.disk.image.TransferImageCommand.handleImageIsReadyForTransfer(TransferImageCommand.java:261)
[bll.jar:]
at
org.ovirt.engine.core.bll.storage.disk.image.TransferImageCommand.handleInitializing(TransferImageCommand.java:232)
[bll.jar:]
at
org.ovirt.engine.core.bll.storage.disk.image.TransferImageCommand.executeStateHandler(TransferImageCommand.java:167)
[bll.jar:]
at
org.ovirt.engine.core.bll.storage.disk.image.TransferImageCommand.proceedCommandExecution(TransferImageCommand.java:154)
[bll.jar:]
at
org.ovirt.engine.core.bll.storage.disk.image.TransferImageCommandCallback.doPolling(TransferImageCommandCallback.java:21)
[bll.jar:]
at
org.ovirt.engine.core.bll.tasks.CommandCallbacksPoller.invokeCallbackMethodsImpl(CommandCallbacksPoller.java:146)
[bll.jar:]
at
org.ovirt.engine.core.bll.tasks.CommandCallbacksPoller.invokeCallbackMethods(CommandCallbacksPoller.java:107)
[bll.jar:]
at java.util.concurrent.Executors$RunnableAdapter.call(Executors.java:511)
[rt.jar:1.8.0_191]
at java.util.concurrent.FutureTask.runAndReset(FutureTask.java:308) [rt.jar:1.8.0_191]
at
org.glassfish.enterprise.concurrent.internal.ManagedScheduledThreadPoolExecutor$ManagedScheduledFutureTask.access$201(ManagedScheduledThreadPoolExecutor.java:383)
[javax.enterprise.concurrent-1.0.jar:]
at
org.glassfish.enterprise.concurrent.internal.ManagedScheduledThreadPoolExecutor$ManagedScheduledFutureTask.run(ManagedScheduledThreadPoolExecutor.java:534)
[javax.enterprise.concurrent-1.0.jar:]
at java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1149)
[rt.jar:1.8.0_191]
at java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:624)
[rt.jar:1.8.0_191]
at java.lang.Thread.run(Thread.java:748) [rt.jar:1.8.0_191]
at
org.glassfish.enterprise.concurrent.ManagedThreadFactoryImpl$ManagedThread.run(ManagedThreadFactoryImpl.java:250)
[javax.enterprise.concurrent-1.0.jar:]
at
org.jboss.as.ee.concurrent.service.ElytronManagedThreadFactory$ElytronManagedThread.run(ElytronManagedThreadFactory.java:78)
Caused by: sun.security.validator.ValidatorException: PKIX path building failed:
sun.security.provider.certpath.SunCertPathBuilderException: unable to find valid
certification path to requested target
at sun.security.validator.PKIXValidator.doBuild(PKIXValidator.java:397)
[rt.jar:1.8.0_191]
at sun.security.validator.PKIXValidator.engineValidate(PKIXValidator.java:302)
[rt.jar:1.8.0_191]
at sun.security.validator.Validator.validate(Validator.java:262) [rt.jar:1.8.0_191]
at sun.security.ssl.X509TrustManagerImpl.validate(X509TrustManagerImpl.java:324)
[jsse.jar:1.8.0_191]
at sun.security.ssl.X509TrustManagerImpl.checkTrusted(X509TrustManagerImpl.java:229)
[jsse.jar:1.8.0_191]
at
sun.security.ssl.X509TrustManagerImpl.checkServerTrusted(X509TrustManagerImpl.java:124)
[jsse.jar:1.8.0_191]
at sun.security.ssl.ClientHandshaker.serverCertificate(ClientHandshaker.java:1621)
[jsse.jar:1.8.0_191]
... 30 more
Caused by: sun.security.provider.certpath.SunCertPathBuilderException: unable to find
valid certification path to requested target
at sun.security.provider.certpath.SunCertPathBuilder.build(SunCertPathBuilder.java:141)
[rt.jar:1.8.0_191]
at
sun.security.provider.certpath.SunCertPathBuilder.engineBuild(SunCertPathBuilder.java:126)
[rt.jar:1.8.0_191]
at java.security.cert.CertPathBuilder.build(CertPathBuilder.java:280) [rt.jar:1.8.0_191]
at sun.security.validator.PKIXValidator.doBuild(PKIXValidator.java:392)
[rt.jar:1.8.0_191]
... 36 more
########################################################################
I'm guessing that I affected the engine's ability to validate the
public-CA-signed cert on the imageio-proxy? Maybe I just messed
something else up?
> I'm trying to avoid ever needing to install a special CA
cert in
> browsers.
Makes sense.
This is known bug:
https://bugzilla.redhat.com/show_bug.cgi?id=1637809
Before opening it, we had a bug about fixing the documentation you
point at:
https://bugzilla.redhat.com/show_bug.cgi?id=1385617
As mentioned there, what you tried to do should have worked.
I saw the second BZ and read through it. I was taking the approach of
replacing the imageio-proxy key/cert rather than repointing it; I've
switched to just changing the config but have the same issue.
--
Chris Adams <cma(a)cmadams.net>
2:26 p.m.
Digging a little deeper... if I add the Let's Encrypt CA to
/etc/pki/ovirt-engine/.truststore, imageio-proxy works (I can
successfully upload an ISO), so I guess the issue is that imageio-proxy
uses the same cert for web and engine communication and the engine
wasn't happy with the public-CA-signed cert.
So, rather than point part of the engine at a separate trust store (as
the docs recommend), maybe just add the public CA to the engine's
existing trust store?
However, while digging, I also noticed that now the engine is not
communicating with ovirt-provider-ovn, possibly due to a similar issue?
It is having the reverse problem; it rejects the engine's cert.
This is all on 4.2.8 BTW.
--
Chris Adams <cma(a)cmadams.net>
3:19 p.m.
I got it working by changing the SSL certs pointed out in the
/etc/imageio-proxy/imageio-proxy.conf.
BR
Chris Adams <cma(a)cmadams.net> schrieb am Mi., 30. Jan. 2019, 21:28:
Digging a little deeper... if I add the Let's Encrypt CA to
/etc/pki/ovirt-engine/.truststore, imageio-proxy works (I can
successfully upload an ISO), so I guess the issue is that imageio-proxy
uses the same cert for web and engine communication and the engine
wasn't happy with the public-CA-signed cert.
So, rather than point part of the engine at a separate trust store (as
the docs recommend), maybe just add the public CA to the engine's
existing trust store?
However, while digging, I also noticed that now the engine is not
communicating with ovirt-provider-ovn, possibly due to a similar issue?
It is having the reverse problem; it rejects the engine's cert.
This is all on 4.2.8 BTW.
--
Chris Adams <cma(a)cmadams.net>
_______________________________________________
Users mailing list -- users(a)ovirt.org
To unsubscribe send an email to users-leave(a)ovirt.org
Privacy Statement: https://www.ovirt.org/site/privacy-policy/
oVirt Code of Conduct:
https://www.ovirt.org/community/about/community-guidelines/
List Archives:
https://lists.ovirt.org/archives/list/users@ovirt.org/message/FC6FNKINSVQ...
5:21 a.m.
On Wed, Jan 30, 2019 at 10:28 PM Chris Adams <cma(a)cmadams.net> wrote:
Digging a little deeper... if I add the Let's Encrypt CA to
/etc/pki/ovirt-engine/.truststore, imageio-proxy works (I can
successfully upload an ISO), so I guess the issue is that imageio-proxy
uses the same cert for web and engine communication and the engine
wasn't happy with the public-CA-signed cert.
I think I agree with your analysis.
I now reproduced this on a test env.
I started with ovirt-system-tests basic suite deploy, made sure I
can upload an image.
Then I followed the docs about replacing certs, using a temporarily-
created CA for testing (using openssl, actually using a copy of the
engine's pki scripts), including adding 99-custom-truststore.conf,
imported the CA's cert to the browser, and:
1. Connecting with the browser worked, all is green.
2. Logged in, pressed "Disks -> Upload -> Start -> Test Connection",
and it failed.
3. Edited the ovirt-imageio-proxy conf to point key and cert to a
key and cert I created and signed using my temp ca, restarted it,
"Test Connection" worked.
4. Actually uploading the image failed as you describe.
5. Imported my CA's cert to /etc/pki/ovirt-engine/.truststore,
using:
keytool -importcert -trustcacerts -keystore
/etc/pki/ovirt-engine/.truststore -storepass mypass -file
/etc/pki/ovirt-engine/apache-ca.pem
and restarted the engine, and then upload works.
Adding Martin and Nir.
So, rather than point part of the engine at a separate trust store (as
the docs recommend), maybe just add the public CA to the engine's
existing trust store?
I admit I still didn't try to fully analyze this myself, but I tend
to agree with you. Or rather: Our docs should probably support both
options - tell the engine to trust (and use?) the system-wide store,
or manually add a specific cert. Because I guess you can find people
that will prefer either option.
However, while digging, I also noticed that now the engine is not
communicating with ovirt-provider-ovn, possibly due to a similar issue?
It is having the reverse problem; it rejects the engine's cert.
Didn't try this yet, adding Dominik.
This is all on 4.2.8 BTW.
I personally tried this on:
ovirt-engine-4.3.0-0.8.master.20190122121624.git9a8a519.el7.noarch
I guess the behavior didn't change much between them.
Thanks for your debugging and report!
Best regards,
--
Didi
6:44 a.m.
On Mon, 4 Feb 2019 13:21:56 +0200
Yedidyah Bar David <didi(a)redhat.com> wrote:
On Wed, Jan 30, 2019 at 10:28 PM Chris Adams <cma(a)cmadams.net>
wrote:
>
> Digging a little deeper... if I add the Let's Encrypt CA to
> /etc/pki/ovirt-engine/.truststore, imageio-proxy works (I can
> successfully upload an ISO), so I guess the issue is that imageio-proxy
> uses the same cert for web and engine communication and the engine
> wasn't happy with the public-CA-signed cert.
I think I agree with your analysis.
I now reproduced this on a test env.
I started with ovirt-system-tests basic suite deploy, made sure I
can upload an image.
Then I followed the docs about replacing certs, using a temporarily-
created CA for testing (using openssl, actually using a copy of the
engine's pki scripts), including adding 99-custom-truststore.conf,
imported the CA's cert to the browser, and:
1. Connecting with the browser worked, all is green.
2. Logged in, pressed "Disks -> Upload -> Start -> Test Connection",
and it failed.
3. Edited the ovirt-imageio-proxy conf to point key and cert to a
key and cert I created and signed using my temp ca, restarted it,
"Test Connection" worked.
4. Actually uploading the image failed as you describe.
5. Imported my CA's cert to /etc/pki/ovirt-engine/.truststore,
using:
keytool -importcert -trustcacerts -keystore
/etc/pki/ovirt-engine/.truststore -storepass mypass -file
/etc/pki/ovirt-engine/apache-ca.pem
and restarted the engine, and then upload works.
Adding Martin and Nir.
>
> So, rather than point part of the engine at a separate trust store (as
> the docs recommend), maybe just add the public CA to the engine's
> existing trust store?
I admit I still didn't try to fully analyze this myself, but I tend
to agree with you. Or rather: Our docs should probably support both
options - tell the engine to trust (and use?) the system-wide store,
or manually add a specific cert. Because I guess you can find people
that will prefer either option.
>
> However, while digging, I also noticed that now the engine is not
> communicating with ovirt-provider-ovn, possibly due to a similar issue?
> It is having the reverse problem; it rejects the engine's cert.
Didn't try this yet, adding Dominik.
Please ensure that the configured certificates in
/etc/ovirt-provider-ovn/conf.d/10-setup-ovirt-provider-ovn.conf
especially ovirt-ca-file, points to the expected files and restart
ovirt-provider-ovn.
If this does not solve the issue, please share
ovirt-provider-ovn.log.
>
> This is all on 4.2.8 BTW.
I personally tried this on:
ovirt-engine-4.3.0-0.8.master.20190122121624.git9a8a519.el7.noarch
I guess the behavior didn't change much between them.
Thanks for your debugging and report!
Best regards,
5:02 a.m.
On Mon, Feb 4, 2019 at 1:21 PM Yedidyah Bar David <didi(a)redhat.com> wrote:
On Wed, Jan 30, 2019 at 10:28 PM Chris Adams <cma(a)cmadams.net> wrote:
>
> Digging a little deeper... if I add the Let's Encrypt CA to
> /etc/pki/ovirt-engine/.truststore, imageio-proxy works (I can
> successfully upload an ISO), so I guess the issue is that imageio-proxy
> uses the same cert for web and engine communication and the engine
> wasn't happy with the public-CA-signed cert.
I think I agree with your analysis.
I now reproduced this on a test env.
I started with ovirt-system-tests basic suite deploy, made sure I
can upload an image.
Then I followed the docs about replacing certs, using a temporarily-
created CA for testing (using openssl, actually using a copy of the
engine's pki scripts), including adding 99-custom-truststore.conf,
imported the CA's cert to the browser, and:
1. Connecting with the browser worked, all is green.
2. Logged in, pressed "Disks -> Upload -> Start -> Test Connection",
and it failed.
3. Edited the ovirt-imageio-proxy conf to point key and cert to a
key and cert I created and signed using my temp ca, restarted it,
"Test Connection" worked.
4. Actually uploading the image failed as you describe.
5. Imported my CA's cert to /etc/pki/ovirt-engine/.truststore,
using:
keytool -importcert -trustcacerts -keystore
/etc/pki/ovirt-engine/.truststore -storepass mypass -file
/etc/pki/ovirt-engine/apache-ca.pem
and restarted the engine, and then upload works.
Adding Martin and Nir.
>
> So, rather than point part of the engine at a separate trust store (as
> the docs recommend), maybe just add the public CA to the engine's
> existing trust store?
I admit I still didn't try to fully analyze this myself, but I tend
to agree with you. Or rather: Our docs should probably support both
options - tell the engine to trust (and use?) the system-wide store,
or manually add a specific cert. Because I guess you can find people
that will prefer either option.
Decided that only the first makes sense, opened this bug, should be
fixed in 4.3.2:
https://bugzilla.redhat.com/1687301
This is obviously just one step. The next will be:
https://bugzilla.redhat.com/show_bug.cgi?id=1637809
Then, hopefully, following the existing doc to use 3rd-party CA
will "just work" also for imageio.
BTW, of course you can also create another custom truststore
only for https access to the engine, and point
ENGINE_HTTPS_PKI_TRUST_STORE at it - but I wouldn't add this to
the docs before we have automated testing that makes sure this
does not break in the future.
Best regards,
>
> However, while digging, I also noticed that now the engine is not
> communicating with ovirt-provider-ovn, possibly due to a similar issue?
> It is having the reverse problem; it rejects the engine's cert.
Didn't try this yet, adding Dominik.
>
> This is all on 4.2.8 BTW.
I personally tried this on:
ovirt-engine-4.3.0-0.8.master.20190122121624.git9a8a519.el7.noarch
I guess the behavior didn't change much between them.
Thanks for your debugging and report!
Best regards,
--
Didi
--
Didi
3:33 p.m.
Circling back to an old email...
Once upon a time, Yedidyah Bar David <didi(a)redhat.com> said:
On Wed, Jan 30, 2019 at 10:28 PM Chris Adams <cma(a)cmadams.net>
wrote:
> However, while digging, I also noticed that now the engine is not
> communicating with ovirt-provider-ovn, possibly due to a similar issue?
> It is having the reverse problem; it rejects the engine's cert.
Didn't try this yet, adding Dominik.
Was anybody able to look at this? I had to use my dev hardware for
something else for a bit, so re-installed with 4.3.5 yesterday. The
imageio SSL cert issue looks good, but I still can't figure out the
ovirt-provider-ovn CA usage.
My little bit of digging seems to show that the engine connects to the
provider and is using an SSL client cert, and that cert is signed by
something... but I'm not sure what. I think the provider side is trying
to validate with the following setting from
/etc/ovirt-provider-ovn/conf.d/10-setup-ovirt-provider-ovn.conf
[OVIRT]
ovirt-ca-file=/etc/pki/ovirt-engine/apache-ca.pem
Following the general "3rd-party SSL", that is now the Let's Encrypt CA.
I tried changing it to point to the original self-signed oVirt CA (same
directory, just "ca.pem"), but that didn't work either.
Any suggestions?
--
Chris Adams <cma(a)cmadams.net>
8:45 p.m.
I figured it out. When ovirt-provider-ovn attempts to connect back to
the engine via HTTPS, it tells the python requests module to use the
specified CA cert file... but that won't work with most 3rd-party certs
because they have an intermediate cert as well. It appears that the
requests module tries to validate both certs.
Creating /etc/ovirt-provider-ovn/conf.d/99-custom-cert.conf that just
has:
[OVIRT]
ovirt-ca-file=
tells the module to use the regular system CA cert file(s), which works.
This should probably be added to the oVirt doc for using a 3rd-party
cert.
Once upon a time, Chris Adams <cma(a)cmadams.net> said:
Circling back to an old email...
Once upon a time, Yedidyah Bar David <didi(a)redhat.com> said:
> On Wed, Jan 30, 2019 at 10:28 PM Chris Adams <cma(a)cmadams.net> wrote:
> > However, while digging, I also noticed that now the engine is not
> > communicating with ovirt-provider-ovn, possibly due to a similar issue?
> > It is having the reverse problem; it rejects the engine's cert.
>
> Didn't try this yet, adding Dominik.
Was anybody able to look at this? I had to use my dev hardware for
something else for a bit, so re-installed with 4.3.5 yesterday. The
imageio SSL cert issue looks good, but I still can't figure out the
ovirt-provider-ovn CA usage.
My little bit of digging seems to show that the engine connects to the
provider and is using an SSL client cert, and that cert is signed by
something... but I'm not sure what. I think the provider side is trying
to validate with the following setting from
/etc/ovirt-provider-ovn/conf.d/10-setup-ovirt-provider-ovn.conf
[OVIRT]
ovirt-ca-file=/etc/pki/ovirt-engine/apache-ca.pem
Following the general "3rd-party SSL", that is now the Let's Encrypt CA.
I tried changing it to point to the original self-signed oVirt CA (same
directory, just "ca.pem"), but that didn't work either.
Any suggestions?
--
Chris Adams <cma(a)cmadams.net>
9:37 a.m.
On Thu, 1 Aug 2019 20:45:56 -0500
Chris Adams <cma(a)cmadams.net> wrote:
I figured it out. When ovirt-provider-ovn attempts to connect back
to
the engine via HTTPS, it tells the python requests module to use the
specified CA cert file... but that won't work with most 3rd-party certs
because they have an intermediate cert as well. It appears that the
requests module tries to validate both certs.
Creating /etc/ovirt-provider-ovn/conf.d/99-custom-cert.conf that just
has:
[OVIRT]
ovirt-ca-file=
tells the module to use the regular system CA cert file(s), which works.
Thanks for your investigation!
Looks like the empty string is converted implicitly to Boolean in
https://github.com/psf/requests/blob/75bdc998e2d430a35d869b2abf1779bd0d34...
Because bool('') is False in python, the certificate should be checked
at all.
Would
ovirt-ca-file=/etc/pki/tls/certs/ca-bundle.crt
work for you?
(It works for https://helloworld.letsencrypt.org)
This should probably be added to the oVirt doc for using a 3rd-party
cert.
Once upon a time, Chris Adams <cma(a)cmadams.net> said:
> Circling back to an old email...
>
> Once upon a time, Yedidyah Bar David <didi(a)redhat.com> said:
> > On Wed, Jan 30, 2019 at 10:28 PM Chris Adams <cma(a)cmadams.net> wrote:
> > > However, while digging, I also noticed that now the engine is not
> > > communicating with ovirt-provider-ovn, possibly due to a similar issue?
> > > It is having the reverse problem; it rejects the engine's cert.
> >
> > Didn't try this yet, adding Dominik.
>
> Was anybody able to look at this? I had to use my dev hardware for
> something else for a bit, so re-installed with 4.3.5 yesterday. The
> imageio SSL cert issue looks good, but I still can't figure out the
> ovirt-provider-ovn CA usage.
>
> My little bit of digging seems to show that the engine connects to the
> provider and is using an SSL client cert, and that cert is signed by
> something... but I'm not sure what. I think the provider side is trying
> to validate with the following setting from
> /etc/ovirt-provider-ovn/conf.d/10-setup-ovirt-provider-ovn.conf
>
> [OVIRT]
> ovirt-ca-file=/etc/pki/ovirt-engine/apache-ca.pem
>
> Following the general "3rd-party SSL", that is now the Let's Encrypt
CA.
> I tried changing it to point to the original self-signed oVirt CA (same
> directory, just "ca.pem"), but that didn't work either.
>
> Any suggestions?
9:39 a.m.
On Fri, Aug 2, 2019 at 4:37 PM Dominik Holler <dholler(a)redhat.com> wrote:
On Thu, 1 Aug 2019 20:45:56 -0500
Chris Adams <cma(a)cmadams.net> wrote:
> I figured it out. When ovirt-provider-ovn attempts to connect back to
> the engine via HTTPS, it tells the python requests module to use the
> specified CA cert file... but that won't work with most 3rd-party certs
> because they have an intermediate cert as well. It appears that the
> requests module tries to validate both certs.
>
> Creating /etc/ovirt-provider-ovn/conf.d/99-custom-cert.conf that just
> has:
>
> [OVIRT]
> ovirt-ca-file=
>
> tells the module to use the regular system CA cert file(s), which works.
Thanks for your investigation!
Looks like the empty string is converted implicitly to Boolean in
https://github.com/psf/requests/blob/75bdc998e2d430a35d869b2abf1779bd0d34...
Because bool('') is False in python, the certificate should be checked
at all.
Because bool('') is False in python, the certificate should be* not
*checked
at all.
Would
ovirt-ca-file=/etc/pki/tls/certs/ca-bundle.crt
work for you?
(It works for https://helloworld.letsencrypt.org)
> This should probably be added to the oVirt doc for using a 3rd-party
> cert.
>
> Once upon a time, Chris Adams <cma(a)cmadams.net> said:
> > Circling back to an old email...
> >
> > Once upon a time, Yedidyah Bar David <didi(a)redhat.com> said:
> > > On Wed, Jan 30, 2019 at 10:28 PM Chris Adams <cma(a)cmadams.net>
wrote:
> > > > However, while digging, I also noticed that now the engine is not
> > > > communicating with ovirt-provider-ovn, possibly due to a similar
issue?
> > > > It is having the reverse problem; it rejects the engine's cert.
> > >
> > > Didn't try this yet, adding Dominik.
> >
> > Was anybody able to look at this? I had to use my dev hardware for
> > something else for a bit, so re-installed with 4.3.5 yesterday. The
> > imageio SSL cert issue looks good, but I still can't figure out the
> > ovirt-provider-ovn CA usage.
> >
> > My little bit of digging seems to show that the engine connects to the
> > provider and is using an SSL client cert, and that cert is signed by
> > something... but I'm not sure what. I think the provider side is
trying
> > to validate with the following setting from
> > /etc/ovirt-provider-ovn/conf.d/10-setup-ovirt-provider-ovn.conf
> >
> > [OVIRT]
> > ovirt-ca-file=/etc/pki/ovirt-engine/apache-ca.pem
> >
> > Following the general "3rd-party SSL", that is now the Let's
Encrypt
CA.
> > I tried changing it to point to the original self-signed oVirt CA (same
> > directory, just "ca.pem"), but that didn't work either.
> >
> > Any suggestions?
>
2:55 p.m.
Once upon a time, Dominik Holler <dholler(a)redhat.com> said:
Would
ovirt-ca-file=/etc/pki/tls/certs/ca-bundle.crt
work for you?
Yes, that looks like it works correctly.
Still chasing issues with a 3rd-party cert down... now it seems like
there may be an SSL issue between ovirt-provider-ovn and ovsdb-server
(seeing SSL and protocol errors in ovsdb-server-nb.log that weren't
there before changing the cert).
Also, I updated the engine from 4.3.4 to 4.3.5 and it overwrote the
necessary changes in ovirt-imageio-proxy's config.
--
Chris Adams <cma(a)cmadams.net>
1969
days inactive
2154
days old
19 comments
6 participants
participants (6)
-
Chris Adams -
Dominik Holler -
John Florian -
Sandro Emma -
Staniforth, Paul -
Yedidyah Bar David