How to recreate Ovirt CA from scratch
Hi list, during our datacenter lifetime many things changed. We moved the engine twice on different hosts with, of course, different FQDNs, and many other changes. Now we are stuck with an error when we try to upload an image to a data domain. The error is somehow bound to a failure to validate the ovirt-imageio-proxy certificate and, since the current root CA certificate is still signed with sha1WithRSAEncryption we'd like to regenerate the whole CA. That's the steps we've done.. without success... 1) Make a tar.gz of the /etc/pki/ovirt-engine as backup 2) Create a new CA cert using the same private key: openssl req -key /etc/pki/ovirt-engine/private/ca.pem -new -x509 -days 3650 -sha256 -extensions v3_ca -out newca.cert.pem 3) Put the new CA cert in place mv ca.pem ca.pem.orig.20190219;mv newca.cert.pem ca.pem 4) Resign all the other certs names="engine apache websocket-proxy jboss imageio-proxy" for name in $names; do subject="$( openssl \ x509 \ -in /etc/pki/ovirt-engine/certs/"${name}".cer \ -noout \ -subject \ | sed \ 's;subject= \(.*\);\1;' \ )" /usr/share/ovirt-engine/bin/pki-enroll-pkcs12.sh \ --name="${name}" \ --password=mypass \ --subject="${subject}" \ --keep-key done 5) Restart all the services systemctl restart httpd systemctl restart ovirt-engine systemctl restart ovirt-websocket-proxy systemctl restart ovirt-imageio-proxy The following step was to take the backup at 1) and fall back to the initial state because nothing worked as expected. There's any documented procedure about how to recreate Ovirt CA from scratch?? Thanks in advance -- gb PGP Key: http://pgp.mit.edu/ Primary key fingerprint: C510 0765 943E EBED A4F2 69D3 16CC DC90 B9CB 0F34
On Tue, Feb 19, 2019 at 3:18 PM Giorgio Biacchi <giorgio@di.unimi.it> wrote:
Hi list, during our datacenter lifetime many things changed. We moved the engine twice on different hosts with, of course, different FQDNs, and many other changes. Now we are stuck with an error when we try to upload an image to a data domain. The error is somehow bound to a failure to validate the ovirt-imageio-proxy certificate and, since the current root CA certificate is still signed with sha1WithRSAEncryption we'd like to regenerate the whole CA.
Is "sha1" all your problem? You might want to check: https://www.ovirt.org/develop/migrate-pki-to-sha256.html
That's the steps we've done.. without success...
1) Make a tar.gz of the /etc/pki/ovirt-engine as backup 2) Create a new CA cert using the same private key: openssl req -key /etc/pki/ovirt-engine/private/ca.pem -new -x509 -days 3650 -sha256 -extensions v3_ca -out newca.cert.pem 3) Put the new CA cert in place mv ca.pem ca.pem.orig.20190219;mv newca.cert.pem ca.pem 4) Resign all the other certs names="engine apache websocket-proxy jboss imageio-proxy" for name in $names; do subject="$( openssl \ x509 \ -in /etc/pki/ovirt-engine/certs/"${name}".cer \ -noout \ -subject \ | sed \ 's;subject= \(.*\);\1;' \ )" /usr/share/ovirt-engine/bin/pki-enroll-pkcs12.sh \ --name="${name}" \ --password=mypass \ --subject="${subject}" \ --keep-key done 5) Restart all the services systemctl restart httpd systemctl restart ovirt-engine systemctl restart ovirt-websocket-proxy systemctl restart ovirt-imageio-proxy
The following step was to take the backup at 1) and fall back to the initial state because nothing worked as expected.
There's any documented procedure about how to recreate Ovirt CA from scratch??
I do not think so. Seems like this isn't what you want to do, either, as you try to keep the existing keys (and recreate only certs). If that's really what you want to do, you can simply remove everything under /etc/pki/ovirt-engine that's not packaged by any rpm, e.g. using: find . -type f | xargs rpm -qf | awk '/is not owned by any package/ {print $2}' and then run engine-setup. Obviously, this will break everything - you will have to reinstall all hosts (or at least "Enroll Certificates" from the web ui) etc. If it's a hosted-engine, make sure to move it to global maintenance beforehand. See also: https://ovirt.org/develop/release-management/features/infra/pki-renew.html So far, we didn't make engine-setup suggest to renew pki on the grounds of not using sha256, considering it not-yet- mandatory, and only documented the above link (and made new setups use sha256 by default). Feel free to open an RFE for this, and once it's fixed, the answer to your question will be "Run engine-setup"... Best regards, -- Didi
On 2/20/19 7:47 AM, Yedidyah Bar David wrote:
On Tue, Feb 19, 2019 at 3:18 PM Giorgio Biacchi <giorgio@di.unimi.it> wrote:
Hi list, during our datacenter lifetime many things changed. We moved the engine twice on different hosts with, of course, different FQDNs, and many other changes. Now we are stuck with an error when we try to upload an image to a data domain. The error is somehow bound to a failure to validate the ovirt-imageio-proxy certificate and, since the current root CA certificate is still signed with sha1WithRSAEncryption we'd like to regenerate the whole CA.
Is "sha1" all your problem? You might want to check:
Today I repeated the procedure described in the link here above and finally I was successful. Maybe yesterday I was too quick to fall back to the original state but my enviroment is in production and I was scared... I had some problems while enrolling the new certificate on the hypervisors, but removing/rebooting/readding did the trick. Our engine have an SSO_ALTERNATE_ENGINE_FQDN (before it was the real engine FQDN) so I found that ImageProxyAddress was still pointing to the old name. I'm now able (as before) to access the admin portal with both names but only one (the one with the green lock in the browser) is the FQDN in the certificate, so I did: engine-config --set ImageProxyAddress=realFQDN:54323 and now I have sha256 certs and ovirt-imageio-proxy working as expected. :) Thanks for your hints..
That's the steps we've done.. without success...
1) Make a tar.gz of the /etc/pki/ovirt-engine as backup 2) Create a new CA cert using the same private key: openssl req -key /etc/pki/ovirt-engine/private/ca.pem -new -x509 -days 3650 -sha256 -extensions v3_ca -out newca.cert.pem 3) Put the new CA cert in place mv ca.pem ca.pem.orig.20190219;mv newca.cert.pem ca.pem 4) Resign all the other certs names="engine apache websocket-proxy jboss imageio-proxy" for name in $names; do subject="$( openssl \ x509 \ -in /etc/pki/ovirt-engine/certs/"${name}".cer \ -noout \ -subject \ | sed \ 's;subject= \(.*\);\1;' \ )" /usr/share/ovirt-engine/bin/pki-enroll-pkcs12.sh \ --name="${name}" \ --password=mypass \ --subject="${subject}" \ --keep-key done 5) Restart all the services systemctl restart httpd systemctl restart ovirt-engine systemctl restart ovirt-websocket-proxy systemctl restart ovirt-imageio-proxy
The following step was to take the backup at 1) and fall back to the initial state because nothing worked as expected.
There's any documented procedure about how to recreate Ovirt CA from scratch??
I do not think so.
Seems like this isn't what you want to do, either, as you try to keep the existing keys (and recreate only certs).
If that's really what you want to do, you can simply remove everything under /etc/pki/ovirt-engine that's not packaged by any rpm, e.g. using:
find . -type f | xargs rpm -qf | awk '/is not owned by any package/ {print $2}'
and then run engine-setup.
Obviously, this will break everything - you will have to reinstall all hosts (or at least "Enroll Certificates" from the web ui) etc. If it's a hosted-engine, make sure to move it to global maintenance beforehand.
See also:
https://ovirt.org/develop/release-management/features/infra/pki-renew.html
So far, we didn't make engine-setup suggest to renew pki on the grounds of not using sha256, considering it not-yet- mandatory, and only documented the above link (and made new setups use sha256 by default). Feel free to open an RFE for this, and once it's fixed, the answer to your question will be "Run engine-setup"...
Best regards,
-- gb PGP Key: http://pgp.mit.edu/ Primary key fingerprint: C510 0765 943E EBED A4F2 69D3 16CC DC90 B9CB 0F34
On Wed, Feb 20, 2019 at 6:34 PM Giorgio Biacchi <giorgio@di.unimi.it> wrote:
On 2/20/19 7:47 AM, Yedidyah Bar David wrote:
On Tue, Feb 19, 2019 at 3:18 PM Giorgio Biacchi <giorgio@di.unimi.it> wrote:
Hi list, during our datacenter lifetime many things changed. We moved the engine twice on different hosts with, of course, different FQDNs, and many other changes. Now we are stuck with an error when we try to upload an image to a data domain. The error is somehow bound to a failure to validate the ovirt-imageio-proxy certificate and, since the current root CA certificate is still signed with sha1WithRSAEncryption we'd like to regenerate the whole CA.
Is "sha1" all your problem? You might want to check:
Today I repeated the procedure described in the link here above and finally I was successful. Maybe yesterday I was too quick to fall back to the original state but my enviroment is in production and I was scared...
I had some problems while enrolling the new certificate on the hypervisors, but removing/rebooting/readding did the trick.
If you still have logs of the failure, you might want to open a bug. I think 'Enroll Certificate' should have worked.
Our engine have an SSO_ALTERNATE_ENGINE_FQDN (before it was the real engine FQDN) so I found that ImageProxyAddress was still pointing to the old name.
Should be fixed in ovirt-engine-rename in 4.3: https://bugzilla.redhat.com/show_bug.cgi?id=1519194 If you used other means (e.g. only add alternate fqdn but do not run rename), it's up to you to handle, e.g. as you did below.
I'm now able (as before) to access the admin portal with both names but only one (the one with the green lock in the browser) is the FQDN in the certificate, so I did:
engine-config --set ImageProxyAddress=realFQDN:54323
and now I have sha256 certs and ovirt-imageio-proxy working as expected. :)
Glad to hear that, thanks for the report! Best regards, -- Didi
participants (2)
-
Giorgio Biacchi -
Yedidyah Bar David