console breaks with signed SSL certs
Chrome didnt want to talk AT ALL to ovirt with self-signed certs (Because HSTS is enabled) So I installed signed wildcard certs to the engine, and the nodes, following http://187.1.81.65/ovirt-engine/docs/manual/en-US/html/Administration_Guide/... and https://cockpit-project.org/guide/172/https.html and chrome is happy now... except that suddenly, consoles refuse to work. and there are no useful errors that I see, other than "Unable to connect to the graphic server" from the remote viewer app. I see someone not too long ago had the exact same problem, in https://www.mail-archive.com/users@ovirt.org/msg58814.html but.. no answer was given to him? Help please -- Philip Brown| Sr. Linux System Administrator | Medata, Inc. 5 Peters Canyon Rd Suite 250 Irvine CA 92606 Office 714.918.1310| Fax 714.918.1325 pbrown@medata.com| www.medata.com
More detail on the problem. after starting remote-viewer --debug, I get (remote-viewer.exe:18308): virt-viewer-DEBUG: 11:45:30.594: New spice channel 000000000608B240 SpiceMainChannel 0 (remote-viewer.exe:18308): virt-viewer-DEBUG: 11:45:30.594: notebook show status 0000000003479130 (remote-viewer.exe:18308): Spice-WARNING **: 11:45:30.691: ../subprojects/spice-common/common/ssl_verify.c:444:openssl_verify: Error in certificate chain verification: self signed certificate in certificate chain (num=19:depth1:/C=US/O=xxxxxxxxxx.65101) (remote-viewer.exe:18308): GSpice-WARNING **: 11:45:30.692: main-1:0: SSL_connect: error:00000001:lib(0):func(0):reason(1) (remote-viewer.exe:18308): virt-viewer-DEBUG: 11:45:30.693: Destroy SPICE channel SpiceMainChannel 0 So it seems like there's some additional thing that needs telling to use the official signed cert. Any clues for me please?
I assume you are working on linux (for windows you will need to ssh to a linux box or even one ofthe Hosts). When you download the 'console.vv' file for Spice connection - you will have to note several stuff: - host - tls-port (not the plain 'port=' !!! ) - ca Process the CA and replace the '\n' with new lines . Then you can run: openssl s_client -connect <host>:<tls-port> -CAfile <path-to-ca-with-newlines> -showcerts Then you can inspect the certificate chain. I would then grep for the strings from openssl in the engine. In my case I find these containing the line with the 'issuer': /etc/pki/ovirt-engine/certs/websocket-proxy.cer /etc/pki/ovirt-engine/certs/apache.cer /etc/pki/ovirt-engine/certs/reports.cer /etc/pki/ovirt-engine/certs/imageio-proxy.cer /etc/pki/ovirt-engine/certs/ovn-ndb.cer /etc/pki/ovirt-engine/certs/ovn-sdb.cer /etc/pki/ovirt-engine/certs/ovirt-provider-ovn.cer Happy Hunting! Best Regards, Strahil Nikolov В вторник, 22 септември 2020 г., 21:52:10 Гринуич+3, Philip Brown <pbrown@medata.com> написа: More detail on the problem. after starting remote-viewer --debug, I get (remote-viewer.exe:18308): virt-viewer-DEBUG: 11:45:30.594: New spice channel 000000000608B240 SpiceMainChannel 0 (remote-viewer.exe:18308): virt-viewer-DEBUG: 11:45:30.594: notebook show status 0000000003479130 (remote-viewer.exe:18308): Spice-WARNING **: 11:45:30.691: ../subprojects/spice-common/common/ssl_verify.c:444:openssl_verify: Error in certificate chain verification: self signed certificate in certificate chain (num=19:depth1:/C=US/O=xxxxxxxxxx.65101) (remote-viewer.exe:18308): GSpice-WARNING **: 11:45:30.692: main-1:0: SSL_connect: error:00000001:lib(0):func(0):reason(1) (remote-viewer.exe:18308): virt-viewer-DEBUG: 11:45:30.693: Destroy SPICE channel SpiceMainChannel 0 So it seems like there's some additional thing that needs telling to use the official signed cert. Any clues for me please? _______________________________________________ Users mailing list -- users@ovirt.org To unsubscribe send an email to users-leave@ovirt.org Privacy Statement: https://www.ovirt.org/privacy-policy.html oVirt Code of Conduct: https://www.ovirt.org/community/about/community-guidelines/ List Archives: https://lists.ovirt.org/archives/list/users@ovirt.org/message/VKSX7CLJ4N7PNC...
Thanks for the initial start, Strahil, my desktop is windows. but I took apart the console.vv file, and these are my findings: in the console.vv file, there is a valid CA cert, which is for the signing CA for our valid wildcard SSL cert. However, when I connected to the target host, on the tls-port, i noted that it is still using the original self-signed CA, generated by ovirt-engine for the host. Digging with lsof says that the process is qemu-kvm Looking at command line, that has x509-dir=/etc/pki/vdsm/libvirt-spice So... I guess I need to update server.key server.cert and ca-cert in there? except there's a whoole lot of '*key.pem' files under the /etc/pki directory tree. Suggestions on which is best to update? For example, there is also /etc/pki/vdsm/keys/vdsmkey.pem ----- Original Message ----- From: "Strahil Nikolov" <hunter86_bg@yahoo.com> To: "users" <users@ovirt.org>, "Philip Brown" <pbrown@medata.com> Sent: Tuesday, September 22, 2020 12:09:55 PM Subject: Re: [ovirt-users] Re: console breaks with signed SSL certs I assume you are working on linux (for windows you will need to ssh to a linux box or even one ofthe Hosts). When you download the 'console.vv' file for Spice connection - you will have to note several stuff: - host - tls-port (not the plain 'port=' !!! ) - ca Process the CA and replace the '\n' with new lines . Then you can run: openssl s_client -connect <host>:<tls-port> -CAfile <path-to-ca-with-newlines> -showcerts Then you can inspect the certificate chain. I would then grep for the strings from openssl in the engine. In my case I find these containing the line with the 'issuer': /etc/pki/ovirt-engine/certs/websocket-proxy.cer /etc/pki/ovirt-engine/certs/apache.cer /etc/pki/ovirt-engine/certs/reports.cer /etc/pki/ovirt-engine/certs/imageio-proxy.cer /etc/pki/ovirt-engine/certs/ovn-ndb.cer /etc/pki/ovirt-engine/certs/ovn-sdb.cer /etc/pki/ovirt-engine/certs/ovirt-provider-ovn.cer Happy Hunting! Best Regards, Strahil Nikolov В вторник, 22 септември 2020 г., 21:52:10 Гринуич+3, Philip Brown <pbrown@medata.com> написа: More detail on the problem. after starting remote-viewer --debug, I get (remote-viewer.exe:18308): virt-viewer-DEBUG: 11:45:30.594: New spice channel 000000000608B240 SpiceMainChannel 0 (remote-viewer.exe:18308): virt-viewer-DEBUG: 11:45:30.594: notebook show status 0000000003479130 (remote-viewer.exe:18308): Spice-WARNING **: 11:45:30.691: ../subprojects/spice-common/common/ssl_verify.c:444:openssl_verify: Error in certificate chain verification: self signed certificate in certificate chain (num=19:depth1:/C=US/O=xxxxxxxxxx.65101) (remote-viewer.exe:18308): GSpice-WARNING **: 11:45:30.692: main-1:0: SSL_connect: error:00000001:lib(0):func(0):reason(1) (remote-viewer.exe:18308): virt-viewer-DEBUG: 11:45:30.693: Destroy SPICE channel SpiceMainChannel 0 So it seems like there's some additional thing that needs telling to use the official signed cert. Any clues for me please? _______________________________________________ Users mailing list -- users@ovirt.org To unsubscribe send an email to users-leave@ovirt.org Privacy Statement: https://www.ovirt.org/privacy-policy.html oVirt Code of Conduct: https://www.ovirt.org/community/about/community-guidelines/ List Archives: https://lists.ovirt.org/archives/list/users@ovirt.org/message/VKSX7CLJ4N7PNC...
Most probably there is an option to tell it (I mean oVIrt) the exact keys to be used. Yet, give the engine a gentle push and reboot it - just to be sure you are not chasing a ghost. I'm using self-signed certs and I can't help much in this case. Best Regards, Strahil Nikolov В вторник, 22 септември 2020 г., 22:54:28 Гринуич+3, Philip Brown <pbrown@medata.com> написа: Thanks for the initial start, Strahil, my desktop is windows. but I took apart the console.vv file, and these are my findings: in the console.vv file, there is a valid CA cert, which is for the signing CA for our valid wildcard SSL cert. However, when I connected to the target host, on the tls-port, i noted that it is still using the original self-signed CA, generated by ovirt-engine for the host. Digging with lsof says that the process is qemu-kvm Looking at command line, that has x509-dir=/etc/pki/vdsm/libvirt-spice So... I guess I need to update server.key server.cert and ca-cert in there? except there's a whoole lot of '*key.pem' files under the /etc/pki directory tree. Suggestions on which is best to update? For example, there is also /etc/pki/vdsm/keys/vdsmkey.pem ----- Original Message ----- From: "Strahil Nikolov" <hunter86_bg@yahoo.com> To: "users" <users@ovirt.org>, "Philip Brown" <pbrown@medata.com> Sent: Tuesday, September 22, 2020 12:09:55 PM Subject: Re: [ovirt-users] Re: console breaks with signed SSL certs I assume you are working on linux (for windows you will need to ssh to a linux box or even one ofthe Hosts). When you download the 'console.vv' file for Spice connection - you will have to note several stuff: - host - tls-port (not the plain 'port=' !!! ) - ca Process the CA and replace the '\n' with new lines . Then you can run: openssl s_client -connect <host>:<tls-port> -CAfile <path-to-ca-with-newlines> -showcerts Then you can inspect the certificate chain. I would then grep for the strings from openssl in the engine. In my case I find these containing the line with the 'issuer': /etc/pki/ovirt-engine/certs/websocket-proxy.cer /etc/pki/ovirt-engine/certs/apache.cer /etc/pki/ovirt-engine/certs/reports.cer /etc/pki/ovirt-engine/certs/imageio-proxy.cer /etc/pki/ovirt-engine/certs/ovn-ndb.cer /etc/pki/ovirt-engine/certs/ovn-sdb.cer /etc/pki/ovirt-engine/certs/ovirt-provider-ovn.cer Happy Hunting! Best Regards, Strahil Nikolov В вторник, 22 септември 2020 г., 21:52:10 Гринуич+3, Philip Brown <pbrown@medata.com> написа: More detail on the problem. after starting remote-viewer --debug, I get (remote-viewer.exe:18308): virt-viewer-DEBUG: 11:45:30.594: New spice channel 000000000608B240 SpiceMainChannel 0 (remote-viewer.exe:18308): virt-viewer-DEBUG: 11:45:30.594: notebook show status 0000000003479130 (remote-viewer.exe:18308): Spice-WARNING **: 11:45:30.691: ../subprojects/spice-common/common/ssl_verify.c:444:openssl_verify: Error in certificate chain verification: self signed certificate in certificate chain (num=19:depth1:/C=US/O=xxxxxxxxxx.65101) (remote-viewer.exe:18308): GSpice-WARNING **: 11:45:30.692: main-1:0: SSL_connect: error:00000001:lib(0):func(0):reason(1) (remote-viewer.exe:18308): virt-viewer-DEBUG: 11:45:30.693: Destroy SPICE channel SpiceMainChannel 0 So it seems like there's some additional thing that needs telling to use the official signed cert. Any clues for me please? _______________________________________________ Users mailing list -- users@ovirt.org To unsubscribe send an email to users-leave@ovirt.org Privacy Statement: https://www.ovirt.org/privacy-policy.html oVirt Code of Conduct: https://www.ovirt.org/community/about/community-guidelines/ List Archives: https://lists.ovirt.org/archives/list/users@ovirt.org/message/VKSX7CLJ4N7PNC... _______________________________________________ Users mailing list -- users@ovirt.org To unsubscribe send an email to users-leave@ovirt.org Privacy Statement: https://www.ovirt.org/privacy-policy.html oVirt Code of Conduct: https://www.ovirt.org/community/about/community-guidelines/ List Archives: https://lists.ovirt.org/archives/list/users@ovirt.org/message/545XR3UZJ3U4H5...
Hmm. that seems to be half the battle. I updated the filels in /etc/pki/vdsm/libvirt-spice, and the debug output from remote-viewer changes.. but its not entirely happy. (remote-viewer.exe:15808): Spice-WARNING **: 12:55:01.188: ../subprojects/spice-common/common/ssl_verify.c:444:openssl_verify: Error in certificate chain verification: unable to get issuer certificate (num=2:depth1:/C=US/ST=Arizona/L=Scottsdale/O=GoDaddy.com, Inc./OU=http://certs.godaddy.com/repository//CN=Go Daddy Secure Certificate Authority - G2) (remote-viewer.exe:15808): GSpice-WARNING **: 12:55:01.189: main-1:0: SSL_connect: error:00000001:lib(0):func(0):reason(1) (remote-viewer.exe:15808): virt-viewer-DEBUG: 12:55:01.192: Destroy SPICE channel SpiceMainChannel 0 (remote-viewer.exe:15808): virt-viewer-DEBUG: 12:55:01.192: zap main channel I put the cert itself, in server-cert.pem I put the key in server-key.pem I put the bundle file from godaddy, which they call "gd_bundle-g2-g1", in "ca-cert.pem" but its still complaining about error in chain? Ive been updating a whoole bunch of SSL-requiring systems this month, and notice that one or two systems like a different order to the multiple-cert-CA stack. Does libvirt-spice require yet another, different stacking? Can you tell me what needs to be in each, and in what order, please? :-/
On Tue, Sep 22, 2020 at 6:46 PM Philip Brown <pbrown@medata.com> wrote:
Chrome didnt want to talk AT ALL to ovirt with self-signed certs (Because HSTS is enabled)
So I installed signed wildcard certs to the engine, and the nodes, following
http://187.1.81.65/ovirt-engine/docs/manual/en-US/html/Administration_Guide/...
Going to http://187.1.81.65/ovirt-engine/ shows that this is RHEV 3.6.6, and the above document is from the documentation included for it. Is this the machine you work with? Or you simply found it at random and use as doc? Anyway, 3.6.6 is very old and log unsupported. If it's indeed your setup, I recommend to upgrade. Even if it's not, I recommend to check latest (4.4) docs, and compare to yours - and try to guess what also applies in 3.6.6 (I think almost everything does, didn't check).
and https://cockpit-project.org/guide/172/https.html
and chrome is happy now... except that suddenly, consoles refuse to work. and there are no useful errors that I see, other than
"Unable to connect to the graphic server"
from the remote viewer app.
If you are going to continue debugging it yourself, you should also check relevant logs on the engine and the host. Also, assuming you did follow latest docs (as applicable): Please check the cert included inside console.vv. Is it (check "Issuer") the engine-internal CA (/etc/pki/ovirt-engine/ca.pem), or your other CA? It should be the engine's, and (at least for me) remote-viewer accepts it - I do not see with --debug the error you got about self-signed cert. If it's the "other" CA cert, then it's a bug somewhere - either in the software or the doc. I am not sure remote-viewer of any version has a problem with this. If you want a client that strictly uses only CAs you explicitly accepted (not the one inside console.vv), you can use the novnc one - this one connects to websocket-proxy, which (with an up-to-date procedure) uses your other CA.
I see someone not too long ago had the exact same problem, in https://www.mail-archive.com/users@ovirt.org/msg58814.html
Sorry, I didn't notice it. Best regards,
but.. no answer was given to him?
Help please
-- Philip Brown| Sr. Linux System Administrator | Medata, Inc. 5 Peters Canyon Rd Suite 250 Irvine CA 92606 Office 714.918.1310| Fax 714.918.1325 pbrown@medata.com| www.medata.com _______________________________________________ Users mailing list -- users@ovirt.org To unsubscribe send an email to users-leave@ovirt.org Privacy Statement: https://www.ovirt.org/privacy-policy.html oVirt Code of Conduct: https://www.ovirt.org/community/about/community-guidelines/ List Archives: https://lists.ovirt.org/archives/list/users@ovirt.org/message/KNJGW2Z6XPK4CD...
-- Didi
participants (3)
-
Philip Brown -
Strahil Nikolov -
Yedidyah Bar David