Re: [ovirt-users] SSLHandshakeException: Received fatal alert: certificate_expired

On Fri, Sep 22, 2017 at 10:18 AM, Neil <nwilson123(a)gmail.com&gt; wrote: > > Hi Piotr, > > Thank you for the information. > > It looks like something has expired looking in the server.log now that > debug is enabled. > > 2017-09-22 09:35:26,462 INFO [stdout] (MSC service thread 1-4) Version: > V3 > 2017-09-22 09:35:26,464 INFO [stdout] (MSC service thread 1-4) Subject: > CN=engine01.mydomain.za, O=mydomain, C=US > 2017-09-22 09:35:26,467 INFO [stdout] (MSC service thread 1-4) > Signature Algorithm: SHA1withRSA, OID = 1.2.840.113549.1.1.5 > 2017-09-22 09:35:26,471 INFO [stdout] (MSC service thread 1-4) > 2017-09-22 09:35:26,472 INFO [stdout] (MSC service thread 1-4) Key: > Sun RSA public key, 1024 bits > 2017-09-22 09:35:26,474 INFO [stdout] (MSC service thread 1-4) modulus: > 96670613185023785772001656613227416922514371649313203413281121371175732119596513752882171306045450346018887835032223373125981220753972276294203593174404470265593368091683564110524316403260121331609213962612618181708680331850541390318868926054438078223371655800890725486783860059873397983318033852172060923531 > 2017-09-22 09:35:26,476 INFO [stdout] (MSC service thread 1-4) public > exponent: 65537 > 2017-09-22 09:35:26,477 INFO [stdout] (MSC service thread 1-4) > Validity: [From: Sun Oct 14 22:26:46 SAST 2012, > 2017-09-22 09:35:26,478 INFO [stdout] (MSC service thread 1-4) > To: Tue Sep 19 18:26:49 SAST 2017] > 2017-09-22 09:35:26,479 INFO [stdout] (MSC service thread 1-4) Issuer: > CN=CA-engine01.mydomain.za.47472, O=mydomain, C=US > > Any idea how I can generate a new one and what cert it is that's expired? It seems that your engine certificate has expired, but AFAIK this certificate should be automatically renewed during engine-setup. So when did you execute engine-setup for last time? Any info/warning about this shown during invocation?

Also looking at server.log I found JBoss 7.1.1, so you are using really ancient oVirt, version, right? > > Please see the attached log for more info. > > Thank you so much for your assistance. > > Regards. > > Neil Wilson. > > > > > > > On Thu, Sep 21, 2017 at 8:41 PM, Piotr Kliczewski > <piotr.kliczewski(a)gmail.com&gt; wrote: >> >> Neil, >> >> It seems that your engine certificate(s) is/are not ok. I would >> suggest to enable ssl debug in the engine by: >> - add '-Djavax.net.debug=all' to ovirt-engine.py file here [1]. >> - restart your engine >> - check your server.log and check what is the issue. >> >> Hopefully we will be able to understand what happened in your setup. >> >> Thanks, >> Piotr >> >> [1] >> https://github.com/oVirt/ovirt-engine/blob/master/packaging/services/ovir... >> >> On Thu, Sep 21, 2017 at 4:42 PM, Neil <nwilson123(a)gmail.com&gt; wrote: >> > Further to the logs sent, on the nodes I'm also seeing the following >> > error >> > under /var/log/messages... >> > >> > Sep 20 03:43:12 node01 vdsm root ERROR invalid client certificate with >> > subject "/C=US/O=UKDM/CN=engine01.mydomain.za"^C >> > Sep 20 03:43:12 node01 vdsm vds ERROR xml-rpc handler >> > exception#012Traceback >> > (most recent call last):#012 File "/usr/share/vdsm/BindingXMLRPC.py", >> > line >> > 80, in threaded_start#012 self.server.handle_request()#012 File >> > "/usr/lib64/python2.6/SocketServer.py", line 278, in handle_request#012 >> > self._handle_request_noblock()#012 File >> > "/usr/lib64/python2.6/SocketServer.py", line 288, in >> > _handle_request_noblock#012 request, client_address = >> > self.get_request()#012 File "/usr/lib64/python2.6/SocketServer.py", >> > line >> > 456, in get_request#012 return self.socket.accept()#012 File >> > "/usr/lib64/python2.6/site-packages/vdsm/SecureXMLRPCServer.py", line >> > 136, >> > in accept#012 raise SSL.SSLError("%s, client %s" % (e, >> > address[0]))#012SSLError: no certificate returned, client 10.251.193.5 >> > >> > Not sure if this is any further help in diagnosing the issue? >> > >> > Thanks, any assistance is appreciated. >> > >> > Regards. >> > >> > Neil Wilson. >> > >> > >> > On Thu, Sep 21, 2017 at 4:31 PM, Neil <nwilson123(a)gmail.com&gt; wrote: >> >> >> >> Hi Piotr, >> >> >> >> Thank you for the reply. After sending the email I did go and check >> >> the >> >> engine one too.... >> >> >> >> [root@engine01 /]# openssl x509 -in /etc/pki/ovirt-engine/ca.pem >> >> -enddate >> >> -noout >> >> notAfter=Oct 13 16:26:46 2022 GMT >> >> >> >> I'm not sure if this one below is meant to verify or if this output is >> >> expected? >> >> >> >> [root@engine01 /]# openssl x509 -in >> >> /etc/pki/ovirt-engine/private/ca.pem >> >> -enddate -noout >> >> unable to load certificate >> >> 140642165552968:error:0906D06C:PEM routines:PEM_read_bio:no start >> >> line:pem_lib.c:703:Expecting: TRUSTED CERTIFICATE >> >> >> >> My date is correct too Thu Sep 21 16:30:15 SAST 2017 >> >> >> >> Any ideas? >> >> >> >> Googling surprisingly doesn't come up with much. >> >> >> >> Thank you. >> >> >> >> Regards. >> >> >> >> Neil Wilson. >> >> >> >> On Thu, Sep 21, 2017 at 4:16 PM, Piotr Kliczewski >> >> <piotr.kliczewski(a)gmail.com&gt; wrote: >> >>> >> >>> Neil, >> >>> >> >>> You checked both nodes what about the engine? Can you check engine >> >>> certs? >> >>> You can find more info where they are located here [1]. >> >>> >> >>> Thanks, >> >>> Piotr >> >>> >> >>> [1] >> >>> >> >>> https://www.ovirt.org/develop/release-management/features/infra/pki/#ovir... >> >>> >> >>> On Thu, Sep 21, 2017 at 3:26 PM, Neil <nwilson123(a)gmail.com&gt; wrote: >> >>> > Hi guys, >> >>> > >> >>> > Please could someone assist, my cluster is down and I can't access >> >>> > my >> >>> > vm's >> >>> > to switch some of them back on. >> >>> > >> >>> > I'm seeing the following error in the engine.log however I've >> >>> > checked >> >>> > my >> >>> > certs on my hosts (as some of the goolge results said to check), >> >>> > but >> >>> > the >> >>> > certs haven't expired... >> >>> > >> >>> > >> >>> > 2017-09-21 15:09:45,077 ERROR >> >>> > >> >>> > [org.ovirt.engine.core.vdsbroker.vdsbroker.GetCapabilitiesVDSCommand] >> >>> > (DefaultQuartzScheduler_Worker-4) Command >> >>> > GetCapabilitiesVDSCommand(HostName >> >>> > = node02.mydomain.za, HostId = >> >>> > d2debdfe-76e7-40cf-a7fd-78a0f50f14d4, >> >>> > vds=Host[node02.mydomain.za]) execution failed. Exception: >> >>> > VDSNetworkException: javax.net.ssl.SSLHandshakeException: Received >> >>> > fatal >> >>> > alert: certificate_expired >> >>> > 2017-09-21 15:09:45,086 ERROR >> >>> > >> >>> > [org.ovirt.engine.core.vdsbroker.vdsbroker.GetCapabilitiesVDSCommand] >> >>> > (DefaultQuartzScheduler_Worker-10) Command >> >>> > GetCapabilitiesVDSCommand(HostName = node01.mydomain.za, HostId = >> >>> > b108549c-1700-11e2-b936-9f5243b8ce13, vds=Host[node01.mydomain.za]) >> >>> > execution failed. Exception: VDSNetworkException: >> >>> > javax.net.ssl.SSLHandshakeException: Received fatal alert: >> >>> > certificate_expired >> >>> > 2017-09-21 15:09:48,173 ERROR >> >>> > >> >>> > My engine and host info is below... >> >>> > >> >>> > [root@engine01 ovirt-engine]# rpm -qa | grep -i ovirt >> >>> > ovirt-engine-lib-3.4.0-1.el6.noarch >> >>> > ovirt-engine-restapi-3.4.0-1.el6.noarch >> >>> > ovirt-engine-setup-plugin-ovirt-engine-3.4.0-1.el6.noarch >> >>> > ovirt-engine-3.4.0-1.el6.noarch >> >>> > ovirt-engine-setup-plugin-websocket-proxy-3.4.0-1.el6.noarch >> >>> > ovirt-host-deploy-java-1.2.0-1.el6.noarch >> >>> > ovirt-engine-setup-3.4.0-1.el6.noarch >> >>> > ovirt-host-deploy-1.2.0-1.el6.noarch >> >>> > ovirt-engine-backend-3.4.0-1.el6.noarch >> >>> > ovirt-image-uploader-3.4.0-1.el6.noarch >> >>> > ovirt-engine-tools-3.4.0-1.el6.noarch >> >>> > ovirt-engine-sdk-python-3.4.0.7-1.el6.noarch >> >>> > ovirt-engine-webadmin-portal-3.4.0-1.el6.noarch >> >>> > ovirt-engine-cli-3.4.0.5-1.el6.noarch >> >>> > ovirt-engine-setup-base-3.4.0-1.el6.noarch >> >>> > ovirt-iso-uploader-3.4.0-1.el6.noarch >> >>> > ovirt-engine-userportal-3.4.0-1.el6.noarch >> >>> > ovirt-log-collector-3.4.1-1.el6.noarch >> >>> > ovirt-engine-websocket-proxy-3.4.0-1.el6.noarch >> >>> > ovirt-engine-setup-plugin-ovirt-engine-common-3.4.0-1.el6.noarch >> >>> > ovirt-engine-dbscripts-3.4.0-1.el6.noarch >> >>> > [root@engine01 ovirt-engine]# cat /etc/redhat-release >> >>> > CentOS release 6.5 (Final) >> >>> > >> >>> > >> >>> > [root@node02 ~]# openssl x509 -in /etc/pki/vdsm/certs/vdsmcert.pem >> >>> > -enddate >> >>> > -noout ; date >> >>> > notAfter=May 27 08:36:17 2019 GMT >> >>> > Thu Sep 21 15:18:22 SAST 2017 >> >>> > CentOS release 6.5 (Final) >> >>> > [root@node02 ~]# rpm -qa | grep vdsm >> >>> > vdsm-4.14.6-0.el6.x86_64 >> >>> > vdsm-python-4.14.6-0.el6.x86_64 >> >>> > vdsm-cli-4.14.6-0.el6.noarch >> >>> > vdsm-xmlrpc-4.14.6-0.el6.noarch >> >>> > vdsm-python-zombiereaper-4.14.6-0.el6.noarch >> >>> > >> >>> > >> >>> > [root@node01 ~]# openssl x509 -in /etc/pki/vdsm/certs/vdsmcert.pem >> >>> > -enddate >> >>> > -noout ; date >> >>> > notAfter=Jun 13 16:09:41 2018 GMT >> >>> > Thu Sep 21 15:18:52 SAST 2017 >> >>> > CentOS release 6.5 (Final) >> >>> > [root@node01 ~]# rpm -qa | grep -i vdsm >> >>> > vdsm-4.14.6-0.el6.x86_64 >> >>> > vdsm-xmlrpc-4.14.6-0.el6.noarch >> >>> > vdsm-cli-4.14.6-0.el6.noarch >> >>> > vdsm-python-zombiereaper-4.14.6-0.el6.noarch >> >>> > vdsm-python-4.14.6-0.el6.x86_64 >> >>> > >> >>> > Please could I have some assistance, I'm rater desperate. >> >>> > >> >>> > Thank you. >> >>> > >> >>> > Regards. >> >>> > >> >>> > Neil Wilson >> >>> > >> >>> > >> >>> > >> >>> > _______________________________________________ >> >>> > Users mailing list >> >>> > Users(a)ovirt.org >> >>> > http://lists.ovirt.org/mailman/listinfo/users >> >>> > >> >> >> >> >> > > > > > _______________________________________________ > Users mailing list > Users(a)ovirt.org > http://lists.ovirt.org/mailman/listinfo/users >