[ovirt-users] SSLHandshakeException: Received fatal alert: certificate_expired

Fri Sep 22 08:58:17 UTC 2017

Thanks Martin and Piotr,

Correct, this was a very old installation from the old drey repo that was
upgraded gradually over the years.

I have tried engine-setup yesterday, prior to this looking under
/var/log/ovirt-engine/setup it looks like 2014

I've attached a log of the output of running it now, looks like a repo
issue with trying to upgrade to the latest 3.4.x release, but not sure what
else to look for?

Thanks for the assistance.

Regards.

Neil Wilson

On Fri, Sep 22, 2017 at 10:38 AM, Piotr Kliczewski <
piotr.kliczewski at gmail.com> wrote:

> On Fri, Sep 22, 2017 at 10:35 AM, Martin Perina <mperina at redhat.com>
> wrote:
> >
> >
> > On Fri, Sep 22, 2017 at 10:18 AM, Neil <nwilson123 at gmail.com> wrote:
> >>
> >> Hi Piotr,
> >>
> >> Thank you for the information.
> >>
> >> It looks like something has expired looking in the server.log now that
> >> debug is enabled.
> >>
> >> 2017-09-22 09:35:26,462 INFO  [stdout] (MSC service thread 1-4)
>  Version:
> >> V3
> >> 2017-09-22 09:35:26,464 INFO  [stdout] (MSC service thread 1-4)
>  Subject:
> >> CN=engine01.mydomain.za, O=mydomain, C=US
> >> 2017-09-22 09:35:26,467 INFO  [stdout] (MSC service thread 1-4)
> >> Signature Algorithm: SHA1withRSA, OID = 1.2.840.113549.1.1.5
> >> 2017-09-22 09:35:26,471 INFO  [stdout] (MSC service thread 1-4)
> >> 2017-09-22 09:35:26,472 INFO  [stdout] (MSC service thread 1-4)   Key:
> >> Sun RSA public key, 1024 bits
> >> 2017-09-22 09:35:26,474 INFO  [stdout] (MSC service thread 1-4)
>  modulus:
> >> 966706131850237857720016566132274169225143716493132034132811
> 213711757321195965137528821713060454503460188878350322233731
> 259812207539722762942035931744044702655933680916835641105243
> 164032601213316092139626126181817086803318505413903188689260
> 54438078223371655800890725486783860059873397983318033852172060923531
> >> 2017-09-22 09:35:26,476 INFO  [stdout] (MSC service thread 1-4)   public
> >> exponent: 65537
> >> 2017-09-22 09:35:26,477 INFO  [stdout] (MSC service thread 1-4)
> >> Validity: [From: Sun Oct 14 22:26:46 SAST 2012,
> >> 2017-09-22 09:35:26,478 INFO  [stdout] (MSC service thread 1-4)
> >> To: Tue Sep 19 18:26:49 SAST 2017]
> >> 2017-09-22 09:35:26,479 INFO  [stdout] (MSC service thread 1-4)
>  Issuer:
> >> CN=CA-engine01.mydomain.za.47472, O=mydomain, C=US
> >>
> >> Any idea how I can generate a new one and what cert it is that's
> expired?
> >
> >
> > It seems that your engine certificate has expired, but AFAIK this
> > certificate should be automatically renewed during engine-setup. So when
> did
> > you execute engine-setup for last time? Any info/warning about this shown
> > during invocation?
>
> Correct, Martin was a bit faster then me :)
>
> >
> > Also looking at server.log I found JBoss 7.1.1, so you are using really
> > ancient oVirt, version, right?
> >
> >>
> >> Please see the attached log for more info.
> >>
> >> Thank you so much for your assistance.
> >>
> >> Regards.
> >>
> >> Neil Wilson.
> >>
> >>
> >>
> >>
> >>
> >>
> >> On Thu, Sep 21, 2017 at 8:41 PM, Piotr Kliczewski
> >> <piotr.kliczewski at gmail.com> wrote:
> >>>
> >>> Neil,
> >>>
> >>> It seems that your engine certificate(s) is/are not ok. I would
> >>> suggest to enable ssl debug in the engine by:
> >>> - add '-Djavax.net.debug=all' to ovirt-engine.py file here [1].
> >>> - restart your engine
> >>> - check your server.log and check what is the issue.
> >>>
> >>> Hopefully we will be able to understand what happened in your setup.
> >>>
> >>> Thanks,
> >>> Piotr
> >>>
> >>> [1]
> >>> https://github.com/oVirt/ovirt-engine/blob/master/
> packaging/services/ovirt-engine/ovirt-engine.py#L341
> >>>
> >>> On Thu, Sep 21, 2017 at 4:42 PM, Neil <nwilson123 at gmail.com> wrote:
> >>> > Further to the logs sent, on the nodes I'm also seeing the following
> >>> > error
> >>> > under /var/log/messages...
> >>> >
> >>> > Sep 20 03:43:12 node01 vdsm root ERROR invalid client certificate
> with
> >>> > subject "/C=US/O=UKDM/CN=engine01.mydomain.za"^C
> >>> > Sep 20 03:43:12 node01 vdsm vds ERROR xml-rpc handler
> >>> > exception#012Traceback
> >>> > (most recent call last):#012  File "/usr/share/vdsm/
> BindingXMLRPC.py",
> >>> > line
> >>> > 80, in threaded_start#012    self.server.handle_request()#012  File
> >>> > "/usr/lib64/python2.6/SocketServer.py", line 278, in
> handle_request#012
> >>> > self._handle_request_noblock()#012  File
> >>> > "/usr/lib64/python2.6/SocketServer.py", line 288, in
> >>> > _handle_request_noblock#012    request, client_address =
> >>> > self.get_request()#012  File "/usr/lib64/python2.6/SocketServer.py",
> >>> > line
> >>> > 456, in get_request#012    return self.socket.accept()#012  File
> >>> > "/usr/lib64/python2.6/site-packages/vdsm/SecureXMLRPCServer.py",
> line
> >>> > 136,
> >>> > in accept#012    raise SSL.SSLError("%s, client %s" % (e,
> >>> > address[0]))#012SSLError: no certificate returned, client
> 10.251.193.5
> >>> >
> >>> > Not sure if this is any further help in diagnosing the issue?
> >>> >
> >>> > Thanks, any assistance is appreciated.
> >>> >
> >>> > Regards.
> >>> >
> >>> > Neil Wilson.
> >>> >
> >>> >
> >>> > On Thu, Sep 21, 2017 at 4:31 PM, Neil <nwilson123 at gmail.com> wrote:
> >>> >>
> >>> >> Hi Piotr,
> >>> >>
> >>> >> Thank you for the reply. After sending the email I did go and check
> >>> >> the
> >>> >> engine one too....
> >>> >>
> >>> >> [root at engine01 /]# openssl x509 -in /etc/pki/ovirt-engine/ca.pem
> >>> >> -enddate
> >>> >> -noout
> >>> >> notAfter=Oct 13 16:26:46 2022 GMT
> >>> >>
> >>> >> I'm not sure if this one below is meant to verify or if this output
> is
> >>> >> expected?
> >>> >>
> >>> >> [root at engine01 /]# openssl x509 -in
> >>> >> /etc/pki/ovirt-engine/private/ca.pem
> >>> >> -enddate -noout
> >>> >> unable to load certificate
> >>> >> 140642165552968:error:0906D06C:PEM routines:PEM_read_bio:no start
> >>> >> line:pem_lib.c:703:Expecting: TRUSTED CERTIFICATE
> >>> >>
> >>> >> My date is correct too Thu Sep 21 16:30:15 SAST 2017
> >>> >>
> >>> >> Any ideas?
> >>> >>
> >>> >> Googling surprisingly doesn't come up with much.
> >>> >>
> >>> >> Thank you.
> >>> >>
> >>> >> Regards.
> >>> >>
> >>> >> Neil Wilson.
> >>> >>
> >>> >> On Thu, Sep 21, 2017 at 4:16 PM, Piotr Kliczewski
> >>> >> <piotr.kliczewski at gmail.com> wrote:
> >>> >>>
> >>> >>> Neil,
> >>> >>>
> >>> >>> You checked both nodes what about the engine? Can you check engine
> >>> >>> certs?
> >>> >>> You can find more info where they are located here [1].
> >>> >>>
> >>> >>> Thanks,
> >>> >>> Piotr
> >>> >>>
> >>> >>> [1]
> >>> >>>
> >>> >>> https://www.ovirt.org/develop/release-management/features/
> infra/pki/#ovirt-engine
> >>> >>>
> >>> >>> On Thu, Sep 21, 2017 at 3:26 PM, Neil <nwilson123 at gmail.com>
> wrote:
> >>> >>> > Hi guys,
> >>> >>> >
> >>> >>> > Please could someone assist, my cluster is down and I can't
> access
> >>> >>> > my
> >>> >>> > vm's
> >>> >>> > to switch some of them back on.
> >>> >>> >
> >>> >>> > I'm seeing the following error in the engine.log however I've
> >>> >>> > checked
> >>> >>> > my
> >>> >>> > certs on my hosts (as some of the goolge results said to check),
> >>> >>> > but
> >>> >>> > the
> >>> >>> > certs haven't expired...
> >>> >>> >
> >>> >>> >
> >>> >>> > 2017-09-21 15:09:45,077 ERROR
> >>> >>> >
> >>> >>> > [org.ovirt.engine.core.vdsbroker.vdsbroker.
> GetCapabilitiesVDSCommand]
> >>> >>> > (DefaultQuartzScheduler_Worker-4) Command
> >>> >>> > GetCapabilitiesVDSCommand(HostName
> >>> >>> > = node02.mydomain.za, HostId =
> >>> >>> > d2debdfe-76e7-40cf-a7fd-78a0f50f14d4,
> >>> >>> > vds=Host[node02.mydomain.za]) execution failed. Exception:
> >>> >>> > VDSNetworkException: javax.net.ssl.SSLHandshakeException:
> Received
> >>> >>> > fatal
> >>> >>> > alert: certificate_expired
> >>> >>> > 2017-09-21 15:09:45,086 ERROR
> >>> >>> >
> >>> >>> > [org.ovirt.engine.core.vdsbroker.vdsbroker.
> GetCapabilitiesVDSCommand]
> >>> >>> > (DefaultQuartzScheduler_Worker-10) Command
> >>> >>> > GetCapabilitiesVDSCommand(HostName = node01.mydomain.za, HostId
> =
> >>> >>> > b108549c-1700-11e2-b936-9f5243b8ce13, vds=Host[
> node01.mydomain.za])
> >>> >>> > execution failed. Exception: VDSNetworkException:
> >>> >>> > javax.net.ssl.SSLHandshakeException: Received fatal alert:
> >>> >>> > certificate_expired
> >>> >>> > 2017-09-21 15:09:48,173 ERROR
> >>> >>> >
> >>> >>> > My engine and host info is below...
> >>> >>> >
> >>> >>> > [root at engine01 ovirt-engine]# rpm -qa | grep -i ovirt
> >>> >>> > ovirt-engine-lib-3.4.0-1.el6.noarch
> >>> >>> > ovirt-engine-restapi-3.4.0-1.el6.noarch
> >>> >>> > ovirt-engine-setup-plugin-ovirt-engine-3.4.0-1.el6.noarch
> >>> >>> > ovirt-engine-3.4.0-1.el6.noarch
> >>> >>> > ovirt-engine-setup-plugin-websocket-proxy-3.4.0-1.el6.noarch
> >>> >>> > ovirt-host-deploy-java-1.2.0-1.el6.noarch
> >>> >>> > ovirt-engine-setup-3.4.0-1.el6.noarch
> >>> >>> > ovirt-host-deploy-1.2.0-1.el6.noarch
> >>> >>> > ovirt-engine-backend-3.4.0-1.el6.noarch
> >>> >>> > ovirt-image-uploader-3.4.0-1.el6.noarch
> >>> >>> > ovirt-engine-tools-3.4.0-1.el6.noarch
> >>> >>> > ovirt-engine-sdk-python-3.4.0.7-1.el6.noarch
> >>> >>> > ovirt-engine-webadmin-portal-3.4.0-1.el6.noarch
> >>> >>> > ovirt-engine-cli-3.4.0.5-1.el6.noarch
> >>> >>> > ovirt-engine-setup-base-3.4.0-1.el6.noarch
> >>> >>> > ovirt-iso-uploader-3.4.0-1.el6.noarch
> >>> >>> > ovirt-engine-userportal-3.4.0-1.el6.noarch
> >>> >>> > ovirt-log-collector-3.4.1-1.el6.noarch
> >>> >>> > ovirt-engine-websocket-proxy-3.4.0-1.el6.noarch
> >>> >>> > ovirt-engine-setup-plugin-ovirt-engine-common-3.4.0-1.el6.noarch
> >>> >>> > ovirt-engine-dbscripts-3.4.0-1.el6.noarch
> >>> >>> > [root at engine01 ovirt-engine]# cat /etc/redhat-release
> >>> >>> > CentOS release 6.5 (Final)
> >>> >>> >
> >>> >>> >
> >>> >>> > [root at node02 ~]# openssl x509 -in /etc/pki/vdsm/certs/vdsmcert.
> pem
> >>> >>> > -enddate
> >>> >>> > -noout ; date
> >>> >>> > notAfter=May 27 08:36:17 2019 GMT
> >>> >>> > Thu Sep 21 15:18:22 SAST 2017
> >>> >>> > CentOS release 6.5 (Final)
> >>> >>> > [root at node02 ~]# rpm -qa | grep vdsm
> >>> >>> > vdsm-4.14.6-0.el6.x86_64
> >>> >>> > vdsm-python-4.14.6-0.el6.x86_64
> >>> >>> > vdsm-cli-4.14.6-0.el6.noarch
> >>> >>> > vdsm-xmlrpc-4.14.6-0.el6.noarch
> >>> >>> > vdsm-python-zombiereaper-4.14.6-0.el6.noarch
> >>> >>> >
> >>> >>> >
> >>> >>> > [root at node01 ~]# openssl x509 -in /etc/pki/vdsm/certs/vdsmcert.
> pem
> >>> >>> > -enddate
> >>> >>> > -noout ; date
> >>> >>> > notAfter=Jun 13 16:09:41 2018 GMT
> >>> >>> > Thu Sep 21 15:18:52 SAST 2017
> >>> >>> > CentOS release 6.5 (Final)
> >>> >>> > [root at node01 ~]# rpm -qa | grep -i vdsm
> >>> >>> > vdsm-4.14.6-0.el6.x86_64
> >>> >>> > vdsm-xmlrpc-4.14.6-0.el6.noarch
> >>> >>> > vdsm-cli-4.14.6-0.el6.noarch
> >>> >>> > vdsm-python-zombiereaper-4.14.6-0.el6.noarch
> >>> >>> > vdsm-python-4.14.6-0.el6.x86_64
> >>> >>> >
> >>> >>> > Please could I have some assistance, I'm rater desperate.
> >>> >>> >
> >>> >>> > Thank you.
> >>> >>> >
> >>> >>> > Regards.
> >>> >>> >
> >>> >>> > Neil Wilson
> >>> >>> >
> >>> >>> >
> >>> >>> >
> >>> >>> > _______________________________________________
> >>> >>> > Users mailing list
> >>> >>> > Users at ovirt.org
> >>> >>> > http://lists.ovirt.org/mailman/listinfo/users
> >>> >>> >
> >>> >>
> >>> >>
> >>> >
> >>
> >>
> >>
> >> _______________________________________________
> >> Users mailing list
> >> Users at ovirt.org
> >> http://lists.ovirt.org/mailman/listinfo/users
> >>
> >
>
-------------- next part --------------
An HTML attachment was scrubbed...
URL: <http://lists.ovirt.org/pipermail/users/attachments/20170922/e51eebbb/attachment.html>
-------------- next part --------------
A non-text attachment was scrubbed...
Name: setup.log
Type: text/x-log
Size: 145357 bytes
Desc: not available
URL: <http://lists.ovirt.org/pipermail/users/attachments/20170922/e51eebbb/attachment.bin>