[ovirt-users] SSLHandshakeException: Received fatal alert: certificate_expired

Fri Sep 22 09:10:34 UTC 2017

On Fri, Sep 22, 2017 at 10:58 AM, Neil <nwilson123 at gmail.com> wrote:

> Thanks Martin and Piotr,
>
> Correct, this was a very old installation from the old drey repo that was
> upgraded gradually over the years.
>
> I have tried engine-setup yesterday, prior to this looking under
> /var/log/ovirt-engine/setup it looks like 2014
>
> I've attached a log of the output of running it now, looks like a repo
> issue with trying to upgrade to the latest 3.4.x release, but not sure what
> else to look for?
>

Hmm, it's so ancient version that oVirt 3.4 mirrors are probably not
working anymore. You can either:

1. Execute engine-setup --offline to skip updates check or
2. Edit /etc/yum.repos.d/ovirt*.conf files and switch from mirrors to main
site resources.ovirt.org

> Thanks for the assistance.
>
> Regards.
>
> Neil Wilson
>
>
> On Fri, Sep 22, 2017 at 10:38 AM, Piotr Kliczewski <
> piotr.kliczewski at gmail.com> wrote:
>
>> On Fri, Sep 22, 2017 at 10:35 AM, Martin Perina <mperina at redhat.com>
>> wrote:
>> >
>> >
>> > On Fri, Sep 22, 2017 at 10:18 AM, Neil <nwilson123 at gmail.com> wrote:
>> >>
>> >> Hi Piotr,
>> >>
>> >> Thank you for the information.
>> >>
>> >> It looks like something has expired looking in the server.log now that
>> >> debug is enabled.
>> >>
>> >> 2017-09-22 09:35:26,462 INFO  [stdout] (MSC service thread 1-4)
>>  Version:
>> >> V3
>> >> 2017-09-22 09:35:26,464 INFO  [stdout] (MSC service thread 1-4)
>>  Subject:
>> >> CN=engine01.mydomain.za, O=mydomain, C=US
>> >> 2017-09-22 09:35:26,467 INFO  [stdout] (MSC service thread 1-4)
>> >> Signature Algorithm: SHA1withRSA, OID = 1.2.840.113549.1.1.5
>> >> 2017-09-22 09:35:26,471 INFO  [stdout] (MSC service thread 1-4)
>> >> 2017-09-22 09:35:26,472 INFO  [stdout] (MSC service thread 1-4)   Key:
>> >> Sun RSA public key, 1024 bits
>> >> 2017-09-22 09:35:26,474 INFO  [stdout] (MSC service thread 1-4)
>>  modulus:
>> >> 966706131850237857720016566132274169225143716493132034132811
>> 213711757321195965137528821713060454503460188878350322233731
>> 259812207539722762942035931744044702655933680916835641105243
>> 164032601213316092139626126181817086803318505413903188689260
>> 54438078223371655800890725486783860059873397983318033852172060923531
>> >> 2017-09-22 09:35:26,476 INFO  [stdout] (MSC service thread 1-4)
>>  public
>> >> exponent: 65537
>> >> 2017-09-22 09:35:26,477 INFO  [stdout] (MSC service thread 1-4)
>> >> Validity: [From: Sun Oct 14 22:26:46 SAST 2012,
>> >> 2017-09-22 09:35:26,478 INFO  [stdout] (MSC service thread 1-4)
>> >> To: Tue Sep 19 18:26:49 SAST 2017]
>> >> 2017-09-22 09:35:26,479 INFO  [stdout] (MSC service thread 1-4)
>>  Issuer:
>> >> CN=CA-engine01.mydomain.za.47472, O=mydomain, C=US
>> >>
>> >> Any idea how I can generate a new one and what cert it is that's
>> expired?
>> >
>> >
>> > It seems that your engine certificate has expired, but AFAIK this
>> > certificate should be automatically renewed during engine-setup. So
>> when did
>> > you execute engine-setup for last time? Any info/warning about this
>> shown
>> > during invocation?
>>
>> Correct, Martin was a bit faster then me :)
>>
>> >
>> > Also looking at server.log I found JBoss 7.1.1, so you are using really
>> > ancient oVirt, version, right?
>> >
>> >>
>> >> Please see the attached log for more info.
>> >>
>> >> Thank you so much for your assistance.
>> >>
>> >> Regards.
>> >>
>> >> Neil Wilson.
>> >>
>> >>
>> >>
>> >>
>> >>
>> >>
>> >> On Thu, Sep 21, 2017 at 8:41 PM, Piotr Kliczewski
>> >> <piotr.kliczewski at gmail.com> wrote:
>> >>>
>> >>> Neil,
>> >>>
>> >>> It seems that your engine certificate(s) is/are not ok. I would
>> >>> suggest to enable ssl debug in the engine by:
>> >>> - add '-Djavax.net.debug=all' to ovirt-engine.py file here [1].
>> >>> - restart your engine
>> >>> - check your server.log and check what is the issue.
>> >>>
>> >>> Hopefully we will be able to understand what happened in your setup.
>> >>>
>> >>> Thanks,
>> >>> Piotr
>> >>>
>> >>> [1]
>> >>> https://github.com/oVirt/ovirt-engine/blob/master/packaging/
>> services/ovirt-engine/ovirt-engine.py#L341
>> >>>
>> >>> On Thu, Sep 21, 2017 at 4:42 PM, Neil <nwilson123 at gmail.com> wrote:
>> >>> > Further to the logs sent, on the nodes I'm also seeing the following
>> >>> > error
>> >>> > under /var/log/messages...
>> >>> >
>> >>> > Sep 20 03:43:12 node01 vdsm root ERROR invalid client certificate
>> with
>> >>> > subject "/C=US/O=UKDM/CN=engine01.mydomain.za"^C
>> >>> > Sep 20 03:43:12 node01 vdsm vds ERROR xml-rpc handler
>> >>> > exception#012Traceback
>> >>> > (most recent call last):#012  File "/usr/share/vdsm/BindingXMLRPC
>> .py",
>> >>> > line
>> >>> > 80, in threaded_start#012    self.server.handle_request()#012  File
>> >>> > "/usr/lib64/python2.6/SocketServer.py", line 278, in
>> handle_request#012
>> >>> > self._handle_request_noblock()#012  File
>> >>> > "/usr/lib64/python2.6/SocketServer.py", line 288, in
>> >>> > _handle_request_noblock#012    request, client_address =
>> >>> > self.get_request()#012  File "/usr/lib64/python2.6/SocketSe
>> rver.py",
>> >>> > line
>> >>> > 456, in get_request#012    return self.socket.accept()#012  File
>> >>> > "/usr/lib64/python2.6/site-packages/vdsm/SecureXMLRPCServer.py",
>> line
>> >>> > 136,
>> >>> > in accept#012    raise SSL.SSLError("%s, client %s" % (e,
>> >>> > address[0]))#012SSLError: no certificate returned, client
>> 10.251.193.5
>> >>> >
>> >>> > Not sure if this is any further help in diagnosing the issue?
>> >>> >
>> >>> > Thanks, any assistance is appreciated.
>> >>> >
>> >>> > Regards.
>> >>> >
>> >>> > Neil Wilson.
>> >>> >
>> >>> >
>> >>> > On Thu, Sep 21, 2017 at 4:31 PM, Neil <nwilson123 at gmail.com> wrote:
>> >>> >>
>> >>> >> Hi Piotr,
>> >>> >>
>> >>> >> Thank you for the reply. After sending the email I did go and check
>> >>> >> the
>> >>> >> engine one too....
>> >>> >>
>> >>> >> [root at engine01 /]# openssl x509 -in /etc/pki/ovirt-engine/ca.pem
>> >>> >> -enddate
>> >>> >> -noout
>> >>> >> notAfter=Oct 13 16:26:46 2022 GMT
>> >>> >>
>> >>> >> I'm not sure if this one below is meant to verify or if this
>> output is
>> >>> >> expected?
>> >>> >>
>> >>> >> [root at engine01 /]# openssl x509 -in
>> >>> >> /etc/pki/ovirt-engine/private/ca.pem
>> >>> >> -enddate -noout
>> >>> >> unable to load certificate
>> >>> >> 140642165552968:error:0906D06C:PEM routines:PEM_read_bio:no start
>> >>> >> line:pem_lib.c:703:Expecting: TRUSTED CERTIFICATE
>> >>> >>
>> >>> >> My date is correct too Thu Sep 21 16:30:15 SAST 2017
>> >>> >>
>> >>> >> Any ideas?
>> >>> >>
>> >>> >> Googling surprisingly doesn't come up with much.
>> >>> >>
>> >>> >> Thank you.
>> >>> >>
>> >>> >> Regards.
>> >>> >>
>> >>> >> Neil Wilson.
>> >>> >>
>> >>> >> On Thu, Sep 21, 2017 at 4:16 PM, Piotr Kliczewski
>> >>> >> <piotr.kliczewski at gmail.com> wrote:
>> >>> >>>
>> >>> >>> Neil,
>> >>> >>>
>> >>> >>> You checked both nodes what about the engine? Can you check engine
>> >>> >>> certs?
>> >>> >>> You can find more info where they are located here [1].
>> >>> >>>
>> >>> >>> Thanks,
>> >>> >>> Piotr
>> >>> >>>
>> >>> >>> [1]
>> >>> >>>
>> >>> >>> https://www.ovirt.org/develop/release-management/features/in
>> fra/pki/#ovirt-engine
>> >>> >>>
>> >>> >>> On Thu, Sep 21, 2017 at 3:26 PM, Neil <nwilson123 at gmail.com>
>> wrote:
>> >>> >>> > Hi guys,
>> >>> >>> >
>> >>> >>> > Please could someone assist, my cluster is down and I can't
>> access
>> >>> >>> > my
>> >>> >>> > vm's
>> >>> >>> > to switch some of them back on.
>> >>> >>> >
>> >>> >>> > I'm seeing the following error in the engine.log however I've
>> >>> >>> > checked
>> >>> >>> > my
>> >>> >>> > certs on my hosts (as some of the goolge results said to check),
>> >>> >>> > but
>> >>> >>> > the
>> >>> >>> > certs haven't expired...
>> >>> >>> >
>> >>> >>> >
>> >>> >>> > 2017-09-21 15:09:45,077 ERROR
>> >>> >>> >
>> >>> >>> > [org.ovirt.engine.core.vdsbroker.vdsbroker.GetCapabilitiesVD
>> SCommand]
>> >>> >>> > (DefaultQuartzScheduler_Worker-4) Command
>> >>> >>> > GetCapabilitiesVDSCommand(HostName
>> >>> >>> > = node02.mydomain.za, HostId =
>> >>> >>> > d2debdfe-76e7-40cf-a7fd-78a0f50f14d4,
>> >>> >>> > vds=Host[node02.mydomain.za]) execution failed. Exception:
>> >>> >>> > VDSNetworkException: javax.net.ssl.SSLHandshakeException:
>> Received
>> >>> >>> > fatal
>> >>> >>> > alert: certificate_expired
>> >>> >>> > 2017-09-21 15:09:45,086 ERROR
>> >>> >>> >
>> >>> >>> > [org.ovirt.engine.core.vdsbroker.vdsbroker.GetCapabilitiesVD
>> SCommand]
>> >>> >>> > (DefaultQuartzScheduler_Worker-10) Command
>> >>> >>> > GetCapabilitiesVDSCommand(HostName = node01.mydomain.za,
>> HostId =
>> >>> >>> > b108549c-1700-11e2-b936-9f5243b8ce13, vds=Host[
>> node01.mydomain.za])
>> >>> >>> > execution failed. Exception: VDSNetworkException:
>> >>> >>> > javax.net.ssl.SSLHandshakeException: Received fatal alert:
>> >>> >>> > certificate_expired
>> >>> >>> > 2017-09-21 15:09:48,173 ERROR
>> >>> >>> >
>> >>> >>> > My engine and host info is below...
>> >>> >>> >
>> >>> >>> > [root at engine01 ovirt-engine]# rpm -qa | grep -i ovirt
>> >>> >>> > ovirt-engine-lib-3.4.0-1.el6.noarch
>> >>> >>> > ovirt-engine-restapi-3.4.0-1.el6.noarch
>> >>> >>> > ovirt-engine-setup-plugin-ovirt-engine-3.4.0-1.el6.noarch
>> >>> >>> > ovirt-engine-3.4.0-1.el6.noarch
>> >>> >>> > ovirt-engine-setup-plugin-websocket-proxy-3.4.0-1.el6.noarch
>> >>> >>> > ovirt-host-deploy-java-1.2.0-1.el6.noarch
>> >>> >>> > ovirt-engine-setup-3.4.0-1.el6.noarch
>> >>> >>> > ovirt-host-deploy-1.2.0-1.el6.noarch
>> >>> >>> > ovirt-engine-backend-3.4.0-1.el6.noarch
>> >>> >>> > ovirt-image-uploader-3.4.0-1.el6.noarch
>> >>> >>> > ovirt-engine-tools-3.4.0-1.el6.noarch
>> >>> >>> > ovirt-engine-sdk-python-3.4.0.7-1.el6.noarch
>> >>> >>> > ovirt-engine-webadmin-portal-3.4.0-1.el6.noarch
>> >>> >>> > ovirt-engine-cli-3.4.0.5-1.el6.noarch
>> >>> >>> > ovirt-engine-setup-base-3.4.0-1.el6.noarch
>> >>> >>> > ovirt-iso-uploader-3.4.0-1.el6.noarch
>> >>> >>> > ovirt-engine-userportal-3.4.0-1.el6.noarch
>> >>> >>> > ovirt-log-collector-3.4.1-1.el6.noarch
>> >>> >>> > ovirt-engine-websocket-proxy-3.4.0-1.el6.noarch
>> >>> >>> > ovirt-engine-setup-plugin-ovirt-engine-common-3.4.0-1.el6.
>> noarch
>> >>> >>> > ovirt-engine-dbscripts-3.4.0-1.el6.noarch
>> >>> >>> > [root at engine01 ovirt-engine]# cat /etc/redhat-release
>> >>> >>> > CentOS release 6.5 (Final)
>> >>> >>> >
>> >>> >>> >
>> >>> >>> > [root at node02 ~]# openssl x509 -in
>> /etc/pki/vdsm/certs/vdsmcert.pem
>> >>> >>> > -enddate
>> >>> >>> > -noout ; date
>> >>> >>> > notAfter=May 27 08:36:17 2019 GMT
>> >>> >>> > Thu Sep 21 15:18:22 SAST 2017
>> >>> >>> > CentOS release 6.5 (Final)
>> >>> >>> > [root at node02 ~]# rpm -qa | grep vdsm
>> >>> >>> > vdsm-4.14.6-0.el6.x86_64
>> >>> >>> > vdsm-python-4.14.6-0.el6.x86_64
>> >>> >>> > vdsm-cli-4.14.6-0.el6.noarch
>> >>> >>> > vdsm-xmlrpc-4.14.6-0.el6.noarch
>> >>> >>> > vdsm-python-zombiereaper-4.14.6-0.el6.noarch
>> >>> >>> >
>> >>> >>> >
>> >>> >>> > [root at node01 ~]# openssl x509 -in
>> /etc/pki/vdsm/certs/vdsmcert.pem
>> >>> >>> > -enddate
>> >>> >>> > -noout ; date
>> >>> >>> > notAfter=Jun 13 16:09:41 2018 GMT
>> >>> >>> > Thu Sep 21 15:18:52 SAST 2017
>> >>> >>> > CentOS release 6.5 (Final)
>> >>> >>> > [root at node01 ~]# rpm -qa | grep -i vdsm
>> >>> >>> > vdsm-4.14.6-0.el6.x86_64
>> >>> >>> > vdsm-xmlrpc-4.14.6-0.el6.noarch
>> >>> >>> > vdsm-cli-4.14.6-0.el6.noarch
>> >>> >>> > vdsm-python-zombiereaper-4.14.6-0.el6.noarch
>> >>> >>> > vdsm-python-4.14.6-0.el6.x86_64
>> >>> >>> >
>> >>> >>> > Please could I have some assistance, I'm rater desperate.
>> >>> >>> >
>> >>> >>> > Thank you.
>> >>> >>> >
>> >>> >>> > Regards.
>> >>> >>> >
>> >>> >>> > Neil Wilson
>> >>> >>> >
>> >>> >>> >
>> >>> >>> >
>> >>> >>> > _______________________________________________
>> >>> >>> > Users mailing list
>> >>> >>> > Users at ovirt.org
>> >>> >>> > http://lists.ovirt.org/mailman/listinfo/users
>> >>> >>> >
>> >>> >>
>> >>> >>
>> >>> >
>> >>
>> >>
>> >>
>> >> _______________________________________________
>> >> Users mailing list
>> >> Users at ovirt.org
>> >> http://lists.ovirt.org/mailman/listinfo/users
>> >>
>> >
>>
>
>
-------------- next part --------------
An HTML attachment was scrubbed...
URL: <http://lists.ovirt.org/pipermail/users/attachments/20170922/d6d192b9/attachment.html>