[ovirt-users] SSLHandshakeException: Received fatal alert: certificate_expired
Neil
nwilson123 at gmail.com
Fri Sep 22 13:07:58 UTC 2017
Thanks for the guidance everyone.
I've upgraded my engine now to ovirt-engine-3.4.4-1 but I've still got the
same error unfortunately. Below is the output of the upgrade. Should this
have fixed the issue or do I need to upgrade to 3.5 etc?
[ INFO ] Stage: Initializing
[ INFO ] Stage: Environment setup
Configuration files:
['/etc/ovirt-engine-setup.conf.d/10-packaging.conf',
'/etc/ovirt-engine-setup.conf.d/20-setup-ovirt-post.conf']
Log file:
/var/log/ovirt-engine/setup/ovirt-engine-setup-20170922125526-vw5khx.log
Version: otopi-1.2.3 (otopi-1.2.3-1.el6)
[ INFO ] Stage: Environment packages setup
[ INFO ] Yum Downloading: repomdPLa0LXtmp.xml (0%)
[ INFO ] Stage: Programs detection
[ INFO ] Stage: Environment setup
[ INFO ] Stage: Environment customization
--== PRODUCT OPTIONS ==--
--== PACKAGES ==--
[ INFO ] Checking for product updates...
Setup has found updates for some packages, do you wish to update
them now? (Yes, No) [Yes]:
[ INFO ] Checking for an update for Setup...
--== NETWORK CONFIGURATION ==--
[WARNING] Failed to resolve engine01.mydomain.za using DNS, it can be
resolved only locally
Setup can automatically configure the firewall on this system.
Note: automatic configuration of the firewall may overwrite
current settings.
Do you want Setup to configure the firewall? (Yes, No) [Yes]: no
--== DATABASE CONFIGURATION ==--
--== OVIRT ENGINE CONFIGURATION ==--
Skipping storing options as database already prepared
--== PKI CONFIGURATION ==--
PKI is already configured
--== APACHE CONFIGURATION ==--
--== SYSTEM CONFIGURATION ==--
--== MISC CONFIGURATION ==--
--== END OF CONFIGURATION ==--
[ INFO ] Stage: Setup validation
During execution engine service will be stopped (OK, Cancel)
[OK]:
[WARNING] Less than 16384MB of memory is available
[ INFO ] Cleaning stale zombie tasks
--== CONFIGURATION PREVIEW ==--
Engine database name : engine
Engine database secured connection : False
Engine database host : localhost
Engine database user name : engine
Engine database host name validation : False
Engine database port : 5432
Datacenter storage type : False
Update Firewall : False
Configure WebSocket Proxy : True
Host FQDN : engine01.mydomain.za
Upgrade packages : True
Please confirm installation settings (OK, Cancel) [OK]:
[ INFO ] Cleaning async tasks and compensations
[ INFO ] Checking the Engine database consistency
[ INFO ] Stage: Transaction setup
[ INFO ] Stopping engine service
[ INFO ] Stopping websocket-proxy service
[ INFO ] Stage: Misc configuration
[ INFO ] Stage: Package installation
[ INFO ] Yum Status: Downloading Packages
[ INFO ] Yum Download/Verify: ovirt-engine-3.4.4-1.el6.noarch
[ INFO ] Yum Downloading: (2/13):
ovirt-engine-backend-3.4.4-1.el6.noarch.rpm 2.0 M(19%)
[ INFO ] Yum Downloading: (2/13):
ovirt-engine-backend-3.4.4-1.el6.noarch.rpm 4.3 M(41%)
[ INFO ] Yum Downloading: (2/13):
ovirt-engine-backend-3.4.4-1.el6.noarch.rpm 6.3 M(60%)
[ INFO ] Yum Downloading: (2/13):
ovirt-engine-backend-3.4.4-1.el6.noarch.rpm 8.9 M(85%)
[ INFO ] Yum Download/Verify: ovirt-engine-backend-3.4.4-1.el6.noarch
[ INFO ] Yum Download/Verify: ovirt-engine-dbscripts-3.4.4-1.el6.noarch
(I've taken out all the downloading progress)
[ INFO ] Yum Verify: 26/26: ovirt-engine-backend.noarch 0:3.4.0-1.el6 - ud
[ INFO ] Stage: Misc configuration
[ INFO ] Backing up database localhost:engine to
'/var/lib/ovirt-engine/backups/engine-20170922143709.m_8fr_.dump'.
[ INFO ] Updating Engine database schema
[ INFO ] Generating post install configuration file
'/etc/ovirt-engine-setup.conf.d/20-setup-ovirt-post.conf'
[ INFO ] Stage: Transaction commit
[ INFO ] Stage: Closing up
--== SUMMARY ==--
[WARNING] Less than 16384MB of memory is available
SSH fingerprint: 86:C7:AA:35:45:E9:83:3E:16:C9:2A:F5:68:52:68:84
Internal CA
EE:91:B3:E7:40:D7:DD:A7:DD:77:9C:3B:D5:A1:E7:BE:E2:C9:8B:AA
Web access is enabled at:
http://engine01.mydomain.za:80/ovirt-engine
https://engine01.mydomain.za:443/ovirt-engine
In order to configure firewalld, copy the files from
/etc/ovirt-engine/firewalld to /etc/firewalld/services
and execute the following commands:
firewall-cmd -service ovirt-postgres
firewall-cmd -service ovirt-https
firewall-cmd -service ovirt-websocket-proxy
firewall-cmd -service ovirt-http
The following network ports should be opened:
tcp:443
tcp:5432
tcp:6100
tcp:80
An example of the required configuration for iptables can be
found at:
/etc/ovirt-engine/iptables.example
--== END OF SUMMARY ==--
[ INFO ] Starting engine service
[ INFO ] Restarting httpd
[ INFO ] Stage: Clean up
Log file is located at
/var/log/ovirt-engine/setup/ovirt-engine-setup-20170922125526-vw5khx.log
[ INFO ] Generating answer file
'/var/lib/ovirt-engine/setup/answers/20170922143806-setup.conf'
[ INFO ] Stage: Pre-termination
[ INFO ] Stage: Termination
[ INFO ] Execution of setup completed successfully
I'm still seeing the following below, in my engine.log and when I log in,
all my VM's show as unknown.
2017-09-22 15:06:06,060 ERROR
[org.ovirt.engine.core.vdsbroker.vdsbroker.GetCapabilitiesVDSCommand]
(DefaultQuartzScheduler_Worker-57) Command
GetCapabilitiesVDSCommand(HostName = node02.mydomain.za, HostId =
d2debdfe-76e7-40cf-a7fd-78a0f50f14d4,
vds=Host[node02.mydomain.za,d2debdfe-76e7-40cf-a7fd-78a0f50f14d4])
execution failed. Exception: VDSNetworkException:
javax.net.ssl.SSLHandshakeException: Received fatal alert:
certificate_expired
Any ideas?
Thanks!
On Fri, Sep 22, 2017 at 11:10 AM, Martin Perina <mperina at redhat.com> wrote:
>
>
> On Fri, Sep 22, 2017 at 10:58 AM, Neil <nwilson123 at gmail.com> wrote:
>
>> Thanks Martin and Piotr,
>>
>> Correct, this was a very old installation from the old drey repo that was
>> upgraded gradually over the years.
>>
>> I have tried engine-setup yesterday, prior to this looking under
>> /var/log/ovirt-engine/setup it looks like 2014
>>
>> I've attached a log of the output of running it now, looks like a repo
>> issue with trying to upgrade to the latest 3.4.x release, but not sure what
>> else to look for?
>>
>
> Hmm, it's so ancient version that oVirt 3.4 mirrors are probably not
> working anymore. You can either:
>
> 1. Execute engine-setup --offline to skip updates check or
> 2. Edit /etc/yum.repos.d/ovirt*.conf files and switch from mirrors to main
> site resources.ovirt.org
>
>
>> Thanks for the assistance.
>>
>> Regards.
>>
>> Neil Wilson
>>
>>
>> On Fri, Sep 22, 2017 at 10:38 AM, Piotr Kliczewski <
>> piotr.kliczewski at gmail.com> wrote:
>>
>>> On Fri, Sep 22, 2017 at 10:35 AM, Martin Perina <mperina at redhat.com>
>>> wrote:
>>> >
>>> >
>>> > On Fri, Sep 22, 2017 at 10:18 AM, Neil <nwilson123 at gmail.com> wrote:
>>> >>
>>> >> Hi Piotr,
>>> >>
>>> >> Thank you for the information.
>>> >>
>>> >> It looks like something has expired looking in the server.log now that
>>> >> debug is enabled.
>>> >>
>>> >> 2017-09-22 09:35:26,462 INFO [stdout] (MSC service thread 1-4)
>>> Version:
>>> >> V3
>>> >> 2017-09-22 09:35:26,464 INFO [stdout] (MSC service thread 1-4)
>>> Subject:
>>> >> CN=engine01.mydomain.za, O=mydomain, C=US
>>> >> 2017-09-22 09:35:26,467 INFO [stdout] (MSC service thread 1-4)
>>> >> Signature Algorithm: SHA1withRSA, OID = 1.2.840.113549.1.1.5
>>> >> 2017-09-22 09:35:26,471 INFO [stdout] (MSC service thread 1-4)
>>> >> 2017-09-22 09:35:26,472 INFO [stdout] (MSC service thread 1-4) Key:
>>> >> Sun RSA public key, 1024 bits
>>> >> 2017-09-22 09:35:26,474 INFO [stdout] (MSC service thread 1-4)
>>> modulus:
>>> >> 966706131850237857720016566132274169225143716493132034132811
>>> 213711757321195965137528821713060454503460188878350322233731
>>> 259812207539722762942035931744044702655933680916835641105243
>>> 164032601213316092139626126181817086803318505413903188689260
>>> 54438078223371655800890725486783860059873397983318033852172060923531
>>> >> 2017-09-22 09:35:26,476 INFO [stdout] (MSC service thread 1-4)
>>> public
>>> >> exponent: 65537
>>> >> 2017-09-22 09:35:26,477 INFO [stdout] (MSC service thread 1-4)
>>> >> Validity: [From: Sun Oct 14 22:26:46 SAST 2012,
>>> >> 2017-09-22 09:35:26,478 INFO [stdout] (MSC service thread 1-4)
>>> >> To: Tue Sep 19 18:26:49 SAST 2017]
>>> >> 2017-09-22 09:35:26,479 INFO [stdout] (MSC service thread 1-4)
>>> Issuer:
>>> >> CN=CA-engine01.mydomain.za.47472, O=mydomain, C=US
>>> >>
>>> >> Any idea how I can generate a new one and what cert it is that's
>>> expired?
>>> >
>>> >
>>> > It seems that your engine certificate has expired, but AFAIK this
>>> > certificate should be automatically renewed during engine-setup. So
>>> when did
>>> > you execute engine-setup for last time? Any info/warning about this
>>> shown
>>> > during invocation?
>>>
>>> Correct, Martin was a bit faster then me :)
>>>
>>> >
>>> > Also looking at server.log I found JBoss 7.1.1, so you are using really
>>> > ancient oVirt, version, right?
>>> >
>>> >>
>>> >> Please see the attached log for more info.
>>> >>
>>> >> Thank you so much for your assistance.
>>> >>
>>> >> Regards.
>>> >>
>>> >> Neil Wilson.
>>> >>
>>> >>
>>> >>
>>> >>
>>> >>
>>> >>
>>> >> On Thu, Sep 21, 2017 at 8:41 PM, Piotr Kliczewski
>>> >> <piotr.kliczewski at gmail.com> wrote:
>>> >>>
>>> >>> Neil,
>>> >>>
>>> >>> It seems that your engine certificate(s) is/are not ok. I would
>>> >>> suggest to enable ssl debug in the engine by:
>>> >>> - add '-Djavax.net.debug=all' to ovirt-engine.py file here [1].
>>> >>> - restart your engine
>>> >>> - check your server.log and check what is the issue.
>>> >>>
>>> >>> Hopefully we will be able to understand what happened in your setup.
>>> >>>
>>> >>> Thanks,
>>> >>> Piotr
>>> >>>
>>> >>> [1]
>>> >>> https://github.com/oVirt/ovirt-engine/blob/master/packaging/
>>> services/ovirt-engine/ovirt-engine.py#L341
>>> >>>
>>> >>> On Thu, Sep 21, 2017 at 4:42 PM, Neil <nwilson123 at gmail.com> wrote:
>>> >>> > Further to the logs sent, on the nodes I'm also seeing the
>>> following
>>> >>> > error
>>> >>> > under /var/log/messages...
>>> >>> >
>>> >>> > Sep 20 03:43:12 node01 vdsm root ERROR invalid client certificate
>>> with
>>> >>> > subject "/C=US/O=UKDM/CN=engine01.mydomain.za"^C
>>> >>> > Sep 20 03:43:12 node01 vdsm vds ERROR xml-rpc handler
>>> >>> > exception#012Traceback
>>> >>> > (most recent call last):#012 File "/usr/share/vdsm/BindingXMLRPC
>>> .py",
>>> >>> > line
>>> >>> > 80, in threaded_start#012 self.server.handle_request()#012
>>> File
>>> >>> > "/usr/lib64/python2.6/SocketServer.py", line 278, in
>>> handle_request#012
>>> >>> > self._handle_request_noblock()#012 File
>>> >>> > "/usr/lib64/python2.6/SocketServer.py", line 288, in
>>> >>> > _handle_request_noblock#012 request, client_address =
>>> >>> > self.get_request()#012 File "/usr/lib64/python2.6/SocketSe
>>> rver.py",
>>> >>> > line
>>> >>> > 456, in get_request#012 return self.socket.accept()#012 File
>>> >>> > "/usr/lib64/python2.6/site-packages/vdsm/SecureXMLRPCServer.py",
>>> line
>>> >>> > 136,
>>> >>> > in accept#012 raise SSL.SSLError("%s, client %s" % (e,
>>> >>> > address[0]))#012SSLError: no certificate returned, client
>>> 10.251.193.5
>>> >>> >
>>> >>> > Not sure if this is any further help in diagnosing the issue?
>>> >>> >
>>> >>> > Thanks, any assistance is appreciated.
>>> >>> >
>>> >>> > Regards.
>>> >>> >
>>> >>> > Neil Wilson.
>>> >>> >
>>> >>> >
>>> >>> > On Thu, Sep 21, 2017 at 4:31 PM, Neil <nwilson123 at gmail.com>
>>> wrote:
>>> >>> >>
>>> >>> >> Hi Piotr,
>>> >>> >>
>>> >>> >> Thank you for the reply. After sending the email I did go and
>>> check
>>> >>> >> the
>>> >>> >> engine one too....
>>> >>> >>
>>> >>> >> [root at engine01 /]# openssl x509 -in /etc/pki/ovirt-engine/ca.pem
>>> >>> >> -enddate
>>> >>> >> -noout
>>> >>> >> notAfter=Oct 13 16:26:46 2022 GMT
>>> >>> >>
>>> >>> >> I'm not sure if this one below is meant to verify or if this
>>> output is
>>> >>> >> expected?
>>> >>> >>
>>> >>> >> [root at engine01 /]# openssl x509 -in
>>> >>> >> /etc/pki/ovirt-engine/private/ca.pem
>>> >>> >> -enddate -noout
>>> >>> >> unable to load certificate
>>> >>> >> 140642165552968:error:0906D06C:PEM routines:PEM_read_bio:no start
>>> >>> >> line:pem_lib.c:703:Expecting: TRUSTED CERTIFICATE
>>> >>> >>
>>> >>> >> My date is correct too Thu Sep 21 16:30:15 SAST 2017
>>> >>> >>
>>> >>> >> Any ideas?
>>> >>> >>
>>> >>> >> Googling surprisingly doesn't come up with much.
>>> >>> >>
>>> >>> >> Thank you.
>>> >>> >>
>>> >>> >> Regards.
>>> >>> >>
>>> >>> >> Neil Wilson.
>>> >>> >>
>>> >>> >> On Thu, Sep 21, 2017 at 4:16 PM, Piotr Kliczewski
>>> >>> >> <piotr.kliczewski at gmail.com> wrote:
>>> >>> >>>
>>> >>> >>> Neil,
>>> >>> >>>
>>> >>> >>> You checked both nodes what about the engine? Can you check
>>> engine
>>> >>> >>> certs?
>>> >>> >>> You can find more info where they are located here [1].
>>> >>> >>>
>>> >>> >>> Thanks,
>>> >>> >>> Piotr
>>> >>> >>>
>>> >>> >>> [1]
>>> >>> >>>
>>> >>> >>> https://www.ovirt.org/develop/release-management/features/in
>>> fra/pki/#ovirt-engine
>>> >>> >>>
>>> >>> >>> On Thu, Sep 21, 2017 at 3:26 PM, Neil <nwilson123 at gmail.com>
>>> wrote:
>>> >>> >>> > Hi guys,
>>> >>> >>> >
>>> >>> >>> > Please could someone assist, my cluster is down and I can't
>>> access
>>> >>> >>> > my
>>> >>> >>> > vm's
>>> >>> >>> > to switch some of them back on.
>>> >>> >>> >
>>> >>> >>> > I'm seeing the following error in the engine.log however I've
>>> >>> >>> > checked
>>> >>> >>> > my
>>> >>> >>> > certs on my hosts (as some of the goolge results said to
>>> check),
>>> >>> >>> > but
>>> >>> >>> > the
>>> >>> >>> > certs haven't expired...
>>> >>> >>> >
>>> >>> >>> >
>>> >>> >>> > 2017-09-21 15:09:45,077 ERROR
>>> >>> >>> >
>>> >>> >>> > [org.ovirt.engine.core.vdsbroker.vdsbroker.GetCapabilitiesVD
>>> SCommand]
>>> >>> >>> > (DefaultQuartzScheduler_Worker-4) Command
>>> >>> >>> > GetCapabilitiesVDSCommand(HostName
>>> >>> >>> > = node02.mydomain.za, HostId =
>>> >>> >>> > d2debdfe-76e7-40cf-a7fd-78a0f50f14d4,
>>> >>> >>> > vds=Host[node02.mydomain.za]) execution failed. Exception:
>>> >>> >>> > VDSNetworkException: javax.net.ssl.SSLHandshakeException:
>>> Received
>>> >>> >>> > fatal
>>> >>> >>> > alert: certificate_expired
>>> >>> >>> > 2017-09-21 15:09:45,086 ERROR
>>> >>> >>> >
>>> >>> >>> > [org.ovirt.engine.core.vdsbroker.vdsbroker.GetCapabilitiesVD
>>> SCommand]
>>> >>> >>> > (DefaultQuartzScheduler_Worker-10) Command
>>> >>> >>> > GetCapabilitiesVDSCommand(HostName = node01.mydomain.za,
>>> HostId =
>>> >>> >>> > b108549c-1700-11e2-b936-9f5243b8ce13, vds=Host[
>>> node01.mydomain.za])
>>> >>> >>> > execution failed. Exception: VDSNetworkException:
>>> >>> >>> > javax.net.ssl.SSLHandshakeException: Received fatal alert:
>>> >>> >>> > certificate_expired
>>> >>> >>> > 2017-09-21 15:09:48,173 ERROR
>>> >>> >>> >
>>> >>> >>> > My engine and host info is below...
>>> >>> >>> >
>>> >>> >>> > [root at engine01 ovirt-engine]# rpm -qa | grep -i ovirt
>>> >>> >>> > ovirt-engine-lib-3.4.0-1.el6.noarch
>>> >>> >>> > ovirt-engine-restapi-3.4.0-1.el6.noarch
>>> >>> >>> > ovirt-engine-setup-plugin-ovirt-engine-3.4.0-1.el6.noarch
>>> >>> >>> > ovirt-engine-3.4.0-1.el6.noarch
>>> >>> >>> > ovirt-engine-setup-plugin-websocket-proxy-3.4.0-1.el6.noarch
>>> >>> >>> > ovirt-host-deploy-java-1.2.0-1.el6.noarch
>>> >>> >>> > ovirt-engine-setup-3.4.0-1.el6.noarch
>>> >>> >>> > ovirt-host-deploy-1.2.0-1.el6.noarch
>>> >>> >>> > ovirt-engine-backend-3.4.0-1.el6.noarch
>>> >>> >>> > ovirt-image-uploader-3.4.0-1.el6.noarch
>>> >>> >>> > ovirt-engine-tools-3.4.0-1.el6.noarch
>>> >>> >>> > ovirt-engine-sdk-python-3.4.0.7-1.el6.noarch
>>> >>> >>> > ovirt-engine-webadmin-portal-3.4.0-1.el6.noarch
>>> >>> >>> > ovirt-engine-cli-3.4.0.5-1.el6.noarch
>>> >>> >>> > ovirt-engine-setup-base-3.4.0-1.el6.noarch
>>> >>> >>> > ovirt-iso-uploader-3.4.0-1.el6.noarch
>>> >>> >>> > ovirt-engine-userportal-3.4.0-1.el6.noarch
>>> >>> >>> > ovirt-log-collector-3.4.1-1.el6.noarch
>>> >>> >>> > ovirt-engine-websocket-proxy-3.4.0-1.el6.noarch
>>> >>> >>> > ovirt-engine-setup-plugin-ovirt-engine-common-3.4.0-1.el6.no
>>> arch
>>> >>> >>> > ovirt-engine-dbscripts-3.4.0-1.el6.noarch
>>> >>> >>> > [root at engine01 ovirt-engine]# cat /etc/redhat-release
>>> >>> >>> > CentOS release 6.5 (Final)
>>> >>> >>> >
>>> >>> >>> >
>>> >>> >>> > [root at node02 ~]# openssl x509 -in
>>> /etc/pki/vdsm/certs/vdsmcert.pem
>>> >>> >>> > -enddate
>>> >>> >>> > -noout ; date
>>> >>> >>> > notAfter=May 27 08:36:17 2019 GMT
>>> >>> >>> > Thu Sep 21 15:18:22 SAST 2017
>>> >>> >>> > CentOS release 6.5 (Final)
>>> >>> >>> > [root at node02 ~]# rpm -qa | grep vdsm
>>> >>> >>> > vdsm-4.14.6-0.el6.x86_64
>>> >>> >>> > vdsm-python-4.14.6-0.el6.x86_64
>>> >>> >>> > vdsm-cli-4.14.6-0.el6.noarch
>>> >>> >>> > vdsm-xmlrpc-4.14.6-0.el6.noarch
>>> >>> >>> > vdsm-python-zombiereaper-4.14.6-0.el6.noarch
>>> >>> >>> >
>>> >>> >>> >
>>> >>> >>> > [root at node01 ~]# openssl x509 -in
>>> /etc/pki/vdsm/certs/vdsmcert.pem
>>> >>> >>> > -enddate
>>> >>> >>> > -noout ; date
>>> >>> >>> > notAfter=Jun 13 16:09:41 2018 GMT
>>> >>> >>> > Thu Sep 21 15:18:52 SAST 2017
>>> >>> >>> > CentOS release 6.5 (Final)
>>> >>> >>> > [root at node01 ~]# rpm -qa | grep -i vdsm
>>> >>> >>> > vdsm-4.14.6-0.el6.x86_64
>>> >>> >>> > vdsm-xmlrpc-4.14.6-0.el6.noarch
>>> >>> >>> > vdsm-cli-4.14.6-0.el6.noarch
>>> >>> >>> > vdsm-python-zombiereaper-4.14.6-0.el6.noarch
>>> >>> >>> > vdsm-python-4.14.6-0.el6.x86_64
>>> >>> >>> >
>>> >>> >>> > Please could I have some assistance, I'm rater desperate.
>>> >>> >>> >
>>> >>> >>> > Thank you.
>>> >>> >>> >
>>> >>> >>> > Regards.
>>> >>> >>> >
>>> >>> >>> > Neil Wilson
>>> >>> >>> >
>>> >>> >>> >
>>> >>> >>> >
>>> >>> >>> > _______________________________________________
>>> >>> >>> > Users mailing list
>>> >>> >>> > Users at ovirt.org
>>> >>> >>> > http://lists.ovirt.org/mailman/listinfo/users
>>> >>> >>> >
>>> >>> >>
>>> >>> >>
>>> >>> >
>>> >>
>>> >>
>>> >>
>>> >> _______________________________________________
>>> >> Users mailing list
>>> >> Users at ovirt.org
>>> >> http://lists.ovirt.org/mailman/listinfo/users
>>> >>
>>> >
>>>
>>
>>
>
-------------- next part --------------
An HTML attachment was scrubbed...
URL: <http://lists.ovirt.org/pipermail/users/attachments/20170922/419d17bf/attachment.html>
More information about the Users
mailing list