[ovirt-users] ERROR [org.ovirt.vdsm.jsonrpc.client.reactors.Reactor] (SSL Stomp Reactor) [] Unable to process messages

Mon Jul 25 09:40:00 UTC 2016

I remember an issue that engine upgrade corrupted certificates and
"General SSLEngine problem" may be indication that you saw it.
I asked to open BZ for it but was unable to find it.

@Sandro @Simone was it fixed already?

On Thu, Jul 21, 2016 at 3:18 PM, Martin Perina <mperina at redhat.com> wrote:
> Thanks a lot for you effort, I'm glad that you were able to upgrade
> successfully although we were not able to find the cause for the issue :-(
>
> On Thu, Jul 21, 2016 at 2:30 PM, <nicolas at devels.es> wrote:
>>
>> So I gave it another try and this time it worked without any issue (with
>> 4.0.1.1 version). Strange, maybe the first upgrade failure left system in a
>> weird state? Anyhow almost everything ([1]) is working fine now. Thanks for
>> the help!
>>
>>   [1]: https://bugzilla.redhat.com/show_bug.cgi?id=1358737
>
>
> Adding Tomas about this one
>
>>
>>
>> El 2016-07-20 20:23, Martin Perina escribió:
>>>
>>> On Wed, Jul 20, 2016 at 6:18 PM, Nicolás <nicolas at devels.es> wrote:
>>>
>>>> El 20/07/16 a las 16:45, Martin Perina escribió:
>>>>
>>>> On Wed, Jul 20, 2016 at 4:44 PM, Nicolás <nicolas at devels.es> wrote:
>>>>
>>>> Hi Martin,
>>>>
>>>> Actually, up until now we had that cert configured in httpd and in
>>>> websocket proxy. Seems that now in 4.0.x it's not enough, as opening
>>>> the https://fqdn [1] complains about the cert not being imported in
>>>> the key chain.
>>>>
>>>> Yes, there's an updated procedure on using external CA in 4.0,
>>>> for details please take a look at Doc Text in
>>>>
>>>> https://bugzilla.redhat.com/show_bug.cgi?id=1336838 [2]
>>>>
>>>> So I imported it via keytool, but I don't want to use it in the
>>>> engine <-> VDSM communication.
>>>>
>>>> Hmm, so that would imply that we have some issue with existing
>>>> internal enigne CA during upgrade ...
>>>>
>>>> The strange thing is that we test upgrades a lot but so far we
>>>> haven't seen any issues which will broke
>>>>
>>>> SSL setup between engine and VDSM. You said that you had to
>>>> downgrade back to 3.6.7 (so unfortunately for us we cannot
>>>> investigate your nonworking setup more), but how did you do that?
>>>>
>>>> Removing all engine packages and configuration, installing back
>>>> 3.6.7 packaging and restoring configuration form backup?
>>>>
>>>> I'm asking to know what changed in your setup between not working
>>>> 4.0 and working 3.6.7 ...
>>>
>>>
>>>  Indeed, those are the steps I followed to the point.
>>>
>>>  To add more strangeness, previously to upgrading this oVirt
>>> infrastructure, we upgraded another one that we have (also using own
>>> cert, a different one but from the same CA) and everything went
>>> smoothly. And what's more, previously to upgrading the engine that
>>> failed, I created a copy of that engine machine in a sandbox
>>> environment to see if upgrade process would or not success, and it
>>> worked perfectly.
>>>
>>>  The only difference between the sandbox and the real machine's
>>> process was that when upgrading the real one, the first time I run
>>> "engine-setup" it failed because 'systemd' reported PostgreSQL as it
>>> was not running (actually it was, thougg), so everything rolled back.
>>> I had to kill the PostgreSQL process, start it again with systemctl
>>> and then run "engine-setup", where the process completed successfully
>>> but the SSL issue appeared. Not sure if this rollback could have
>>> shattered the whole thing...
>>>
>>>  Anyhow, tomorrow I'm going to create another copy of the engine
>>> machine to a sandbox environment and try again. If it works I'll cross
>>> my fingers and give another try on the real machine...
>>>
>>>  Thanks!
>>>
>>> Thanks a lot for you effort. I will try to perform same upgrade
>>> tomorrow in my test env.
>>>
>>>
>>>> Thanks
>>>>
>>>> Martin
>>>>
>>>> Thanks!
>>>> En 20/7/2016 2:48 p. m., Martin Perina <mperina at redhat.com>
>>>> escribió:
>>>>
>>>> Hi,
>>>>
>>>> sorry for late response, I overlook your reply :-(
>>>>
>>>> I looked at your logs and it seems to me that there's SSL
>>>> error when engine tries to contact VDSM.
>>>>
>>>> You have mentioned that your are using your own custom CA. Are
>>>> you using it only for HTTPS certificate or do you want to use it
>>>> also for Engine <-> VDSM communication?
>>>>
>>>>
>>>> Martin Perina
>>>>
>>>>
>>>>
>>>> On Wed, Jul 20, 2016 at 9:18 AM, <nicolas at devels.es> wrote:
>>>> Any hints about this?
>>>>
>>>> El 2016-07-13 11:13, nicolas at devels.es escribió:
>>>> Hi,
>>>>
>>>> Unfortunately, upgrading to 4.0.1RC didn't solve the problem.
>>>> Actually, the error changed to 'General SSLEngine problem', but the
>>>> result was the same, like this:
>>>>
>>>> 2016-07-13 09:52:22,010 INFO
>>>> [org.ovirt.vdsm.jsonrpc.client.reactors.ReactorClient] (SSL Stomp
>>>> Reactor) [] Connecting to /10.X.X.X
>>>> 2016-07-13 09:52:22,018 ERROR
>>>> [org.ovirt.vdsm.jsonrpc.client.reactors.Reactor] (SSL Stomp
>>>> Reactor)
>>>> [] Unable to process messages: General SSLEngine problem
>>>>
>>>> It's worth mentioning that we're using our own SSL certificates
>>>> (not
>>>> self-signed), and I imported the combined certificate into the
>>>> /etc/pki/ovirt-engine/.truststore key file. Not sure if related,
>>>> but
>>>> just in case.
>>>
>>>
>>>> I had to downgrade to 3.6.7. I'm attaching requested logs, if you
>>>> need
>>>> anything else don't hesitate to ask.
>>>>
>>>> Regards.
>>>>
>>>> El 2016-07-13 09:45, Martin Perina escribió:
>>>> Hi,
>>>>
>>>> could you please share also vdsm.log from your hosts and also
>>>> server.log and setup logs from /var/log/ovirt-engine/setup
>>>> directory?
>>>>
>>>> Thanks
>>>>
>>>> Martin Perina
>>>>
>>>> On Wed, Jul 13, 2016 at 10:36 AM, <nicolas at devels.es> wrote:
>>>>
>>>> Hi,
>>>>
>>>> We upgraded from 3.6.6 to 4.0.0 and we have a big issue since the
>>>> engine cannot connect to hosts. In the logs all we see is this
>>>> error:
>>>>
>>>>     ERROR [org.ovirt.vdsm.jsonrpc.client.reactors.Reactor] (SSL
>>>> Stomp Reactor) [] Unable to process messages
>>>>
>>>> I'm attaching full logs.
>>>>
>>>> Could someone help please?
>>>>
>>>> Thanks.
>>>> _______________________________________________
>>>> Users mailing list
>>>> Users at ovirt.org
>>>> http://lists.ovirt.org/mailman/listinfo/users [3] [1]
>>>>
>>>> Links:
>>>> ------
>>>> [1] http://lists.ovirt.org/mailman/listinfo/users [3]
>>>
>>>
>>>  _______________________________________________
>>>  Users mailing list
>>>  Users at ovirt.org
>>>  http://lists.ovirt.org/mailman/listinfo/users [3]
>>>
>>>
>>>
>>> Links:
>>> ------
>>> [1] https://fqdn
>>> [2] https://bugzilla.redhat.com/show_bug.cgi?id=1336838
>>> [3] http://lists.ovirt.org/mailman/listinfo/users
>
>
>
> _______________________________________________
> Users mailing list
> Users at ovirt.org
> http://lists.ovirt.org/mailman/listinfo/users
>