[ovirt-users] ERROR [org.ovirt.vdsm.jsonrpc.client.reactors.Reactor] (SSL Stomp Reactor) [] Unable to process messages

Thu Jul 21 13:18:49 UTC 2016

Thanks a lot for you effort, I'm glad that you were able to upgrade
successfully although we were not able to find the cause for the issue :-(

On Thu, Jul 21, 2016 at 2:30 PM, <nicolas at devels.es> wrote:

> So I gave it another try and this time it worked without any issue (with
> 4.0.1.1 version). Strange, maybe the first upgrade failure left system in a
> weird state? Anyhow almost everything ([1]) is working fine now. Thanks for
> the help!
>
>   [1]: https://bugzilla.redhat.com/show_bug.cgi?id=1358737
>

Adding Tomas about this one

>
> El 2016-07-20 20:23, Martin Perina escribió:
>
>> On Wed, Jul 20, 2016 at 6:18 PM, Nicolás <nicolas at devels.es> wrote:
>>
>> El 20/07/16 a las 16:45, Martin Perina escribió:
>>>
>>> On Wed, Jul 20, 2016 at 4:44 PM, Nicolás <nicolas at devels.es> wrote:
>>>
>>> Hi Martin,
>>>
>>> Actually, up until now we had that cert configured in httpd and in
>>> websocket proxy. Seems that now in 4.0.x it's not enough, as opening
>>> the https://fqdn [1] complains about the cert not being imported in
>>> the key chain.
>>>
>>> Yes, there's an updated procedure on using external CA in 4.0,
>>> for details please take a look at Doc Text in
>>>
>>> https://bugzilla.redhat.com/show_bug.cgi?id=1336838 [2]
>>> 
>>> So I imported it via keytool, but I don't want to use it in the
>>> engine <-> VDSM communication.
>>>
>>> Hmm, so that would imply that we have some issue with existing
>>> internal enigne CA during upgrade ...
>>>
>>> The strange thing is that we test upgrades a lot but so far we
>>> haven't seen any issues which will broke
>>>
>>> SSL setup between engine and VDSM. You said that you had to
>>> downgrade back to 3.6.7 (so unfortunately for us we cannot
>>> investigate your nonworking setup more), but how did you do that?
>>>
>>> Removing all engine packages and configuration, installing back
>>> 3.6.7 packaging and restoring configuration form backup?
>>>
>>> I'm asking to know what changed in your setup between not working
>>> 4.0 and working 3.6.7 ...
>>>
>>
>>  Indeed, those are the steps I followed to the point.
>>
>>  To add more strangeness, previously to upgrading this oVirt
>> infrastructure, we upgraded another one that we have (also using own
>> cert, a different one but from the same CA) and everything went
>> smoothly. And what's more, previously to upgrading the engine that
>> failed, I created a copy of that engine machine in a sandbox
>> environment to see if upgrade process would or not success, and it
>> worked perfectly.
>>
>>  The only difference between the sandbox and the real machine's
>> process was that when upgrading the real one, the first time I run
>> "engine-setup" it failed because 'systemd' reported PostgreSQL as it
>> was not running (actually it was, thougg), so everything rolled back.
>> I had to kill the PostgreSQL process, start it again with systemctl
>> and then run "engine-setup", where the process completed successfully
>> but the SSL issue appeared. Not sure if this rollback could have
>> shattered the whole thing...
>>
>>  Anyhow, tomorrow I'm going to create another copy of the engine
>> machine to a sandbox environment and try again. If it works I'll cross
>> my fingers and give another try on the real machine...
>>
>>  Thanks!
>>
>> Thanks a lot for you effort. I will try to perform same upgrade
>> tomorrow in my test env.
>> 
>>
>> Thanks
>>>
>>> Martin
>>>
>>> Thanks!
>>> En 20/7/2016 2:48 p. m., Martin Perina <mperina at redhat.com>
>>> escribió:
>>>
>>> Hi,
>>>
>>> sorry for late response, I overlook your reply :-(
>>>
>>> I looked at your logs and it seems to me that there's SSL
>>> error when engine tries to contact VDSM.
>>>
>>> You have mentioned that your are using your own custom CA. Are
>>> you using it only for HTTPS certificate or do you want to use it
>>> also for Engine <-> VDSM communication?
>>> 
>>>
>>> Martin Perina
>>>
>>>
>>>
>>> On Wed, Jul 20, 2016 at 9:18 AM, <nicolas at devels.es> wrote:
>>> Any hints about this?
>>>
>>> El 2016-07-13 11:13, nicolas at devels.es escribió:
>>> Hi,
>>>
>>> Unfortunately, upgrading to 4.0.1RC didn't solve the problem.
>>> Actually, the error changed to 'General SSLEngine problem', but the
>>> result was the same, like this:
>>>
>>> 2016-07-13 09:52:22,010 INFO
>>> [org.ovirt.vdsm.jsonrpc.client.reactors.ReactorClient] (SSL Stomp
>>> Reactor) [] Connecting to /10.X.X.X
>>> 2016-07-13 09:52:22,018 ERROR
>>> [org.ovirt.vdsm.jsonrpc.client.reactors.Reactor] (SSL Stomp
>>> Reactor)
>>> [] Unable to process messages: General SSLEngine problem
>>>
>>> It's worth mentioning that we're using our own SSL certificates
>>> (not
>>> self-signed), and I imported the combined certificate into the
>>> /etc/pki/ovirt-engine/.truststore key file. Not sure if related,
>>> but
>>> just in case.
>>>
>>
>> I had to downgrade to 3.6.7. I'm attaching requested logs, if you
>>> need
>>> anything else don't hesitate to ask.
>>>
>>> Regards.
>>>
>>> El 2016-07-13 09:45, Martin Perina escribió:
>>> Hi,
>>>
>>> could you please share also vdsm.log from your hosts and also
>>> server.log and setup logs from /var/log/ovirt-engine/setup
>>> directory?
>>>
>>> Thanks
>>>
>>> Martin Perina
>>>
>>> On Wed, Jul 13, 2016 at 10:36 AM, <nicolas at devels.es> wrote:
>>>
>>> Hi,
>>>
>>> We upgraded from 3.6.6 to 4.0.0 and we have a big issue since the
>>> engine cannot connect to hosts. In the logs all we see is this
>>> error:
>>>
>>>     ERROR [org.ovirt.vdsm.jsonrpc.client.reactors.Reactor] (SSL
>>> Stomp Reactor) [] Unable to process messages
>>>
>>> I'm attaching full logs.
>>>
>>> Could someone help please?
>>>
>>> Thanks.
>>> _______________________________________________
>>> Users mailing list
>>> Users at ovirt.org
>>> http://lists.ovirt.org/mailman/listinfo/users [3] [1]
>>>
>>> Links:
>>> ------
>>> [1] http://lists.ovirt.org/mailman/listinfo/users [3]
>>>
>>
>>  _______________________________________________
>>  Users mailing list
>>  Users at ovirt.org
>>  http://lists.ovirt.org/mailman/listinfo/users [3]
>>
>>
>>
>> Links:
>> ------
>> [1] https://fqdn
>> [2] https://bugzilla.redhat.com/show_bug.cgi?id=1336838
>> [3] http://lists.ovirt.org/mailman/listinfo/users
>>
>
-------------- next part --------------
An HTML attachment was scrubbed...
URL: <http://lists.ovirt.org/pipermail/users/attachments/20160721/8b43394d/attachment-0001.html>