<div dir="ltr"><br><div class="gmail_extra"><br><div class="gmail_quote">On Mon, Jul 25, 2016 at 11:40 AM, Piotr Kliczewski <span dir="ltr"><<a href="mailto:piotr.kliczewski@gmail.com" target="_blank">piotr.kliczewski@gmail.com</a>></span> wrote:<br><blockquote class="gmail_quote" style="margin:0 0 0 .8ex;border-left:1px #ccc solid;padding-left:1ex">I remember an issue that engine upgrade corrupted certificates and<br>
"General SSLEngine problem" may be indication that you saw it.<br>
I asked to open BZ for it but was unable to find it.<br>
<br>
@Sandro @Simone was it fixed already?<br></blockquote><div><br></div><div>I've vague memories of something related being fixed, but without a bug number I can't tell for sure.</div><div>Adding also Didi, since ssl / pki is his area and he may be aware.</div><div><br></div><div><br></div><div> </div><blockquote class="gmail_quote" style="margin:0 0 0 .8ex;border-left:1px #ccc solid;padding-left:1ex">
<div class="HOEnZb"><div class="h5"><br>
On Thu, Jul 21, 2016 at 3:18 PM, Martin Perina <<a href="mailto:mperina@redhat.com">mperina@redhat.com</a>> wrote:<br>
> Thanks a lot for you effort, I'm glad that you were able to upgrade<br>
> successfully although we were not able to find the cause for the issue :-(<br>
><br>
> On Thu, Jul 21, 2016 at 2:30 PM, <<a href="mailto:nicolas@devels.es">nicolas@devels.es</a>> wrote:<br>
>><br>
>> So I gave it another try and this time it worked without any issue (with<br>
>> 4.0.1.1 version). Strange, maybe the first upgrade failure left system in a<br>
>> weird state? Anyhow almost everything ([1]) is working fine now. Thanks for<br>
>> the help!<br>
>><br>
>> [1]: <a href="https://bugzilla.redhat.com/show_bug.cgi?id=1358737" rel="noreferrer" target="_blank">https://bugzilla.redhat.com/show_bug.cgi?id=1358737</a><br>
><br>
><br>
> Adding Tomas about this one<br>
><br>
>><br>
>><br>
>> El 2016-07-20 20:23, Martin Perina escribió:<br>
>>><br>
>>> On Wed, Jul 20, 2016 at 6:18 PM, Nicolás <<a href="mailto:nicolas@devels.es">nicolas@devels.es</a>> wrote:<br>
>>><br>
>>>> El 20/07/16 a las 16:45, Martin Perina escribió:<br>
>>>><br>
>>>> On Wed, Jul 20, 2016 at 4:44 PM, Nicolás <<a href="mailto:nicolas@devels.es">nicolas@devels.es</a>> wrote:<br>
>>>><br>
>>>> Hi Martin,<br>
>>>><br>
>>>> Actually, up until now we had that cert configured in httpd and in<br>
>>>> websocket proxy. Seems that now in 4.0.x it's not enough, as opening<br>
>>>> the <a href="https://fqdn" rel="noreferrer" target="_blank">https://fqdn</a> [1] complains about the cert not being imported in<br>
>>>> the key chain.<br>
>>>><br>
>>>> Yes, there's an updated procedure on using external CA in 4.0,<br>
>>>> for details please take a look at Doc Text in<br>
>>>><br>
>>>> <a href="https://bugzilla.redhat.com/show_bug.cgi?id=1336838" rel="noreferrer" target="_blank">https://bugzilla.redhat.com/show_bug.cgi?id=1336838</a> [2]<br>
>>>><br>
>>>> So I imported it via keytool, but I don't want to use it in the<br>
>>>> engine <-> VDSM communication.<br>
>>>><br>
>>>> Hmm, so that would imply that we have some issue with existing<br>
>>>> internal enigne CA during upgrade ...<br>
>>>><br>
>>>> The strange thing is that we test upgrades a lot but so far we<br>
>>>> haven't seen any issues which will broke<br>
>>>><br>
>>>> SSL setup between engine and VDSM. You said that you had to<br>
>>>> downgrade back to 3.6.7 (so unfortunately for us we cannot<br>
>>>> investigate your nonworking setup more), but how did you do that?<br>
>>>><br>
>>>> Removing all engine packages and configuration, installing back<br>
>>>> 3.6.7 packaging and restoring configuration form backup?<br>
>>>><br>
>>>> I'm asking to know what changed in your setup between not working<br>
>>>> 4.0 and working 3.6.7 ...<br>
>>><br>
>>><br>
>>> Indeed, those are the steps I followed to the point.<br>
>>><br>
>>> To add more strangeness, previously to upgrading this oVirt<br>
>>> infrastructure, we upgraded another one that we have (also using own<br>
>>> cert, a different one but from the same CA) and everything went<br>
>>> smoothly. And what's more, previously to upgrading the engine that<br>
>>> failed, I created a copy of that engine machine in a sandbox<br>
>>> environment to see if upgrade process would or not success, and it<br>
>>> worked perfectly.<br>
>>><br>
>>> The only difference between the sandbox and the real machine's<br>
>>> process was that when upgrading the real one, the first time I run<br>
>>> "engine-setup" it failed because 'systemd' reported PostgreSQL as it<br>
>>> was not running (actually it was, thougg), so everything rolled back.<br>
>>> I had to kill the PostgreSQL process, start it again with systemctl<br>
>>> and then run "engine-setup", where the process completed successfully<br>
>>> but the SSL issue appeared. Not sure if this rollback could have<br>
>>> shattered the whole thing...<br>
>>><br>
>>> Anyhow, tomorrow I'm going to create another copy of the engine<br>
>>> machine to a sandbox environment and try again. If it works I'll cross<br>
>>> my fingers and give another try on the real machine...<br>
>>><br>
>>> Thanks!<br>
>>><br>
>>> Thanks a lot for you effort. I will try to perform same upgrade<br>
>>> tomorrow in my test env.<br>
>>><br>
>>><br>
>>>> Thanks<br>
>>>><br>
>>>> Martin<br>
>>>><br>
>>>> Thanks!<br>
>>>> En 20/7/2016 2:48 p. m., Martin Perina <<a href="mailto:mperina@redhat.com">mperina@redhat.com</a>><br>
>>>> escribió:<br>
>>>><br>
>>>> Hi,<br>
>>>><br>
>>>> sorry for late response, I overlook your reply :-(<br>
>>>><br>
>>>> I looked at your logs and it seems to me that there's SSL<br>
>>>> error when engine tries to contact VDSM.<br>
>>>><br>
>>>> You have mentioned that your are using your own custom CA. Are<br>
>>>> you using it only for HTTPS certificate or do you want to use it<br>
>>>> also for Engine <-> VDSM communication?<br>
>>>><br>
>>>><br>
>>>> Martin Perina<br>
>>>><br>
>>>><br>
>>>><br>
>>>> On Wed, Jul 20, 2016 at 9:18 AM, <<a href="mailto:nicolas@devels.es">nicolas@devels.es</a>> wrote:<br>
>>>> Any hints about this?<br>
>>>><br>
>>>> El 2016-07-13 11:13, <a href="mailto:nicolas@devels.es">nicolas@devels.es</a> escribió:<br>
>>>> Hi,<br>
>>>><br>
>>>> Unfortunately, upgrading to 4.0.1RC didn't solve the problem.<br>
>>>> Actually, the error changed to 'General SSLEngine problem', but the<br>
>>>> result was the same, like this:<br>
>>>><br>
>>>> 2016-07-13 09:52:22,010 INFO<br>
>>>> [org.ovirt.vdsm.jsonrpc.client.reactors.ReactorClient] (SSL Stomp<br>
>>>> Reactor) [] Connecting to /10.X.X.X<br>
>>>> 2016-07-13 09:52:22,018 ERROR<br>
>>>> [org.ovirt.vdsm.jsonrpc.client.reactors.Reactor] (SSL Stomp<br>
>>>> Reactor)<br>
>>>> [] Unable to process messages: General SSLEngine problem<br>
>>>><br>
>>>> It's worth mentioning that we're using our own SSL certificates<br>
>>>> (not<br>
>>>> self-signed), and I imported the combined certificate into the<br>
>>>> /etc/pki/ovirt-engine/.truststore key file. Not sure if related,<br>
>>>> but<br>
>>>> just in case.<br>
>>><br>
>>><br>
>>>> I had to downgrade to 3.6.7. I'm attaching requested logs, if you<br>
>>>> need<br>
>>>> anything else don't hesitate to ask.<br>
>>>><br>
>>>> Regards.<br>
>>>><br>
>>>> El 2016-07-13 09:45, Martin Perina escribió:<br>
>>>> Hi,<br>
>>>><br>
>>>> could you please share also vdsm.log from your hosts and also<br>
>>>> server.log and setup logs from /var/log/ovirt-engine/setup<br>
>>>> directory?<br>
>>>><br>
>>>> Thanks<br>
>>>><br>
>>>> Martin Perina<br>
>>>><br>
>>>> On Wed, Jul 13, 2016 at 10:36 AM, <<a href="mailto:nicolas@devels.es">nicolas@devels.es</a>> wrote:<br>
>>>><br>
>>>> Hi,<br>
>>>><br>
>>>> We upgraded from 3.6.6 to 4.0.0 and we have a big issue since the<br>
>>>> engine cannot connect to hosts. In the logs all we see is this<br>
>>>> error:<br>
>>>><br>
>>>> ERROR [org.ovirt.vdsm.jsonrpc.client.reactors.Reactor] (SSL<br>
>>>> Stomp Reactor) [] Unable to process messages<br>
>>>><br>
>>>> I'm attaching full logs.<br>
>>>><br>
>>>> Could someone help please?<br>
>>>><br>
>>>> Thanks.<br>
>>>> _______________________________________________<br>
>>>> Users mailing list<br>
>>>> <a href="mailto:Users@ovirt.org">Users@ovirt.org</a><br>
>>>> <a href="http://lists.ovirt.org/mailman/listinfo/users" rel="noreferrer" target="_blank">http://lists.ovirt.org/mailman/listinfo/users</a> [3] [1]<br>
>>>><br>
>>>> Links:<br>
>>>> ------<br>
>>>> [1] <a href="http://lists.ovirt.org/mailman/listinfo/users" rel="noreferrer" target="_blank">http://lists.ovirt.org/mailman/listinfo/users</a> [3]<br>
>>><br>
>>><br>
>>> _______________________________________________<br>
>>> Users mailing list<br>
>>> <a href="mailto:Users@ovirt.org">Users@ovirt.org</a><br>
>>> <a href="http://lists.ovirt.org/mailman/listinfo/users" rel="noreferrer" target="_blank">http://lists.ovirt.org/mailman/listinfo/users</a> [3]<br>
>>><br>
>>><br>
>>><br>
>>> Links:<br>
>>> ------<br>
>>> [1] <a href="https://fqdn" rel="noreferrer" target="_blank">https://fqdn</a><br>
>>> [2] <a href="https://bugzilla.redhat.com/show_bug.cgi?id=1336838" rel="noreferrer" target="_blank">https://bugzilla.redhat.com/show_bug.cgi?id=1336838</a><br>
>>> [3] <a href="http://lists.ovirt.org/mailman/listinfo/users" rel="noreferrer" target="_blank">http://lists.ovirt.org/mailman/listinfo/users</a><br>
><br>
><br>
><br>
> _______________________________________________<br>
> Users mailing list<br>
> <a href="mailto:Users@ovirt.org">Users@ovirt.org</a><br>
> <a href="http://lists.ovirt.org/mailman/listinfo/users" rel="noreferrer" target="_blank">http://lists.ovirt.org/mailman/listinfo/users</a><br>
><br>
</div></div></blockquote></div><br><br clear="all"><div><br></div>-- <br><div class="gmail_signature" data-smartmail="gmail_signature"><div dir="ltr"><div><div dir="ltr">Sandro Bonazzola<br>Better technology. Faster innovation. Powered by community collaboration.<br>See how it works at <a href="http://redhat.com" target="_blank">redhat.com</a><br></div></div></div></div>
</div></div>