This is a multi-part message in MIME format. --------------4A7D47D278593ED9C71C5805 Content-Type: text/plain; charset=utf-8; format=flowed Content-Transfer-Encoding: 8bit El 20/07/16 a las 16:45, Martin Perina escribió:
On Wed, Jul 20, 2016 at 4:44 PM, Nicolás <nicolas@devels.es <mailto:nicolas@devels.es>> wrote:
Hi Martin,
Actually, up until now we had that cert configured in httpd and in websocket proxy. Seems that now in 4.0.x it's not enough, as opening the https://fqdn complains about the cert not being imported in the key chain.
Yes, there's an updated procedure on using external CA in 4.0, for details please take a look at Doc Text in
https://bugzilla.redhat.com/show_bug.cgi?id=1336838
So I imported it via keytool, but I don't want to use it in the engine <-> VDSM communication.
Hmm, so that would imply that we have some issue with existing internal enigne CA during upgrade ... The strange thing is that we test upgrades a lot but so far we haven't seen any issues which will broke SSL setup between engine and VDSM. You said that you had to downgrade back to 3.6.7 (so unfortunately for us we cannot investigate your nonworking setup more), but how did you do that? Removing all engine packages and configuration, installing back 3.6.7 packaging and restoring configuration form backup? I'm asking to know what changed in your setup between not working 4.0 and working 3.6.7 ...
Indeed, those are the steps I followed to the point. To add more strangeness, previously to upgrading this oVirt infrastructure, we upgraded another one that we have (also using own cert, a different one but from the same CA) and everything went smoothly. And what's more, previously to upgrading the engine that failed, I created a copy of that engine machine in a sandbox environment to see if upgrade process would or not success, and it worked perfectly. The only difference between the sandbox and the real machine's process was that when upgrading the real one, the first time I run "engine-setup" it failed because 'systemd' reported PostgreSQL as it was not running (actually it was, thougg), so everything rolled back. I had to kill the PostgreSQL process, start it again with systemctl and then run "engine-setup", where the process completed successfully but the SSL issue appeared. Not sure if this rollback could have shattered the whole thing... Anyhow, tomorrow I'm going to create another copy of the engine machine to a sandbox environment and try again. If it works I'll cross my fingers and give another try on the real machine... Thanks!
Thanks
Martin
Thanks! En 20/7/2016 2:48 p. m., Martin Perina <mperina@redhat.com <mailto:mperina@redhat.com>> escribió:
Hi,
sorry for late response, I overlook your reply :-(
I looked at your logs and it seems to me that there's SSL error when engine tries to contact VDSM. You have mentioned that your are using your own custom CA. Are you using it only for HTTPS certificate or do you want to use it also for Engine <-> VDSM communication? Martin Perina
On Wed, Jul 20, 2016 at 9:18 AM, <nicolas@devels.es <mailto:nicolas@devels.es>> wrote:
Any hints about this?
El 2016-07-13 11:13, nicolas@devels.es <mailto:nicolas@devels.es> escribió:
Hi,
Unfortunately, upgrading to 4.0.1RC didn't solve the problem. Actually, the error changed to 'General SSLEngine problem', but the result was the same, like this:
2016-07-13 09:52:22,010 INFO [org.ovirt.vdsm.jsonrpc.client.reactors.ReactorClient] (SSL Stomp Reactor) [] Connecting to /10.X.X.X 2016-07-13 09:52:22,018 ERROR [org.ovirt.vdsm.jsonrpc.client.reactors.Reactor] (SSL Stomp Reactor) [] Unable to process messages: General SSLEngine problem
It's worth mentioning that we're using our own SSL certificates (not self-signed), and I imported the combined certificate into the /etc/pki/ovirt-engine/.truststore key file. Not sure if related, but just in case.
I had to downgrade to 3.6.7. I'm attaching requested logs, if you need anything else don't hesitate to ask.
Regards.
El 2016-07-13 09:45, Martin Perina escribió:
Hi,
could you please share also vdsm.log from your hosts and also server.log and setup logs from /var/log/ovirt-engine/setup directory?
Thanks
Martin Perina
On Wed, Jul 13, 2016 at 10:36 AM, <nicolas@devels.es <mailto:nicolas@devels.es>> wrote:
Hi,
We upgraded from 3.6.6 to 4.0.0 and we have a big issue since the engine cannot connect to hosts. In the logs all we see is this error:
ERROR [org.ovirt.vdsm.jsonrpc.client.reactors.Reactor] (SSL Stomp Reactor) [] Unable to process messages
I'm attaching full logs.
Could someone help please?
Thanks. _______________________________________________ Users mailing list Users@ovirt.org <mailto:Users@ovirt.org> http://lists.ovirt.org/mailman/listinfo/users [1]
Links: ------ [1] http://lists.ovirt.org/mailman/listinfo/users
_______________________________________________ Users mailing list Users@ovirt.org <mailto:Users@ovirt.org> http://lists.ovirt.org/mailman/listinfo/users
--------------4A7D47D278593ED9C71C5805 Content-Type: text/html; charset=utf-8 Content-Transfer-Encoding: 8bit <html> <head> <meta content="text/html; charset=utf-8" http-equiv="Content-Type"> </head> <body bgcolor="#FFFFFF" text="#000000"> <p><br> </p> <br> <div class="moz-cite-prefix">El 20/07/16 a las 16:45, Martin Perina escribió:<br> </div> <blockquote cite="mid:CAP5iht6TsJm=Wc+=gGNttr4sNmVeH-QU+uBmY+uz4GLdipeXdw@mail.gmail.com" type="cite"> <div dir="ltr"> <div class="gmail_default" style="font-family:arial,helvetica,sans-serif"><br> </div> <div class="gmail_extra"><br> <div class="gmail_quote">On Wed, Jul 20, 2016 at 4:44 PM, Nicolás <span dir="ltr"><<a moz-do-not-send="true" href="mailto:nicolas@devels.es" target="_blank">nicolas@devels.es</a>></span> wrote:<br> <blockquote class="gmail_quote" style="margin:0px 0px 0px 0.8ex;border-left:1px solid rgb(204,204,204);padding-left:1ex"> <div dir="ltr">Hi Martin,<br> </div> <div dir="ltr"><br> </div> <div dir="ltr">Actually, up until now we had that cert configured in httpd and in websocket proxy. Seems that now in 4.0.x it's not enough, as opening the <a moz-do-not-send="true" href="https://fqdn" target="_blank">https://fqdn</a> complains about the cert not being imported in the key chain. </div> </blockquote> <div><br> <div class="gmail_default" style="font-family:arial,helvetica,sans-serif;display:inline">Yes, there's an updated procedure on using external CA in 4.0, for details please take a look at Doc Text in<br> <br> <a moz-do-not-send="true" href="https://bugzilla.redhat.com/show_bug.cgi?id=1336838">https://bugzilla.redhat.com/show_bug.cgi?id=1336838</a><br> </div> </div> <blockquote class="gmail_quote" style="margin:0px 0px 0px 0.8ex;border-left:1px solid rgb(204,204,204);padding-left:1ex"> <div dir="ltr">So I imported it via keytool, but I don't want to use it in the engine <-> VDSM communication.<br> </div> </blockquote> <div><br> <div class="gmail_default" style="font-family:arial,helvetica,sans-serif;display:inline">Hmm, so that would imply that we have some issue with existing internal enigne CA during upgrade ...<br> </div> <div class="gmail_default" style="font-family:arial,helvetica,sans-serif;display:inline">The strange thing is that we test upgrades a lot but so far we haven't seen any issues which will broke<br> </div> <div class="gmail_default" style="font-family:arial,helvetica,sans-serif;display:inline">SSL setup between engine and VDSM. You said that you had to downgrade back to 3.6.7 (so unfortunately for us we cannot investigate your nonworking setup more), but how did you do that?<br> </div> <div class="gmail_default" style="font-family:arial,helvetica,sans-serif;display:inline">Removing all engine packages and configuration, installing back 3.6.7 packaging and restoring configuration form backup?<br> </div> <div class="gmail_default" style="font-family:arial,helvetica,sans-serif;display:inline">I'm asking to know what changed in your setup between not working 4.0 and working 3.6.7 ...<br> <br> </div> </div> </div> </div> </div> </blockquote> <br> Indeed, those are the steps I followed to the point.<br> <br> To add more strangeness, previously to upgrading this oVirt infrastructure, we upgraded another one that we have (also using own cert, a different one but from the same CA) and everything went smoothly. And what's more, previously to upgrading the engine that failed, I created a copy of that engine machine in a sandbox environment to see if upgrade process would or not success, and it worked perfectly.<br> <br> The only difference between the sandbox and the real machine's process was that when upgrading the real one, the first time I run "engine-setup" it failed because 'systemd' reported PostgreSQL as it was not running (actually it was, thougg), so everything rolled back. I had to kill the PostgreSQL process, start it again with systemctl and then run "engine-setup", where the process completed successfully but the SSL issue appeared. Not sure if this rollback could have shattered the whole thing...<br> <br> Anyhow, tomorrow I'm going to create another copy of the engine machine to a sandbox environment and try again. If it works I'll cross my fingers and give another try on the real machine...<br> <br> Thanks!<br> <br> <blockquote cite="mid:CAP5iht6TsJm=Wc+=gGNttr4sNmVeH-QU+uBmY+uz4GLdipeXdw@mail.gmail.com" type="cite"> <div dir="ltr"> <div class="gmail_extra"> <div class="gmail_quote"> <div> <div class="gmail_default" style="font-family:arial,helvetica,sans-serif;display:inline">Thanks<br> <br> </div> <div class="gmail_default" style="font-family:arial,helvetica,sans-serif;display:inline">Martin<br> <br> </div> </div> <blockquote class="gmail_quote" style="margin:0px 0px 0px 0.8ex;border-left:1px solid rgb(204,204,204);padding-left:1ex"> <div dir="ltr"> </div> <div dir="ltr"><br> </div> <div dir="ltr">Thanks!</div> <div>En 20/7/2016 2:48 p. m., Martin Perina <<a moz-do-not-send="true" href="mailto:mperina@redhat.com" target="_blank">mperina@redhat.com</a>> escribió:<br type="attribution"> <blockquote style="margin:0px 0px 0px 0.8ex;border-left:1px solid rgb(204,204,204);padding-left:1ex"> <div dir="ltr"> <div style="font-family:arial,helvetica,sans-serif">Hi,<br> <br> </div> <div style="font-family:arial,helvetica,sans-serif">sorry for late response, I overlook your reply :-(<br> <br> </div> <div class="gmail_extra"> <div style="font-family:arial,helvetica,sans-serif">I looked at your logs and it seems to me that there's SSL error when engine tries to contact VDSM.<br> </div> <div style="font-family:arial,helvetica,sans-serif;display:inline">You have mentioned that your are using your own custom CA. Are you using it only for HTTPS certificate or do you want to use it also for Engine <-> VDSM communication?<br> <br> </div> <div style="font-family:arial,helvetica,sans-serif;display:inline">Martin Perina<br> </div> <div style="font-family:arial,helvetica,sans-serif"> <br> </div> <br> <div class="gmail_quote">On Wed, Jul 20, 2016 at 9:18 AM, <span dir="ltr"><<a moz-do-not-send="true" href="mailto:nicolas@devels.es" target="_blank">nicolas@devels.es</a>></span> wrote:<br> <blockquote class="gmail_quote" style="margin:0px 0px 0px 0.8ex;border-left:1px solid rgb(204,204,204);padding-left:1ex">Any hints about this?<br> <br> El 2016-07-13 11:13, <a moz-do-not-send="true" href="mailto:nicolas@devels.es" target="_blank">nicolas@devels.es</a> escribió:<br> <blockquote class="gmail_quote" style="margin:0px 0px 0px 0.8ex;border-left:1px solid rgb(204,204,204);padding-left:1ex"> Hi,<br> <br> Unfortunately, upgrading to 4.0.1RC didn't solve the problem.<br> Actually, the error changed to 'General SSLEngine problem', but the<br> result was the same, like this:<br> <br> 2016-07-13 09:52:22,010 INFO<br> [org.ovirt.vdsm.jsonrpc.client.reactors.ReactorClient] (SSL Stomp<br> Reactor) [] Connecting to /10.X.X.X<br> 2016-07-13 09:52:22,018 ERROR<br> [org.ovirt.vdsm.jsonrpc.client.reactors.Reactor] (SSL Stomp Reactor)<br> [] Unable to process messages: General SSLEngine problem<br> <br> It's worth mentioning that we're using our own SSL certificates (not<br> self-signed), and I imported the combined certificate into the<br> /etc/pki/ovirt-engine/.truststore key file. Not sure if related, but<br> just in case.<br> </blockquote> </blockquote> <blockquote class="gmail_quote" style="margin:0px 0px 0px 0.8ex;border-left:1px solid rgb(204,204,204);padding-left:1ex"> <blockquote class="gmail_quote" style="margin:0px 0px 0px 0.8ex;border-left:1px solid rgb(204,204,204);padding-left:1ex"> <br> I had to downgrade to 3.6.7. I'm attaching requested logs, if you need<br> anything else don't hesitate to ask.<br> <br> Regards.<br> <br> El 2016-07-13 09:45, Martin Perina escribió:<br> <blockquote class="gmail_quote" style="margin:0px 0px 0px 0.8ex;border-left:1px solid rgb(204,204,204);padding-left:1ex"> Hi,<br> <br> could you please share also vdsm.log from your hosts and also<br> server.log and setup logs from /var/log/ovirt-engine/setup directory?<br> <br> Thanks<br> <br> Martin Perina<br> <br> On Wed, Jul 13, 2016 at 10:36 AM, <<a moz-do-not-send="true" href="mailto:nicolas@devels.es" target="_blank">nicolas@devels.es</a>> wrote:<br> <br> <blockquote class="gmail_quote" style="margin:0px 0px 0px 0.8ex;border-left:1px solid rgb(204,204,204);padding-left:1ex"> Hi,<br> <br> We upgraded from 3.6.6 to 4.0.0 and we have a big issue since the<br> engine cannot connect to hosts. In the logs all we see is this<br> error:<br> <br> ERROR [org.ovirt.vdsm.jsonrpc.client.reactors.Reactor] (SSL<br> Stomp Reactor) [] Unable to process messages<br> <br> I'm attaching full logs.<br> <br> Could someone help please?<br> <br> Thanks.<br> _______________________________________________<br> Users mailing list<br> <a moz-do-not-send="true" href="mailto:Users@ovirt.org" target="_blank">Users@ovirt.org</a><br> <a moz-do-not-send="true" href="http://lists.ovirt.org/mailman/listinfo/users" rel="noreferrer" target="_blank">http://lists.ovirt.org/mailman/listinfo/users</a> [1]<br> </blockquote> <br> <br> <br> Links:<br> ------<br> [1] <a moz-do-not-send="true" href="http://lists.ovirt.org/mailman/listinfo/users" rel="noreferrer" target="_blank">http://lists.ovirt.org/mailman/listinfo/users</a><br> </blockquote> <br> _______________________________________________<br> Users mailing list<br> <a moz-do-not-send="true" href="mailto:Users@ovirt.org" target="_blank">Users@ovirt.org</a><br> <a moz-do-not-send="true" href="http://lists.ovirt.org/mailman/listinfo/users" rel="noreferrer" target="_blank">http://lists.ovirt.org/mailman/listinfo/users</a><br> </blockquote> </blockquote> </div> <br> </div> </div> </blockquote> </div> </blockquote> </div> <br> </div> </div> </blockquote> <br> </body> </html> --------------4A7D47D278593ED9C71C5805--