On April 1, 2020 11:21:34 PM GMT+03:00, Mark Steele <msteele@telvue.com> wrote:
More information:
- the host that ovirt hosted engine is running on is not running libvirtd. When we attempt to start it, it complains about expired certs and fails. Any ideas on this new development?
*** *Mark Steele* CIO / VP Technical Operations | TelVue Corporation TelVue - We Share Your Vision 16000 Horizon Way, Suite 100 | Mt. Laurel, NJ 08054 800.885.8886 | msteele@telvue.com | http://www.telvue.com twitter: http://twitter.com/telvue | facebook: https://www.facebook.com/telvue
On Wed, Apr 1, 2020 at 1:43 PM Mark Steele <msteele@telvue.com> wrote:
A million years ago - we are going to be upgrading with a new cluster but for now, we have this one.
It's production and we cannot take it down unless absolutely necessary.
*** *Mark Steele* CIO / VP Technical Operations | TelVue Corporation TelVue - We Share Your Vision 16000 Horizon Way, Suite 100 | Mt. Laurel, NJ 08054 800.885.8886 | msteele@telvue.com | http://www.telvue.com twitter: http://twitter.com/telvue | facebook: https://www.facebook.com/telvue
On Wed, Apr 1, 2020 at 1:17 PM Strahil Nikolov <hunter86_bg@yahoo.com> wrote:
On April 1, 2020 5:40:27 PM GMT+03:00, Mark Steele <msteele@telvue.com> wrote:
Two other symptoms:
- Some of the HV's respond 'Destination Host Prohibited' on ping, but SSH works fine
- Seeing this in the log:
2020-04-01 10:34:31,839 ERROR [org.ovirt.vdsm.jsonrpc.client.reactors.Reactor] (SSL Stomp Reactor) Unable to process messages: javax.net.ssl.SSLException: Received fatal alert: certificate_expired
*** *Mark Steele* CIO / VP Technical Operations | TelVue Corporation TelVue - We Share Your Vision 16000 Horizon Way, Suite 100 | Mt. Laurel, NJ 08054 800.885.8886 | msteele@telvue.com | http://www.telvue.com twitter: http://twitter.com/telvue | facebook: https://www.facebook.com/telvue
On Wed, Apr 1, 2020 at 10:25 AM Mark Steele <msteele@telvue.com> wrote:
Yesterday we had a storage crash that caused our hosted engine to stop and not auto-restart. We manually restarted it and this morning, all elements are showing a status of either question mark '?' or a red down chevron.
We are running version oVirt Engine Version: 3.5.0.1-1.el6.
We are chasing it down and see this in the log:
2020-04-01 10:15:53,060 WARN [org.ovirt.engine.core.vdsbroker.VdsManager] (DefaultQuartzScheduler_Worker-23) Failed to refresh VDS , vds = e98f8f1d-dd1d-47c4-b039-d6eaaf578317 : hv-06, VDS Network Error, continuing
Any insight into what might be causing this or the fix would be appreciated.
*** *Mark Steele*
Hey Mark, When was the oVirt last updated ?
Best Regards, Strahil Nikolov
That's why I asked about the update. The update process renews somee certificates, and as it wasn't updated for a long time - now you have issues. I hope someone from the community can assist you. My recommendations: 1. Check on all hosts all vdsm.log and extract the VMs' xml 2. Get all network's xml Best Regards, Strahil Nikolov