As a followup - is it possible to move the hosted-engine to another HV manually? Meaning we shutdown the hosted-engine VM, and copy it's XML to another HV and fire it up over there where libvirt IS running? If that is possible, where do we find the live / running XML files on a host? Mark *** *Mark Steele* CIO / VP Technical Operations | TelVue Corporation TelVue - We Share Your Vision 16000 Horizon Way, Suite 100 | Mt. Laurel, NJ 08054 800.885.8886 | msteele@telvue.com | http://www.telvue.com twitter: http://twitter.com/telvue | facebook: https://www.facebook.com/telvue On Thu, Apr 2, 2020 at 7:35 AM Mark Steele <msteele@telvue.com> wrote:
All,
We've read some things stating to try running the engine-setup again to get the certs inline. That is failing due to missing repo's - does anyone know of alternative or archived repo's we could point at to get past that part?
[root@ovirt-01 ~]# engine-setup [ INFO ] Stage: Initializing [ INFO ] Stage: Environment setup Configuration files: ['/etc/ovirt-engine-setup.conf.d/10-packaging-jboss.conf', '/etc/ovirt-engine-setup.conf.d/10-packaging.conf', '/etc/ovirt-engine-setup.conf.d/20-setup-ovirt-post.conf'] Log file: /var/log/ovirt-engine/setup/ovirt-engine-setup-20200402072811-h72nc0.log Version: otopi-1.3.0 (otopi-1.3.0-1.el6) [ INFO ] Stage: Environment packages setup [ ERROR ] Yum Cannot queue package iproute: Cannot find a valid baseurl for repo: ovirt-3.5-jpackage-6.0-generic [ ERROR ] Failed to execute stage 'Environment packages setup': Cannot find a valid baseurl for repo: ovirt-3.5-jpackage-6.0-generic [ INFO ] Yum Performing yum transaction rollback [ INFO ] Stage: Clean up Log file is located at /var/log/ovirt-engine/setup/ovirt-engine-setup-20200402072811-h72nc0.log [ INFO ] Generating answer file '/var/lib/ovirt-engine/setup/answers/20200402072821-setup.conf' [ INFO ] Stage: Pre-termination [ INFO ] Stage: Termination [ ERROR ] Execution of setup failed
*** *Mark Steele* CIO / VP Technical Operations | TelVue Corporation TelVue - We Share Your Vision 16000 Horizon Way, Suite 100 | Mt. Laurel, NJ 08054 800.885.8886 | msteele@telvue.com | http://www.telvue.com twitter: http://twitter.com/telvue | facebook: https://www.facebook.com/telvue
On Thu, Apr 2, 2020 at 12:43 AM Strahil Nikolov <hunter86_bg@yahoo.com> wrote:
On April 1, 2020 11:21:34 PM GMT+03:00, Mark Steele <msteele@telvue.com> wrote:
More information:
- the host that ovirt hosted engine is running on is not running libvirtd. When we attempt to start it, it complains about expired certs and fails. Any ideas on this new development?
*** *Mark Steele* CIO / VP Technical Operations | TelVue Corporation TelVue - We Share Your Vision 16000 Horizon Way, Suite 100 | Mt. Laurel, NJ 08054 800.885.8886 | msteele@telvue.com | http://www.telvue.com twitter: http://twitter.com/telvue | facebook: https://www.facebook.com/telvue
On Wed, Apr 1, 2020 at 1:43 PM Mark Steele <msteele@telvue.com> wrote:
A million years ago - we are going to be upgrading with a new cluster but for now, we have this one.
It's production and we cannot take it down unless absolutely necessary.
*** *Mark Steele* CIO / VP Technical Operations | TelVue Corporation TelVue - We Share Your Vision 16000 Horizon Way, Suite 100 | Mt. Laurel, NJ 08054 800.885.8886 | msteele@telvue.com | http://www.telvue.com twitter: http://twitter.com/telvue | facebook: https://www.facebook.com/telvue
On Wed, Apr 1, 2020 at 1:17 PM Strahil Nikolov <hunter86_bg@yahoo.com> wrote:
On April 1, 2020 5:40:27 PM GMT+03:00, Mark Steele <msteele@telvue.com> wrote:
Two other symptoms:
- Some of the HV's respond 'Destination Host Prohibited' on ping, but SSH works fine
- Seeing this in the log:
2020-04-01 10:34:31,839 ERROR [org.ovirt.vdsm.jsonrpc.client.reactors.Reactor] (SSL Stomp Reactor) Unable to process messages: javax.net.ssl.SSLException: Received fatal alert: certificate_expired
*** *Mark Steele* CIO / VP Technical Operations | TelVue Corporation TelVue - We Share Your Vision 16000 Horizon Way, Suite 100 | Mt. Laurel, NJ 08054 800.885.8886 | msteele@telvue.com | http://www.telvue.com twitter: http://twitter.com/telvue | facebook: https://www.facebook.com/telvue
On Wed, Apr 1, 2020 at 10:25 AM Mark Steele <msteele@telvue.com> wrote:
> Yesterday we had a storage crash that caused our hosted engine to stop and > not auto-restart. We manually restarted it and this morning, all elements > are showing a status of either question mark '?' or a red down chevron. > > We are running version oVirt Engine Version: 3.5.0.1-1.el6. > > We are chasing it down and see this in the log: > > 2020-04-01 10:15:53,060 WARN [org.ovirt.engine.core.vdsbroker.VdsManager] (DefaultQuartzScheduler_Worker-23) Failed to refresh VDS , vds = e98f8f1d-dd1d-47c4-b039-d6eaaf578317 : hv-06, VDS Network Error, continuing > > > Any insight into what might be causing this or the fix would be > appreciated. > > *** > *Mark Steele* > >
Hey Mark, When was the oVirt last updated ?
Best Regards, Strahil Nikolov
That's why I asked about the update. The update process renews somee certificates, and as it wasn't updated for a long time - now you have issues.
I hope someone from the community can assist you.
My recommendations: 1. Check on all hosts all vdsm.log and extract the VMs' xml 2. Get all network's xml
Best Regards, Strahil Nikolov