Re: [ovirt-users] Unable to get HE up after update

Hi, i did run that command from the engine (now the hostname is changed to susin.myftp.org -> 192.168.0.101 ) and got: [root@susin ~]# cat < /dev/tcp/susin/54321 -bash: connect: Connection refused -bash: /dev/tcp/susin/54321: Connection refused [root@susin ~]# cat < /dev/tcp/susin.myftp.org/54321 -bash: connect: Connection refused -bash: /dev/tcp/susin.myftp.org/54321: Connection refused [root@susin ~]# cat < /dev/tcp/192.168.0.101/54321 -bash: connect: Connection refused -bash: /dev/tcp/192.168.0.101/54321: Connection refused [root@susin ~]# Both host and engine is behing a NAT and I've configured /etc/hosts correctly so the hosts ping by name from the engine and host. The hostname of the engine is susin.myftp.org so using dig or host, it will resolve to my public IP, and pinging will resolve correctly. But now i came over http://www.ovirt.org/documentation/how-to/networking/changing-engine-host... since i actually changed the hostname for the engine to be able to login through the web UI. Especially the following "The bigger concern is with the engine's certificate. Currently, to the best of our knowledge, there is no component that actually checks this trust. But it's possible, that in some future version of one of the relevant tools - vdsm, libvirt, etc. - such a check will actually be made, and even prevent connections. If this happens, the engine might not be able to connect to the hosts, and the worst case is that they will have to be reinstalled, thus loosing all the configuration and data accumulated by then." tail -f /var/log/ovirt-engine/engine.log 2016-10-21 11:05:16,888 ERROR [org.ovirt.engine.core.vdsbroker.vdsbroker.GetAllVmStatsVDSCommand] (DefaultQuartzScheduler1) [] Command 'GetAllVmStatsVDSCommand(HostName = hosted_engine_1, VdsIdAndVdsVDSCommandParametersBase:{runAsync='true', hostId='826a8da5-74c1-4002-ab7b-e6e32be94fe6', vds='Host[hosted_engine_1,826a8da5-74c1-4002-ab7b-e6e32be94fe6]'})' execution failed: VDSGenericException: VDSNetworkException: Vds timeout occured 2016-10-21 11:05:16,888 INFO [org.ovirt.engine.core.vdsbroker.monitoring.PollVmStatsRefresher] (DefaultQuartzScheduler1) [] Failed to fetch vms info for host 'hosted_engine_1' - skipping VMs monitoring. 2016-10-21 11:05:16,918 ERROR [org.ovirt.engine.core.dal.dbbroker.auditloghandling.AuditLogDirector] (DefaultQuartzScheduler4) [] Correlation ID: null, Call Stack: null, Custom Event ID: -1, Message: VDSM hosted_engine_1 command failed: Message timeout which can be caused by communication issues 2016-10-21 11:05:16,918 INFO [org.ovirt.engine.core.vdsbroker.vdsbroker.GetCapabilitiesVDSCommand] (DefaultQuartzScheduler4) [] Command 'org.ovirt.engine.core.vdsbroker.vdsbroker.GetCapabilitiesVDSCommand' return value 'org.ovirt.engine.core.vdsbroker.vdsbroker.VDSInfoReturnForXmlRpc@1f2e4065' 2016-10-21 11:05:16,918 INFO [org.ovirt.engine.core.vdsbroker.vdsbroker.GetCapabilitiesVDSCommand] (DefaultQuartzScheduler4) [] HostName = hosted_engine_1 2016-10-21 11:05:16,919 ERROR [org.ovirt.engine.core.vdsbroker.vdsbroker.GetCapabilitiesVDSCommand] (DefaultQuartzScheduler4) [] Command 'GetCapabilitiesVDSCommand(HostName = hosted_engine_1, VdsIdAndVdsVDSCommandParametersBase:{runAsync='true', hostId='826a8da5-74c1-4002-ab7b-e6e32be94fe6', vds='Host[hosted_engine_1,826a8da5-74c1-4002-ab7b-e6e32be94fe6]'})' execution failed: VDSGenericException: VDSNetworkException: Message timeout which can be caused by communication issues 2016-10-21 11:05:16,919 ERROR [org.ovirt.engine.core.vdsbroker.monitoring.HostMonitoring] (DefaultQuartzScheduler4) [] Failure to refresh Vds runtime info: VDSGenericException: VDSNetworkException: Message timeout which can be caused by communication issues 2016-10-21 11:05:16,919 ERROR [org.ovirt.engine.core.vdsbroker.monitoring.HostMonitoring] (DefaultQuartzScheduler4) [] Exception: org.ovirt.engine.core.vdsbroker.vdsbroker.VDSNetworkException: VDSGenericException: VDSNetworkException: Message timeout which can be caused by communication issues at org.ovirt.engine.core.vdsbroker.vdsbroker.BrokerCommandBase.proceedProxyReturnValue(BrokerCommandBase.java:188) [vdsbroker.jar:] at org.ovirt.engine.core.vdsbroker.vdsbroker.GetCapabilitiesVDSCommand.executeVdsBrokerCommand(GetCapabilitiesVDSCommand.java:16) [vdsbroker.jar:] at org.ovirt.engine.core.vdsbroker.vdsbroker.VdsBrokerCommand.executeVDSCommand(VdsBrokerCommand.java:110) [vdsbroker.jar:] at org.ovirt.engine.core.vdsbroker.VDSCommandBase.executeCommand(VDSCommandBase.java:73) [vdsbroker.jar:] at org.ovirt.engine.core.dal.VdcCommandBase.execute(VdcCommandBase.java:33) [dal.jar:] at org.ovirt.engine.core.vdsbroker.ResourceManager.runVdsCommand(ResourceManager.java:451) [vdsbroker.jar:] at org.ovirt.engine.core.vdsbroker.VdsManager.refreshCapabilities(VdsManager.java:653) [vdsbroker.jar:] at org.ovirt.engine.core.vdsbroker.monitoring.HostMonitoring.refreshVdsRunTimeInfo(HostMonitoring.java:121) [vdsbroker.jar:] at org.ovirt.engine.core.vdsbroker.monitoring.HostMonitoring.refresh(HostMonitoring.java:85) [vdsbroker.jar:] at org.ovirt.engine.core.vdsbroker.VdsManager.onTimer(VdsManager.java:238) [vdsbroker.jar:] at sun.reflect.NativeMethodAccessorImpl.invoke0(Native Method) [rt.jar:1.8.0_102] at sun.reflect.NativeMethodAccessorImpl.invoke(NativeMethodAccessorImpl.java:62) [rt.jar:1.8.0_102] at sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:43) [rt.jar:1.8.0_102] at java.lang.reflect.Method.invoke(Method.java:498) [rt.jar:1.8.0_102] at org.ovirt.engine.core.utils.timer.JobWrapper.invokeMethod(JobWrapper.java:77) [scheduler.jar:] at org.ovirt.engine.core.utils.timer.JobWrapper.execute(JobWrapper.java:51) [scheduler.jar:] at org.quartz.core.JobRunShell.run(JobRunShell.java:213) [quartz.jar:] at java.util.concurrent.Executors$RunnableAdapter.call(Executors.java:511) [rt.jar:1.8.0_102] at java.util.concurrent.FutureTask.run(FutureTask.java:266) [rt.jar:1.8.0_102] at java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1142) [rt.jar:1.8.0_102] at java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:617) [rt.jar:1.8.0_102] at java.lang.Thread.run(Thread.java:745) [rt.jar:1.8.0_102] 2016-10-21 11:05:16,921 WARN [org.ovirt.engine.core.vdsbroker.VdsManager] (DefaultQuartzScheduler4) [] Failed to refresh VDS, network error, continuing, vds='hosted_engine_1'(826a8da5-74c1-4002-ab7b-e6e32be94fe6): VDSGenericException: VDSNetworkException: Message timeout which can be caused by communication issues 2016-10-21 11:05:16,921 WARN [org.ovirt.engine.core.vdsbroker.VdsManager] (org.ovirt.thread.pool-8-thread-1) [] Host 'hosted_engine_1' is not responding. 2016-10-21 11:05:16,993 WARN [org.ovirt.engine.core.dal.dbbroker.auditloghandling.AuditLogDirector] (org.ovirt.thread.pool-8-thread-1) [] Correlation ID: null, Call Stack: null, Custom Event ID: -1, Message: Host hosted_engine_1 is not responding. Host cannot be fenced automatically because power management for the host is disabled. 2016-10-21 11:05:19,943 INFO [org.ovirt.vdsm.jsonrpc.client.reactors.ReactorClient] (SSL Stomp Reactor) [] Connecting to ovirt01/192.168.0.100 2016-10-21 11:05:19,960 ERROR [org.ovirt.vdsm.jsonrpc.client.reactors.Reactor] (SSL Stomp Reactor) [] Unable to process messages: General SSLEngine problem .. 2016-10-21 11:08:40,871 WARN [org.ovirt.engine.core.bll.pm.VdsNotRespondingTreatmentCommand] (org.ovirt.thread.pool-8-thread-6) [6d681722] Validation of action 'VdsNotRespondingTreatment' failed for user SYSTEM. Reasons: VAR__ACTION__RESTART,POWER_MANAGEMENT_ACTION_ON_ENTITY_ALREADY_IN_PROGRESS 2016-10-21 11:08:41,023 INFO [org.ovirt.engine.core.bll.pm.VdsNotRespondingTreatmentCommand] (org.ovirt.thread.pool-8-thread-5) [193e2e22] Running command: VdsNotRespondingTreatmentCommand internal: true. Entities affected : ID: 826a8da5-74c1-4002-ab7b-e6e32be94fe6 Type: VDS 2016-10-21 11:08:41,088 INFO [org.ovirt.engine.core.bll.pm.SshSoftFencingCommand] (org.ovirt.thread.pool-8-thread-5) [193e2e22] Running command: SshSoftFencingCommand internal: true. Entities affected : ID: 826a8da5-74c1-4002-ab7b-e6e32be94fe6 Type: VDS 2016-10-21 11:08:41,116 INFO [org.ovirt.engine.core.bll.pm.SshSoftFencingCommand] (org.ovirt.thread.pool-8-thread-5) [193e2e22] Opening SSH Soft Fencing session on host 'ovirt01' 2016-10-21 11:08:41,470 ERROR [org.ovirt.engine.core.bll.pm.SshSoftFencingCommand] (org.ovirt.thread.pool-8-thread-5) [193e2e22] SSH Soft Fencing command failed on host 'ovirt01': SSH authentication to 'root@ovirt01' failed. Please verify provided credentials. Make sure key is authorized at host Stdout: Stderr: 2016-10-21 11:08:41,483 INFO [org.ovirt.engine.core.bll.pm.SshSoftFencingCommand] (org.ovirt.thread.pool-8-thread-5) [193e2e22] Lock freed to object 'EngineLock:{exclusiveLocks='[826a8da5-74c1-4002-ab7b-e6e32be94fe6=<VDS_FENCE, POWER_MANAGEMENT_ACTION_ON_ENTITY_ALREADY_IN_PROGRESS>]', sharedLocks='null'}' 2016-10-21 11:08:41,545 WARN [org.ovirt.engine.core.bll.lock.InMemoryLockManager] (org.ovirt.thread.pool-8-thread-5) [193e2e22] Trying to release exclusive lock which does not exist, lock key: '826a8da5-74c1-4002-ab7b-e6e32be94fe6VDS_FENCE' 2016-10-21 11:08:41,547 INFO [org.ovirt.engine.core.bll.pm.VdsNotRespondingTreatmentCommand] (org.ovirt.thread.pool-8-thread-5) [193e2e22] Lock freed to object 'EngineLock:{exclusiveLocks='[826a8da5-74c1-4002-ab7b-e6e32be94fe6=<VDS_FENCE, POWER_MANAGEMENT_ACTION_ON_ENTITY_ALREADY_IN_PROGRESS>]', sharedLocks='null'}' 2016-10-21 11:08:43,503 INFO [org.ovirt.vdsm.jsonrpc.client.reactors.ReactorClient] (SSL Stomp Reactor) [] Connecting to ovirt01/192.168.0.100 2016-10-21 11:08:43,517 ERROR [org.ovirt.vdsm.jsonrpc.client.reactors.Reactor] (SSL Stomp Reactor) [] Unable to process messages: General SSLEngine problem 2016-10-21 11:08:55,446 INFO [org.ovirt.vdsm.jsonrpc.client.reactors.ReactorClient] (SSL Stomp Reactor) [] Connecting to ovirt01/192.168.0.100 2016-10-21 11:08:55,461 ERROR [org.ovirt.vdsm.jsonrpc.client.reactors.Reactor] (SSL Stomp Reactor) [] Unable to process messages: General SSLEngine problem Since the engine is complaining about SSL communication error, i suspect the problem is there. Is there still any ways to save my VMs or do i have to reinstall it? On Thu, Oct 20, 2016 at 6:27 PM, Simone Tiraboschi <stirabos(a)redhat.com&gt; wrote: -- Susinthiran Sithamparanathan