[ovirt-users] Re: VDSM HOST ISSUE - Message timeout which can be caused by communication issues

2020-07-15 11:41:58,968+02 ERROR [org.ovirt.engine.core.vdsbroker.monitoring.HostMonitoring] (EE-ManagedThreadFactory-engineScheduled-Thread-79) [] Unable to RefreshCapabilities: VDSNetworkException: VDSGenericException: VDSNetworkException: Message timeout which can be caused by communication issues 2020-07-15 11:41:59,350+02 WARN [org.ovirt.vdsm.jsonrpc.client.utils.retry.Retryable] (SSL Stomp Reactor) [] Retry failed 2020-07-15 11:41:59,350+02 ERROR [org.ovirt.vdsm.jsonrpc.client.reactors.ReactorClient] (EE-ManagedThreadFactory-engineScheduled-Thread-59) [] Exception during connection 2020-07-15 11:41:59,351+02 ERROR [org.ovirt.engine.core.vdsbroker.monitoring.HostMonitoring] (EE-ManagedThreadFactory-engineScheduled-Thread-59) [] Unable to RefreshCapabilities: ConnectException: Connection timeout 2020-07-15 11:41:59,354+02 ERROR [org.ovirt.engine.core.vdsbroker.vdsbroker.GetCapabilitiesAsyncVDSCommand] (EE-ManagedThreadFactory-engineScheduled-Thread-59) [] Command 'GetCapabilitiesAsyncVDSCommand(HostName = sia-svr-ct02, VdsIdAndVdsVDSCommandParametersBase:{hostId='9e65ad71-f633-4be3-aa74-a7fc51b285e6', vds='Host[sia-svr-ct02,9e65ad71-f633-4be3-aa74-a7fc51b285e6]'})' execution failed: java.rmi.ConnectException: Connection timeout 2020-07-15 11:42:00,465+02 ERROR [org.ovirt.engine.core.vdsbroker.vdsbroker.SpmStatusVDSCommand] (EE-ManagedThreadFactory-engineScheduled-Thread-62) [] Command 'SpmStatusVDSCommand(HostName = sia-svr-ct01, SpmStatusVDSCommandParameters:{hostId='2a97ece4-c83e-4107-ac0d-915757c17f16', storagePoolId='cdeb9ed3-60ab-441e-8026-de62e6aa8022'})' execution failed: VDSGenericException: VDSNetworkException: Message timeout which can be caused by communication issues 2020-07-15 11:42:00,484+02 INFO [org.ovirt.vdsm.jsonrpc.client.reactors.ReactorClient] (SSL Stomp Reactor) [] Connecting to sia-svr-ct01.afis/10.234.50.133 2020-07-15 11:42:02,370+02 INFO [org.ovirt.vdsm.jsonrpc.client.reactors.ReactorClient] (SSL Stomp Reactor) [] Connecting to sia-svr-ct02.afis/10.234.50.134 2020-07-15 11:42:02,745+02 INFO [org.ovirt.engine.core.vdsbroker.monitoring.VmAnalyzer] (EE-ManagedThreadFactory-engineScheduled-Thread-64) [] VM 'faf4149e-4e31-40a1-a4ce-f08f03b282c9' is migrating to VDS 'b8e7c982-f816-4eb0-9634-762cd6956fac'(sia-svr-rc02) ignoring it in the refresh until migration is done 2020-07-15 11:42:17,771+02 INFO [org.ovirt.engine.core.vdsbroker.monitoring.VmAnalyzer] (EE-ManagedThreadFactory-engineScheduled-Thread-80) [] VM 'faf4149e-4e31-40a1-a4ce-f08f03b282c9' is migrating to VDS 'b8e7c982-f816-4eb0-9634-762cd6956fac'(sia-svr-rc02) ignoring it in the refresh until migration is done 2020-07-15 11:42:20,485+02 WARN [org.ovirt.vdsm.jsonrpc.client.utils.retry.Retryable] (SSL Stomp Reactor) [] Retry failed 2020-07-15 11:42:20,486+02 ERROR [org.ovirt.vdsm.jsonrpc.client.reactors.ReactorClient] (EE-ManagedThreadFactory-engineScheduled-Thread-62) [] Exception during connection 2020-07-15 11:42:20,487+02 ERROR [org.ovirt.engine.core.vdsbroker.irsbroker.IrsBrokerCommand] (EE-ManagedThreadFactory-engineScheduled-Thread-62) [] IrsBroker::Failed::GetStoragePoolInfoVDS 2020-07-15 11:42:20,487+02 ERROR [org.ovirt.engine.core.vdsbroker.irsbroker.GetStoragePoolInfoVDSCommand] (EE-ManagedThreadFactory-engineScheduled-Thread-62) [] Command 'GetStoragePoolInfoVDSCommand( GetStoragePoolInfoVDSCommandParameters:{storagePoolId='cdeb9ed3-60ab-441e-8026-de62e6aa8022', ignoreFailoverLimit='true'})' execution failed: IRSProtocolException: 2020-07-15 11:42:20,497+02 ERROR [org.ovirt.engine.core.vdsbroker.monitoring.HostMonitoring] (EE-ManagedThreadFactory-engineScheduled-Thread-81) [] Unable to RefreshCapabilities: VDSNetworkException: VDSGenericException: VDSNetworkException: Connection issue Connection timeout 2020-07-15 11:42:22,370+02 WARN [org.ovirt.vdsm.jsonrpc.client.utils.retry.Retryable] (SSL Stomp Reactor) [] Retry failed 2020-07-15 11:42:22,370+02 ERROR [org.ovirt.vdsm.jsonrpc.client.reactors.ReactorClient] (EE-ManagedThreadFactory-engineScheduled-Thread-37) [] Exception during connection 2020-07-15 11:42:22,371+02 ERROR [org.ovirt.engine.core.vdsbroker.monitoring.HostMonitoring] (EE-ManagedThreadFactory-engineScheduled-Thread-37) [] Unable to RefreshCapabilities: ConnectException: Connection timeout 2020-07-15 11:42:22,373+02 ERROR [org.ovirt.engine.core.vdsbroker.vdsbroker.GetCapabilitiesAsyncVDSCommand] (EE-ManagedThreadFactory-engineScheduled-Thread-37) [] Command 'GetCapabilitiesAsyncVDSCommand(HostName = sia-svr-ct02, VdsIdAndVdsVDSCommandParametersBase:{hostId='9e65ad71-f633-4be3-aa74-a7fc51b285e6', vds='Host[sia-svr-ct02,9e65ad71-f633-4be3-aa74-a7fc51b285e6]'})' execution failed: java.rmi.ConnectException: Connection timeout 2020-07-15 11:42:25,384+02 INFO [org.ovirt.vdsm.jsonrpc.client.reactors.ReactorClient] (SSL Stomp Reactor) [] Connecting to sia-svr-ct02.afis/10.234.50.134 2020-07-15 11:42:32,805+02 INFO [org.ovirt.engine.core.vdsbroker.monitoring.VmAnalyzer] (EE-ManagedThreadFactory-engineScheduled-Thread-90) [] VM 'faf4149e-4e31-40a1-a4ce-f08f03b282c9' is migrating to VDS 'b8e7c982-f816-4eb0-9634-762cd6956fac'(sia-svr-rc02) ignoring it in the refresh until migration is done 2020-07-15 11:42:43,163+02 ERROR [org.ovirt.vdsm.jsonrpc.client.reactors.ReactorClient] (SSL Stomp Reactor) [] Connection timeout for host 'sia-svr-rc02.afis', last response arrived 1501 ms ago.