[ovirt-users] oVirt Engine 4.0.5 and CentOS 7.3 Instability
Rogério Ceni Coelho
rogeriocenicoelho at gmail.com
Fri Jan 6 19:02:45 UTC 2017
Hi Pavel,
I do not have a host with non-responsive status as you can see below. But
the symptons are the same ... :-(
[root at prd-rbs-ovirt01-poa ~]# rpm -qa | grep -i vdsm-jsonrpc
vdsm-jsonrpc-java-1.2.10-1.20161211091707.gitaf70e3f.el7.centos.noarch
I have some nodes running 4.0.4. This can be a problem ?
[image: pasted2]
[image: pasted1]
Em sex, 6 de jan de 2017 às 16:55, Pavel Gashev <Pax at acronis.com> escreveu:
> Rogério,
>
>
>
> This bug is fixed in vdsm-jsonrpc-java-1.2.9.
>
> Do you have a really non-responsive host?
>
>
>
> *From: *Rogério Ceni Coelho <rogeriocenicoelho at gmail.com>
>
> *Date: *Friday 6 January 2017 at 21:35
>
>
> *To: *Pavel Gashev <Pax at acronis.com>, users <users at ovirt.org>
> *Subject: *Re: [ovirt-users] oVirt Engine 4.0.5 and CentOS 7.3 Instability
>
>
>
> I found https://bugzilla.redhat.com/show_bug.cgi?id=1393714 which seem to
> be the same problem that occour with me ...
>
>
>
>
>
> Em sex, 6 de jan de 2017 às 16:28, Rogério Ceni Coelho <
> rogeriocenicoelho at gmail.com> escreveu:
>
> Pavel,
>
>
>
> Take a look ...
>
>
>
> [root at prd-rbs-ovirt01-poa ~]# grep -i error
> /var/log/ovirt-engine/engine.log | grep -v
> org.ovirt.engine.core.vdsbroker.HostDevListByCapsVDSCommand | tail -50
>
> 2017-01-06 16:20:06,897 ERROR
> [org.ovirt.engine.core.vdsbroker.vdsbroker.GetAllVmStatsVDSCommand]
> (DefaultQuartzScheduler6) [a6fc972] Command
> 'GetAllVmStatsVDSCommand(HostName = prd-rbs-ovirt-kvm10-poa.rbs.com.br,
> VdsIdAndVdsVDSCommandParametersBase:{runAsync='true',
> hostId='8b4bc7dc-af8c-4415-a2ef-bccc11ddf23a', vds='Host[
> prd-rbs-ovirt-kvm10-poa.rbs.com.br,8b4bc7dc-af8c-4415-a2ef-bccc11ddf23a]'})'
> execution failed: VDSGenericException: VDSNetworkException: Heartbeat
> exceeded
>
> 2017-01-06 16:20:06,899 ERROR
> [org.ovirt.engine.core.vdsbroker.vdsbroker.GetAllVmStatsVDSCommand]
> (DefaultQuartzScheduler10) [19b85a77] Command
> 'GetAllVmStatsVDSCommand(HostName = prd-rbs-ovirt-kvm17-poa.rbs.com.br,
> VdsIdAndVdsVDSCommandParametersBase:{runAsync='true',
> hostId='795b917b-ea5b-499d-80c5-aa3aad4f2537', vds='Host[
> prd-rbs-ovirt-kvm17-poa.rbs.com.br,795b917b-ea5b-499d-80c5-aa3aad4f2537]'})'
> execution failed: VDSGenericException: VDSNetworkException: Heartbeat
> exceeded
>
> 2017-01-06 16:20:06,901 ERROR
> [org.ovirt.engine.core.vdsbroker.vdsbroker.GetAllVmStatsVDSCommand]
> (DefaultQuartzScheduler8) [1bccfc8a] Command
> 'GetAllVmStatsVDSCommand(HostName = prd-rbs-ovirt-kvm19-poa.rbs.com.br,
> VdsIdAndVdsVDSCommandParametersBase:{runAsync='true',
> hostId='887b6e35-1fd1-4cd6-9e78-05bcab12a417', vds='Host[
> prd-rbs-ovirt-kvm19-poa.rbs.com.br,887b6e35-1fd1-4cd6-9e78-05bcab12a417]'})'
> execution failed: VDSGenericException: VDSNetworkException: Heartbeat
> exceeded
>
> 2017-01-06 16:23:26,020 ERROR
> [org.ovirt.engine.core.vdsbroker.vdsbroker.GetAllVmStatsVDSCommand]
> (DefaultQuartzScheduler1) [7771902b] Command
> 'GetAllVmStatsVDSCommand(HostName = prd-rbs-ovirt-kvm06-poa.rbs.com.br,
> VdsIdAndVdsVDSCommandParametersBase:{runAsync='true',
> hostId='23b750ed-4ba6-4782-a9bd-c018c0f36e44', vds='Host[
> prd-rbs-ovirt-kvm06-poa.rbs.com.br,23b750ed-4ba6-4782-a9bd-c018c0f36e44]'})'
> execution failed: VDSGenericException: VDSNetworkException: Heartbeat
> exceeded
>
> 2017-01-06 16:23:26,043 ERROR
> [org.ovirt.vdsm.jsonrpc.client.JsonRpcClient] (ResponseWorker) [] Not able
> to update response for "e5f7a478-841d-413c-baa1-d63632be7748"
>
> 2017-01-06 16:24:32,480 ERROR
> [org.ovirt.vdsm.jsonrpc.client.JsonRpcClient] (ResponseWorker) [] Not able
> to update response for "75bf7e9a-fb5f-4253-bdf6-ff0fcbf8c876"
>
> 2017-01-06 16:24:32,485 ERROR
> [org.ovirt.engine.core.dal.dbbroker.auditloghandling.AuditLogDirector]
> (DefaultQuartzScheduler6) [35cc958c] Correlation ID: null, Call Stack:
> null, Custom Event ID: -1, Message: VDSM
> prd-rbs-ovirt-kvm06-poa.rbs.com.br command failed: Heartbeat exceeded
>
> 2017-01-06 16:24:32,485 ERROR
> [org.ovirt.engine.core.vdsbroker.vdsbroker.SpmStatusVDSCommand]
> (DefaultQuartzScheduler6) [35cc958c] Command 'SpmStatusVDSCommand(HostName
> = prd-rbs-ovirt-kvm06-poa.rbs.com.br,
> SpmStatusVDSCommandParameters:{runAsync='true',
> hostId='23b750ed-4ba6-4782-a9bd-c018c0f36e44',
> storagePoolId='00000001-0001-0001-0001-000000000198'})' execution failed:
> VDSGenericException: VDSNetworkException: Heartbeat exceeded
>
> 2017-01-06 16:24:32,524 WARN
> [org.ovirt.engine.core.dal.dbbroker.auditloghandling.AuditLogDirector]
> (DefaultQuartzScheduler6) [152cde35] Correlation ID: 152cde35, Call Stack:
> null, Custom Event ID: -1, Message: Invalid status on Data Center RBS.
> Setting Data Center status to Non Responsive (On host
> prd-rbs-ovirt-kvm06-poa.rbs.com.br, Error: Network error during
> communication with the Host.).
>
> 2017-01-06 16:25:37,493 ERROR
> [org.ovirt.vdsm.jsonrpc.client.reactors.Reactor] (SSL Stomp Reactor) []
> Internal server error: null
>
> 2017-01-06 16:25:37,494 ERROR
> [org.ovirt.vdsm.jsonrpc.client.reactors.Reactor] (SSL Stomp Reactor) []
> Internal server error: null
>
> 2017-01-06 16:25:37,494 ERROR
> [org.ovirt.vdsm.jsonrpc.client.reactors.Reactor] (SSL Stomp Reactor) []
> Internal server error: null
>
> 2017-01-06 16:25:37,494 ERROR
> [org.ovirt.vdsm.jsonrpc.client.reactors.Reactor] (SSL Stomp Reactor) []
> Internal server error: null
>
> 2017-01-06 16:25:37,494 ERROR
> [org.ovirt.vdsm.jsonrpc.client.reactors.Reactor] (SSL Stomp Reactor) []
> Internal server error: null
>
> 2017-01-06 16:25:37,494 ERROR
> [org.ovirt.vdsm.jsonrpc.client.reactors.Reactor] (SSL Stomp Reactor) []
> Internal server error: null
>
> 2017-01-06 16:25:37,495 ERROR
> [org.ovirt.engine.core.vdsbroker.vdsbroker.GetAllVmStatsVDSCommand]
> (DefaultQuartzScheduler4) [44e1d6ed] Command
> 'GetAllVmStatsVDSCommand(HostName = prd-rbs-ovirt-kvm03-poa.rbs.com.br,
> VdsIdAndVdsVDSCommandParametersBase:{runAsync='true',
> hostId='f7842244-646c-400a-9736-f8d4aa9b1cef', vds='Host[
> prd-rbs-ovirt-kvm03-poa.rbs.com.br,f7842244-646c-400a-9736-f8d4aa9b1cef]'})'
> execution failed: VDSGenericException: VDSNetworkException: Heartbeat
> exceeded
>
> 2017-01-06 16:25:37,499 ERROR
> [org.ovirt.engine.core.vdsbroker.vdsbroker.GetAllVmStatsVDSCommand]
> (DefaultQuartzScheduler5) [4fb9c0c9] Command
> 'GetAllVmStatsVDSCommand(HostName = prd-rbs-ovirt-kvm17-poa.rbs.com.br,
> VdsIdAndVdsVDSCommandParametersBase:{runAsync='true',
> hostId='795b917b-ea5b-499d-80c5-aa3aad4f2537', vds='Host[
> prd-rbs-ovirt-kvm17-poa.rbs.com.br,795b917b-ea5b-499d-80c5-aa3aad4f2537]'})'
> execution failed: VDSGenericException: VDSNetworkException: Heartbeat
> exceeded
>
> 2017-01-06 16:25:37,499 ERROR
> [org.ovirt.engine.core.vdsbroker.vdsbroker.GetAllVmStatsVDSCommand]
> (DefaultQuartzScheduler7) [7bc94bc6] Command
> 'GetAllVmStatsVDSCommand(HostName = prd-rbs-ovirt-kvm16-poa.rbs.com.br,
> VdsIdAndVdsVDSCommandParametersBase:{runAsync='true',
> hostId='e2e148b4-00b2-444a-b935-633882e840af', vds='Host[
> prd-rbs-ovirt-kvm16-poa.rbs.com.br,e2e148b4-00b2-444a-b935-633882e840af]'})'
> execution failed: VDSGenericException: VDSNetworkException: Heartbeat
> exceeded
>
> 2017-01-06 16:25:37,500 ERROR
> [org.ovirt.engine.core.vdsbroker.vdsbroker.GetAllVmStatsVDSCommand]
> (DefaultQuartzScheduler10) [19b85a77] Command
> 'GetAllVmStatsVDSCommand(HostName = prd-rbs-ovirt-kvm08-poa.rbs.com.br,
> VdsIdAndVdsVDSCommandParametersBase:{runAsync='true',
> hostId='91dde882-0c86-4330-a206-499275557534', vds='Host[
> prd-rbs-ovirt-kvm08-poa.rbs.com.br,91dde882-0c86-4330-a206-499275557534]'})'
> execution failed: VDSGenericException: VDSNetworkException: Heartbeat
> exceeded
>
> 2017-01-06 16:25:37,501 ERROR
> [org.ovirt.engine.core.vdsbroker.vdsbroker.GetAllVmStatsVDSCommand]
> (DefaultQuartzScheduler8) [1bccfc8a] Command
> 'GetAllVmStatsVDSCommand(HostName = prd-rbs-ovirt-kvm10-poa.rbs.com.br,
> VdsIdAndVdsVDSCommandParametersBase:{runAsync='true',
> hostId='8b4bc7dc-af8c-4415-a2ef-bccc11ddf23a', vds='Host[
> prd-rbs-ovirt-kvm10-poa.rbs.com.br,8b4bc7dc-af8c-4415-a2ef-bccc11ddf23a]'})'
> execution failed: VDSGenericException: VDSNetworkException: Heartbeat
> exceeded
>
> 2017-01-06 16:25:37,501 ERROR
> [org.ovirt.engine.core.vdsbroker.vdsbroker.GetAllVmStatsVDSCommand]
> (DefaultQuartzScheduler6) [152cde35] Command
> 'GetAllVmStatsVDSCommand(HostName = prd-rbs-ovirt-kvm01-poa.rbs.com.br,
> VdsIdAndVdsVDSCommandParametersBase:{runAsync='true',
> hostId='c4a21d0b-f003-4ee1-8b6b-2b26671d410f', vds='Host[
> prd-rbs-ovirt-kvm01-poa.rbs.com.br,c4a21d0b-f003-4ee1-8b6b-2b26671d410f]'})'
> execution failed: VDSGenericException: VDSNetworkException: Heartbeat
> exceeded
>
> 2017-01-06 16:25:37,504 ERROR
> [org.ovirt.engine.core.vdsbroker.vdsbroker.GetAllVmStatsVDSCommand]
> (DefaultQuartzScheduler9) [2c0caad] Command
> 'GetAllVmStatsVDSCommand(HostName = prd-rbs-ovirt-kvm12-poa.rbs.com.br,
> VdsIdAndVdsVDSCommandParametersBase:{runAsync='true',
> hostId='46a28187-ab69-4871-9182-24dd910d2784', vds='Host[
> prd-rbs-ovirt-kvm12-poa.rbs.com.br,46a28187-ab69-4871-9182-24dd910d2784]'})'
> execution failed: VDSGenericException: VDSNetworkException: Heartbeat
> exceeded
>
> 2017-01-06 16:25:37,558 ERROR
> [org.ovirt.vdsm.jsonrpc.client.JsonRpcClient] (ResponseWorker) [] Not able
> to update response for "c5556060-f18c-4021-a4b2-8300c9137133"
>
> 2017-01-06 16:26:43,588 ERROR
> [org.ovirt.vdsm.jsonrpc.client.reactors.Reactor] (SSL Stomp Reactor) []
> Internal server error: null
>
> 2017-01-06 16:26:43,588 ERROR
> [org.ovirt.vdsm.jsonrpc.client.reactors.Reactor] (SSL Stomp Reactor) []
> Internal server error: null
>
> 2017-01-06 16:26:43,589 ERROR
> [org.ovirt.vdsm.jsonrpc.client.reactors.Reactor] (SSL Stomp Reactor) []
> Internal server error: null
>
> 2017-01-06 16:26:43,589 ERROR
> [org.ovirt.vdsm.jsonrpc.client.reactors.Reactor] (SSL Stomp Reactor) []
> Internal server error: null
>
> 2017-01-06 16:26:43,589 ERROR
> [org.ovirt.vdsm.jsonrpc.client.reactors.Reactor] (SSL Stomp Reactor) []
> Internal server error: null
>
> 2017-01-06 16:26:43,589 ERROR
> [org.ovirt.vdsm.jsonrpc.client.reactors.Reactor] (SSL Stomp Reactor) []
> Internal server error: null
>
> 2017-01-06 16:26:43,590 ERROR
> [org.ovirt.engine.core.vdsbroker.vdsbroker.GetAllVmStatsVDSCommand]
> (DefaultQuartzScheduler1) [585c6b32] Command
> 'GetAllVmStatsVDSCommand(HostName = prd-rbs-ovirt-kvm09-poa.rbs.com.br,
> VdsIdAndVdsVDSCommandParametersBase:{runAsync='true',
> hostId='da563ca2-eb39-451d-8ee8-20853b87d341', vds='Host[
> prd-rbs-ovirt-kvm09-poa.rbs.com.br,da563ca2-eb39-451d-8ee8-20853b87d341]'})'
> execution failed: VDSGenericException: VDSNetworkException: Heartbeat
> exceeded
>
> 2017-01-06 16:26:43,589 ERROR
> [org.ovirt.engine.core.vdsbroker.vdsbroker.GetAllVmStatsVDSCommand]
> (DefaultQuartzScheduler10) [3acdc247] Command
> 'GetAllVmStatsVDSCommand(HostName = prd-rbs-ovirt-kvm04-poa.rbs.com.br,
> VdsIdAndVdsVDSCommandParametersBase:{runAsync='true',
> hostId='c0a42c7f-f8fc-4a7e-815a-aed9cf490180', vds='Host[
> prd-rbs-ovirt-kvm04-poa.rbs.com.br,c0a42c7f-f8fc-4a7e-815a-aed9cf490180]'})'
> execution failed: VDSGenericException: VDSNetworkException: Heartbeat
> exceeded
>
> 2017-01-06 16:26:43,590 ERROR
> [org.ovirt.engine.core.vdsbroker.vdsbroker.GetAllVmStatsVDSCommand]
> (DefaultQuartzScheduler3) [71319dca] Command
> 'GetAllVmStatsVDSCommand(HostName = prd-rbs-ovirt-kvm15-poa.rbs.com.br,
> VdsIdAndVdsVDSCommandParametersBase:{runAsync='true',
> hostId='9e2642de-91b2-40ec-aec3-b03be2b00e93', vds='Host[
> prd-rbs-ovirt-kvm15-poa.rbs.com.br,9e2642de-91b2-40ec-aec3-b03be2b00e93]'})'
> execution failed: VDSGenericException: VDSNetworkException: Heartbeat
> exceeded
>
> 2017-01-06 16:26:43,591 ERROR
> [org.ovirt.engine.core.vdsbroker.vdsbroker.GetAllVmStatsVDSCommand]
> (DefaultQuartzScheduler2) [2e9a9316] Command
> 'GetAllVmStatsVDSCommand(HostName = prd-rbs-ovirt-kvm02-poa.rbs.com.br,
> VdsIdAndVdsVDSCommandParametersBase:{runAsync='true',
> hostId='21116080-5e7d-40b4-ba59-056e55eadb5b', vds='Host[
> prd-rbs-ovirt-kvm02-poa.rbs.com.br,21116080-5e7d-40b4-ba59-056e55eadb5b]'})'
> execution failed: VDSGenericException: VDSNetworkException: Heartbeat
> exceeded
>
> 2017-01-06 16:26:43,592 ERROR
> [org.ovirt.engine.core.vdsbroker.vdsbroker.GetAllVmStatsVDSCommand]
> (DefaultQuartzScheduler6) [f0fd53d] Command
> 'GetAllVmStatsVDSCommand(HostName = prd-rbs-ovirt-kvm01-poa.rbs.com.br,
> VdsIdAndVdsVDSCommandParametersBase:{runAsync='true',
> hostId='c4a21d0b-f003-4ee1-8b6b-2b26671d410f', vds='Host[
> prd-rbs-ovirt-kvm01-poa.rbs.com.br,c4a21d0b-f003-4ee1-8b6b-2b26671d410f]'})'
> execution failed: VDSGenericException: VDSNetworkException: Heartbeat
> exceeded
>
> 2017-01-06 16:26:43,616 ERROR
> [org.ovirt.engine.core.vdsbroker.vdsbroker.GetAllVmStatsVDSCommand]
> (DefaultQuartzScheduler5) [4fb9c0c9] Command
> 'GetAllVmStatsVDSCommand(HostName = prd-rbs-ovirt-kvm05-poa.rbs.com.br,
> VdsIdAndVdsVDSCommandParametersBase:{runAsync='true',
> hostId='445946b6-6c1a-4faf-a705-e9ba23261c7f', vds='Host[
> prd-rbs-ovirt-kvm05-poa.rbs.com.br,445946b6-6c1a-4faf-a705-e9ba23261c7f]'})'
> execution failed: VDSGenericException: VDSNetworkException: Heartbeat
> exceeded
>
> 2017-01-06 16:26:39,409 ERROR
> [org.ovirt.engine.core.dal.dbbroker.auditloghandling.AuditLogDirector]
> (DefaultQuartzScheduler10) [32ab9c87] Correlation ID: null, Call Stack:
> null, Custom Event ID: -1, Message: VDSM
> prd-rbs-ovirt-kvm01-poa.rbs.com.br command failed: Heartbeat exceeded
>
> 2017-01-06 16:26:39,410 ERROR
> [org.ovirt.engine.core.vdsbroker.vdsbroker.GetStatsVDSCommand]
> (DefaultQuartzScheduler10) [32ab9c87] Command 'GetStatsVDSCommand(HostName
> = prd-rbs-ovirt-kvm01-poa.rbs.com.br,
> VdsIdAndVdsVDSCommandParametersBase:{runAsync='true',
> hostId='c4a21d0b-f003-4ee1-8b6b-2b26671d410f', vds='Host[
> prd-rbs-ovirt-kvm01-poa.rbs.com.br,c4a21d0b-f003-4ee1-8b6b-2b26671d410f]'})'
> execution failed: VDSGenericException: VDSNetworkException: Heartbeat
> exceeded
>
> 2017-01-06 16:26:39,411 ERROR
> [org.ovirt.engine.core.vdsbroker.monitoring.HostMonitoring]
> (DefaultQuartzScheduler10) [32ab9c87] Failed getting vds stats, host='
> prd-rbs-ovirt-kvm01-poa.rbs.com.br'(c4a21d0b-f003-4ee1-8b6b-2b26671d410f):
> org.ovirt.engine.core.vdsbroker.vdsbroker.VDSNetworkException:
> VDSGenericException: VDSNetworkException: Heartbeat exceeded
>
> 2017-01-06 16:26:39,411 ERROR
> [org.ovirt.engine.core.vdsbroker.monitoring.HostMonitoring]
> (DefaultQuartzScheduler10) [32ab9c87] Failure to refresh host '
> prd-rbs-ovirt-kvm01-poa.rbs.com.br' runtime info: VDSGenericException:
> VDSNetworkException: Heartbeat exceeded
>
> 2017-01-06 16:26:39,411 WARN [org.ovirt.engine.core.vdsbroker.VdsManager]
> (DefaultQuartzScheduler10) [32ab9c87] Failed to refresh VDS, network error,
> continuing, vds='prd-rbs-ovirt-kvm01-poa.rbs.com.br'(c4a21d0b-f003-4ee1-8b6b-2b26671d410f):
> VDSGenericException: VDSNetworkException: Heartbeat exceeded
>
> 2017-01-06 16:27:49,661 ERROR
> [org.ovirt.vdsm.jsonrpc.client.reactors.Reactor] (SSL Stomp Reactor) []
> Internal server error: null
>
> 2017-01-06 16:27:49,661 ERROR
> [org.ovirt.vdsm.jsonrpc.client.reactors.Reactor] (SSL Stomp Reactor) []
> Internal server error: null
>
> 2017-01-06 16:27:49,661 ERROR
> [org.ovirt.vdsm.jsonrpc.client.reactors.Reactor] (SSL Stomp Reactor) []
> Internal server error: null
>
> 2017-01-06 16:27:49,661 ERROR
> [org.ovirt.vdsm.jsonrpc.client.reactors.Reactor] (SSL Stomp Reactor) []
> Internal server error: null
>
> 2017-01-06 16:27:49,663 ERROR
> [org.ovirt.engine.core.vdsbroker.vdsbroker.GetAllVmStatsVDSCommand]
> (DefaultQuartzScheduler6) [54a978b8] Command
> 'GetAllVmStatsVDSCommand(HostName = prd-rbs-ovirt-kvm06-poa.rbs.com.br,
> VdsIdAndVdsVDSCommandParametersBase:{runAsync='true',
> hostId='23b750ed-4ba6-4782-a9bd-c018c0f36e44', vds='Host[
> prd-rbs-ovirt-kvm06-poa.rbs.com.br,23b750ed-4ba6-4782-a9bd-c018c0f36e44]'})'
> execution failed: VDSGenericException: VDSNetworkException: Heartbeat
> exceeded
>
> 2017-01-06 16:27:49,663 ERROR
> [org.ovirt.engine.core.vdsbroker.vdsbroker.GetAllVmStatsVDSCommand]
> (DefaultQuartzScheduler9) [27c382ce] Command
> 'GetAllVmStatsVDSCommand(HostName = prd-rbs-ovirt-kvm14-poa.rbs.com.br,
> VdsIdAndVdsVDSCommandParametersBase:{runAsync='true',
> hostId='60997c2f-3ba0-4911-928a-b70f640f30b8', vds='Host[
> prd-rbs-ovirt-kvm14-poa.rbs.com.br,60997c2f-3ba0-4911-928a-b70f640f30b8]'})'
> execution failed: VDSGenericException: VDSNetworkException: Heartbeat
> exceeded
>
> 2017-01-06 16:27:49,666 ERROR
> [org.ovirt.engine.core.vdsbroker.vdsbroker.GetAllVmStatsVDSCommand]
> (DefaultQuartzScheduler6) [54a978b8] Command
> 'GetAllVmStatsVDSCommand(HostName = prd-rbs-ovirt-kvm03-poa.rbs.com.br,
> VdsIdAndVdsVDSCommandParametersBase:{runAsync='true',
> hostId='f7842244-646c-400a-9736-f8d4aa9b1cef', vds='Host[
> prd-rbs-ovirt-kvm03-poa.rbs.com.br,f7842244-646c-400a-9736-f8d4aa9b1cef]'})'
> execution failed: VDSGenericException: VDSNetworkException: Heartbeat
> exceeded
>
> 2017-01-06 16:27:49,666 ERROR
> [org.ovirt.engine.core.vdsbroker.vdsbroker.GetAllVmStatsVDSCommand]
> (DefaultQuartzScheduler7) [68fa51f0] Command
> 'GetAllVmStatsVDSCommand(HostName = prd-rbs-ovirt-kvm11-poa.rbs.com.br,
> VdsIdAndVdsVDSCommandParametersBase:{runAsync='true',
> hostId='d3c5cf1f-b203-4818-9767-5249387e3553', vds='Host[
> prd-rbs-ovirt-kvm11-poa.rbs.com.br,d3c5cf1f-b203-4818-9767-5249387e3553]'})'
> execution failed: VDSGenericException: VDSNetworkException: Heartbeat
> exceeded
>
> 2017-01-06 16:27:49,667 ERROR
> [org.ovirt.engine.core.vdsbroker.vdsbroker.GetAllVmStatsVDSCommand]
> (DefaultQuartzScheduler11) [2d5a3838] Command
> 'GetAllVmStatsVDSCommand(HostName = prd-rbs-ovirt-kvm19-poa.rbs.com.br,
> VdsIdAndVdsVDSCommandParametersBase:{runAsync='true',
> hostId='887b6e35-1fd1-4cd6-9e78-05bcab12a417', vds='Host[
> prd-rbs-ovirt-kvm19-poa.rbs.com.br,887b6e35-1fd1-4cd6-9e78-05bcab12a417]'})'
> execution failed: VDSGenericException: VDSNetworkException: Heartbeat
> exceeded
>
> 2017-01-06 16:27:49,682 ERROR
> [org.ovirt.vdsm.jsonrpc.client.JsonRpcClient] (ResponseWorker) [] Not able
> to update response for "edd8b375-4598-4671-a2cf-0d62fc9bdd0a"
>
>
>
>
>
> Em sex, 6 de jan de 2017 às 15:16, Rogério Ceni Coelho <
> rogeriocenicoelho at gmail.com> escreveu:
>
> After apply the new vdsm-json package ? Yes, restart hole vm. :-(
>
>
>
> [root at prd-rbs-ovirt01-poa ~]# rpm -q
> vdsm-jsonrpc-java-1.2.10-1.20161211091707.gitaf70e3f.el7.centos.noarch
>
> vdsm-jsonrpc-java-1.2.10-1.20161211091707.gitaf70e3f.el7.centos.noarch
>
>
>
> [root at prd-rbs-ovirt01-poa ~]# rpm -qi
> vdsm-jsonrpc-java-1.2.10-1.20161211091707.gitaf70e3f.el7.centos.noarch
>
> Name : vdsm-jsonrpc-java
>
> Version : 1.2.10
>
> Release : 1.20161211091707.gitaf70e3f.el7.centos
>
> Architecture: noarch
>
> Install Date: Fri Jan 6 09:57:25 2017
>
> Group : Development/Libraries
>
> Size : 144305
>
> License : LGPLv2+
>
> Signature : (none)
>
> Source RPM :
> vdsm-jsonrpc-java-1.2.10-1.20161211091707.gitaf70e3f.el7.centos.src.rpm
>
> Build Date : Sun Dec 11 07:17:40 2016
>
> Build Host : vm0065.workers-phx.ovirt.org
>
> Relocations : (not relocatable)
>
> URL : http://www.ovirt.org
>
> Summary : JsonRpc java client (vdsm-jsonrpc-java) for oVirt
>
> Description :
>
> vdsm jsonrpc java
>
> [root at prd-rbs-ovirt01-poa ~]# uptime
>
> 15:15:11 up 19 min, 1 user, load average: 0.50, 0.46, 0.37
>
> [root at prd-rbs-ovirt01-poa ~]# last | head
>
> rogerio_ pts/0 10.40.134.150 Fri Jan 6 15:06 still logged in
>
>
> reboot system boot 3.10.0-514.2.2.e Fri Jan 6 14:56 - 15:15 (00:19)
>
>
> rogerio_ pts/0 10.40.134.150 Fri Jan 6 14:46 - 14:55 (00:08)
>
>
> reboot system boot 3.10.0-514.2.2.e Fri Jan 6 10:11 - 14:55 (04:44)
>
>
> rogerio_ pts/0 10.40.134.150 Fri Jan 6 09:56 - 10:10 (00:14)
>
>
> reboot system boot 3.10.0-514.2.2.e Fri Jan 6 09:55 - 10:11 (00:15)
>
>
> rogerio_ pts/0 10.40.134.150 Fri Jan 6 09:53 - 09:53 (00:00)
>
>
> reboot system boot 3.10.0-514.2.2.e Thu Jan 5 10:00 - 10:11 (1+00:10)
>
>
> rogerio_ pts/0 10.40.134.150 Thu Jan 5 09:59 - 09:59 (00:00)
>
>
> reboot system boot 3.10.0-514.2.2.e Wed Jan 4 14:53 - 10:11 (1+19:18)
>
>
> [root at prd-rbs-ovirt01-poa ~]#
>
>
>
>
>
> Em sex, 6 de jan de 2017 às 15:12, Pavel Gashev <Pax at acronis.com>
> escreveu:
>
> Rogério,
>
>
>
> Did you restart the engine?
>
>
>
> *From: *Rogério Ceni Coelho <rogeriocenicoelho at gmail.com>
> *Date: *Friday 6 January 2017 at 19:56
> *To: *Pavel Gashev <Pax at acronis.com>, users <users at ovirt.org>
> *Subject: *Re: [ovirt-users] oVirt Engine 4.0.5 and CentOS 7.3 Instability
>
>
>
> Pavel and oVirt Admins,
>
>
>
> Problem still the same ... Take a look ...
>
>
>
> *Error! Filename not specified.*
>
> [root at prd-rbs-ovirt01-poa ~]# tail -f /var/log/ovirt-engine/engine.log
>
> 2017-01-06 14:24:48,145 WARN [org.ovirt.engine.core.vdsbroker.VdsManager]
> (org.ovirt.thread.pool-6-thread-34) [] Host '
> prd-rbs-ovirt-kvm19-poa.rbs.com.br' is not responding. It will stay in
> Connecting state for a grace period of 62 seconds and after that an attempt
> to fence the host will be issued.
>
> 2017-01-06 14:24:48,168 WARN
> [org.ovirt.engine.core.dal.dbbroker.auditloghandling.AuditLogDirector]
> (org.ovirt.thread.pool-6-thread-34) [] Correlation ID: null, Call Stack:
> null, Custom Event ID: -1, Message: Host
> prd-rbs-ovirt-kvm19-poa.rbs.com.br is not responding. It will stay in
> Connecting state for a grace period of 62 seconds and after that an attempt
> to fence the host will be issued.
>
> 2017-01-06 14:26:00,521 ERROR
> [org.ovirt.engine.core.vdsbroker.vdsbroker.GetAllVmStatsVDSCommand]
> (DefaultQuartzScheduler17) [] Command 'GetAllVmStatsVDSCommand(HostName =
> prd-rbs-ovirt-kvm07-poa.rbs.com.br,
> VdsIdAndVdsVDSCommandParametersBase:{runAsync='true',
> hostId='1f128680-6152-4273-bb1d-8be545b43461', vds='Host[
> prd-rbs-ovirt-kvm07-poa.rbs.com.br,1f128680-6152-4273-bb1d-8be545b43461]'})'
> execution failed: VDSGenericException: VDSNetworkException: Message timeout
> which can be caused by communication issues
>
> 2017-01-06 14:26:00,521 INFO
> [org.ovirt.engine.core.vdsbroker.monitoring.PollVmStatsRefresher]
> (DefaultQuartzScheduler17) [] Failed to fetch vms info for host '
> prd-rbs-ovirt-kvm07-poa.rbs.com.br' - skipping VMs monitoring.
>
> 2017-01-06 14:26:00,521 WARN [org.ovirt.engine.core.vdsbroker.VdsManager]
> (org.ovirt.thread.pool-6-thread-27) [] Host '
> prd-rbs-ovirt-kvm07-poa.rbs.com.br' is not responding. It will stay in
> Connecting state for a grace period of 62 seconds and after that an attempt
> to fence the host will be issued.
>
> 2017-01-06 14:26:00,545 WARN
> [org.ovirt.engine.core.dal.dbbroker.auditloghandling.AuditLogDirector]
> (org.ovirt.thread.pool-6-thread-27) [] Correlation ID: null, Call Stack:
> null, Custom Event ID: -1, Message: Host
> prd-rbs-ovirt-kvm07-poa.rbs.com.br is not responding. It will stay in
> Connecting state for a grace period of 62 seconds and after that an attempt
> to fence the host will be issued.
>
> 2017-01-06 14:44:37,136 INFO
> [org.ovirt.engine.core.sso.utils.AuthenticationUtils] (default task-97) []
> User adm_coelho at rbs.net successfully logged in with scopes:
> ovirt-app-admin ovirt-app-api ovirt-app-portal
> ovirt-ext=auth:sequence-priority=~ ovirt-ext=revoke:revoke-all
> ovirt-ext=token-info:authz-search ovirt-ext=token-info:public-authz-search
> ovirt-ext=token-info:validate ovirt-ext=token:password-access
>
> 2017-01-06 14:44:37,245 INFO
> [org.ovirt.engine.core.bll.aaa.CreateUserSessionCommand] (default task-98)
> [316f2664] Running command: CreateUserSessionCommand internal: false.
>
> 2017-01-06 14:44:39,137 INFO
> [org.ovirt.engine.docs.utils.servlet.ContextSensitiveHelpMappingServlet]
> (default task-100) [] Context-sensitive help is not installed. Manual
> directory doesn't exist: /usr/share/ovirt-engine/manual
>
> 2017-01-06 14:44:39,138 ERROR
> [org.ovirt.engine.core.utils.servlet.ServletUtils] (default task-102) []
> Can't read file '/usr/share/ovirt-engine/files/spice/SpiceVersion.txt' for
> request '/ovirt-engine/services/files/spice/SpiceVersion.txt', will send a
> 404 error response.
>
>
>
>
>
>
>
>
>
> Em qui, 5 de jan de 2017 às 17:58, Rogério Ceni Coelho <
> rogeriocenicoelho at gmail.com> escreveu:
>
> Hi Pavel and oVirt Elves,
>
>
>
> Today, I install vdsm-jsonrpc-java. Take a look. Let´s see if problem are
> solve. Thanks again.
>
>
>
> [root at hlg-rbs-ovirt01-poa ~]# rpm -qa | grep -i vdsm-jsonrpc
>
> vdsm-jsonrpc-java-1.2.7-1.el7.centos.noarch
>
> [root at hlg-rbs-ovirt01-poa ~]# wget
> http://resources.ovirt.org/pub/ovirt-4.0-snapshot/rpm/el7/noarch/vdsm-jsonrpc-java-1.2.10-1.20161211091707.gitaf70e3f.el7.centos.noarch.rpm
>
> --2017-01-05 17:52:51--
> http://resources.ovirt.org/pub/ovirt-4.0-snapshot/rpm/el7/noarch/vdsm-jsonrpc-java-1.2.10-1.20161211091707.gitaf70e3f.el7.centos.noarch.rpm
>
> Resolving resources.ovirt.org (resources.ovirt.org)... 66.187.230.28
>
> Connecting to resources.ovirt.org (resources.ovirt.org)|66.187.230.28|:80...
> connected.
>
> HTTP request sent, awaiting response... 200 OK
>
> Length: 123420 (121K) [application/x-rpm]
>
> Saving to:
> 'vdsm-jsonrpc-java-1.2.10-1.20161211091707.gitaf70e3f.el7.centos.noarch.rpm'
>
>
>
> 100%[=====================================================================================================================================================>]
> 123,420 202KB/s in 0.6s
>
>
>
> 2017-01-05 17:52:52 (202 KB/s) -
> 'vdsm-jsonrpc-java-1.2.10-1.20161211091707.gitaf70e3f.el7.centos.noarch.rpm'
> saved [123420/123420]
>
>
>
> [root at hlg-rbs-ovirt01-poa ~]# ll
> vdsm-jsonrpc-java-1.2.10-1.20161211091707.gitaf70e3f.el7.centos.noarch.rpm
>
> -rw-r--r--. 1 root root 123420 Jan 4 22:39
> vdsm-jsonrpc-java-1.2.10-1.20161211091707.gitaf70e3f.el7.centos.noarch.rpm
>
> [root at hlg-rbs-ovirt01-poa ~]# rpm -Uvh
> vdsm-jsonrpc-java-1.2.10-1.20161211091707.gitaf70e3f.el7.centos.noarch.rpm
>
> Preparing... #################################
> [100%]
>
> Updating / installing...
>
> 1:vdsm-jsonrpc-java-1.2.10-1.201612################################# [
> 50%]
>
> Cleaning up / removing...
>
> 2:vdsm-jsonrpc-java-1.2.7-1.el7.cen#################################
> [100%]
>
> [root at hlg-rbs-ovirt01-poa ~]#
>
>
>
>
>
> Em qua, 4 de jan de 2017 às 17:17, Rogério Ceni Coelho <
> rogeriocenicoelho at gmail.com> escreveu:
>
> Wonderfull ... Seems to be the original problem with me on 4.0.5 ... I
> will try this and do some feedback about soon ...
>
>
>
> Thanks !!!!
>
>
>
> Em qua, 4 de jan de 2017 às 17:08, Pavel Gashev <Pax at acronis.com>
> escreveu:
>
> Rogério,
>
>
>
> It looks like https://bugzilla.redhat.com/show_bug.cgi?id=1401585
>
>
>
> Please try to install the following package, and restart the engine.
>
>
> http://resources.ovirt.org/pub/ovirt-4.0-snapshot/rpm/el7/noarch/vdsm-jsonrpc-java-1.2.10-1.20161211091707.gitaf70e3f.el7.centos.noarch.rpm
>
>
>
>
>
> *From: *<users-bounces at ovirt.org> on behalf of Rogério Ceni Coelho <
> rogeriocenicoelho at gmail.com>
> *Date: *Monday 2 January 2017 at 21:51
> *To: *users <users at ovirt.org>
> *Subject: *[ovirt-users] oVirt Engine 4.0.5 and CentOS 7.3 Instability
>
>
>
> Hi oVirt Gurus,
>
>
>
> Happy new year to everyone !!!
>
>
>
> I update oVirt Engine to 4.0.5 from 4.0.4 and Centos to 7.3 from 7.2 last
> week and after that I have instability four times. Every time ovirt engine
> seems to loose communication with one or more node servers like this image
> below. Every time I rebooted oVirt engine server and everything came back
> to normal.
>
>
>
> Anyone with this kind of problem ???
>
>
>
> *Error! Filename not specified.*
>
>
>
> After reboot :
>
>
>
> *Error! Filename not specified.*
>
>
>
>
>
>
-------------- next part --------------
An HTML attachment was scrubbed...
URL: <http://lists.ovirt.org/pipermail/users/attachments/20170106/b0a00e44/attachment-0001.html>
-------------- next part --------------
A non-text attachment was scrubbed...
Name: pasted1
Type: image/png
Size: 301840 bytes
Desc: not available
URL: <http://lists.ovirt.org/pipermail/users/attachments/20170106/b0a00e44/attachment-0002.png>
-------------- next part --------------
A non-text attachment was scrubbed...
Name: pasted2
Type: image/png
Size: 230509 bytes
Desc: not available
URL: <http://lists.ovirt.org/pipermail/users/attachments/20170106/b0a00e44/attachment-0003.png>
More information about the Users
mailing list