<div dir="ltr">Here VDSM logs ...<div><br></div><div><br><br><div class="gmail_quote"><div dir="ltr">Em ter, 3 de jan de 2017 às 09:58, Rogério Ceni Coelho <<a href="mailto:rogeriocenicoelho@gmail.com">rogeriocenicoelho@gmail.com</a>> escreveu:<br></div><blockquote class="gmail_quote" style="margin:0 0 0 .8ex;border-left:1px #ccc solid;padding-left:1ex"><div dir="ltr" class="gmail_msg"><span style="color:rgb(33,33,33);font-size:13px" class="gmail_msg">Hi Everyone,</span><div class="gmail_msg" style="color:rgb(33,33,33);font-size:13px"><br class="gmail_msg"></div><div class="gmail_msg" style="color:rgb(33,33,33);font-size:13px">I found a lot of Heartbeat exceeded like below ... I attach some logs ...</div><div class="gmail_msg" style="color:rgb(33,33,33);font-size:13px"><br class="gmail_msg"></div><div class="gmail_msg" style="color:rgb(33,33,33);font-size:13px">I am thinking to do rollback to 4.0.4 and CentOS 7.2 ...</div><div class="gmail_msg" style="color:rgb(33,33,33);font-size:13px"><br class="gmail_msg"></div></div><div dir="ltr" class="gmail_msg"><div class="gmail_msg" style="color:rgb(33,33,33);font-size:13px"><div class="gmail_msg">2016-12-28 03:49:14,962 ERROR [org.ovirt.vdsm.jsonrpc.client.reactors.Reactor] (SSL Stomp Reactor) [] Internal server error: null</div><div class="gmail_msg">2016-12-28 03:49:14,986 INFO [org.ovirt.vdsm.jsonrpc.client.reactors.ReactorClient] (SSL Stomp Reactor) [] Connecting to<span class="m_6107469398387908964inbox-inbox-Apple-converted-space gmail_msg"> </span><a href="http://prd-rbs-ovirt-kvm15-poa.rbs.com.br/10.151.252.235" class="gmail_msg" target="_blank">prd-rbs-ovirt-kvm15-poa.rbs.com.br/10.151.252.235</a></div><div class="gmail_msg">2016-12-28 03:49:14,993 INFO [org.ovirt.vdsm.jsonrpc.client.reactors.ReactorClient] (SSL Stomp Reactor) [] Connecting to<span class="m_6107469398387908964inbox-inbox-Apple-converted-space gmail_msg"> </span><a href="http://prd-rbs-ovirt-kvm06-poa.rbs.com.br/10.151.252.226" class="gmail_msg" target="_blank">prd-rbs-ovirt-kvm06-poa.rbs.com.br/10.151.252.226</a></div><div class="gmail_msg">2016-12-28 03:49:14,997 INFO [org.ovirt.vdsm.jsonrpc.client.reactors.ReactorClient] (SSL Stomp Reactor) [] Connecting to<span class="m_6107469398387908964inbox-inbox-Apple-converted-space gmail_msg"> </span><a href="http://prd-rbs-ovirt-kvm08-poa.rbs.com.br/10.151.252.228" class="gmail_msg" target="_blank">prd-rbs-ovirt-kvm08-poa.rbs.com.br/10.151.252.228</a></div><div class="gmail_msg">2016-12-28 03:49:15,001 INFO [org.ovirt.vdsm.jsonrpc.client.reactors.ReactorClient] (SSL Stomp Reactor) [] Connecting to<span class="m_6107469398387908964inbox-inbox-Apple-converted-space gmail_msg"> </span><a href="http://prd-rbs-ovirt-kvm03-poa.rbs.com.br/10.151.252.30" class="gmail_msg" target="_blank">prd-rbs-ovirt-kvm03-poa.rbs.com.br/10.151.252.30</a></div><div class="gmail_msg">2016-12-28 03:49:15,006 INFO [org.ovirt.vdsm.jsonrpc.client.reactors.ReactorClient] (SSL Stomp Reactor) [] Connecting to<span class="m_6107469398387908964inbox-inbox-Apple-converted-space gmail_msg"> </span><a href="http://prd-rbs-ovirt-kvm17-poa.rbs.com.br/10.151.252.237" class="gmail_msg" target="_blank">prd-rbs-ovirt-kvm17-poa.rbs.com.br/10.151.252.237</a></div><div class="gmail_msg">2016-12-28 03:49:15,012 INFO [org.ovirt.vdsm.jsonrpc.client.reactors.ReactorClient] (SSL Stomp Reactor) [] Connecting to<span class="m_6107469398387908964inbox-inbox-Apple-converted-space gmail_msg"> </span><a href="http://prd-rbs-ovirt-kvm02-poa.rbs.com.br/10.151.252.223" class="gmail_msg" target="_blank">prd-rbs-ovirt-kvm02-poa.rbs.com.br/10.151.252.223</a></div><div class="gmail_msg">2016-12-28 03:49:15,018 ERROR [org.ovirt.engine.core.vdsbroker.vdsbroker.GetAllVmStatsVDSCommand] (DefaultQuartzScheduler11) [36d1c163] Command 'GetAllVmStatsVDSCommand(HostName =<span class="m_6107469398387908964inbox-inbox-Apple-converted-space gmail_msg"> </span><a href="http://prd-rbs-ovirt-kvm07-poa.rbs.com.br/" class="gmail_msg" target="_blank">prd-rbs-ovirt-kvm07-poa.rbs.com.br</a>, VdsIdAndVdsVDSCommandParametersBase:{runAsync='true', hostId='1f128680-6152-4273-bb1d-8be545b43461', vds='Host[<a href="http://prd-rbs-ovirt-kvm07-poa.rbs.com.br/" class="gmail_msg" target="_blank">prd-rbs-ovirt-kvm07-poa.rbs.com.br</a>,1f128680-6152-4273-bb1d-8be545b43461]'})' execution failed: VDSGenericException: VDSNetworkException: Heartbeat exceeded</div></div></div><div dir="ltr" class="gmail_msg"><div class="gmail_msg" style="color:rgb(33,33,33);font-size:13px"><div class="gmail_msg">2016-12-28 03:49:15,018 INFO [org.ovirt.engine.core.vdsbroker.monitoring.PollVmStatsRefresher] (DefaultQuartzScheduler11) [36d1c163] Failed to fetch vms info for host '<a href="http://prd-rbs-ovirt-kvm07-poa.rbs.com.br/" class="gmail_msg" target="_blank">prd-rbs-ovirt-kvm07-poa.rbs.com.br</a>' - skipping VMs monitoring.</div><div class="gmail_msg">2016-12-28 03:49:15,018 WARN [org.ovirt.engine.core.vdsbroker.VdsManager] (org.ovirt.thread.pool-6-thread-8) [36d1c163] Host '<a href="http://prd-rbs-ovirt-kvm07-poa.rbs.com.br/" class="gmail_msg" target="_blank">prd-rbs-ovirt-kvm07-poa.rbs.com.br</a>' is not responding. It will stay in Connecting state for a grace period of 62 seconds and after that an attempt to fence the host will be issued.</div></div></div><div dir="ltr" class="gmail_msg"><div class="gmail_msg" style="color:rgb(33,33,33);font-size:13px"><div class="gmail_msg">2016-12-28 03:49:15,025 ERROR [org.ovirt.engine.core.vdsbroker.vdsbroker.GetAllVmStatsVDSCommand] (DefaultQuartzScheduler9) [13bbd638] Command 'GetAllVmStatsVDSCommand(HostName =<span class="m_6107469398387908964inbox-inbox-Apple-converted-space gmail_msg"> </span><a href="http://prd-rbs-ovirt-kvm10-poa.rbs.com.br/" class="gmail_msg" target="_blank">prd-rbs-ovirt-kvm10-poa.rbs.com.br</a>, VdsIdAndVdsVDSCommandParametersBase:{runAsync='true', hostId='8b4bc7dc-af8c-4415-a2ef-bccc11ddf23a', vds='Host[<a href="http://prd-rbs-ovirt-kvm10-poa.rbs.com.br/" class="gmail_msg" target="_blank">prd-rbs-ovirt-kvm10-poa.rbs.com.br</a>,8b4bc7dc-af8c-4415-a2ef-bccc11ddf23a]'})' execution failed: VDSGenericException: VDSNetworkException: Heartbeat exceeded</div></div></div><div dir="ltr" class="gmail_msg"><div class="gmail_msg" style="color:rgb(33,33,33);font-size:13px"><div class="gmail_msg">2016-12-28 03:49:15,025 INFO [org.ovirt.engine.core.vdsbroker.monitoring.PollVmStatsRefresher] (DefaultQuartzScheduler9) [13bbd638] Failed to fetch vms info for host '<a href="http://prd-rbs-ovirt-kvm10-poa.rbs.com.br/" class="gmail_msg" target="_blank">prd-rbs-ovirt-kvm10-poa.rbs.com.br</a>' - skipping VMs monitoring.</div></div></div><div dir="ltr" class="gmail_msg"><div class="gmail_msg" style="color:rgb(33,33,33);font-size:13px"><div class="gmail_msg">2016-12-28 03:49:15,026 INFO [org.ovirt.vdsm.jsonrpc.client.reactors.ReactorClient] (SSL Stomp Reactor) [] Connecting to<span class="m_6107469398387908964inbox-inbox-Apple-converted-space gmail_msg"> </span><a href="http://prd-rbs-ovirt-kvm05-poa.rbs.com.br/10.151.252.225" class="gmail_msg" target="_blank">prd-rbs-ovirt-kvm05-poa.rbs.com.br/10.151.252.225</a></div><div class="gmail_msg">2016-12-28 03:49:15,027 ERROR [org.ovirt.engine.core.dal.dbbroker.auditloghandling.AuditLogDirector] (DefaultQuartzScheduler32) [5b86105] Correlation ID: null, Call Stack: null, Custom Event ID: -1, Message: VDSM<span class="m_6107469398387908964inbox-inbox-Apple-converted-space gmail_msg"> </span><a href="http://prd-rbs-ovirt-kvm03-poa.rbs.com.br/" class="gmail_msg" target="_blank">prd-rbs-ovirt-kvm03-poa.rbs.com.br</a><span class="m_6107469398387908964inbox-inbox-Apple-converted-space gmail_msg"> </span>command failed: Heartbeat exceeded</div><div class="gmail_msg">2016-12-28 03:49:15,027 ERROR [org.ovirt.engine.core.vdsbroker.vdsbroker.SpmStatusVDSCommand] (DefaultQuartzScheduler32) [5b86105] Command 'SpmStatusVDSCommand(HostName =<span class="m_6107469398387908964inbox-inbox-Apple-converted-space gmail_msg"> </span><a href="http://prd-rbs-ovirt-kvm03-poa.rbs.com.br/" class="gmail_msg" target="_blank">prd-rbs-ovirt-kvm03-poa.rbs.com.br</a>, SpmStatusVDSCommandParameters:{runAsync='true', hostId='f7842244-646c-400a-9736-f8d4aa9b1cef', storagePoolId='98867d75-9c43-46b4-891a-ff3a5eb0f06e'})' execution failed: VDSGenericException: VDSNetworkException: Heartbeat exceeded</div></div></div><div dir="ltr" class="gmail_msg"><div class="gmail_msg" style="color:rgb(33,33,33);font-size:13px"><div class="gmail_msg">2016-12-28 03:49:15,027 WARN [org.ovirt.engine.core.vdsbroker.VdsManager] (org.ovirt.thread.pool-6-thread-10) [5b86105] Host '<a href="http://prd-rbs-ovirt-kvm03-poa.rbs.com.br/" class="gmail_msg" target="_blank">prd-rbs-ovirt-kvm03-poa.rbs.com.br</a>' is not responding. It will stay in Connecting state for a grace period of 81 seconds and after that an attempt to fence the host will be issued.</div><div class="gmail_msg">2016-12-28 03:49:15,029 INFO [org.ovirt.engine.core.bll.storage.pool.SetStoragePoolStatusCommand] (DefaultQuartzScheduler32) [613ae8a1] Running command: SetStoragePoolStatusCommand internal: true. Entities affected : ID: 98867d75-9c43-46b4-891a-ff3a5eb0f06e Type: StoragePool</div><div class="gmail_msg">2016-12-28 03:49:15,030 INFO [org.ovirt.engine.core.vdsbroker.storage.StoragePoolDomainHelper] (DefaultQuartzScheduler32) [613ae8a1] Storage Pool '98867d75-9c43-46b4-891a-ff3a5eb0f06e' - Updating Storage Domain '7b8c9293-f103-401a-93ac-550981837224' status from 'Active' to 'Unknown', reason: null</div><div class="gmail_msg">2016-12-28 03:49:15,030 INFO [org.ovirt.engine.core.vdsbroker.storage.StoragePoolDomainHelper] (DefaultQuartzScheduler32) [613ae8a1] Storage Pool '98867d75-9c43-46b4-891a-ff3a5eb0f06e' - Updating Storage Domain 'cfdbbda4-bd72-4c58-af73-8aa89d62ff01' status from 'Active' to 'Unknown', reason: null</div><div class="gmail_msg">2016-12-28 03:49:15,030 INFO [org.ovirt.engine.core.vdsbroker.storage.StoragePoolDomainHelper] (DefaultQuartzScheduler32) [613ae8a1] Storage Pool '98867d75-9c43-46b4-891a-ff3a5eb0f06e' - Updating Storage Domain '0b5015d2-8f05-44c5-9e5a-d732b0b0e419' status from 'Active' to 'Unknown', reason: null</div></div></div><div dir="ltr" class="gmail_msg"><div class="gmail_msg" style="color:rgb(33,33,33);font-size:13px"><div class="gmail_msg">2016-12-28 03:49:15,031 INFO [org.ovirt.vdsm.jsonrpc.client.reactors.ReactorClient] (SSL Stomp Reactor) [] Connecting to<span class="m_6107469398387908964inbox-inbox-Apple-converted-space gmail_msg"> </span><a href="http://prd-rbs-ovirt-kvm13-poa.rbs.com.br/10.151.252.233" class="gmail_msg" target="_blank">prd-rbs-ovirt-kvm13-poa.rbs.com.br/10.151.252.233</a></div><div class="gmail_msg">2016-12-28 03:49:15,033 ERROR [org.ovirt.engine.core.vdsbroker.vdsbroker.GetAllVmStatsVDSCommand] (DefaultQuartzScheduler21) [b33cc28] Command 'GetAllVmStatsVDSCommand(HostName =<span class="m_6107469398387908964inbox-inbox-Apple-converted-space gmail_msg"> </span><a href="http://prd-rbs-ovirt-kvm19-poa.rbs.com.br/" class="gmail_msg" target="_blank">prd-rbs-ovirt-kvm19-poa.rbs.com.br</a>, VdsIdAndVdsVDSCommandParametersBase:{runAsync='true', hostId='887b6e35-1fd1-4cd6-9e78-05bcab12a417', vds='Host[<a href="http://prd-rbs-ovirt-kvm19-poa.rbs.com.br/" class="gmail_msg" target="_blank">prd-rbs-ovirt-kvm19-poa.rbs.com.br</a>,887b6e35-1fd1-4cd6-9e78-05bcab12a417]'})' execution failed: VDSGenericException: VDSNetworkException: Heartbeat exceeded</div><div class="gmail_msg"><br class="gmail_msg"></div><div class="gmail_msg"><br class="gmail_msg"></div></div><br class="gmail_msg"><div class="gmail_quote gmail_msg"></div></div><div dir="ltr" class="gmail_msg"><div class="gmail_quote gmail_msg"><div dir="ltr" class="gmail_msg">Em seg, 2 de jan de 2017 às 17:18, Yaniv Kaul <<a href="mailto:ykaul@redhat.com" class="gmail_msg" target="_blank">ykaul@redhat.com</a>> escreveu:<br class="gmail_msg"></div></div></div><div dir="ltr" class="gmail_msg"><div class="gmail_quote gmail_msg"><blockquote class="gmail_quote gmail_msg" style="margin:0 0 0 .8ex;border-left:1px #ccc solid;padding-left:1ex"><div dir="ltr" class="gmail_msg"><div class="gmail_extra gmail_msg"><div class="gmail_quote gmail_msg">On Mon, Jan 2, 2017 at 8:51 PM, Rogério Ceni Coelho <span dir="ltr" class="gmail_msg"><<a href="mailto:rogeriocenicoelho@gmail.com" class="gmail_msg" target="_blank">rogeriocenicoelho@gmail.com</a>></span> wrote:<br class="gmail_msg"><blockquote class="gmail_quote gmail_msg" style="margin:0 0 0 .8ex;border-left:1px #ccc solid;padding-left:1ex"><div dir="ltr" class="gmail_msg"><p class="MsoNormal gmail_msg">Hi oVirt Gurus,</p>
<p class="MsoNormal gmail_msg"><br class="gmail_msg"></p><p class="MsoNormal gmail_msg">Happy new
year to everyone !!!</p>
<p class="MsoNormal gmail_msg"><span lang="EN-US" class="gmail_msg"> </span></p>
<p class="MsoNormal gmail_msg"><span lang="EN-US" class="gmail_msg">I update
oVirt Engine to 4.0.5 from 4.0.4 and Centos to 7.3 from 7.2 last week and after
that I have instability four times. Every time ovirt engine seems to loose
communication with one or more node servers like this image below. Every time I
rebooted oVirt engine server and everything came back to normal.</span></p></div></blockquote><div class="gmail_msg"><br class="gmail_msg"></div></div></div></div><div dir="ltr" class="gmail_msg"><div class="gmail_extra gmail_msg"><div class="gmail_quote gmail_msg"><div class="gmail_msg">It'd be great if you could share logs - engine.log from the Engine and vdsm.log from the host(s).</div><div class="gmail_msg">Y.</div><div class="gmail_msg"> </div><blockquote class="gmail_quote gmail_msg" style="margin:0 0 0 .8ex;border-left:1px #ccc solid;padding-left:1ex"></blockquote></div></div></div><div dir="ltr" class="gmail_msg"><div class="gmail_extra gmail_msg"><div class="gmail_quote gmail_msg"><blockquote class="gmail_quote gmail_msg" style="margin:0 0 0 .8ex;border-left:1px #ccc solid;padding-left:1ex"><div dir="ltr" class="gmail_msg">
<p class="MsoNormal gmail_msg"><span lang="EN-US" class="gmail_msg"> </span></p>
<p class="MsoNormal gmail_msg"><span lang="EN-US" class="gmail_msg">Anyone with
this kind of problem ???</span></p>
<p class="MsoNormal gmail_msg"><span lang="EN-US" class="gmail_msg"> </span></p>
<p class="MsoNormal gmail_msg"></p><img src="cid:159608366c3d0cad2201" alt="pasted1" class="" style="max-width: 100%;"><p class="gmail_msg"></p><p class="MsoNormal gmail_msg"><br class="gmail_msg"></p>
<p class="MsoNormal gmail_msg">After reboot :</p>
<p class="MsoNormal gmail_msg"> </p>
<p class="MsoNormal gmail_msg"></p><img src="cid:159608409c5d0cad2212" alt="pasted2" class="" style="max-width: 100%;"><br class="gmail_msg"><p class="gmail_msg"></p><p class="MsoNormal gmail_msg"> <br class="gmail_msg"></p>
<br class="gmail_msg"></div>
<br class="gmail_msg"></blockquote></div></div></div><div dir="ltr" class="gmail_msg"><div class="gmail_extra gmail_msg"><div class="gmail_quote gmail_msg"><blockquote class="gmail_quote gmail_msg" style="margin:0 0 0 .8ex;border-left:1px #ccc solid;padding-left:1ex">_______________________________________________<br class="gmail_msg">
Users mailing list<br class="gmail_msg">
<a href="mailto:Users@ovirt.org" class="gmail_msg" target="_blank">Users@ovirt.org</a><br class="gmail_msg">
<a href="http://lists.ovirt.org/mailman/listinfo/users" rel="noreferrer" class="gmail_msg" target="_blank">http://lists.ovirt.org/mailman/listinfo/users</a><br class="gmail_msg">
<br class="gmail_msg"></blockquote></div><br class="gmail_msg"></div></div>
</blockquote></div></div></blockquote></div></div></div>