1:33 p.m.
On Tue, Mar 19, 2019 at 12:46 PM Juhani Rautiainen
<juhani.rautiainen(a)gmail.com> wrote:
Couldn't find anything that jumps as problem but another post in list
made me check ha-agent logs. This is the reason for reboot:
MainThread::INFO::2019-03-19
12:04:41,262::states::135::ovirt_hosted_engine_ha.agent.hosted_engine.HostedEngine::(score)
Penalizing score by 1600 due to gateway status
MainThread::INFO::2019-03-19
12:04:41,263::hosted_engine::493::ovirt_hosted_engine_ha.agent.hosted_engine.HostedEngine::(_monitoring_loop)
Current state EngineUp (score: 1800)
MainThread::ERROR::2019-03-19
12:04:51,283::states::435::ovirt_hosted_engine_ha.agent.hosted_engine.HostedEngine::(consume)
Host ovirt02.virt.local (id 2) score is significantly better than
local score, shutting down VM on this host
MainThread::INFO::2019-03-19
12:04:51,467::brokerlink::68::ovirt_hosted_engine_ha.lib.brokerlink.BrokerLink::(notify)
Success, was notification of state_transition (EngineUp-EngineStop)
sent? sent
MainThread::INFO::2019-03-19
12:04:51,624::hosted_engine::493::ovirt_hosted_engine_ha.agent.hosted_engine.HostedEngine::(_monitoring_loop)
Current state EngineStop (score: 3400)
So HA-agent does the reboot. Now the question is: What that
'Penalizing score by 1600 due to gateway status' means? Other HA VM's
don't seen to have any problems.
It seems that either our firewall is not responding to pings or
something else is wrong. Looking at the broker.log this can be seen.
Curious thing is that the reboot happens even when ping comes back in
couple of seconds. Is there timeout in ping or does it fire them in
quick succession?
Thread-1::INFO::2019-03-19 12:04:20,244::ping::60::ping.Ping::(action)
Successfully pinged 10.168.8.1
Thread-2::INFO::2019-03-19
12:04:20,567::mgmt_bridge::62::mgmt_bridge.MgmtBridge::(action) Found
bridge ovirtmgmt with ports
Thread-5::INFO::2019-03-19
12:04:24,729::engine_health::242::engine_health.EngineHealth::(_result_from_stats)
VM is up on this host with healthy engine
Thread-2::INFO::2019-03-19
12:04:29,745::mgmt_bridge::62::mgmt_bridge.MgmtBridge::(action) Found
bridge ovirtmgmt with ports
Thread-3::INFO::2019-03-19
12:04:30,166::mem_free::51::mem_free.MemFree::(action) memFree: 340451
Thread-5::INFO::2019-03-19
12:04:34,843::engine_health::242::engine_health.EngineHealth::(_result_from_stats)
VM is up on this host with healthy engine
Thread-2::INFO::2019-03-19
12:04:39,926::mgmt_bridge::62::mgmt_bridge.MgmtBridge::(action) Found
bridge ovirtmgmt with ports
Thread-3::INFO::2019-03-19
12:04:40,287::mem_free::51::mem_free.MemFree::(action) memFree: 340450
Thread-1::WARNING::2019-03-19
12:04:40,389::ping::63::ping.Ping::(action) Failed to ping 10.168.8.1,
(0 out of 5)
Thread-1::INFO::2019-03-19 12:04:43,474::ping::60::ping.Ping::(action)
Successfully pinged 10.168.8.1
Thread-5::INFO::2019-03-19
12:04:44,961::engine_health::242::engine_health.EngineHealth::(_result_from_stats)
VM is up on this host with healthy engine
Thread-2::INFO::2019-03-19
12:04:50,154::mgmt_bridge::62::mgmt_bridge.MgmtBridge::(action) Found
bridge ovirtmgmt with ports
Thread-3::INFO::2019-03-19
12:04:50,415::mem_free::51::mem_free.MemFree::(action) memFree: 340454
Thread-1::INFO::2019-03-19 12:04:51,616::ping::60::ping.Ping::(action)
Successfully pinged 10.168.8.1
Thread-5::INFO::2019-03-19
12:04:55,076::engine_health::242::engine_health.EngineHealth::(_result_from_stats)
VM is up on this host with healthy engine
Thread-4::INFO::2019-03-19
12:04:59,197::cpu_load_no_engine::126::cpu_load_no_engine.CpuLoadNoEngine::(calculate_load)
System load total=0.0247, engine=0.0004, non-engine=0.0243
Thread-2::INFO::2019-03-19
12:05:00,434::mgmt_bridge::62::mgmt_bridge.MgmtBridge::(action) Found
bridge ovirtmgmt with ports
Thread-3::INFO::2019-03-19
12:05:00,541::mem_free::51::mem_free.MemFree::(action) memFree: 340433
Thread-1::INFO::2019-03-19 12:05:01,763::ping::60::ping.Ping::(action)
Successfully pinged 10.168.8.1
Thread-7::INFO::2019-03-19
12:05:06,692::engine_health::203::engine_health.EngineHealth::(_result_from_stats)
VM not running on this host, status Down
Thanks,
Juhani