6:52 a.m.
--_000_CFEE1CAF46A1194DABFD2C2628D324067A591Dmcexch01mcongroup_
Content-Type: text/plain; charset="iso-8859-1"
Content-Transfer-Encoding: quoted-printable
Dear Experts,
I am experiencing a problem with ovirt, every hour the hosted engine will s=
hutdown and reboot. The engine-status value will move to "unknown stale-dat=
a" every second minute, and then the hosted engine will be again operative =
14 minutes after that. As far as I can see the scores remain in 2400 at all=
times, and seems I have a liveliness check failing, but I am not able to f=
ind why.
Why I have this problem every hour exactly?
Why the liveliness check fails?
I would appreciate if someone can bring some light, I am new to ovirt but I=
really like it so far.
During the period the machine is down I can see this messages on the /var/l=
og/ovirt-hosted-engine-ha/broker.log :
Thread-170803::INFO::2015-07-01 11:06:47,335::listener::134::ovirt_hosted_e=
ngine_ha.broker.listener.ConnectionHandler::(setup) Connection established
Thread-170803::INFO::2015-07-01 11:06:47,342::listener::186::ovirt_hosted_e=
ngine_ha.broker.listener.ConnectionHandler::(handle) Connection closed
Thread-170804::INFO::2015-07-01 11:06:47,343::listener::134::ovirt_hosted_e=
ngine_ha.broker.listener.ConnectionHandler::(setup) Connection established
Thread-170804::INFO::2015-07-01 11:06:47,344::listener::186::ovirt_hosted_e=
ngine_ha.broker.listener.ConnectionHandler::(handle) Connection closed
Thread-170805::INFO::2015-07-01 11:06:47,344::listener::134::ovirt_hosted_e=
ngine_ha.broker.listener.ConnectionHandler::(setup) Connection established
Thread-170805::INFO::2015-07-01 11:06:47,346::listener::186::ovirt_hosted_e=
ngine_ha.broker.listener.ConnectionHandler::(handle) Connection closed
Thread-170806::INFO::2015-07-01 11:06:47,346::listener::134::ovirt_hosted_e=
ngine_ha.broker.listener.ConnectionHandler::(setup) Connection established
Thread-170806::INFO::2015-07-01 11:06:47,348::listener::186::ovirt_hosted_e=
ngine_ha.broker.listener.ConnectionHandler::(handle) Connection closed
Thread-170807::INFO::2015-07-01 11:06:47,348::listener::134::ovirt_hosted_e=
ngine_ha.broker.listener.ConnectionHandler::(setup) Connection established
Thread-170807::INFO::2015-07-01 11:06:47,350::listener::186::ovirt_hosted_e=
ngine_ha.broker.listener.ConnectionHandler::(handle) Connection closed
Thread-7::INFO::2015-07-01 11:06:50,394::cpu_load_no_engine::121::cpu_load_=
no_engine.EngineHealth::(calculate_load) System load total=3D0.0095, engine=
=3D0.0046, non-engine=3D0.0049
Thread-8::WARNING::2015-07-01 11:06:50,464::engine_health::116::engine_heal=
th.CpuLoadNoEngine::(action) bad health status: Hosted Engine is not up!
and here the /var/log/ovirt-hosted-engine-ha/agent.log :
MainThread::INFO::2015-07-01 11:01:04,216::hosted_engine::327::ovirt_hosted=
_engine_ha.agent.hosted_engine.HostedEngine::(start_monitoring) Current sta=
te EngineUp (score: 2400)
MainThread::INFO::2015-07-01 11:01:04,217::hosted_engine::332::ovirt_hosted=
_engine_ha.agent.hosted_engine.HostedEngine::(start_monitoring) Best remote=
host mc-place-compute-02-live.mc.mcon.net (id: 2, score: 240
0)
MainThread::INFO::2015-07-01 11:01:14,682::hosted_engine::327::ovirt_hosted=
_engine_ha.agent.hosted_engine.HostedEngine::(start_monitoring) Current sta=
te EngineUp (score: 2400)
MainThread::INFO::2015-07-01 11:01:14,682::hosted_engine::332::ovirt_hosted=
_engine_ha.agent.hosted_engine.HostedEngine::(start_monitoring) Best remote=
host mc-place-compute-02-live.mc.mcon.net (id: 2, score: 240
0)
MainThread::INFO::2015-07-01 11:01:24,724::states::393::ovirt_hosted_engine=
_ha.agent.hosted_engine.HostedEngine::(consume) Engine vm running on localh=
ost
MainThread::INFO::2015-07-01 11:01:25,174::hosted_engine::327::ovirt_hosted=
_engine_ha.agent.hosted_engine.HostedEngine::(start_monitoring) Current sta=
te EngineUp (score: 2400)
MainThread::INFO::2015-07-01 11:01:25,174::hosted_engine::332::ovirt_hosted=
_engine_ha.agent.hosted_engine.HostedEngine::(start_monitoring) Best remote=
host mc-place-compute-02-live.mc.mcon.net (id: 2, score: 240
0)
MainThread::INFO::2015-07-01 11:01:35,234::brokerlink::111::ovirt_hosted_en=
gine_ha.lib.brokerlink.BrokerLink::(notify) Trying: notify time=3D143571969=
5.23 type=3Dstate_transition detail=3DEngineUp-EngineUpBadHealth ho
stname=3D'mc-place-compute-01-live.mc.mcon.net'
MainThread::INFO::2015-07-01 11:03:42,536::brokerlink::120::ovirt_hosted_en=
gine_ha.lib.brokerlink.BrokerLink::(notify) Success, was notification of st=
ate_transition (EngineUp-EngineUpBadHealth) sent? ignored
MainThread::INFO::2015-07-01 11:03:43,018::hosted_engine::327::ovirt_hosted=
_engine_ha.agent.hosted_engine.HostedEngine::(start_monitoring) Current sta=
te EngineUpBadHealth (score: 2400)
MainThread::INFO::2015-07-01 11:03:43,018::hosted_engine::332::ovirt_hosted=
_engine_ha.agent.hosted_engine.HostedEngine::(start_monitoring) Best remote=
host mc-place-compute-02-live.mc.mcon.net (id: 2, score: 240
0)
MainThread::INFO::2015-07-01 11:03:53,060::state_machine::160::ovirt_hosted=
_engine_ha.agent.hosted_engine.HostedEngine::(refresh) Global metadata: {'m=
aintenance': False}
MainThread::INFO::2015-07-01 11:03:53,060::state_machine::165::ovirt_hosted=
_engine_ha.agent.hosted_engine.HostedEngine::(refresh) Host mc-place-comput=
e-02-live.mc.mcon.net (id 2): {'extra': 'metadata_parse_versi
on=3D1\nmetadata_feature_version=3D1\ntimestamp=3D226246 (Wed Jul 1 11:02:=
11 2015)\nhost-id=3D2\nscore=3D2400\nmaintenance=3DFalse\nstate=3DEngineDow=
n\n', 'hostname': 'mc-place-compute-02-live.mc.mcon.net',
'alive': True, 'h
ost-id': 2, 'engine-status': {'reason': 'vm not running on this
host', 'hea=
lth': 'bad', 'vm': 'down', 'detail':
'unknown'}, 'score': 2400, 'maintenanc=
e': False, 'host-ts': 226246}
MainThread::INFO::2015-07-01 11:03:53,060::state_machine::165::ovirt_hosted=
_engine_ha.agent.hosted_engine.HostedEngine::(refresh) Host mc-place-comput=
e-03-live.mc.mcon.net (id 3): {'extra': 'metadata_parse_versi
on=3D1\nmetadata_feature_version=3D1\ntimestamp=3D226256 (Wed Jul 1 11:02:=
15 2015)\nhost-id=3D3\nscore=3D2400\nmaintenance=3DFalse\nstate=3DEngineDow=
n\n', 'hostname': 'mc-place-compute-03-live.mc.mcon.net',
'alive': True, 'h
ost-id': 3, 'engine-status': {'reason': 'vm not running on this
host', 'hea=
lth': 'bad', 'vm': 'down', 'detail':
'unknown'}, 'score': 2400, 'maintenanc=
e': False, 'host-ts': 226256}
MainThread::INFO::2015-07-01 11:03:53,061::state_machine::165::ovirt_hosted=
_engine_ha.agent.hosted_engine.HostedEngine::(refresh) Host mc-place-comput=
e-04-live.mc.mcon.net (id 4): {'extra': 'metadata_parse_versi
on=3D1\nmetadata_feature_version=3D1\ntimestamp=3D226300 (Wed Jul 1 11:02:=
11 2015)\nhost-id=3D4\nscore=3D2400\nmaintenance=3DFalse\nstate=3DEngineDow=
n\n', 'hostname': 'mc-place-compute-04-live.mc.mcon.net',
'alive': True, 'h
ost-id': 4, 'engine-status': {'reason': 'vm not running on this
host', 'hea=
lth': 'bad', 'vm': 'down', 'detail':
'unknown'}, 'score': 2400, 'maintenanc=
e': False, 'host-ts': 226300}
MainThread::INFO::2015-07-01 11:03:53,061::state_machine::168::ovirt_hosted=
_engine_ha.agent.hosted_engine.HostedEngine::(refresh) Local (id 1): {'engi=
ne-health': {'reason': 'failed liveliness check', 'health':
'
bad', 'vm': 'up', 'detail': 'up'}, 'bridge':
True, 'mem-free': 136637.0, 'm=
aintenance': False, 'cpu-load': 0.0035, 'gateway': True}
MainThread::ERROR::2015-07-01 11:03:53,061::states::562::ovirt_hosted_engin=
e_ha.agent.hosted_engine.HostedEngine::(consume) Engine VM has bad health s=
tatus, timeout in 300 seconds
MainThread::INFO::2015-07-01 11:03:53,081::state_decorators::95::ovirt_host=
ed_engine_ha.agent.hosted_engine.HostedEngine::(check) Timeout set to Wed J=
ul 1 11:08:53 2015 while transitioning <class 'ovirt_hosted_
engine_ha.agent.states.EngineUpBadHealth'> -> <class
'ovirt_hosted_engine_h=
a.agent.states.EngineUpBadHealth'
MainThread::INFO::2015-07-01 11:03:53,530::hosted_engine::327::ovirt_hosted=
_engine_ha.agent.hosted_engine.HostedEngine::(start_monitoring) Current sta=
te EngineUpBadHealth (score: 2400)
MainThread::INFO::2015-07-01 11:03:53,530::hosted_engine::332::ovirt_hosted=
_engine_ha.agent.hosted_engine.HostedEngine::(start_monitoring) Best remote=
host mc-place-compute-02-live.mc.mcon.net (id: 2, score: 240
0)
MainThread::ERROR::2015-07-01 11:04:03,559::states::562::ovirt_hosted_engin=
e_ha.agent.hosted_engine.HostedEngine::(consume) Engine VM has bad health s=
tatus, timeout in 289 seconds
MainThread::INFO::2015-07-01 11:04:03,980::hosted_engine::327::ovirt_hosted=
_engine_ha.agent.hosted_engine.HostedEngine::(start_monitoring) Current sta=
te EngineUpBadHealth (score: 2400)
MainThread::INFO::2015-07-01 11:04:03,980::hosted_engine::332::ovirt_hosted=
_engine_ha.agent.hosted_engine.HostedEngine::(start_monitoring) Best remote=
host mc-place-compute-02-live.mc.mcon.net (id: 2, score: 240
0)
MainThread::ERROR::2015-07-01 11:04:14,007::states::562::ovirt_hosted_engin=
e_ha.agent.hosted_engine.HostedEngine::(consume) Engine VM has bad health s=
tatus, timeout in 279 seconds
MainThread::INFO::2015-07-01 11:04:14,478::hosted_engine::327::ovirt_hosted=
_engine_ha.agent.hosted_engine.HostedEngine::(start_monitoring) Current sta=
te EngineUpBadHealth (score: 2400)
MainThread::INFO::2015-07-01 11:04:14,478::hosted_engine::332::ovirt_hosted=
_engine_ha.agent.hosted_engine.HostedEngine::(start_monitoring) Best remote=
host mc-place-compute-02-live.mc.mcon.net (id: 2, score: 240
0)
MainThread::ERROR::2015-07-01 11:04:24,505::states::562::ovirt_hosted_engin=
e_ha.agent.hosted_engine.HostedEngine::(consume) Engine VM has bad health s=
tatus, timeout in 268 seconds
MainThread::INFO::2015-07-01 11:04:24,994::hosted_engine::327::ovirt_hosted=
_engine_ha.agent.hosted_engine.HostedEngine::(start_monitoring) Current sta=
te EngineUpBadHealth (score: 2400)
MainThread::INFO::2015-07-01 11:04:24,994::hosted_engine::332::ovirt_hosted=
_engine_ha.agent.hosted_engine.HostedEngine::(start_monitoring) Best remote=
host mc-place-compute-02-live.mc.mcon.net (id: 2, score: 240
Best Regards
Carles Cortes Costa
--_000_CFEE1CAF46A1194DABFD2C2628D324067A591Dmcexch01mcongroup_
Content-Type: text/html; charset="iso-8859-1"
Content-Transfer-Encoding: quoted-printable
<html dir=3D"ltr" <head <meta http-equiv=3D"Content-Type"
content=3D"text/html; charset=3Diso-8859-=
1" <style id=3D"owaParaStyle"
type=3D"text/css">P {margin-top:0;margin-bottom:=
0;}</style </head <body ocsi=3D"0" fpstyle=3D"1" <div style=3D"direction: ltr;font-family: Tahoma;color:
#000000;font-size: =
10pt;" <div
class=3D"-x-evo-paragraph" id=3D"-x-evo-input-start"
style=3D"color: r=
gb(46, 52, 54); font-family: Cantarell; font-size: 15px; font-style: normal=
; font-variant: normal; font-weight: normal; letter-spacing: normal; line-h=
eight: normal; orphans: auto; text-align: start; text-indent: 0px; text-tra=
nsform: none; white-space: normal; widows: auto; word-spacing: 0px; -webkit=
-text-stroke-width: 0px;" Dear Experts,</div <div class=3D"-x-evo-paragraph"
id=3D"-x-evo-input-start" style=3D"color: r=
gb(46, 52, 54); font-family: Cantarell; font-size: 15px; font-style: normal=
; font-variant: normal; font-weight: normal; letter-spacing: normal; line-h=
eight: normal; orphans: auto; text-align: start; text-indent: 0px; text-tra=
nsform: none; white-space: normal; widows: auto; word-spacing: 0px; -webkit=
-text-stroke-width: 0px;" <br
</div <div
class=3D"-x-evo-paragraph" id=3D"-x-evo-input-start"
style=3D"color: r=
gb(46, 52, 54); font-family: Cantarell; font-size: 15px; font-style: normal=
; font-variant: normal; font-weight: normal; letter-spacing: normal; line-h=
eight: normal; orphans: auto; text-align: start; text-indent: 0px; text-tra=
nsform: none; white-space: normal; widows: auto; word-spacing: 0px; -webkit=
-text-stroke-width: 0px;" I am experiencing a problem with
ovirt, every hour the hosted engine will s=
hutdown and reboot. The engine-status value will move to "unknown stal=
e-data" every second minute, and then the hosted engine will be again =
operative 14 minutes after that. As far as
I can see the scores remain in 2400 at all times, and seems I have a livel=
iness check failing, but I am not able to find why.</div <div
class=3D"-x-evo-paragraph" id=3D"-x-evo-input-start"
style=3D"color: r=
gb(46, 52, 54); font-family: Cantarell; font-size: 15px; font-style: normal=
; font-variant: normal; font-weight: normal; letter-spacing: normal; line-h=
eight: normal; orphans: auto; text-align: start; text-indent: 0px; text-tra=
nsform: none; white-space: normal; widows: auto; word-spacing: 0px; -webkit=
-text-stroke-width: 0px;" <br
</div <div
class=3D"-x-evo-paragraph" id=3D"-x-evo-input-start"
style=3D"color: r=
gb(46, 52, 54); font-family: Cantarell; font-size: 15px; font-style: normal=
; font-variant: normal; font-weight: normal; letter-spacing: normal; line-h=
eight: normal; orphans: auto; text-align: start; text-indent: 0px; text-tra=
nsform: none; white-space: normal; widows: auto; word-spacing: 0px; -webkit=
-text-stroke-width: 0px;" Why I have this problem every hour
exactly?</div <div
class=3D"-x-evo-paragraph" id=3D"-x-evo-input-start"
style=3D"color: r=
gb(46, 52, 54); font-family: Cantarell; font-size: 15px; font-style: normal=
; font-variant: normal; font-weight: normal; letter-spacing: normal; line-h=
eight: normal; orphans: auto; text-align: start; text-indent: 0px; text-tra=
nsform: none; white-space: normal; widows: auto; word-spacing: 0px; -webkit=
-text-stroke-width: 0px;" Why the liveliness check
fails?</div <div
class=3D"-x-evo-paragraph" id=3D"-x-evo-input-start"
style=3D"color: r=
gb(46, 52, 54); font-family: Cantarell; font-size: 15px; font-style: normal=
; font-variant: normal; font-weight: normal; letter-spacing: normal; line-h=
eight: normal; orphans: auto; text-align: start; text-indent: 0px; text-tra=
nsform: none; white-space: normal; widows: auto; word-spacing: 0px; -webkit=
-text-stroke-width: 0px;" <br
</div <div
class=3D"-x-evo-paragraph" id=3D"-x-evo-input-start"
style=3D"color: r=
gb(46, 52, 54); font-family: Cantarell; font-size: 15px; font-style: normal=
; font-variant: normal; font-weight: normal; letter-spacing: normal; line-h=
eight: normal; orphans: auto; text-align: start; text-indent: 0px; text-tra=
nsform: none; white-space: normal; widows: auto; word-spacing: 0px; -webkit=
-text-stroke-width: 0px;" I would appreciate if someone can
bring some light, I am new to ovirt but I=
really like it so far. </div <div
class=3D"-x-evo-paragraph" id=3D"-x-evo-input-start"
style=3D"color: r=
gb(46, 52, 54); font-family: Cantarell; font-size: 15px; font-style: normal=
; font-variant: normal; font-weight: normal; letter-spacing: normal; line-h=
eight: normal; orphans: auto; text-align: start; text-indent: 0px; text-tra=
nsform: none; white-space: normal; widows: auto; word-spacing: 0px; -webkit=
-text-stroke-width: 0px;" <br
</div <div
class=3D"-x-evo-paragraph" id=3D"-x-evo-input-start"
style=3D"color: r=
gb(46, 52, 54); font-family: Cantarell; font-size: 15px; font-style: normal=
; font-variant: normal; font-weight: normal; letter-spacing: normal; line-h=
eight: normal; orphans: auto; text-align: start; text-indent: 0px; text-tra=
nsform: none; white-space: normal; widows: auto; word-spacing: 0px; -webkit=
-text-stroke-width: 0px;" During the period the machine is
down I can see this messages on the /var/l=
og/ovirt-hosted-engine-ha/broker.log :</div <div
class=3D"-x-evo-paragraph" id=3D"-x-evo-input-start"
style=3D"color: r=
gb(46, 52, 54); font-family: Cantarell; font-size: 15px; font-style: normal=
; font-variant: normal; font-weight: normal; letter-spacing: normal; line-h=
eight: normal; orphans: auto; text-align: start; text-indent: 0px; text-tra=
nsform: none; white-space: normal; widows: auto; word-spacing: 0px; -webkit=
-text-stroke-width: 0px;" <br
</div <div
class=3D"-x-evo-paragraph" style=3D"color: rgb(46, 52, 54); font-famil=
y: Cantarell; font-size: 15px; font-style: normal; font-variant: normal; fo=
nt-weight: normal; letter-spacing: normal; line-height: normal; orphans: au=
to; text-align: start; text-indent: 0px; text-transform: none; white-space:=
normal; widows: auto; word-spacing: 0px; -webkit-text-stroke-width: 0px;" Thread-170803::INFO::2015-07-01
11:06:47,335::listener::134::ovirt_hosted_e=
ngine_ha.broker.listener.ConnectionHandler::(setup) Connection established<=
/div <div class=3D"-x-evo-paragraph"
style=3D"color: rgb(46, 52, 54); font-famil=
y: Cantarell; font-size: 15px; font-style: normal; font-variant: normal; fo=
nt-weight: normal; letter-spacing: normal; line-height: normal; orphans: au=
to; text-align: start; text-indent: 0px; text-transform: none; white-space:=
normal; widows: auto; word-spacing: 0px; -webkit-text-stroke-width: 0px;" Thread-170803::INFO::2015-07-01
11:06:47,342::listener::186::ovirt_hosted_e=
ngine_ha.broker.listener.ConnectionHandler::(handle) Connection closed</div=
<div class=3D"-x-evo-paragraph"
style=3D"color: rgb(46, 52, 54); font-famil=
y: Cantarell; font-size: 15px; font-style: normal; font-variant: normal; fo=
nt-weight: normal; letter-spacing: normal; line-height: normal; orphans: au=
to; text-align: start; text-indent: 0px; text-transform: none; white-space:=
normal; widows: auto; word-spacing: 0px; -webkit-text-stroke-width: 0px;" Thread-170804::INFO::2015-07-01
11:06:47,343::listener::134::ovirt_hosted_e=
ngine_ha.broker.listener.ConnectionHandler::(setup) Connection established<=
/div <div class=3D"-x-evo-paragraph"
style=3D"color: rgb(46, 52, 54); font-famil=
y: Cantarell; font-size: 15px; font-style: normal; font-variant: normal; fo=
nt-weight: normal; letter-spacing: normal; line-height: normal; orphans: au=
to; text-align: start; text-indent: 0px; text-transform: none; white-space:=
normal; widows: auto; word-spacing: 0px; -webkit-text-stroke-width: 0px;" Thread-170804::INFO::2015-07-01
11:06:47,344::listener::186::ovirt_hosted_e=
ngine_ha.broker.listener.ConnectionHandler::(handle) Connection closed</div=
<div class=3D"-x-evo-paragraph"
style=3D"color: rgb(46, 52, 54); font-famil=
y: Cantarell; font-size: 15px; font-style: normal; font-variant: normal; fo=
nt-weight: normal; letter-spacing: normal; line-height: normal; orphans: au=
to; text-align: start; text-indent: 0px; text-transform: none; white-space:=
normal; widows: auto; word-spacing: 0px; -webkit-text-stroke-width: 0px;" Thread-170805::INFO::2015-07-01
11:06:47,344::listener::134::ovirt_hosted_e=
ngine_ha.broker.listener.ConnectionHandler::(setup) Connection established<=
/div <div class=3D"-x-evo-paragraph"
style=3D"color: rgb(46, 52, 54); font-famil=
y: Cantarell; font-size: 15px; font-style: normal; font-variant: normal; fo=
nt-weight: normal; letter-spacing: normal; line-height: normal; orphans: au=
to; text-align: start; text-indent: 0px; text-transform: none; white-space:=
normal; widows: auto; word-spacing: 0px; -webkit-text-stroke-width: 0px;" Thread-170805::INFO::2015-07-01
11:06:47,346::listener::186::ovirt_hosted_e=
ngine_ha.broker.listener.ConnectionHandler::(handle) Connection closed</div=
<div class=3D"-x-evo-paragraph"
style=3D"color: rgb(46, 52, 54); font-famil=
y: Cantarell; font-size: 15px; font-style: normal; font-variant: normal; fo=
nt-weight: normal; letter-spacing: normal; line-height: normal; orphans: au=
to; text-align: start; text-indent: 0px; text-transform: none; white-space:=
normal; widows: auto; word-spacing: 0px; -webkit-text-stroke-width: 0px;" Thread-170806::INFO::2015-07-01
11:06:47,346::listener::134::ovirt_hosted_e=
ngine_ha.broker.listener.ConnectionHandler::(setup) Connection established<=
/div <div class=3D"-x-evo-paragraph"
style=3D"color: rgb(46, 52, 54); font-famil=
y: Cantarell; font-size: 15px; font-style: normal; font-variant: normal; fo=
nt-weight: normal; letter-spacing: normal; line-height: normal; orphans: au=
to; text-align: start; text-indent: 0px; text-transform: none; white-space:=
normal; widows: auto; word-spacing: 0px; -webkit-text-stroke-width: 0px;" Thread-170806::INFO::2015-07-01
11:06:47,348::listener::186::ovirt_hosted_e=
ngine_ha.broker.listener.ConnectionHandler::(handle) Connection closed</div=
<div class=3D"-x-evo-paragraph"
style=3D"color: rgb(46, 52, 54); font-famil=
y: Cantarell; font-size: 15px; font-style: normal; font-variant: normal; fo=
nt-weight: normal; letter-spacing: normal; line-height: normal; orphans: au=
to; text-align: start; text-indent: 0px; text-transform: none; white-space:=
normal; widows: auto; word-spacing: 0px; -webkit-text-stroke-width: 0px;" Thread-170807::INFO::2015-07-01
11:06:47,348::listener::134::ovirt_hosted_e=
ngine_ha.broker.listener.ConnectionHandler::(setup) Connection established<=
/div <div class=3D"-x-evo-paragraph"
style=3D"color: rgb(46, 52, 54); font-famil=
y: Cantarell; font-size: 15px; font-style: normal; font-variant: normal; fo=
nt-weight: normal; letter-spacing: normal; line-height: normal; orphans: au=
to; text-align: start; text-indent: 0px; text-transform: none; white-space:=
normal; widows: auto; word-spacing: 0px; -webkit-text-stroke-width: 0px;" Thread-170807::INFO::2015-07-01
11:06:47,350::listener::186::ovirt_hosted_e=
ngine_ha.broker.listener.ConnectionHandler::(handle) Connection closed</div=
<div class=3D"-x-evo-paragraph"
style=3D"color: rgb(46, 52, 54); font-famil=
y: Cantarell; font-size: 15px; font-style: normal; font-variant: normal; fo=
nt-weight: normal; letter-spacing: normal; line-height: normal; orphans: au=
to; text-align: start; text-indent: 0px; text-transform: none; white-space:=
normal; widows: auto; word-spacing: 0px; -webkit-text-stroke-width: 0px;" Thread-7::INFO::2015-07-01
11:06:50,394::cpu_load_no_engine::121::cpu_load_=
no_engine.EngineHealth::(calculate_load) System load total=3D0.0095, engine=
=3D0.0046, non-engine=3D0.0049</div <div
class=3D"-x-evo-paragraph" style=3D"color: rgb(46, 52, 54); font-famil=
y: Cantarell; font-size: 15px; font-style: normal; font-variant: normal; fo=
nt-weight: normal; letter-spacing: normal; line-height: normal; orphans: au=
to; text-align: start; text-indent: 0px; text-transform: none; white-space:=
normal; widows: auto; word-spacing: 0px; -webkit-text-stroke-width: 0px;" Thread-8::WARNING::2015-07-01
11:06:50,464::engine_health::116::engine_heal=
th.CpuLoadNoEngine::(action) bad health status: Hosted Engine is not up!</d=
iv <div class=3D"-x-evo-paragraph"
style=3D"color: rgb(46, 52, 54); font-famil=
y: Cantarell; font-size: 15px; font-style: normal; font-variant: normal; fo=
nt-weight: normal; letter-spacing: normal; line-height: normal; orphans: au=
to; text-align: start; text-indent: 0px; text-transform: none; white-space:=
normal; widows: auto; word-spacing: 0px; -webkit-text-stroke-width: 0px;" <br </div <div class=3D"-x-evo-paragraph"
style=3D"color: rgb(46, 52, 54); font-famil=
y: Cantarell; font-size: 15px; font-style: normal; font-variant: normal; fo=
nt-weight: normal; letter-spacing: normal; line-height: normal; orphans: au=
to; text-align: start; text-indent: 0px; text-transform: none; white-space:=
normal; widows: auto; word-spacing: 0px; -webkit-text-stroke-width: 0px;" and here the /var/log/ovirt-hosted-engine-ha/agent.log
:</div <div
class=3D"-x-evo-paragraph" style=3D"color: rgb(46, 52, 54); font-famil=
y: Cantarell; font-size: 15px; font-style: normal; font-variant: normal; fo=
nt-weight: normal; letter-spacing: normal; line-height: normal; orphans: au=
to; text-align: start; text-indent: 0px; text-transform: none; white-space:=
normal; widows: auto; word-spacing: 0px; -webkit-text-stroke-width: 0px;" <br </div <div class=3D"-x-evo-paragraph"
style=3D"color: rgb(46, 52, 54); font-famil=
y: Cantarell; font-size: 15px; font-style: normal; font-variant: normal; fo=
nt-weight: normal; letter-spacing: normal; line-height: normal; orphans: au=
to; text-align: start; text-indent: 0px; text-transform: none; white-space:=
normal; widows: auto; word-spacing: 0px; -webkit-text-stroke-width: 0px;" MainThread::INFO::2015-07-01
11:01:04,216::hosted_engine::327::ovirt_hosted=
_engine_ha.agent.hosted_engine.HostedEngine::(start_monitoring) Current sta=
te EngineUp (score: 2400)</div <div
class=3D"-x-evo-paragraph" style=3D"color: rgb(46, 52, 54); font-famil=
y: Cantarell; font-size: 15px; font-style: normal; font-variant: normal; fo=
nt-weight: normal; letter-spacing: normal; line-height: normal; orphans: au=
to; text-align: start; text-indent: 0px; text-transform: none; white-space:=
normal; widows: auto; word-spacing: 0px; -webkit-text-stroke-width: 0px;" MainThread::INFO::2015-07-01
11:01:04,217::hosted_engine::332::ovirt_hosted=
_engine_ha.agent.hosted_engine.HostedEngine::(start_monitoring) Best remote=
host mc-place-compute-02-live.mc.mcon.net (id: 2, score: 240</div <div class=3D"-x-evo-paragraph"
style=3D"color: rgb(46, 52, 54); font-famil=
y: Cantarell; font-size: 15px; font-style: normal; font-variant: normal; fo=
nt-weight: normal; letter-spacing: normal; line-height: normal; orphans: au=
to; text-align: start; text-indent: 0px; text-transform: none; white-space:=
normal; widows: auto; word-spacing: 0px; -webkit-text-stroke-width: 0px;" 0)</div <div
class=3D"-x-evo-paragraph" style=3D"color: rgb(46, 52, 54); font-famil=
y: Cantarell; font-size: 15px; font-style: normal; font-variant: normal; fo=
nt-weight: normal; letter-spacing: normal; line-height: normal; orphans: au=
to; text-align: start; text-indent: 0px; text-transform: none; white-space:=
normal; widows: auto; word-spacing: 0px; -webkit-text-stroke-width: 0px;" MainThread::INFO::2015-07-01
11:01:14,682::hosted_engine::327::ovirt_hosted=
_engine_ha.agent.hosted_engine.HostedEngine::(start_monitoring) Current sta=
te EngineUp (score: 2400)</div <div
class=3D"-x-evo-paragraph" style=3D"color: rgb(46, 52, 54); font-famil=
y: Cantarell; font-size: 15px; font-style: normal; font-variant: normal; fo=
nt-weight: normal; letter-spacing: normal; line-height: normal; orphans: au=
to; text-align: start; text-indent: 0px; text-transform: none; white-space:=
normal; widows: auto; word-spacing: 0px; -webkit-text-stroke-width: 0px;" MainThread::INFO::2015-07-01
11:01:14,682::hosted_engine::332::ovirt_hosted=
_engine_ha.agent.hosted_engine.HostedEngine::(start_monitoring) Best remote=
host mc-place-compute-02-live.mc.mcon.net (id: 2, score: 240</div <div class=3D"-x-evo-paragraph"
style=3D"color: rgb(46, 52, 54); font-famil=
y: Cantarell; font-size: 15px; font-style: normal; font-variant: normal; fo=
nt-weight: normal; letter-spacing: normal; line-height: normal; orphans: au=
to; text-align: start; text-indent: 0px; text-transform: none; white-space:=
normal; widows: auto; word-spacing: 0px; -webkit-text-stroke-width: 0px;" 0)</div <div
class=3D"-x-evo-paragraph" style=3D"color: rgb(46, 52, 54); font-famil=
y: Cantarell; font-size: 15px; font-style: normal; font-variant: normal; fo=
nt-weight: normal; letter-spacing: normal; line-height: normal; orphans: au=
to; text-align: start; text-indent: 0px; text-transform: none; white-space:=
normal; widows: auto; word-spacing: 0px; -webkit-text-stroke-width: 0px;" MainThread::INFO::2015-07-01
11:01:24,724::states::393::ovirt_hosted_engine=
_ha.agent.hosted_engine.HostedEngine::(consume) Engine vm running on localh=
ost</div <div
class=3D"-x-evo-paragraph" style=3D"color: rgb(46, 52, 54); font-famil=
y: Cantarell; font-size: 15px; font-style: normal; font-variant: normal; fo=
nt-weight: normal; letter-spacing: normal; line-height: normal; orphans: au=
to; text-align: start; text-indent: 0px; text-transform: none; white-space:=
normal; widows: auto; word-spacing: 0px; -webkit-text-stroke-width: 0px;" MainThread::INFO::2015-07-01
11:01:25,174::hosted_engine::327::ovirt_hosted=
_engine_ha.agent.hosted_engine.HostedEngine::(start_monitoring) Current sta=
te EngineUp (score: 2400)</div <div
class=3D"-x-evo-paragraph" style=3D"color: rgb(46, 52, 54); font-famil=
y: Cantarell; font-size: 15px; font-style: normal; font-variant: normal; fo=
nt-weight: normal; letter-spacing: normal; line-height: normal; orphans: au=
to; text-align: start; text-indent: 0px; text-transform: none; white-space:=
normal; widows: auto; word-spacing: 0px; -webkit-text-stroke-width: 0px;" MainThread::INFO::2015-07-01
11:01:25,174::hosted_engine::332::ovirt_hosted=
_engine_ha.agent.hosted_engine.HostedEngine::(start_monitoring) Best remote=
host mc-place-compute-02-live.mc.mcon.net (id: 2, score: 240</div <div class=3D"-x-evo-paragraph"
style=3D"color: rgb(46, 52, 54); font-famil=
y: Cantarell; font-size: 15px; font-style: normal; font-variant: normal; fo=
nt-weight: normal; letter-spacing: normal; line-height: normal; orphans: au=
to; text-align: start; text-indent: 0px; text-transform: none; white-space:=
normal; widows: auto; word-spacing: 0px; -webkit-text-stroke-width: 0px;" 0)</div <div
class=3D"-x-evo-paragraph" style=3D"color: rgb(46, 52, 54); font-famil=
y: Cantarell; font-size: 15px; font-style: normal; font-variant: normal; fo=
nt-weight: normal; letter-spacing: normal; line-height: normal; orphans: au=
to; text-align: start; text-indent: 0px; text-transform: none; white-space:=
normal; widows: auto; word-spacing: 0px; -webkit-text-stroke-width: 0px;" MainThread::INFO::2015-07-01
11:01:35,234::brokerlink::111::ovirt_hosted_en=
gine_ha.lib.brokerlink.BrokerLink::(notify) Trying: notify time=3D143571969=
5.23 type=3Dstate_transition detail=3DEngineUp-EngineUpBadHealth ho</div <div class=3D"-x-evo-paragraph"
style=3D"color: rgb(46, 52, 54); font-famil=
y: Cantarell; font-size: 15px; font-style: normal; font-variant: normal; fo=
nt-weight: normal; letter-spacing: normal; line-height: normal; orphans: au=
to; text-align: start; text-indent: 0px; text-transform: none; white-space:=
normal; widows: auto; word-spacing: 0px; -webkit-text-stroke-width: 0px;"
stname=3D'mc-place-compute-01-live.mc.mcon.net'</div <div class=3D"-x-evo-paragraph"
style=3D"color: rgb(46, 52, 54); font-famil=
y: Cantarell; font-size: 15px; font-style: normal; font-variant: normal; fo=
nt-weight: normal; letter-spacing: normal; line-height: normal; orphans: au=
to; text-align: start; text-indent: 0px; text-transform: none; white-space:=
normal; widows: auto; word-spacing: 0px; -webkit-text-stroke-width: 0px;" MainThread::INFO::2015-07-01
11:03:42,536::brokerlink::120::ovirt_hosted_en=
gine_ha.lib.brokerlink.BrokerLink::(notify) Success, was notification of st=
ate_transition (EngineUp-EngineUpBadHealth) sent? ignored</div <div class=3D"-x-evo-paragraph"
style=3D"color: rgb(46, 52, 54); font-famil=
y: Cantarell; font-size: 15px; font-style: normal; font-variant: normal; fo=
nt-weight: normal; letter-spacing: normal; line-height: normal; orphans: au=
to; text-align: start; text-indent: 0px; text-transform: none; white-space:=
normal; widows: auto; word-spacing: 0px; -webkit-text-stroke-width: 0px;" MainThread::INFO::2015-07-01
11:03:43,018::hosted_engine::327::ovirt_hosted=
_engine_ha.agent.hosted_engine.HostedEngine::(start_monitoring) Current sta=
te EngineUpBadHealth (score: 2400)</div <div
class=3D"-x-evo-paragraph" style=3D"color: rgb(46, 52, 54); font-famil=
y: Cantarell; font-size: 15px; font-style: normal; font-variant: normal; fo=
nt-weight: normal; letter-spacing: normal; line-height: normal; orphans: au=
to; text-align: start; text-indent: 0px; text-transform: none; white-space:=
normal; widows: auto; word-spacing: 0px; -webkit-text-stroke-width: 0px;" MainThread::INFO::2015-07-01
11:03:43,018::hosted_engine::332::ovirt_hosted=
_engine_ha.agent.hosted_engine.HostedEngine::(start_monitoring) Best remote=
host mc-place-compute-02-live.mc.mcon.net (id: 2, score: 240</div <div class=3D"-x-evo-paragraph"
style=3D"color: rgb(46, 52, 54); font-famil=
y: Cantarell; font-size: 15px; font-style: normal; font-variant: normal; fo=
nt-weight: normal; letter-spacing: normal; line-height: normal; orphans: au=
to; text-align: start; text-indent: 0px; text-transform: none; white-space:=
normal; widows: auto; word-spacing: 0px; -webkit-text-stroke-width: 0px;" 0)</div <div
class=3D"-x-evo-paragraph" style=3D"color: rgb(46, 52, 54); font-famil=
y: Cantarell; font-size: 15px; font-style: normal; font-variant: normal; fo=
nt-weight: normal; letter-spacing: normal; line-height: normal; orphans: au=
to; text-align: start; text-indent: 0px; text-transform: none; white-space:=
normal; widows: auto; word-spacing: 0px; -webkit-text-stroke-width: 0px;" MainThread::INFO::2015-07-01
11:03:53,060::state_machine::160::ovirt_hosted=
_engine_ha.agent.hosted_engine.HostedEngine::(refresh) Global metadata: {'m=
aintenance': False}</div <div
class=3D"-x-evo-paragraph" style=3D"color: rgb(46, 52, 54); font-famil=
y: Cantarell; font-size: 15px; font-style: normal; font-variant: normal; fo=
nt-weight: normal; letter-spacing: normal; line-height: normal; orphans: au=
to; text-align: start; text-indent: 0px; text-transform: none; white-space:=
normal; widows: auto; word-spacing: 0px; -webkit-text-stroke-width: 0px;" MainThread::INFO::2015-07-01
11:03:53,060::state_machine::165::ovirt_hosted=
_engine_ha.agent.hosted_engine.HostedEngine::(refresh) Host mc-place-comput=
e-02-live.mc.mcon.net (id 2): {'extra': 'metadata_parse_versi</div <div class=3D"-x-evo-paragraph"
style=3D"color: rgb(46, 52, 54); font-famil=
y: Cantarell; font-size: 15px; font-style: normal; font-variant: normal; fo=
nt-weight: normal; letter-spacing: normal; line-height: normal; orphans: au=
to; text-align: start; text-indent: 0px; text-transform: none; white-space:=
normal; widows: auto; word-spacing: 0px; -webkit-text-stroke-width: 0px;" on=3D1\nmetadata_feature_version=3D1\ntimestamp=3D226246 (Wed
Jul &nbs=
p;1 11:02:11 2015)\nhost-id=3D2\nscore=3D2400\nmaintenance=3DFalse\nstate=
=3DEngineDown\n', 'hostname': 'mc-place-compute-02-live.mc.mcon.net',
'aliv=
e': True, 'h</div <div
class=3D"-x-evo-paragraph" style=3D"color: rgb(46, 52, 54); font-famil=
y: Cantarell; font-size: 15px; font-style: normal; font-variant: normal; fo=
nt-weight: normal; letter-spacing: normal; line-height: normal; orphans: au=
to; text-align: start; text-indent: 0px; text-transform: none; white-space:=
normal; widows: auto; word-spacing: 0px; -webkit-text-stroke-width: 0px;" ost-id': 2, 'engine-status': {'reason':
'vm not running on this host', 'hea=
lth': 'bad', 'vm': 'down', 'detail':
'unknown'}, 'score': 2400, 'maintenanc=
e': False, 'host-ts': 226246}</div <div
class=3D"-x-evo-paragraph" style=3D"color: rgb(46, 52, 54); font-famil=
y: Cantarell; font-size: 15px; font-style: normal; font-variant: normal; fo=
nt-weight: normal; letter-spacing: normal; line-height: normal; orphans: au=
to; text-align: start; text-indent: 0px; text-transform: none; white-space:=
normal; widows: auto; word-spacing: 0px; -webkit-text-stroke-width: 0px;" MainThread::INFO::2015-07-01
11:03:53,060::state_machine::165::ovirt_hosted=
_engine_ha.agent.hosted_engine.HostedEngine::(refresh) Host mc-place-comput=
e-03-live.mc.mcon.net (id 3): {'extra': 'metadata_parse_versi</div <div class=3D"-x-evo-paragraph"
style=3D"color: rgb(46, 52, 54); font-famil=
y: Cantarell; font-size: 15px; font-style: normal; font-variant: normal; fo=
nt-weight: normal; letter-spacing: normal; line-height: normal; orphans: au=
to; text-align: start; text-indent: 0px; text-transform: none; white-space:=
normal; widows: auto; word-spacing: 0px; -webkit-text-stroke-width: 0px;" on=3D1\nmetadata_feature_version=3D1\ntimestamp=3D226256 (Wed
Jul &nbs=
p;1 11:02:15 2015)\nhost-id=3D3\nscore=3D2400\nmaintenance=3DFalse\nstate=
=3DEngineDown\n', 'hostname': 'mc-place-compute-03-live.mc.mcon.net',
'aliv=
e': True, 'h</div <div
class=3D"-x-evo-paragraph" style=3D"color: rgb(46, 52, 54); font-famil=
y: Cantarell; font-size: 15px; font-style: normal; font-variant: normal; fo=
nt-weight: normal; letter-spacing: normal; line-height: normal; orphans: au=
to; text-align: start; text-indent: 0px; text-transform: none; white-space:=
normal; widows: auto; word-spacing: 0px; -webkit-text-stroke-width: 0px;" ost-id': 3, 'engine-status': {'reason':
'vm not running on this host', 'hea=
lth': 'bad', 'vm': 'down', 'detail':
'unknown'}, 'score': 2400, 'maintenanc=
e': False, 'host-ts': 226256}</div <div
class=3D"-x-evo-paragraph" style=3D"color: rgb(46, 52, 54); font-famil=
y: Cantarell; font-size: 15px; font-style: normal; font-variant: normal; fo=
nt-weight: normal; letter-spacing: normal; line-height: normal; orphans: au=
to; text-align: start; text-indent: 0px; text-transform: none; white-space:=
normal; widows: auto; word-spacing: 0px; -webkit-text-stroke-width: 0px;" MainThread::INFO::2015-07-01
11:03:53,061::state_machine::165::ovirt_hosted=
_engine_ha.agent.hosted_engine.HostedEngine::(refresh) Host mc-place-comput=
e-04-live.mc.mcon.net (id 4): {'extra': 'metadata_parse_versi</div <div class=3D"-x-evo-paragraph"
style=3D"color: rgb(46, 52, 54); font-famil=
y: Cantarell; font-size: 15px; font-style: normal; font-variant: normal; fo=
nt-weight: normal; letter-spacing: normal; line-height: normal; orphans: au=
to; text-align: start; text-indent: 0px; text-transform: none; white-space:=
normal; widows: auto; word-spacing: 0px; -webkit-text-stroke-width: 0px;" on=3D1\nmetadata_feature_version=3D1\ntimestamp=3D226300 (Wed
Jul &nbs=
p;1 11:02:11 2015)\nhost-id=3D4\nscore=3D2400\nmaintenance=3DFalse\nstate=
=3DEngineDown\n', 'hostname': 'mc-place-compute-04-live.mc.mcon.net',
'aliv=
e': True, 'h</div <div
class=3D"-x-evo-paragraph" style=3D"color: rgb(46, 52, 54); font-famil=
y: Cantarell; font-size: 15px; font-style: normal; font-variant: normal; fo=
nt-weight: normal; letter-spacing: normal; line-height: normal; orphans: au=
to; text-align: start; text-indent: 0px; text-transform: none; white-space:=
normal; widows: auto; word-spacing: 0px; -webkit-text-stroke-width: 0px;" ost-id': 4, 'engine-status': {'reason':
'vm not running on this host', 'hea=
lth': 'bad', 'vm': 'down', 'detail':
'unknown'}, 'score': 2400, 'maintenanc=
e': False, 'host-ts': 226300}</div <div
class=3D"-x-evo-paragraph" style=3D"color: rgb(46, 52, 54); font-famil=
y: Cantarell; font-size: 15px; font-style: normal; font-variant: normal; fo=
nt-weight: normal; letter-spacing: normal; line-height: normal; orphans: au=
to; text-align: start; text-indent: 0px; text-transform: none; white-space:=
normal; widows: auto; word-spacing: 0px; -webkit-text-stroke-width: 0px;" MainThread::INFO::2015-07-01
11:03:53,061::state_machine::168::ovirt_hosted=
_engine_ha.agent.hosted_engine.HostedEngine::(refresh) Local (id 1): {'engi=
ne-health': {'reason': 'failed liveliness check', 'health':
'</div <div
class=3D"-x-evo-paragraph" style=3D"color: rgb(46, 52, 54); font-famil=
y: Cantarell; font-size: 15px; font-style: normal; font-variant: normal; fo=
nt-weight: normal; letter-spacing: normal; line-height: normal; orphans: au=
to; text-align: start; text-indent: 0px; text-transform: none; white-space:=
normal; widows: auto; word-spacing: 0px; -webkit-text-stroke-width: 0px;" bad', 'vm': 'up', 'detail':
'up'}, 'bridge': True, 'mem-free': 136637.0, 'm=
aintenance': False, 'cpu-load': 0.0035, 'gateway': True}</div <div class=3D"-x-evo-paragraph"
style=3D"color: rgb(46, 52, 54); font-famil=
y: Cantarell; font-size: 15px; font-style: normal; font-variant: normal; fo=
nt-weight: normal; letter-spacing: normal; line-height: normal; orphans: au=
to; text-align: start; text-indent: 0px; text-transform: none; white-space:=
normal; widows: auto; word-spacing: 0px; -webkit-text-stroke-width: 0px;" MainThread::ERROR::2015-07-01
11:03:53,061::states::562::ovirt_hosted_engin=
e_ha.agent.hosted_engine.HostedEngine::(consume) Engine VM has bad health s=
tatus, timeout in 300 seconds</div <div
class=3D"-x-evo-paragraph" style=3D"color: rgb(46, 52, 54); font-famil=
y: Cantarell; font-size: 15px; font-style: normal; font-variant: normal; fo=
nt-weight: normal; letter-spacing: normal; line-height: normal; orphans: au=
to; text-align: start; text-indent: 0px; text-transform: none; white-space:=
normal; widows: auto; word-spacing: 0px; -webkit-text-stroke-width: 0px;" MainThread::INFO::2015-07-01
11:03:53,081::state_decorators::95::ovirt_host=
ed_engine_ha.agent.hosted_engine.HostedEngine::(check) Timeout set to Wed J=
ul 1 11:08:53 2015 while transitioning <class
'ovirt_hosted_<=
/div <div class=3D"-x-evo-paragraph"
style=3D"color: rgb(46, 52, 54); font-famil=
y: Cantarell; font-size: 15px; font-style: normal; font-variant: normal; fo=
nt-weight: normal; letter-spacing: normal; line-height: normal; orphans: au=
to; text-align: start; text-indent: 0px; text-transform: none; white-space:=
normal; widows: auto; word-spacing: 0px; -webkit-text-stroke-width: 0px;" engine_ha.agent.states.EngineUpBadHealth'> ->
<class 'ovirt_hosted=
_engine_ha.agent.states.EngineUpBadHealth'></div <div class=3D"-x-evo-paragraph"
style=3D"color: rgb(46, 52, 54); font-famil=
y: Cantarell; font-size: 15px; font-style: normal; font-variant: normal; fo=
nt-weight: normal; letter-spacing: normal; line-height: normal; orphans: au=
to; text-align: start; text-indent: 0px; text-transform: none; white-space:=
normal; widows: auto; word-spacing: 0px; -webkit-text-stroke-width: 0px;" MainThread::INFO::2015-07-01
11:03:53,530::hosted_engine::327::ovirt_hosted=
_engine_ha.agent.hosted_engine.HostedEngine::(start_monitoring) Current sta=
te EngineUpBadHealth (score: 2400)</div <div
class=3D"-x-evo-paragraph" style=3D"color: rgb(46, 52, 54); font-famil=
y: Cantarell; font-size: 15px; font-style: normal; font-variant: normal; fo=
nt-weight: normal; letter-spacing: normal; line-height: normal; orphans: au=
to; text-align: start; text-indent: 0px; text-transform: none; white-space:=
normal; widows: auto; word-spacing: 0px; -webkit-text-stroke-width: 0px;" MainThread::INFO::2015-07-01
11:03:53,530::hosted_engine::332::ovirt_hosted=
_engine_ha.agent.hosted_engine.HostedEngine::(start_monitoring) Best remote=
host mc-place-compute-02-live.mc.mcon.net (id: 2, score: 240</div <div class=3D"-x-evo-paragraph"
style=3D"color: rgb(46, 52, 54); font-famil=
y: Cantarell; font-size: 15px; font-style: normal; font-variant: normal; fo=
nt-weight: normal; letter-spacing: normal; line-height: normal; orphans: au=
to; text-align: start; text-indent: 0px; text-transform: none; white-space:=
normal; widows: auto; word-spacing: 0px; -webkit-text-stroke-width: 0px;" 0)</div <div
class=3D"-x-evo-paragraph" style=3D"color: rgb(46, 52, 54); font-famil=
y: Cantarell; font-size: 15px; font-style: normal; font-variant: normal; fo=
nt-weight: normal; letter-spacing: normal; line-height: normal; orphans: au=
to; text-align: start; text-indent: 0px; text-transform: none; white-space:=
normal; widows: auto; word-spacing: 0px; -webkit-text-stroke-width: 0px;" MainThread::ERROR::2015-07-01
11:04:03,559::states::562::ovirt_hosted_engin=
e_ha.agent.hosted_engine.HostedEngine::(consume) Engine VM has bad health s=
tatus, timeout in 289 seconds</div <div
class=3D"-x-evo-paragraph" style=3D"color: rgb(46, 52, 54); font-famil=
y: Cantarell; font-size: 15px; font-style: normal; font-variant: normal; fo=
nt-weight: normal; letter-spacing: normal; line-height: normal; orphans: au=
to; text-align: start; text-indent: 0px; text-transform: none; white-space:=
normal; widows: auto; word-spacing: 0px; -webkit-text-stroke-width: 0px;" MainThread::INFO::2015-07-01
11:04:03,980::hosted_engine::327::ovirt_hosted=
_engine_ha.agent.hosted_engine.HostedEngine::(start_monitoring) Current sta=
te EngineUpBadHealth (score: 2400)</div <div
class=3D"-x-evo-paragraph" style=3D"color: rgb(46, 52, 54); font-famil=
y: Cantarell; font-size: 15px; font-style: normal; font-variant: normal; fo=
nt-weight: normal; letter-spacing: normal; line-height: normal; orphans: au=
to; text-align: start; text-indent: 0px; text-transform: none; white-space:=
normal; widows: auto; word-spacing: 0px; -webkit-text-stroke-width: 0px;" MainThread::INFO::2015-07-01
11:04:03,980::hosted_engine::332::ovirt_hosted=
_engine_ha.agent.hosted_engine.HostedEngine::(start_monitoring) Best remote=
host mc-place-compute-02-live.mc.mcon.net (id: 2, score: 240</div <div class=3D"-x-evo-paragraph"
style=3D"color: rgb(46, 52, 54); font-famil=
y: Cantarell; font-size: 15px; font-style: normal; font-variant: normal; fo=
nt-weight: normal; letter-spacing: normal; line-height: normal; orphans: au=
to; text-align: start; text-indent: 0px; text-transform: none; white-space:=
normal; widows: auto; word-spacing: 0px; -webkit-text-stroke-width: 0px;" 0)</div <div
class=3D"-x-evo-paragraph" style=3D"color: rgb(46, 52, 54); font-famil=
y: Cantarell; font-size: 15px; font-style: normal; font-variant: normal; fo=
nt-weight: normal; letter-spacing: normal; line-height: normal; orphans: au=
to; text-align: start; text-indent: 0px; text-transform: none; white-space:=
normal; widows: auto; word-spacing: 0px; -webkit-text-stroke-width: 0px;" MainThread::ERROR::2015-07-01
11:04:14,007::states::562::ovirt_hosted_engin=
e_ha.agent.hosted_engine.HostedEngine::(consume) Engine VM has bad health s=
tatus, timeout in 279 seconds</div <div
class=3D"-x-evo-paragraph" style=3D"color: rgb(46, 52, 54); font-famil=
y: Cantarell; font-size: 15px; font-style: normal; font-variant: normal; fo=
nt-weight: normal; letter-spacing: normal; line-height: normal; orphans: au=
to; text-align: start; text-indent: 0px; text-transform: none; white-space:=
normal; widows: auto; word-spacing: 0px; -webkit-text-stroke-width: 0px;" MainThread::INFO::2015-07-01
11:04:14,478::hosted_engine::327::ovirt_hosted=
_engine_ha.agent.hosted_engine.HostedEngine::(start_monitoring) Current sta=
te EngineUpBadHealth (score: 2400)</div <div
class=3D"-x-evo-paragraph" style=3D"color: rgb(46, 52, 54); font-famil=
y: Cantarell; font-size: 15px; font-style: normal; font-variant: normal; fo=
nt-weight: normal; letter-spacing: normal; line-height: normal; orphans: au=
to; text-align: start; text-indent: 0px; text-transform: none; white-space:=
normal; widows: auto; word-spacing: 0px; -webkit-text-stroke-width: 0px;" MainThread::INFO::2015-07-01
11:04:14,478::hosted_engine::332::ovirt_hosted=
_engine_ha.agent.hosted_engine.HostedEngine::(start_monitoring) Best remote=
host mc-place-compute-02-live.mc.mcon.net (id: 2, score: 240</div <div class=3D"-x-evo-paragraph"
style=3D"color: rgb(46, 52, 54); font-famil=
y: Cantarell; font-size: 15px; font-style: normal; font-variant: normal; fo=
nt-weight: normal; letter-spacing: normal; line-height: normal; orphans: au=
to; text-align: start; text-indent: 0px; text-transform: none; white-space:=
normal; widows: auto; word-spacing: 0px; -webkit-text-stroke-width: 0px;" 0)</div <div
class=3D"-x-evo-paragraph" style=3D"color: rgb(46, 52, 54); font-famil=
y: Cantarell; font-size: 15px; font-style: normal; font-variant: normal; fo=
nt-weight: normal; letter-spacing: normal; line-height: normal; orphans: au=
to; text-align: start; text-indent: 0px; text-transform: none; white-space:=
normal; widows: auto; word-spacing: 0px; -webkit-text-stroke-width: 0px;" MainThread::ERROR::2015-07-01
11:04:24,505::states::562::ovirt_hosted_engin=
e_ha.agent.hosted_engine.HostedEngine::(consume) Engine VM has bad health s=
tatus, timeout in 268 seconds</div <div
class=3D"-x-evo-paragraph" style=3D"color: rgb(46, 52, 54); font-famil=
y: Cantarell; font-size: 15px; font-style: normal; font-variant: normal; fo=
nt-weight: normal; letter-spacing: normal; line-height: normal; orphans: au=
to; text-align: start; text-indent: 0px; text-transform: none; white-space:=
normal; widows: auto; word-spacing: 0px; -webkit-text-stroke-width: 0px;" MainThread::INFO::2015-07-01
11:04:24,994::hosted_engine::327::ovirt_hosted=
_engine_ha.agent.hosted_engine.HostedEngine::(start_monitoring) Current sta=
te EngineUpBadHealth (score: 2400)</div <div
class=3D"-x-evo-paragraph" style=3D"color: rgb(46, 52, 54); font-famil=
y: Cantarell; font-size: 15px; font-style: normal; font-variant: normal; fo=
nt-weight: normal; letter-spacing: normal; line-height: normal; orphans: au=
to; text-align: start; text-indent: 0px; text-transform: none; white-space:=
normal; widows: auto; word-spacing: 0px; -webkit-text-stroke-width: 0px;" MainThread::INFO::2015-07-01
11:04:24,994::hosted_engine::332::ovirt_hosted=
_engine_ha.agent.hosted_engine.HostedEngine::(start_monitoring) Best remote=
host mc-place-compute-02-live.mc.mcon.net (id: 2, score: 240</div <br class=3D"Apple-interchange-newline" <div><br <div
style=3D"font-family:Tahoma; font-size:13px" <div
style=3D"font-family:Tahoma; font-size:13px">Best Regards<br <br Carles Cortes Costa<br <br <p><br </p </div </div
</div </div </body
</html
--_000_CFEE1CAF46A1194DABFD2C2628D324067A591Dmcexch01mcongroup_--
11:16 a.m.
This is a multi-part message in MIME format.
--------------080404070003050909080805
Content-Type: text/plain; charset=UTF-8; format=flowed
Content-Transfer-Encoding: 7bit
On 07/01/2015 06:52 AM, Carles Costa wrote:
Dear Experts,
I am experiencing a problem with ovirt, every hour the hosted engine
will shutdown and reboot. The engine-status value will move to
"unknown stale-data" every second minute, and then the hosted engine
will be again operative 14 minutes after that. As far as I can see the
scores remain in 2400 at all times, and seems I have a liveliness
check failing, but I am not able to find why.
Why I have this problem every hour exactly?
Why the liveliness check fails?
I would appreciate if someone can bring some light, I am new to ovirt
but I really like it so far.
Hi Carles, and welcome.
The agent will try to a servlet running on the engine VM in
http://{ENGINE_IP}/OvirtEngineWeb/HealthStatus
Also debug log will help if we can't resolve this - see the conf the
change it
/etc/ovirt-hosted-engine-ha/broker-log.conf
/etc/ovirt-hosted-engine-ha/agent-log.conf
During the period the machine is down I can see this messages on the
/var/log/ovirt-hosted-engine-ha/broker.log :
Thread-170803::INFO::2015-07-01
11:06:47,335::listener::134::ovirt_hosted_engine_ha.broker.listener.ConnectionHandler::(setup)
Connection established
Thread-170803::INFO::2015-07-01
11:06:47,342::listener::186::ovirt_hosted_engine_ha.broker.listener.ConnectionHandler::(handle)
Connection closed
Thread-170804::INFO::2015-07-01
11:06:47,343::listener::134::ovirt_hosted_engine_ha.broker.listener.ConnectionHandler::(setup)
Connection established
Thread-170804::INFO::2015-07-01
11:06:47,344::listener::186::ovirt_hosted_engine_ha.broker.listener.ConnectionHandler::(handle)
Connection closed
Thread-170805::INFO::2015-07-01
11:06:47,344::listener::134::ovirt_hosted_engine_ha.broker.listener.ConnectionHandler::(setup)
Connection established
Thread-170805::INFO::2015-07-01
11:06:47,346::listener::186::ovirt_hosted_engine_ha.broker.listener.ConnectionHandler::(handle)
Connection closed
Thread-170806::INFO::2015-07-01
11:06:47,346::listener::134::ovirt_hosted_engine_ha.broker.listener.ConnectionHandler::(setup)
Connection established
Thread-170806::INFO::2015-07-01
11:06:47,348::listener::186::ovirt_hosted_engine_ha.broker.listener.ConnectionHandler::(handle)
Connection closed
Thread-170807::INFO::2015-07-01
11:06:47,348::listener::134::ovirt_hosted_engine_ha.broker.listener.ConnectionHandler::(setup)
Connection established
Thread-170807::INFO::2015-07-01
11:06:47,350::listener::186::ovirt_hosted_engine_ha.broker.listener.ConnectionHandler::(handle)
Connection closed
Thread-7::INFO::2015-07-01
11:06:50,394::cpu_load_no_engine::121::cpu_load_no_engine.EngineHealth::(calculate_load)
System load total=0.0095, engine=0.0046, non-engine=0.0049
Thread-8::WARNING::2015-07-01
11:06:50,464::engine_health::116::engine_health.CpuLoadNoEngine::(action)
bad health status: Hosted Engine is not up!
and here the /var/log/ovirt-hosted-engine-ha/agent.log :
MainThread::INFO::2015-07-01
11:01:04,216::hosted_engine::327::ovirt_hosted_engine_ha.agent.hosted_engine.HostedEngine::(start_monitoring)
Current state EngineUp (score: 2400)
MainThread::INFO::2015-07-01
11:01:04,217::hosted_engine::332::ovirt_hosted_engine_ha.agent.hosted_engine.HostedEngine::(start_monitoring)
Best remote host mc-place-compute-02-live.mc.mcon.net (id: 2, score: 240
0)
MainThread::INFO::2015-07-01
11:01:14,682::hosted_engine::327::ovirt_hosted_engine_ha.agent.hosted_engine.HostedEngine::(start_monitoring)
Current state EngineUp (score: 2400)
MainThread::INFO::2015-07-01
11:01:14,682::hosted_engine::332::ovirt_hosted_engine_ha.agent.hosted_engine.HostedEngine::(start_monitoring)
Best remote host mc-place-compute-02-live.mc.mcon.net (id: 2, score: 240
0)
MainThread::INFO::2015-07-01
11:01:24,724::states::393::ovirt_hosted_engine_ha.agent.hosted_engine.HostedEngine::(consume)
Engine vm running on localhost
MainThread::INFO::2015-07-01
11:01:25,174::hosted_engine::327::ovirt_hosted_engine_ha.agent.hosted_engine.HostedEngine::(start_monitoring)
Current state EngineUp (score: 2400)
MainThread::INFO::2015-07-01
11:01:25,174::hosted_engine::332::ovirt_hosted_engine_ha.agent.hosted_engine.HostedEngine::(start_monitoring)
Best remote host mc-place-compute-02-live.mc.mcon.net (id: 2, score: 240
0)
MainThread::INFO::2015-07-01
11:01:35,234::brokerlink::111::ovirt_hosted_engine_ha.lib.brokerlink.BrokerLink::(notify)
Trying: notify time=1435719695.23 type=state_transition
detail=EngineUp-EngineUpBadHealth ho
stname='mc-place-compute-01-live.mc.mcon.net'
MainThread::INFO::2015-07-01
11:03:42,536::brokerlink::120::ovirt_hosted_engine_ha.lib.brokerlink.BrokerLink::(notify)
Success, was notification of state_transition
(EngineUp-EngineUpBadHealth) sent? ignored
MainThread::INFO::2015-07-01
11:03:43,018::hosted_engine::327::ovirt_hosted_engine_ha.agent.hosted_engine.HostedEngine::(start_monitoring)
Current state EngineUpBadHealth (score: 2400)
MainThread::INFO::2015-07-01
11:03:43,018::hosted_engine::332::ovirt_hosted_engine_ha.agent.hosted_engine.HostedEngine::(start_monitoring)
Best remote host mc-place-compute-02-live.mc.mcon.net (id: 2, score: 240
0)
MainThread::INFO::2015-07-01
11:03:53,060::state_machine::160::ovirt_hosted_engine_ha.agent.hosted_engine.HostedEngine::(refresh)
Global metadata: {'maintenance': False}
MainThread::INFO::2015-07-01
11:03:53,060::state_machine::165::ovirt_hosted_engine_ha.agent.hosted_engine.HostedEngine::(refresh)
Host mc-place-compute-02-live.mc.mcon.net (id 2): {'extra':
'metadata_parse_versi
on=1\nmetadata_feature_version=1\ntimestamp=226246 (Wed Jul 1
11:02:11
2015)\nhost-id=2\nscore=2400\nmaintenance=False\nstate=EngineDown\n',
'hostname': 'mc-place-compute-02-live.mc.mcon.net', 'alive':
True, 'h
ost-id': 2, 'engine-status': {'reason': 'vm not running on this
host',
'health': 'bad', 'vm': 'down', 'detail':
'unknown'}, 'score': 2400,
'maintenance': False, 'host-ts': 226246}
MainThread::INFO::2015-07-01
11:03:53,060::state_machine::165::ovirt_hosted_engine_ha.agent.hosted_engine.HostedEngine::(refresh)
Host mc-place-compute-03-live.mc.mcon.net (id 3): {'extra':
'metadata_parse_versi
on=1\nmetadata_feature_version=1\ntimestamp=226256 (Wed Jul 1
11:02:15
2015)\nhost-id=3\nscore=2400\nmaintenance=False\nstate=EngineDown\n',
'hostname': 'mc-place-compute-03-live.mc.mcon.net', 'alive':
True, 'h
ost-id': 3, 'engine-status': {'reason': 'vm not running on this
host',
'health': 'bad', 'vm': 'down', 'detail':
'unknown'}, 'score': 2400,
'maintenance': False, 'host-ts': 226256}
MainThread::INFO::2015-07-01
11:03:53,061::state_machine::165::ovirt_hosted_engine_ha.agent.hosted_engine.HostedEngine::(refresh)
Host mc-place-compute-04-live.mc.mcon.net (id 4): {'extra':
'metadata_parse_versi
on=1\nmetadata_feature_version=1\ntimestamp=226300 (Wed Jul 1
11:02:11
2015)\nhost-id=4\nscore=2400\nmaintenance=False\nstate=EngineDown\n',
'hostname': 'mc-place-compute-04-live.mc.mcon.net', 'alive':
True, 'h
ost-id': 4, 'engine-status': {'reason': 'vm not running on this
host',
'health': 'bad', 'vm': 'down', 'detail':
'unknown'}, 'score': 2400,
'maintenance': False, 'host-ts': 226300}
MainThread::INFO::2015-07-01
11:03:53,061::state_machine::168::ovirt_hosted_engine_ha.agent.hosted_engine.HostedEngine::(refresh)
Local (id 1): {'engine-health': {'reason': 'failed liveliness
check',
'health': '
bad', 'vm': 'up', 'detail': 'up'}, 'bridge':
True, 'mem-free':
136637.0, 'maintenance': False, 'cpu-load': 0.0035, 'gateway':
True}
MainThread::ERROR::2015-07-01
11:03:53,061::states::562::ovirt_hosted_engine_ha.agent.hosted_engine.HostedEngine::(consume)
Engine VM has bad health status, timeout in 300 seconds
MainThread::INFO::2015-07-01
11:03:53,081::state_decorators::95::ovirt_hosted_engine_ha.agent.hosted_engine.HostedEngine::(check)
Timeout set to Wed Jul 1 11:08:53 2015 while transitioning <class
'ovirt_hosted_
engine_ha.agent.states.EngineUpBadHealth'> -> <class
'ovirt_hosted_engine_ha.agent.states.EngineUpBadHealth'>
MainThread::INFO::2015-07-01
11:03:53,530::hosted_engine::327::ovirt_hosted_engine_ha.agent.hosted_engine.HostedEngine::(start_monitoring)
Current state EngineUpBadHealth (score: 2400)
MainThread::INFO::2015-07-01
11:03:53,530::hosted_engine::332::ovirt_hosted_engine_ha.agent.hosted_engine.HostedEngine::(start_monitoring)
Best remote host mc-place-compute-02-live.mc.mcon.net (id: 2, score: 240
0)
MainThread::ERROR::2015-07-01
11:04:03,559::states::562::ovirt_hosted_engine_ha.agent.hosted_engine.HostedEngine::(consume)
Engine VM has bad health status, timeout in 289 seconds
MainThread::INFO::2015-07-01
11:04:03,980::hosted_engine::327::ovirt_hosted_engine_ha.agent.hosted_engine.HostedEngine::(start_monitoring)
Current state EngineUpBadHealth (score: 2400)
MainThread::INFO::2015-07-01
11:04:03,980::hosted_engine::332::ovirt_hosted_engine_ha.agent.hosted_engine.HostedEngine::(start_monitoring)
Best remote host mc-place-compute-02-live.mc.mcon.net (id: 2, score: 240
0)
MainThread::ERROR::2015-07-01
11:04:14,007::states::562::ovirt_hosted_engine_ha.agent.hosted_engine.HostedEngine::(consume)
Engine VM has bad health status, timeout in 279 seconds
MainThread::INFO::2015-07-01
11:04:14,478::hosted_engine::327::ovirt_hosted_engine_ha.agent.hosted_engine.HostedEngine::(start_monitoring)
Current state EngineUpBadHealth (score: 2400)
MainThread::INFO::2015-07-01
11:04:14,478::hosted_engine::332::ovirt_hosted_engine_ha.agent.hosted_engine.HostedEngine::(start_monitoring)
Best remote host mc-place-compute-02-live.mc.mcon.net (id: 2, score: 240
0)
MainThread::ERROR::2015-07-01
11:04:24,505::states::562::ovirt_hosted_engine_ha.agent.hosted_engine.HostedEngine::(consume)
Engine VM has bad health status, timeout in 268 seconds
MainThread::INFO::2015-07-01
11:04:24,994::hosted_engine::327::ovirt_hosted_engine_ha.agent.hosted_engine.HostedEngine::(start_monitoring)
Current state EngineUpBadHealth (score: 2400)
MainThread::INFO::2015-07-01
11:04:24,994::hosted_engine::332::ovirt_hosted_engine_ha.agent.hosted_engine.HostedEngine::(start_monitoring)
Best remote host mc-place-compute-02-live.mc.mcon.net (id: 2, score: 240
Best Regards
Carles Cortes Costa
_______________________________________________
Users mailing list
Users(a)ovirt.org
http://lists.ovirt.org/mailman/listinfo/users
--------------080404070003050909080805
Content-Type: text/html; charset=UTF-8
Content-Transfer-Encoding: 8bit
<html>
<head>
<meta content="text/html; charset=ISO-8859-1"
http-equiv="Content-Type">
</head>
<body>
<div class="moz-cite-prefix">On 07/01/2015 06:52 AM, Carles Costa
wrote:<br>
</div>
<blockquote
cite="mid:CFEE1CAF46A1194DABFD2C2628D324067A591D@mc-exch-01.mcon-group.net"
type="cite">
<meta http-equiv="Content-Type" content="text/html;
charset=ISO-8859-1">
<style id="owaParaStyle" type="text/css">P
{margin-top:0;margin-bottom:0;}</style>
<div style="direction: ltr;font-family: Tahoma;color:
#000000;font-size: 10pt;">
<div class="-x-evo-paragraph" id="-x-evo-input-start"
style="color: rgb(46, 52, 54); font-family: Cantarell;
font-size: 15px; font-style: normal; font-variant: normal;
font-weight: normal; letter-spacing: normal; line-height:
normal; orphans: auto; text-align: start; text-indent: 0px;
text-transform: none; white-space: normal; widows: auto;
word-spacing: 0px; -webkit-text-stroke-width: 0px;">
Dear Experts,</div>
<div class="-x-evo-paragraph" id="-x-evo-input-start"
style="color: rgb(46, 52, 54); font-family: Cantarell;
font-size: 15px; font-style: normal; font-variant: normal;
font-weight: normal; letter-spacing: normal; line-height:
normal; orphans: auto; text-align: start; text-indent: 0px;
text-transform: none; white-space: normal; widows: auto;
word-spacing: 0px; -webkit-text-stroke-width: 0px;">
<br>
</div>
<div class="-x-evo-paragraph" id="-x-evo-input-start"
style="color: rgb(46, 52, 54); font-family: Cantarell;
font-size: 15px; font-style: normal; font-variant: normal;
font-weight: normal; letter-spacing: normal; line-height:
normal; orphans: auto; text-align: start; text-indent: 0px;
text-transform: none; white-space: normal; widows: auto;
word-spacing: 0px; -webkit-text-stroke-width: 0px;">
I am experiencing a problem with ovirt, every hour the hosted
engine will shutdown and reboot. The engine-status value will
move to "unknown stale-data" every second minute, and then the
hosted engine will be again operative 14 minutes after that.
As far as I can see the scores remain in 2400 at all times,
and seems I have a liveliness check failing, but I am not able
to find why.</div>
<div class="-x-evo-paragraph" id="-x-evo-input-start"
style="color: rgb(46, 52, 54); font-family: Cantarell;
font-size: 15px; font-style: normal; font-variant: normal;
font-weight: normal; letter-spacing: normal; line-height:
normal; orphans: auto; text-align: start; text-indent: 0px;
text-transform: none; white-space: normal; widows: auto;
word-spacing: 0px; -webkit-text-stroke-width: 0px;">
<br>
</div>
<div class="-x-evo-paragraph" id="-x-evo-input-start"
style="color: rgb(46, 52, 54); font-family: Cantarell;
font-size: 15px; font-style: normal; font-variant: normal;
font-weight: normal; letter-spacing: normal; line-height:
normal; orphans: auto; text-align: start; text-indent: 0px;
text-transform: none; white-space: normal; widows: auto;
word-spacing: 0px; -webkit-text-stroke-width: 0px;">
Why I have this problem every hour exactly?</div>
<div class="-x-evo-paragraph" id="-x-evo-input-start"
style="color: rgb(46, 52, 54); font-family: Cantarell;
font-size: 15px; font-style: normal; font-variant: normal;
font-weight: normal; letter-spacing: normal; line-height:
normal; orphans: auto; text-align: start; text-indent: 0px;
text-transform: none; white-space: normal; widows: auto;
word-spacing: 0px; -webkit-text-stroke-width: 0px;">
Why the liveliness check fails?</div>
<div class="-x-evo-paragraph" id="-x-evo-input-start"
style="color: rgb(46, 52, 54); font-family: Cantarell;
font-size: 15px; font-style: normal; font-variant: normal;
font-weight: normal; letter-spacing: normal; line-height:
normal; orphans: auto; text-align: start; text-indent: 0px;
text-transform: none; white-space: normal; widows: auto;
word-spacing: 0px; -webkit-text-stroke-width: 0px;">
<br>
</div>
<div class="-x-evo-paragraph" id="-x-evo-input-start"
style="color: rgb(46, 52, 54); font-family: Cantarell;
font-size: 15px; font-style: normal; font-variant: normal;
font-weight: normal; letter-spacing: normal; line-height:
normal; orphans: auto; text-align: start; text-indent: 0px;
text-transform: none; white-space: normal; widows: auto;
word-spacing: 0px; -webkit-text-stroke-width: 0px;">
I would appreciate if someone can bring some light, I am new
to ovirt but I really like it so far. </div>
<div class="-x-evo-paragraph" id="-x-evo-input-start"
style="color: rgb(46, 52, 54); font-family: Cantarell;
font-size: 15px; font-style: normal; font-variant: normal;
font-weight: normal; letter-spacing: normal; line-height:
normal; orphans: auto; text-align: start; text-indent: 0px;
text-transform: none; white-space: normal; widows: auto;
word-spacing: 0px; -webkit-text-stroke-width: 0px;">
<br>
</div>
</div>
</blockquote>
<br>
Hi Carles, and welcome.<br>
<br>
The agent will try to a servlet running on the engine VM in
<a class="moz-txt-link-freetext"
href="http://">http://</a>{ENGINE_IP}/OvirtEngineWeb/HealthStatus<br>
<br>
Also debug log will help if we can't resolve this - see the conf
the change it<br>
/etc/ovirt-hosted-engine-ha/broker-log.conf<br>
/etc/ovirt-hosted-engine-ha/agent-log.conf<br>
<br>
<br>
<blockquote
cite="mid:CFEE1CAF46A1194DABFD2C2628D324067A591D@mc-exch-01.mcon-group.net"
type="cite">
<div style="direction: ltr;font-family: Tahoma;color:
#000000;font-size: 10pt;">
<div class="-x-evo-paragraph" id="-x-evo-input-start"
style="color: rgb(46, 52, 54); font-family: Cantarell;
font-size: 15px; font-style: normal; font-variant: normal;
font-weight: normal; letter-spacing: normal; line-height:
normal; orphans: auto; text-align: start; text-indent: 0px;
text-transform: none; white-space: normal; widows: auto;
word-spacing: 0px; -webkit-text-stroke-width: 0px;">
</div>
<div class="-x-evo-paragraph" id="-x-evo-input-start"
style="color: rgb(46, 52, 54); font-family: Cantarell;
font-size: 15px; font-style: normal; font-variant: normal;
font-weight: normal; letter-spacing: normal; line-height:
normal; orphans: auto; text-align: start; text-indent: 0px;
text-transform: none; white-space: normal; widows: auto;
word-spacing: 0px; -webkit-text-stroke-width: 0px;">
During the period the machine is down I can see this messages
on the /var/log/ovirt-hosted-engine-ha/broker.log :</div>
<div class="-x-evo-paragraph" id="-x-evo-input-start"
style="color: rgb(46, 52, 54); font-family: Cantarell;
font-size: 15px; font-style: normal; font-variant: normal;
font-weight: normal; letter-spacing: normal; line-height:
normal; orphans: auto; text-align: start; text-indent: 0px;
text-transform: none; white-space: normal; widows: auto;
word-spacing: 0px; -webkit-text-stroke-width: 0px;">
<br>
</div>
<div class="-x-evo-paragraph" style="color: rgb(46, 52, 54);
font-family: Cantarell; font-size: 15px; font-style: normal;
font-variant: normal; font-weight: normal; letter-spacing:
normal; line-height: normal; orphans: auto; text-align: start;
text-indent: 0px; text-transform: none; white-space: normal;
widows: auto; word-spacing: 0px; -webkit-text-stroke-width:
0px;">
Thread-170803::<a class="moz-txt-link-freetext"
href="INFO::2015-07-01">INFO::2015-07-01</a>
11:06:47,335::listener::134::ovirt_hosted_engine_ha.broker.listener.ConnectionHandler::(setup)
Connection established</div>
<div class="-x-evo-paragraph" style="color: rgb(46, 52, 54);
font-family: Cantarell; font-size: 15px; font-style: normal;
font-variant: normal; font-weight: normal; letter-spacing:
normal; line-height: normal; orphans: auto; text-align: start;
text-indent: 0px; text-transform: none; white-space: normal;
widows: auto; word-spacing: 0px; -webkit-text-stroke-width:
0px;">
Thread-170803::<a class="moz-txt-link-freetext"
href="INFO::2015-07-01">INFO::2015-07-01</a>
11:06:47,342::listener::186::ovirt_hosted_engine_ha.broker.listener.ConnectionHandler::(handle)
Connection closed</div>
<div class="-x-evo-paragraph" style="color: rgb(46, 52, 54);
font-family: Cantarell; font-size: 15px; font-style: normal;
font-variant: normal; font-weight: normal; letter-spacing:
normal; line-height: normal; orphans: auto; text-align: start;
text-indent: 0px; text-transform: none; white-space: normal;
widows: auto; word-spacing: 0px; -webkit-text-stroke-width:
0px;">
Thread-170804::<a class="moz-txt-link-freetext"
href="INFO::2015-07-01">INFO::2015-07-01</a>
11:06:47,343::listener::134::ovirt_hosted_engine_ha.broker.listener.ConnectionHandler::(setup)
Connection established</div>
<div class="-x-evo-paragraph" style="color: rgb(46, 52, 54);
font-family: Cantarell; font-size: 15px; font-style: normal;
font-variant: normal; font-weight: normal; letter-spacing:
normal; line-height: normal; orphans: auto; text-align: start;
text-indent: 0px; text-transform: none; white-space: normal;
widows: auto; word-spacing: 0px; -webkit-text-stroke-width:
0px;">
Thread-170804::<a class="moz-txt-link-freetext"
href="INFO::2015-07-01">INFO::2015-07-01</a>
11:06:47,344::listener::186::ovirt_hosted_engine_ha.broker.listener.ConnectionHandler::(handle)
Connection closed</div>
<div class="-x-evo-paragraph" style="color: rgb(46, 52, 54);
font-family: Cantarell; font-size: 15px; font-style: normal;
font-variant: normal; font-weight: normal; letter-spacing:
normal; line-height: normal; orphans: auto; text-align: start;
text-indent: 0px; text-transform: none; white-space: normal;
widows: auto; word-spacing: 0px; -webkit-text-stroke-width:
0px;">
Thread-170805::<a class="moz-txt-link-freetext"
href="INFO::2015-07-01">INFO::2015-07-01</a>
11:06:47,344::listener::134::ovirt_hosted_engine_ha.broker.listener.ConnectionHandler::(setup)
Connection established</div>
<div class="-x-evo-paragraph" style="color: rgb(46, 52, 54);
font-family: Cantarell; font-size: 15px; font-style: normal;
font-variant: normal; font-weight: normal; letter-spacing:
normal; line-height: normal; orphans: auto; text-align: start;
text-indent: 0px; text-transform: none; white-space: normal;
widows: auto; word-spacing: 0px; -webkit-text-stroke-width:
0px;">
Thread-170805::<a class="moz-txt-link-freetext"
href="INFO::2015-07-01">INFO::2015-07-01</a>
11:06:47,346::listener::186::ovirt_hosted_engine_ha.broker.listener.ConnectionHandler::(handle)
Connection closed</div>
<div class="-x-evo-paragraph" style="color: rgb(46, 52, 54);
font-family: Cantarell; font-size: 15px; font-style: normal;
font-variant: normal; font-weight: normal; letter-spacing:
normal; line-height: normal; orphans: auto; text-align: start;
text-indent: 0px; text-transform: none; white-space: normal;
widows: auto; word-spacing: 0px; -webkit-text-stroke-width:
0px;">
Thread-170806::<a class="moz-txt-link-freetext"
href="INFO::2015-07-01">INFO::2015-07-01</a>
11:06:47,346::listener::134::ovirt_hosted_engine_ha.broker.listener.ConnectionHandler::(setup)
Connection established</div>
<div class="-x-evo-paragraph" style="color: rgb(46, 52, 54);
font-family: Cantarell; font-size: 15px; font-style: normal;
font-variant: normal; font-weight: normal; letter-spacing:
normal; line-height: normal; orphans: auto; text-align: start;
text-indent: 0px; text-transform: none; white-space: normal;
widows: auto; word-spacing: 0px; -webkit-text-stroke-width:
0px;">
Thread-170806::<a class="moz-txt-link-freetext"
href="INFO::2015-07-01">INFO::2015-07-01</a>
11:06:47,348::listener::186::ovirt_hosted_engine_ha.broker.listener.ConnectionHandler::(handle)
Connection closed</div>
<div class="-x-evo-paragraph" style="color: rgb(46, 52, 54);
font-family: Cantarell; font-size: 15px; font-style: normal;
font-variant: normal; font-weight: normal; letter-spacing:
normal; line-height: normal; orphans: auto; text-align: start;
text-indent: 0px; text-transform: none; white-space: normal;
widows: auto; word-spacing: 0px; -webkit-text-stroke-width:
0px;">
Thread-170807::<a class="moz-txt-link-freetext"
href="INFO::2015-07-01">INFO::2015-07-01</a>
11:06:47,348::listener::134::ovirt_hosted_engine_ha.broker.listener.ConnectionHandler::(setup)
Connection established</div>
<div class="-x-evo-paragraph" style="color: rgb(46, 52, 54);
font-family: Cantarell; font-size: 15px; font-style: normal;
font-variant: normal; font-weight: normal; letter-spacing:
normal; line-height: normal; orphans: auto; text-align: start;
text-indent: 0px; text-transform: none; white-space: normal;
widows: auto; word-spacing: 0px; -webkit-text-stroke-width:
0px;">
Thread-170807::<a class="moz-txt-link-freetext"
href="INFO::2015-07-01">INFO::2015-07-01</a>
11:06:47,350::listener::186::ovirt_hosted_engine_ha.broker.listener.ConnectionHandler::(handle)
Connection closed</div>
<div class="-x-evo-paragraph" style="color: rgb(46, 52, 54);
font-family: Cantarell; font-size: 15px; font-style: normal;
font-variant: normal; font-weight: normal; letter-spacing:
normal; line-height: normal; orphans: auto; text-align: start;
text-indent: 0px; text-transform: none; white-space: normal;
widows: auto; word-spacing: 0px; -webkit-text-stroke-width:
0px;">
Thread-7::<a class="moz-txt-link-freetext"
href="INFO::2015-07-01">INFO::2015-07-01</a>
11:06:50,394::cpu_load_no_engine::121::cpu_load_no_engine.EngineHealth::(calculate_load)
System load total=0.0095, engine=0.0046, non-engine=0.0049</div>
<div class="-x-evo-paragraph" style="color: rgb(46, 52, 54);
font-family: Cantarell; font-size: 15px; font-style: normal;
font-variant: normal; font-weight: normal; letter-spacing:
normal; line-height: normal; orphans: auto; text-align: start;
text-indent: 0px; text-transform: none; white-space: normal;
widows: auto; word-spacing: 0px; -webkit-text-stroke-width:
0px;">
Thread-8::WARNING::2015-07-01
11:06:50,464::engine_health::116::engine_health.CpuLoadNoEngine::(action)
bad health status: Hosted Engine is not up!</div>
<div class="-x-evo-paragraph" style="color: rgb(46, 52, 54);
font-family: Cantarell; font-size: 15px; font-style: normal;
font-variant: normal; font-weight: normal; letter-spacing:
normal; line-height: normal; orphans: auto; text-align: start;
text-indent: 0px; text-transform: none; white-space: normal;
widows: auto; word-spacing: 0px; -webkit-text-stroke-width:
0px;">
<br>
</div>
<div class="-x-evo-paragraph" style="color: rgb(46, 52, 54);
font-family: Cantarell; font-size: 15px; font-style: normal;
font-variant: normal; font-weight: normal; letter-spacing:
normal; line-height: normal; orphans: auto; text-align: start;
text-indent: 0px; text-transform: none; white-space: normal;
widows: auto; word-spacing: 0px; -webkit-text-stroke-width:
0px;">
and here the /var/log/ovirt-hosted-engine-ha/agent.log :</div>
<div class="-x-evo-paragraph" style="color: rgb(46, 52, 54);
font-family: Cantarell; font-size: 15px; font-style: normal;
font-variant: normal; font-weight: normal; letter-spacing:
normal; line-height: normal; orphans: auto; text-align: start;
text-indent: 0px; text-transform: none; white-space: normal;
widows: auto; word-spacing: 0px; -webkit-text-stroke-width:
0px;">
<br>
</div>
<div class="-x-evo-paragraph" style="color: rgb(46, 52, 54);
font-family: Cantarell; font-size: 15px; font-style: normal;
font-variant: normal; font-weight: normal; letter-spacing:
normal; line-height: normal; orphans: auto; text-align: start;
text-indent: 0px; text-transform: none; white-space: normal;
widows: auto; word-spacing: 0px; -webkit-text-stroke-width:
0px;">
MainThread::<a class="moz-txt-link-freetext"
href="INFO::2015-07-01">INFO::2015-07-01</a>
11:01:04,216::hosted_engine::327::ovirt_hosted_engine_ha.agent.hosted_engine.HostedEngine::(start_monitoring)
Current state EngineUp (score: 2400)</div>
<div class="-x-evo-paragraph" style="color: rgb(46, 52, 54);
font-family: Cantarell; font-size: 15px; font-style: normal;
font-variant: normal; font-weight: normal; letter-spacing:
normal; line-height: normal; orphans: auto; text-align: start;
text-indent: 0px; text-transform: none; white-space: normal;
widows: auto; word-spacing: 0px; -webkit-text-stroke-width:
0px;">
MainThread::<a class="moz-txt-link-freetext"
href="INFO::2015-07-01">INFO::2015-07-01</a>
11:01:04,217::hosted_engine::332::ovirt_hosted_engine_ha.agent.hosted_engine.HostedEngine::(start_monitoring)
Best remote host mc-place-compute-02-live.mc.mcon.net (id: 2,
score: 240</div>
<div class="-x-evo-paragraph" style="color: rgb(46, 52, 54);
font-family: Cantarell; font-size: 15px; font-style: normal;
font-variant: normal; font-weight: normal; letter-spacing:
normal; line-height: normal; orphans: auto; text-align: start;
text-indent: 0px; text-transform: none; white-space: normal;
widows: auto; word-spacing: 0px; -webkit-text-stroke-width:
0px;">
0)</div>
<div class="-x-evo-paragraph" style="color: rgb(46, 52, 54);
font-family: Cantarell; font-size: 15px; font-style: normal;
font-variant: normal; font-weight: normal; letter-spacing:
normal; line-height: normal; orphans: auto; text-align: start;
text-indent: 0px; text-transform: none; white-space: normal;
widows: auto; word-spacing: 0px; -webkit-text-stroke-width:
0px;">
MainThread::<a class="moz-txt-link-freetext"
href="INFO::2015-07-01">INFO::2015-07-01</a>
11:01:14,682::hosted_engine::327::ovirt_hosted_engine_ha.agent.hosted_engine.HostedEngine::(start_monitoring)
Current state EngineUp (score: 2400)</div>
<div class="-x-evo-paragraph" style="color: rgb(46, 52, 54);
font-family: Cantarell; font-size: 15px; font-style: normal;
font-variant: normal; font-weight: normal; letter-spacing:
normal; line-height: normal; orphans: auto; text-align: start;
text-indent: 0px; text-transform: none; white-space: normal;
widows: auto; word-spacing: 0px; -webkit-text-stroke-width:
0px;">
MainThread::<a class="moz-txt-link-freetext"
href="INFO::2015-07-01">INFO::2015-07-01</a>
11:01:14,682::hosted_engine::332::ovirt_hosted_engine_ha.agent.hosted_engine.HostedEngine::(start_monitoring)
Best remote host mc-place-compute-02-live.mc.mcon.net (id: 2,
score: 240</div>
<div class="-x-evo-paragraph" style="color: rgb(46, 52, 54);
font-family: Cantarell; font-size: 15px; font-style: normal;
font-variant: normal; font-weight: normal; letter-spacing:
normal; line-height: normal; orphans: auto; text-align: start;
text-indent: 0px; text-transform: none; white-space: normal;
widows: auto; word-spacing: 0px; -webkit-text-stroke-width:
0px;">
0)</div>
<div class="-x-evo-paragraph" style="color: rgb(46, 52, 54);
font-family: Cantarell; font-size: 15px; font-style: normal;
font-variant: normal; font-weight: normal; letter-spacing:
normal; line-height: normal; orphans: auto; text-align: start;
text-indent: 0px; text-transform: none; white-space: normal;
widows: auto; word-spacing: 0px; -webkit-text-stroke-width:
0px;">
MainThread::<a class="moz-txt-link-freetext"
href="INFO::2015-07-01">INFO::2015-07-01</a>
11:01:24,724::states::393::ovirt_hosted_engine_ha.agent.hosted_engine.HostedEngine::(consume)
Engine vm running on localhost</div>
<div class="-x-evo-paragraph" style="color: rgb(46, 52, 54);
font-family: Cantarell; font-size: 15px; font-style: normal;
font-variant: normal; font-weight: normal; letter-spacing:
normal; line-height: normal; orphans: auto; text-align: start;
text-indent: 0px; text-transform: none; white-space: normal;
widows: auto; word-spacing: 0px; -webkit-text-stroke-width:
0px;">
MainThread::<a class="moz-txt-link-freetext"
href="INFO::2015-07-01">INFO::2015-07-01</a>
11:01:25,174::hosted_engine::327::ovirt_hosted_engine_ha.agent.hosted_engine.HostedEngine::(start_monitoring)
Current state EngineUp (score: 2400)</div>
<div class="-x-evo-paragraph" style="color: rgb(46, 52, 54);
font-family: Cantarell; font-size: 15px; font-style: normal;
font-variant: normal; font-weight: normal; letter-spacing:
normal; line-height: normal; orphans: auto; text-align: start;
text-indent: 0px; text-transform: none; white-space: normal;
widows: auto; word-spacing: 0px; -webkit-text-stroke-width:
0px;">
MainThread::<a class="moz-txt-link-freetext"
href="INFO::2015-07-01">INFO::2015-07-01</a>
11:01:25,174::hosted_engine::332::ovirt_hosted_engine_ha.agent.hosted_engine.HostedEngine::(start_monitoring)
Best remote host mc-place-compute-02-live.mc.mcon.net (id: 2,
score: 240</div>
<div class="-x-evo-paragraph" style="color: rgb(46, 52, 54);
font-family: Cantarell; font-size: 15px; font-style: normal;
font-variant: normal; font-weight: normal; letter-spacing:
normal; line-height: normal; orphans: auto; text-align: start;
text-indent: 0px; text-transform: none; white-space: normal;
widows: auto; word-spacing: 0px; -webkit-text-stroke-width:
0px;">
0)</div>
<div class="-x-evo-paragraph" style="color: rgb(46, 52, 54);
font-family: Cantarell; font-size: 15px; font-style: normal;
font-variant: normal; font-weight: normal; letter-spacing:
normal; line-height: normal; orphans: auto; text-align: start;
text-indent: 0px; text-transform: none; white-space: normal;
widows: auto; word-spacing: 0px; -webkit-text-stroke-width:
0px;">
MainThread::<a class="moz-txt-link-freetext"
href="INFO::2015-07-01">INFO::2015-07-01</a>
11:01:35,234::brokerlink::111::ovirt_hosted_engine_ha.lib.brokerlink.BrokerLink::(notify)
Trying: notify time=1435719695.23 type=state_transition
detail=EngineUp-EngineUpBadHealth ho</div>
<div class="-x-evo-paragraph" style="color: rgb(46, 52, 54);
font-family: Cantarell; font-size: 15px; font-style: normal;
font-variant: normal; font-weight: normal; letter-spacing:
normal; line-height: normal; orphans: auto; text-align: start;
text-indent: 0px; text-transform: none; white-space: normal;
widows: auto; word-spacing: 0px; -webkit-text-stroke-width:
0px;">
stname='mc-place-compute-01-live.mc.mcon.net'</div>
<div class="-x-evo-paragraph" style="color: rgb(46, 52, 54);
font-family: Cantarell; font-size: 15px; font-style: normal;
font-variant: normal; font-weight: normal; letter-spacing:
normal; line-height: normal; orphans: auto; text-align: start;
text-indent: 0px; text-transform: none; white-space: normal;
widows: auto; word-spacing: 0px; -webkit-text-stroke-width:
0px;">
MainThread::<a class="moz-txt-link-freetext"
href="INFO::2015-07-01">INFO::2015-07-01</a>
11:03:42,536::brokerlink::120::ovirt_hosted_engine_ha.lib.brokerlink.BrokerLink::(notify)
Success, was notification of state_transition
(EngineUp-EngineUpBadHealth) sent? ignored</div>
<div class="-x-evo-paragraph" style="color: rgb(46, 52, 54);
font-family: Cantarell; font-size: 15px; font-style: normal;
font-variant: normal; font-weight: normal; letter-spacing:
normal; line-height: normal; orphans: auto; text-align: start;
text-indent: 0px; text-transform: none; white-space: normal;
widows: auto; word-spacing: 0px; -webkit-text-stroke-width:
0px;">
MainThread::<a class="moz-txt-link-freetext"
href="INFO::2015-07-01">INFO::2015-07-01</a>
11:03:43,018::hosted_engine::327::ovirt_hosted_engine_ha.agent.hosted_engine.HostedEngine::(start_monitoring)
Current state EngineUpBadHealth (score: 2400)</div>
<div class="-x-evo-paragraph" style="color: rgb(46, 52, 54);
font-family: Cantarell; font-size: 15px; font-style: normal;
font-variant: normal; font-weight: normal; letter-spacing:
normal; line-height: normal; orphans: auto; text-align: start;
text-indent: 0px; text-transform: none; white-space: normal;
widows: auto; word-spacing: 0px; -webkit-text-stroke-width:
0px;">
MainThread::<a class="moz-txt-link-freetext"
href="INFO::2015-07-01">INFO::2015-07-01</a>
11:03:43,018::hosted_engine::332::ovirt_hosted_engine_ha.agent.hosted_engine.HostedEngine::(start_monitoring)
Best remote host mc-place-compute-02-live.mc.mcon.net (id: 2,
score: 240</div>
<div class="-x-evo-paragraph" style="color: rgb(46, 52, 54);
font-family: Cantarell; font-size: 15px; font-style: normal;
font-variant: normal; font-weight: normal; letter-spacing:
normal; line-height: normal; orphans: auto; text-align: start;
text-indent: 0px; text-transform: none; white-space: normal;
widows: auto; word-spacing: 0px; -webkit-text-stroke-width:
0px;">
0)</div>
<div class="-x-evo-paragraph" style="color: rgb(46, 52, 54);
font-family: Cantarell; font-size: 15px; font-style: normal;
font-variant: normal; font-weight: normal; letter-spacing:
normal; line-height: normal; orphans: auto; text-align: start;
text-indent: 0px; text-transform: none; white-space: normal;
widows: auto; word-spacing: 0px; -webkit-text-stroke-width:
0px;">
MainThread::<a class="moz-txt-link-freetext"
href="INFO::2015-07-01">INFO::2015-07-01</a>
11:03:53,060::state_machine::160::ovirt_hosted_engine_ha.agent.hosted_engine.HostedEngine::(refresh)
Global metadata: {'maintenance': False}</div>
<div class="-x-evo-paragraph" style="color: rgb(46, 52, 54);
font-family: Cantarell; font-size: 15px; font-style: normal;
font-variant: normal; font-weight: normal; letter-spacing:
normal; line-height: normal; orphans: auto; text-align: start;
text-indent: 0px; text-transform: none; white-space: normal;
widows: auto; word-spacing: 0px; -webkit-text-stroke-width:
0px;">
MainThread::<a class="moz-txt-link-freetext"
href="INFO::2015-07-01">INFO::2015-07-01</a>
11:03:53,060::state_machine::165::ovirt_hosted_engine_ha.agent.hosted_engine.HostedEngine::(refresh)
Host mc-place-compute-02-live.mc.mcon.net (id 2): {'extra':
'metadata_parse_versi</div>
<div class="-x-evo-paragraph" style="color: rgb(46, 52, 54);
font-family: Cantarell; font-size: 15px; font-style: normal;
font-variant: normal; font-weight: normal; letter-spacing:
normal; line-height: normal; orphans: auto; text-align: start;
text-indent: 0px; text-transform: none; white-space: normal;
widows: auto; word-spacing: 0px; -webkit-text-stroke-width:
0px;">
on=1\nmetadata_feature_version=1\ntimestamp=226246 (Wed Jul 1
11:02:11
2015)\nhost-id=2\nscore=2400\nmaintenance=False\nstate=EngineDown\n',
'hostname': 'mc-place-compute-02-live.mc.mcon.net',
'alive':
True, 'h</div>
<div class="-x-evo-paragraph" style="color: rgb(46, 52, 54);
font-family: Cantarell; font-size: 15px; font-style: normal;
font-variant: normal; font-weight: normal; letter-spacing:
normal; line-height: normal; orphans: auto; text-align: start;
text-indent: 0px; text-transform: none; white-space: normal;
widows: auto; word-spacing: 0px; -webkit-text-stroke-width:
0px;">
ost-id': 2, 'engine-status': {'reason': 'vm not running
on
this host', 'health': 'bad', 'vm': 'down',
'detail':
'unknown'}, 'score': 2400, 'maintenance': False,
'host-ts':
226246}</div>
<div class="-x-evo-paragraph" style="color: rgb(46, 52, 54);
font-family: Cantarell; font-size: 15px; font-style: normal;
font-variant: normal; font-weight: normal; letter-spacing:
normal; line-height: normal; orphans: auto; text-align: start;
text-indent: 0px; text-transform: none; white-space: normal;
widows: auto; word-spacing: 0px; -webkit-text-stroke-width:
0px;">
MainThread::<a class="moz-txt-link-freetext"
href="INFO::2015-07-01">INFO::2015-07-01</a>
11:03:53,060::state_machine::165::ovirt_hosted_engine_ha.agent.hosted_engine.HostedEngine::(refresh)
Host mc-place-compute-03-live.mc.mcon.net (id 3): {'extra':
'metadata_parse_versi</div>
<div class="-x-evo-paragraph" style="color: rgb(46, 52, 54);
font-family: Cantarell; font-size: 15px; font-style: normal;
font-variant: normal; font-weight: normal; letter-spacing:
normal; line-height: normal; orphans: auto; text-align: start;
text-indent: 0px; text-transform: none; white-space: normal;
widows: auto; word-spacing: 0px; -webkit-text-stroke-width:
0px;">
on=1\nmetadata_feature_version=1\ntimestamp=226256 (Wed Jul 1
11:02:15
2015)\nhost-id=3\nscore=2400\nmaintenance=False\nstate=EngineDown\n',
'hostname': 'mc-place-compute-03-live.mc.mcon.net',
'alive':
True, 'h</div>
<div class="-x-evo-paragraph" style="color: rgb(46, 52, 54);
font-family: Cantarell; font-size: 15px; font-style: normal;
font-variant: normal; font-weight: normal; letter-spacing:
normal; line-height: normal; orphans: auto; text-align: start;
text-indent: 0px; text-transform: none; white-space: normal;
widows: auto; word-spacing: 0px; -webkit-text-stroke-width:
0px;">
ost-id': 3, 'engine-status': {'reason': 'vm not running
on
this host', 'health': 'bad', 'vm': 'down',
'detail':
'unknown'}, 'score': 2400, 'maintenance': False,
'host-ts':
226256}</div>
<div class="-x-evo-paragraph" style="color: rgb(46, 52, 54);
font-family: Cantarell; font-size: 15px; font-style: normal;
font-variant: normal; font-weight: normal; letter-spacing:
normal; line-height: normal; orphans: auto; text-align: start;
text-indent: 0px; text-transform: none; white-space: normal;
widows: auto; word-spacing: 0px; -webkit-text-stroke-width:
0px;">
MainThread::<a class="moz-txt-link-freetext"
href="INFO::2015-07-01">INFO::2015-07-01</a>
11:03:53,061::state_machine::165::ovirt_hosted_engine_ha.agent.hosted_engine.HostedEngine::(refresh)
Host mc-place-compute-04-live.mc.mcon.net (id 4): {'extra':
'metadata_parse_versi</div>
<div class="-x-evo-paragraph" style="color: rgb(46, 52, 54);
font-family: Cantarell; font-size: 15px; font-style: normal;
font-variant: normal; font-weight: normal; letter-spacing:
normal; line-height: normal; orphans: auto; text-align: start;
text-indent: 0px; text-transform: none; white-space: normal;
widows: auto; word-spacing: 0px; -webkit-text-stroke-width:
0px;">
on=1\nmetadata_feature_version=1\ntimestamp=226300 (Wed Jul 1
11:02:11
2015)\nhost-id=4\nscore=2400\nmaintenance=False\nstate=EngineDown\n',
'hostname': 'mc-place-compute-04-live.mc.mcon.net',
'alive':
True, 'h</div>
<div class="-x-evo-paragraph" style="color: rgb(46, 52, 54);
font-family: Cantarell; font-size: 15px; font-style: normal;
font-variant: normal; font-weight: normal; letter-spacing:
normal; line-height: normal; orphans: auto; text-align: start;
text-indent: 0px; text-transform: none; white-space: normal;
widows: auto; word-spacing: 0px; -webkit-text-stroke-width:
0px;">
ost-id': 4, 'engine-status': {'reason': 'vm not running
on
this host', 'health': 'bad', 'vm': 'down',
'detail':
'unknown'}, 'score': 2400, 'maintenance': False,
'host-ts':
226300}</div>
<div class="-x-evo-paragraph" style="color: rgb(46, 52, 54);
font-family: Cantarell; font-size: 15px; font-style: normal;
font-variant: normal; font-weight: normal; letter-spacing:
normal; line-height: normal; orphans: auto; text-align: start;
text-indent: 0px; text-transform: none; white-space: normal;
widows: auto; word-spacing: 0px; -webkit-text-stroke-width:
0px;">
MainThread::<a class="moz-txt-link-freetext"
href="INFO::2015-07-01">INFO::2015-07-01</a>
11:03:53,061::state_machine::168::ovirt_hosted_engine_ha.agent.hosted_engine.HostedEngine::(refresh)
Local (id 1): {'engine-health': {'reason': 'failed
liveliness
check', 'health': '</div>
<div class="-x-evo-paragraph" style="color: rgb(46, 52, 54);
font-family: Cantarell; font-size: 15px; font-style: normal;
font-variant: normal; font-weight: normal; letter-spacing:
normal; line-height: normal; orphans: auto; text-align: start;
text-indent: 0px; text-transform: none; white-space: normal;
widows: auto; word-spacing: 0px; -webkit-text-stroke-width:
0px;">
bad', 'vm': 'up', 'detail': 'up'},
'bridge': True, 'mem-free':
136637.0, 'maintenance': False, 'cpu-load': 0.0035,
'gateway':
True}</div>
<div class="-x-evo-paragraph" style="color: rgb(46, 52, 54);
font-family: Cantarell; font-size: 15px; font-style: normal;
font-variant: normal; font-weight: normal; letter-spacing:
normal; line-height: normal; orphans: auto; text-align: start;
text-indent: 0px; text-transform: none; white-space: normal;
widows: auto; word-spacing: 0px; -webkit-text-stroke-width:
0px;">
MainThread::ERROR::2015-07-01
11:03:53,061::states::562::ovirt_hosted_engine_ha.agent.hosted_engine.HostedEngine::(consume)
Engine VM has bad health status, timeout in 300 seconds</div>
<div class="-x-evo-paragraph" style="color: rgb(46, 52, 54);
font-family: Cantarell; font-size: 15px; font-style: normal;
font-variant: normal; font-weight: normal; letter-spacing:
normal; line-height: normal; orphans: auto; text-align: start;
text-indent: 0px; text-transform: none; white-space: normal;
widows: auto; word-spacing: 0px; -webkit-text-stroke-width:
0px;">
MainThread::<a class="moz-txt-link-freetext"
href="INFO::2015-07-01">INFO::2015-07-01</a>
11:03:53,081::state_decorators::95::ovirt_hosted_engine_ha.agent.hosted_engine.HostedEngine::(check)
Timeout set to Wed Jul 1 11:08:53 2015 while transitioning
<class 'ovirt_hosted_</div>
<div class="-x-evo-paragraph" style="color: rgb(46, 52, 54);
font-family: Cantarell; font-size: 15px; font-style: normal;
font-variant: normal; font-weight: normal; letter-spacing:
normal; line-height: normal; orphans: auto; text-align: start;
text-indent: 0px; text-transform: none; white-space: normal;
widows: auto; word-spacing: 0px; -webkit-text-stroke-width:
0px;">
engine_ha.agent.states.EngineUpBadHealth'> -> <class
'ovirt_hosted_engine_ha.agent.states.EngineUpBadHealth'></div>
<div class="-x-evo-paragraph" style="color: rgb(46, 52, 54);
font-family: Cantarell; font-size: 15px; font-style: normal;
font-variant: normal; font-weight: normal; letter-spacing:
normal; line-height: normal; orphans: auto; text-align: start;
text-indent: 0px; text-transform: none; white-space: normal;
widows: auto; word-spacing: 0px; -webkit-text-stroke-width:
0px;">
MainThread::<a class="moz-txt-link-freetext"
href="INFO::2015-07-01">INFO::2015-07-01</a>
11:03:53,530::hosted_engine::327::ovirt_hosted_engine_ha.agent.hosted_engine.HostedEngine::(start_monitoring)
Current state EngineUpBadHealth (score: 2400)</div>
<div class="-x-evo-paragraph" style="color: rgb(46, 52, 54);
font-family: Cantarell; font-size: 15px; font-style: normal;
font-variant: normal; font-weight: normal; letter-spacing:
normal; line-height: normal; orphans: auto; text-align: start;
text-indent: 0px; text-transform: none; white-space: normal;
widows: auto; word-spacing: 0px; -webkit-text-stroke-width:
0px;">
MainThread::<a class="moz-txt-link-freetext"
href="INFO::2015-07-01">INFO::2015-07-01</a>
11:03:53,530::hosted_engine::332::ovirt_hosted_engine_ha.agent.hosted_engine.HostedEngine::(start_monitoring)
Best remote host mc-place-compute-02-live.mc.mcon.net (id: 2,
score: 240</div>
<div class="-x-evo-paragraph" style="color: rgb(46, 52, 54);
font-family: Cantarell; font-size: 15px; font-style: normal;
font-variant: normal; font-weight: normal; letter-spacing:
normal; line-height: normal; orphans: auto; text-align: start;
text-indent: 0px; text-transform: none; white-space: normal;
widows: auto; word-spacing: 0px; -webkit-text-stroke-width:
0px;">
0)</div>
<div class="-x-evo-paragraph" style="color: rgb(46, 52, 54);
font-family: Cantarell; font-size: 15px; font-style: normal;
font-variant: normal; font-weight: normal; letter-spacing:
normal; line-height: normal; orphans: auto; text-align: start;
text-indent: 0px; text-transform: none; white-space: normal;
widows: auto; word-spacing: 0px; -webkit-text-stroke-width:
0px;">
MainThread::ERROR::2015-07-01
11:04:03,559::states::562::ovirt_hosted_engine_ha.agent.hosted_engine.HostedEngine::(consume)
Engine VM has bad health status, timeout in 289 seconds</div>
<div class="-x-evo-paragraph" style="color: rgb(46, 52, 54);
font-family: Cantarell; font-size: 15px; font-style: normal;
font-variant: normal; font-weight: normal; letter-spacing:
normal; line-height: normal; orphans: auto; text-align: start;
text-indent: 0px; text-transform: none; white-space: normal;
widows: auto; word-spacing: 0px; -webkit-text-stroke-width:
0px;">
MainThread::<a class="moz-txt-link-freetext"
href="INFO::2015-07-01">INFO::2015-07-01</a>
11:04:03,980::hosted_engine::327::ovirt_hosted_engine_ha.agent.hosted_engine.HostedEngine::(start_monitoring)
Current state EngineUpBadHealth (score: 2400)</div>
<div class="-x-evo-paragraph" style="color: rgb(46, 52, 54);
font-family: Cantarell; font-size: 15px; font-style: normal;
font-variant: normal; font-weight: normal; letter-spacing:
normal; line-height: normal; orphans: auto; text-align: start;
text-indent: 0px; text-transform: none; white-space: normal;
widows: auto; word-spacing: 0px; -webkit-text-stroke-width:
0px;">
MainThread::<a class="moz-txt-link-freetext"
href="INFO::2015-07-01">INFO::2015-07-01</a>
11:04:03,980::hosted_engine::332::ovirt_hosted_engine_ha.agent.hosted_engine.HostedEngine::(start_monitoring)
Best remote host mc-place-compute-02-live.mc.mcon.net (id: 2,
score: 240</div>
<div class="-x-evo-paragraph" style="color: rgb(46, 52, 54);
font-family: Cantarell; font-size: 15px; font-style: normal;
font-variant: normal; font-weight: normal; letter-spacing:
normal; line-height: normal; orphans: auto; text-align: start;
text-indent: 0px; text-transform: none; white-space: normal;
widows: auto; word-spacing: 0px; -webkit-text-stroke-width:
0px;">
0)</div>
<div class="-x-evo-paragraph" style="color: rgb(46, 52, 54);
font-family: Cantarell; font-size: 15px; font-style: normal;
font-variant: normal; font-weight: normal; letter-spacing:
normal; line-height: normal; orphans: auto; text-align: start;
text-indent: 0px; text-transform: none; white-space: normal;
widows: auto; word-spacing: 0px; -webkit-text-stroke-width:
0px;">
MainThread::ERROR::2015-07-01
11:04:14,007::states::562::ovirt_hosted_engine_ha.agent.hosted_engine.HostedEngine::(consume)
Engine VM has bad health status, timeout in 279 seconds</div>
<div class="-x-evo-paragraph" style="color: rgb(46, 52, 54);
font-family: Cantarell; font-size: 15px; font-style: normal;
font-variant: normal; font-weight: normal; letter-spacing:
normal; line-height: normal; orphans: auto; text-align: start;
text-indent: 0px; text-transform: none; white-space: normal;
widows: auto; word-spacing: 0px; -webkit-text-stroke-width:
0px;">
MainThread::<a class="moz-txt-link-freetext"
href="INFO::2015-07-01">INFO::2015-07-01</a>
11:04:14,478::hosted_engine::327::ovirt_hosted_engine_ha.agent.hosted_engine.HostedEngine::(start_monitoring)
Current state EngineUpBadHealth (score: 2400)</div>
<div class="-x-evo-paragraph" style="color: rgb(46, 52, 54);
font-family: Cantarell; font-size: 15px; font-style: normal;
font-variant: normal; font-weight: normal; letter-spacing:
normal; line-height: normal; orphans: auto; text-align: start;
text-indent: 0px; text-transform: none; white-space: normal;
widows: auto; word-spacing: 0px; -webkit-text-stroke-width:
0px;">
MainThread::<a class="moz-txt-link-freetext"
href="INFO::2015-07-01">INFO::2015-07-01</a>
11:04:14,478::hosted_engine::332::ovirt_hosted_engine_ha.agent.hosted_engine.HostedEngine::(start_monitoring)
Best remote host mc-place-compute-02-live.mc.mcon.net (id: 2,
score: 240</div>
<div class="-x-evo-paragraph" style="color: rgb(46, 52, 54);
font-family: Cantarell; font-size: 15px; font-style: normal;
font-variant: normal; font-weight: normal; letter-spacing:
normal; line-height: normal; orphans: auto; text-align: start;
text-indent: 0px; text-transform: none; white-space: normal;
widows: auto; word-spacing: 0px; -webkit-text-stroke-width:
0px;">
0)</div>
<div class="-x-evo-paragraph" style="color: rgb(46, 52, 54);
font-family: Cantarell; font-size: 15px; font-style: normal;
font-variant: normal; font-weight: normal; letter-spacing:
normal; line-height: normal; orphans: auto; text-align: start;
text-indent: 0px; text-transform: none; white-space: normal;
widows: auto; word-spacing: 0px; -webkit-text-stroke-width:
0px;">
MainThread::ERROR::2015-07-01
11:04:24,505::states::562::ovirt_hosted_engine_ha.agent.hosted_engine.HostedEngine::(consume)
Engine VM has bad health status, timeout in 268 seconds</div>
<div class="-x-evo-paragraph" style="color: rgb(46, 52, 54);
font-family: Cantarell; font-size: 15px; font-style: normal;
font-variant: normal; font-weight: normal; letter-spacing:
normal; line-height: normal; orphans: auto; text-align: start;
text-indent: 0px; text-transform: none; white-space: normal;
widows: auto; word-spacing: 0px; -webkit-text-stroke-width:
0px;">
MainThread::<a class="moz-txt-link-freetext"
href="INFO::2015-07-01">INFO::2015-07-01</a>
11:04:24,994::hosted_engine::327::ovirt_hosted_engine_ha.agent.hosted_engine.HostedEngine::(start_monitoring)
Current state EngineUpBadHealth (score: 2400)</div>
<div class="-x-evo-paragraph" style="color: rgb(46, 52, 54);
font-family: Cantarell; font-size: 15px; font-style: normal;
font-variant: normal; font-weight: normal; letter-spacing:
normal; line-height: normal; orphans: auto; text-align: start;
text-indent: 0px; text-transform: none; white-space: normal;
widows: auto; word-spacing: 0px; -webkit-text-stroke-width:
0px;">
MainThread::<a class="moz-txt-link-freetext"
href="INFO::2015-07-01">INFO::2015-07-01</a>
11:04:24,994::hosted_engine::332::ovirt_hosted_engine_ha.agent.hosted_engine.HostedEngine::(start_monitoring)
Best remote host mc-place-compute-02-live.mc.mcon.net (id: 2,
score: 240</div>
<br class="Apple-interchange-newline">
<div><br>
<div style="font-family:Tahoma; font-size:13px">
<div style="font-family:Tahoma; font-size:13px">Best
Regards<br>
<br>
Carles Cortes Costa<br>
<br>
<p><br>
</p>
</div>
</div>
</div>
</div>
<br>
<fieldset class="mimeAttachmentHeader"></fieldset>
<br>
<pre wrap="">_______________________________________________
Users mailing list
<a class="moz-txt-link-abbreviated"
href="mailto:Users@ovirt.org">Users@ovirt.org</a>
<a class="moz-txt-link-freetext"
href="http://lists.ovirt.org/mailman/listinfo/users">http://...
</pre>
</blockquote>
<br>
</body>
</html>
--------------080404070003050909080805--
3432
days inactive
3432
days old
1 comments
2 participants
participants (2)
-
Carles Costa -
Roy Golan