In borker.log I found this, just before 05:59am:
Thread-3::INFO::2018-12-13 05:58:45,634::mem_free::51::mem_free.MemFree::(action) memFree: 82101"Host is not set" ???
Thread-1::INFO::2018-12-13 05:58:46,322::ping::60::ping.Ping::(action) Successfully pinged 10.0.1.254
Thread-5::INFO::2018-12-13 05:58:46,611::engine_health::241::engine_health.EngineHealth::(_result_from_stats) VM is up on this host with healthy engine
Thread-2::INFO::2018-12-13 05:58:49,144::mgmt_bridge::62::mgmt_bridge.MgmtBridge::(action) Found bridge ovirtmgmt with ports
StatusStorageThread::ERROR::2018-12-13 05:58:54,935::status_broker::90::ovirt_hosted_engine_ha.broker.status_broker.StatusBroker.Update::(run) Failed to update state.
Traceback (most recent call last):
File "/usr/lib/python2.7/site-packages/ovirt_hosted_engine_ha/broker/status_broker.py", line 82, in run
if (self._status_broker._inquire_whiteboard_lock() or
File "/usr/lib/python2.7/site-packages/ovirt_hosted_engine_ha/broker/status_broker.py", line 190, in _inquire_whiteboard_lock
self.host_id, self._lease_file)
File "/usr/lib/python2.7/site-packages/ovirt_hosted_engine_ha/broker/status_broker.py", line 128, in host_id
raise ex.HostIdNotLockedError("Host id is not set")
HostIdNotLockedError: Host id is not set
StatusStorageThread::ERROR::2018-12-13 05:58:54,937::status_broker::70::ovirt_hosted_engine_ha.broker.status_broker.StatusBroker.Update::(trigger_restart) Trying to restart the broker
Regards,
Frank
Hi,check the broker.log as well. The connect is used to talk to ovirt-ha-broker service socket.Best regardsMartin SivakOn Fri, Dec 14, 2018 at 12:20 PM fsoyer <fsoyer@systea.fr> wrote:I think I have it in agent.log. What can be this "file not found" ?MainThread::ERROR::2018-12-13 05:59:03,909::hosted_engine::431::ovirt_hosted_engine_ha.agent.hosted_engine.HostedEngine::(start_monitoring) Unhandled monitoring loop exception
Traceback (most recent call last):
File "/usr/lib/python2.7/site-packages/ovirt_hosted_engine_ha/agent/hosted_engine.py", line 428, in start_monitoring
self._monitoring_loop()
File "/usr/lib/python2.7/site-packages/ovirt_hosted_engine_ha/agent/hosted_engine.py", line 447, in _monitoring_loop
for old_state, state, delay in self.fsm:
File "/usr/lib/python2.7/site-packages/ovirt_hosted_engine_ha/lib/fsm/machine.py", line 127, in next
new_data = self.refresh(self._state.data)
File "/usr/lib/python2.7/site-packages/ovirt_hosted_engine_ha/agent/state_machine.py", line 81, in refresh
stats.update(self.hosted_engine.collect_stats())
File "/usr/lib/python2.7/site-packages/ovirt_hosted_engine_ha/agent/hosted_engine.py", line 736, in collect_stats
all_stats = self._broker.get_stats_from_storage()
File "/usr/lib/python2.7/site-packages/ovirt_hosted_engine_ha/lib/brokerlink.py", line 135, in get_stats_from_storage
result = self._proxy.get_stats()
File "/usr/lib64/python2.7/xmlrpclib.py", line 1233, in __call__
return self.__send(self.__name, args)
File "/usr/lib64/python2.7/xmlrpclib.py", line 1591, in __request
verbose=self.__verbose
File "/usr/lib64/python2.7/xmlrpclib.py", line 1273, in request
return self.single_request(host, handler, request_body, verbose)
File "/usr/lib64/python2.7/xmlrpclib.py", line 1301, in single_request
self.send_content(h, request_body)
File "/usr/lib64/python2.7/xmlrpclib.py", line 1448, in send_content
connection.endheaders(request_body)
File "/usr/lib64/python2.7/httplib.py", line 1037, in endheaders
self._send_output(message_body)
File "/usr/lib64/python2.7/httplib.py", line 881, in _send_output
self.send(msg)
File "/usr/lib64/python2.7/httplib.py", line 843, in send
self.connect()
File "/usr/lib/python2.7/site-packages/ovirt_hosted_engine_ha/lib/unixrpc.py", line 52, in connect
self.sock.connect(base64.b16decode(self.host))
File "/usr/lib64/python2.7/socket.py", line 224, in meth
return getattr(self._sock,name)(*args)
error: [Errno 2] No such file or directory
MainThread::ERROR::2018-12-13 05:59:04,043::agent::144::ovirt_hosted_engine_ha.agent.agent.Agent::(_run_agent) Traceback (most recent call last):
File "/usr/lib/python2.7/site-packages/ovirt_hosted_engine_ha/agent/agent.py", line 131, in _run_agent
return action(he)
File "/usr/lib/python2.7/site-packages/ovirt_hosted_engine_ha/agent/agent.py", line 55, in action_proper
return he.start_monitoring()
File "/usr/lib/python2.7/site-packages/ovirt_hosted_engine_ha/agent/hosted_engine.py", line 435, in start_monitoring
self.publish(stopped)
File "/usr/lib/python2.7/site-packages/ovirt_hosted_engine_ha/agent/hosted_engine.py", line 337, in publish
self._push_to_storage(blocks)
File "/usr/lib/python2.7/site-packages/ovirt_hosted_engine_ha/agent/hosted_engine.py", line 707, in _push_to_storage
self._broker.put_stats_on_storage(self.host_id, blocks)
File "/usr/lib/python2.7/site-packages/ovirt_hosted_engine_ha/lib/brokerlink.py", line 105, in put_stats_on_storage
self._proxy.put_stats(host_id, xmlrpclib.Binary(data))
File "/usr/lib64/python2.7/xmlrpclib.py", line 1233, in __call__
return self.__send(self.__name, args)
File "/usr/lib64/python2.7/xmlrpclib.py", line 1591, in __request
verbose=self.__verbose
File "/usr/lib64/python2.7/xmlrpclib.py", line 1273, in request
return self.single_request(host, handler, request_body, verbose)
File "/usr/lib64/python2.7/xmlrpclib.py", line 1301, in single_request
self.send_content(h, request_body)
File "/usr/lib64/python2.7/xmlrpclib.py", line 1448, in send_content
connection.endheaders(request_body)
File "/usr/lib64/python2.7/httplib.py", line 1037, in endheaders
self._send_output(message_body)
File "/usr/lib64/python2.7/httplib.py", line 881, in _send_output
self.send(msg)
File "/usr/lib64/python2.7/httplib.py", line 843, in send
self.connect()
File "/usr/lib/python2.7/site-packages/ovirt_hosted_engine_ha/lib/unixrpc.py", line 52, in connect
self.sock.connect(base64.b16decode(self.host))
File "/usr/lib64/python2.7/socket.py", line 224, in meth
return getattr(self._sock,name)(*args)
error: [Errno 2] No such file or directory
MainThread::ERROR::2018-12-13 05:59:04,043::agent::145::ovirt_hosted_engine_ha.agent.agent.Agent::(_run_agent) Trying to restart agent
MainThread::INFO::2018-12-13 05:59:04,044::agent::89::ovirt_hosted_engine_ha.agent.agent.Agent::(run) Agent shutting down
MainThread::INFO::2018-12-13 05:59:14,923::agent::67::ovirt_hosted_engine_ha.agent.agent.Agent::(run) ovirt-hosted-engine-ha agent 2.2.16 started
--Cordialement,
Frank Soyer
Mob. 06 72 28 38 53 - Fix. 05 49 50 52 34Systea IG
Administration systèmes, réseaux et bases de données
www.systea.net
Membre du réseau Les Professionnels du NumériqueKoGite
Hébergement de proximité
www.kogite.fr
Le Vendredi, Décembre 14, 2018 12:11 CET, Martin Sivak <msivak@redhat.com> a écrit:
Hi,
no StartState is not common, it is only ever entered when the agent
boots up. So something restarted or killed the agent process. Check
the agent log in /var/log/ovirt-hosted-engine-ha for errors.
Best regards
Martin Sivak
On Fri, Dec 14, 2018 at 12:05 PM fsoyer <fsoyer@systea.fr> wrote:
>
> Hi Martin,
> my problem is that nobody restarted the agent. Do you mean that this is not a normal behavior ? Is it possible that it restarts itself ?
>
> Thanks
> --
>
> Regards,
>
> Frank
>
>
>
> Le Jeudi, Décembre 13, 2018 15:25 CET, Martin Sivak <msivak@redhat.com> a écrit:
>
>
> Hi,
>
> those are state change notifications from the hosted engine agent. It
> basically means somebody restarted the ha-agent process and it found
> out the VM is still running fine and returned to the proper state.
>
> Configuring it is possible using the broker.conf file in
> /etc/ovirt-hosted-engine-ha (look for the notification section) or the
> hosted-engine tool (search --help for set config) depending on the
> version of hosted engine you are using.
>
> Best regards
>
> --
> Martin Sivak
>
>
> On Thu, Dec 13, 2018 at 3:10 PM fsoyer <fsoyer@systea.fr> wrote:
> >
> > Hi,
> > I don't find revelant answer about this. Sorry il this was already asked.
> > I receive randomly (one or two tims a week, differents hours) 3 mails with this subjects :
> > first : ovirt-hosted-engine state transition StartState-ReinitializeFSM
> > second : ovirt-hosted-engine state transition ReinitializeFSM-EngineStarting
> > third : ovirt-hosted-engine state transition EngineStarting-EngineUp
> > all at exactly the same time. The "events" in GUI doesn't indicate anything about this. No impact on engine or VMs.
> > So I wonder what this messages means ? And, if case of just "info" messages, is there a way to disable them ?
> >
> > Thanks.
> > --
> >
> > Reagrds,
> >
> > Frank
> >
> > _______________________________________________
> > Users mailing list -- users@ovirt.org
> > To unsubscribe send an email to users-leave@ovirt.org
> > Privacy Statement: https://www.ovirt.org/site/privacy-policy/
> > oVirt Code of Conduct: https://www.ovirt.org/community/about/community-guidelines/
> > List Archives: https://lists.ovirt.org/archives/list/users@ovirt.org/message/CVEHTWILWDEHASTCQHFHX62U4K4ZCOSK/
>
>
>
>