Hi,
Host id is not set
This is an internal error report that should normally not happen. It means the ovirt-ha-agent asked for a storage operation before it registered itself with the broker. If this happens seldom then it looks like a race condition. I would recommend opening a bug report with all the logs we talked about and all the RPM versions (ovirt-hosted-engine-ha and ovirt-hosted-engine-setup packages). Use this link to go directly to the right component: https://bugzilla.redhat.com/enter_bug.cgi?product=ovirt-hosted-engine-ha Since all is well except the emails I recommend filtering out the emails as a workaround before this can be fully investigated and fixed. Simone, Denis: I can't do more here, looks like a race in agent - broker initialization and host id management. Best regards Martin Sivak On Fri, Dec 14, 2018 at 12:35 PM fsoyer <fsoyer@systea.fr> wrote:
In borker.log I found this, just before 05:59am:
Thread-3::INFO::2018-12-13 05:58:45,634::mem_free::51::mem_free.MemFree::(action) memFree: 82101 Thread-1::INFO::2018-12-13 05:58:46,322::ping::60::ping.Ping::(action) Successfully pinged 10.0.1.254 Thread-5::INFO::2018-12-13 05:58:46,611::engine_health::241::engine_health.EngineHealth::(_result_from_stats) VM is up on this host with healthy engine Thread-2::INFO::2018-12-13 05:58:49,144::mgmt_bridge::62::mgmt_bridge.MgmtBridge::(action) Found bridge ovirtmgmt with ports StatusStorageThread::ERROR::2018-12-13 05:58:54,935::status_broker::90::ovirt_hosted_engine_ha.broker.status_broker.StatusBroker.Update::(run) Failed to update state. Traceback (most recent call last): File "/usr/lib/python2.7/site-packages/ovirt_hosted_engine_ha/broker/status_broker.py", line 82, in run if (self._status_broker._inquire_whiteboard_lock() or File "/usr/lib/python2.7/site-packages/ovirt_hosted_engine_ha/broker/status_broker.py", line 190, in _inquire_whiteboard_lock self.host_id, self._lease_file) File "/usr/lib/python2.7/site-packages/ovirt_hosted_engine_ha/broker/status_broker.py", line 128, in host_id raise ex.HostIdNotLockedError("Host id is not set") HostIdNotLockedError: Host id is not set StatusStorageThread::ERROR::2018-12-13 05:58:54,937::status_broker::70::ovirt_hosted_engine_ha.broker.status_broker.StatusBroker.Update::(trigger_restart) Trying to restart the broker
"Host is not set" ??? --
Regards,
*Frank*
Le Vendredi, Décembre 14, 2018 12:27 CET, Martin Sivak <msivak@redhat.com> a écrit:
Hi,
check the broker.log as well. The connect is used to talk to ovirt-ha-broker service socket.
Best regards
Martin Sivak
On Fri, Dec 14, 2018 at 12:20 PM fsoyer <fsoyer@systea.fr> wrote:
I think I have it in agent.log. What can be this "file not found" ?
MainThread::ERROR::2018-12-13 05:59:03,909::hosted_engine::431::ovirt_hosted_engine_ha.agent.hosted_engine.HostedEngine::(start_monitoring) Unhandled monitoring loop exception Traceback (most recent call last): File "/usr/lib/python2.7/site-packages/ovirt_hosted_engine_ha/agent/hosted_engine.py", line 428, in start_monitoring self._monitoring_loop() File "/usr/lib/python2.7/site-packages/ovirt_hosted_engine_ha/agent/hosted_engine.py", line 447, in _monitoring_loop for old_state, state, delay in self.fsm: File "/usr/lib/python2.7/site-packages/ovirt_hosted_engine_ha/lib/fsm/machine.py", line 127, in next new_data = self.refresh(self._state.data) File "/usr/lib/python2.7/site-packages/ovirt_hosted_engine_ha/agent/state_machine.py", line 81, in refresh stats.update(self.hosted_engine.collect_stats()) File "/usr/lib/python2.7/site-packages/ovirt_hosted_engine_ha/agent/hosted_engine.py", line 736, in collect_stats all_stats = self._broker.get_stats_from_storage() File "/usr/lib/python2.7/site-packages/ovirt_hosted_engine_ha/lib/brokerlink.py", line 135, in get_stats_from_storage result = self._proxy.get_stats() File "/usr/lib64/python2.7/xmlrpclib.py", line 1233, in __call__ return self.__send(self.__name, args) File "/usr/lib64/python2.7/xmlrpclib.py", line 1591, in __request verbose=self.__verbose File "/usr/lib64/python2.7/xmlrpclib.py", line 1273, in request return self.single_request(host, handler, request_body, verbose) File "/usr/lib64/python2.7/xmlrpclib.py", line 1301, in single_request self.send_content(h, request_body) File "/usr/lib64/python2.7/xmlrpclib.py", line 1448, in send_content connection.endheaders(request_body) File "/usr/lib64/python2.7/httplib.py", line 1037, in endheaders self._send_output(message_body) File "/usr/lib64/python2.7/httplib.py", line 881, in _send_output self.send(msg) File "/usr/lib64/python2.7/httplib.py", line 843, in send self.connect() File "/usr/lib/python2.7/site-packages/ovirt_hosted_engine_ha/lib/unixrpc.py", line 52, in connect self.sock.connect(base64.b16decode(self.host)) File "/usr/lib64/python2.7/socket.py", line 224, in meth return getattr(self._sock,name)(*args) error: [Errno 2] No such file or directory MainThread::ERROR::2018-12-13 05:59:04,043::agent::144::ovirt_hosted_engine_ha.agent.agent.Agent::(_run_agent) Traceback (most recent call last): File "/usr/lib/python2.7/site-packages/ovirt_hosted_engine_ha/agent/agent.py", line 131, in _run_agent return action(he) File "/usr/lib/python2.7/site-packages/ovirt_hosted_engine_ha/agent/agent.py", line 55, in action_proper return he.start_monitoring() File "/usr/lib/python2.7/site-packages/ovirt_hosted_engine_ha/agent/hosted_engine.py", line 435, in start_monitoring self.publish(stopped) File "/usr/lib/python2.7/site-packages/ovirt_hosted_engine_ha/agent/hosted_engine.py", line 337, in publish self._push_to_storage(blocks) File "/usr/lib/python2.7/site-packages/ovirt_hosted_engine_ha/agent/hosted_engine.py", line 707, in _push_to_storage self._broker.put_stats_on_storage(self.host_id, blocks) File "/usr/lib/python2.7/site-packages/ovirt_hosted_engine_ha/lib/brokerlink.py", line 105, in put_stats_on_storage self._proxy.put_stats(host_id, xmlrpclib.Binary(data)) File "/usr/lib64/python2.7/xmlrpclib.py", line 1233, in __call__ return self.__send(self.__name, args) File "/usr/lib64/python2.7/xmlrpclib.py", line 1591, in __request verbose=self.__verbose File "/usr/lib64/python2.7/xmlrpclib.py", line 1273, in request return self.single_request(host, handler, request_body, verbose) File "/usr/lib64/python2.7/xmlrpclib.py", line 1301, in single_request self.send_content(h, request_body) File "/usr/lib64/python2.7/xmlrpclib.py", line 1448, in send_content connection.endheaders(request_body) File "/usr/lib64/python2.7/httplib.py", line 1037, in endheaders self._send_output(message_body) File "/usr/lib64/python2.7/httplib.py", line 881, in _send_output self.send(msg) File "/usr/lib64/python2.7/httplib.py", line 843, in send self.connect() File "/usr/lib/python2.7/site-packages/ovirt_hosted_engine_ha/lib/unixrpc.py", line 52, in connect self.sock.connect(base64.b16decode(self.host)) File "/usr/lib64/python2.7/socket.py", line 224, in meth return getattr(self._sock,name)(*args) error: [Errno 2] No such file or directory
MainThread::ERROR::2018-12-13 05:59:04,043::agent::145::ovirt_hosted_engine_ha.agent.agent.Agent::(_run_agent) Trying to restart agent MainThread::INFO::2018-12-13 05:59:04,044::agent::89::ovirt_hosted_engine_ha.agent.agent.Agent::(run) Agent shutting down MainThread::INFO::2018-12-13 05:59:14,923::agent::67::ovirt_hosted_engine_ha.agent.agent.Agent::(run) ovirt-hosted-engine-ha agent 2.2.16 started
--
Cordialement,
*Frank Soyer * Mob. 06 72 28 38 53 - Fix. 05 49 50 52 34 <http://www.systea.net>
*Systea IG* Administration systèmes, réseaux et bases de données www.systea.net Membre du réseau Les Professionnels du Numérique <http://www.spn.asso.fr> <http://www.kogite.fr>
*KoGite* Hébergement de proximité www.kogite.fr
Le Vendredi, Décembre 14, 2018 12:11 CET, Martin Sivak <msivak@redhat.com> a écrit:
Hi,
no StartState is not common, it is only ever entered when the agent boots up. So something restarted or killed the agent process. Check the agent log in /var/log/ovirt-hosted-engine-ha for errors.
Best regards
Martin Sivak
On Fri, Dec 14, 2018 at 12:05 PM fsoyer <fsoyer@systea.fr> wrote:
Hi Martin, my problem is that nobody restarted the agent. Do you mean that this is
not a normal behavior ? Is it possible that it restarts itself ?
Thanks --
Regards,
Frank
Le Jeudi, Décembre 13, 2018 15:25 CET, Martin Sivak <msivak@redhat.com>
a écrit:
Hi,
those are state change notifications from the hosted engine agent. It basically means somebody restarted the ha-agent process and it found out the VM is still running fine and returned to the proper state.
Configuring it is possible using the broker.conf file in /etc/ovirt-hosted-engine-ha (look for the notification section) or the hosted-engine tool (search --help for set config) depending on the version of hosted engine you are using.
Best regards
-- Martin Sivak
On Thu, Dec 13, 2018 at 3:10 PM fsoyer <fsoyer@systea.fr> wrote:
Hi, I don't find revelant answer about this. Sorry il this was already
asked.
I receive randomly (one or two tims a week, differents hours) 3 mails with this subjects : first : ovirt-hosted-engine state transition StartState-ReinitializeFSM second : ovirt-hosted-engine state transition ReinitializeFSM-EngineStarting third : ovirt-hosted-engine state transition EngineStarting-EngineUp all at exactly the same time. The "events" in GUI doesn't indicate anything about this. No impact on engine or VMs. So I wonder what this messages means ? And, if case of just "info" messages, is there a way to disable them ?
Thanks. --
Reagrds,
Frank
_______________________________________________ Users mailing list -- users@ovirt.org To unsubscribe send an email to users-leave@ovirt.org Privacy Statement: https://www.ovirt.org/site/privacy-policy/ oVirt Code of Conduct: https://www.ovirt.org/community/about/community-guidelines/ List Archives: https://lists.ovirt.org/archives/list/users@ovirt.org/message/CVEHTWILWDEHAS...