1:52 p.m.
Hi,
Host id is not set
This is an internal error report that should normally not happen. It means
the ovirt-ha-agent asked for a storage operation before it registered
itself with the broker. If this happens seldom then it looks like a race
condition.
I would recommend opening a bug report with all the logs we talked about
and all the RPM versions (ovirt-hosted-engine-ha and
ovirt-hosted-engine-setup packages). Use this link to go directly to the
right component:
https://bugzilla.redhat.com/enter_bug.cgi?product=ovirt-hosted-engine-ha
Since all is well except the emails I recommend filtering out the emails as
a workaround before this can be fully investigated and fixed.
Simone, Denis: I can't do more here, looks like a race in agent - broker
initialization and host id management.
Best regards
Martin Sivak
On Fri, Dec 14, 2018 at 12:35 PM fsoyer <fsoyer(a)systea.fr> wrote:
In borker.log I found this, just before 05:59am:
Thread-3::INFO::2018-12-13
05:58:45,634::mem_free::51::mem_free.MemFree::(action) memFree: 82101
Thread-1::INFO::2018-12-13 05:58:46,322::ping::60::ping.Ping::(action)
Successfully pinged 10.0.1.254
Thread-5::INFO::2018-12-13
05:58:46,611::engine_health::241::engine_health.EngineHealth::(_result_from_stats)
VM is up on this host with healthy engine
Thread-2::INFO::2018-12-13
05:58:49,144::mgmt_bridge::62::mgmt_bridge.MgmtBridge::(action) Found
bridge ovirtmgmt with ports
StatusStorageThread::ERROR::2018-12-13
05:58:54,935::status_broker::90::ovirt_hosted_engine_ha.broker.status_broker.StatusBroker.Update::(run)
Failed to update state.
Traceback (most recent call last):
File
"/usr/lib/python2.7/site-packages/ovirt_hosted_engine_ha/broker/status_broker.py",
line 82, in run
if (self._status_broker._inquire_whiteboard_lock() or
File
"/usr/lib/python2.7/site-packages/ovirt_hosted_engine_ha/broker/status_broker.py",
line 190, in _inquire_whiteboard_lock
self.host_id, self._lease_file)
File
"/usr/lib/python2.7/site-packages/ovirt_hosted_engine_ha/broker/status_broker.py",
line 128, in host_id
raise ex.HostIdNotLockedError("Host id is not set")
HostIdNotLockedError: Host id is not set
StatusStorageThread::ERROR::2018-12-13
05:58:54,937::status_broker::70::ovirt_hosted_engine_ha.broker.status_broker.StatusBroker.Update::(trigger_restart)
Trying to restart the broker
"Host is not set" ???
--
Regards,
*Frank*
Le Vendredi, Décembre 14, 2018 12:27 CET, Martin Sivak <msivak(a)redhat.com>
a écrit:
Hi,
check the broker.log as well. The connect is used to talk to
ovirt-ha-broker service socket.
Best regards
Martin Sivak
On Fri, Dec 14, 2018 at 12:20 PM fsoyer <fsoyer(a)systea.fr> wrote:
> I think I have it in agent.log. What can be this "file not found" ?
>
> MainThread::ERROR::2018-12-13
>
05:59:03,909::hosted_engine::431::ovirt_hosted_engine_ha.agent.hosted_engine.HostedEngine::(start_monitoring)
> Unhandled monitoring loop exception
> Traceback (most recent call last):
> File
>
"/usr/lib/python2.7/site-packages/ovirt_hosted_engine_ha/agent/hosted_engine.py",
> line 428, in start_monitoring
> self._monitoring_loop()
> File
>
"/usr/lib/python2.7/site-packages/ovirt_hosted_engine_ha/agent/hosted_engine.py",
> line 447, in _monitoring_loop
> for old_state, state, delay in self.fsm:
> File
>
"/usr/lib/python2.7/site-packages/ovirt_hosted_engine_ha/lib/fsm/machine.py",
> line 127, in next
> new_data = self.refresh(self._state.data)
> File
>
"/usr/lib/python2.7/site-packages/ovirt_hosted_engine_ha/agent/state_machine.py",
> line 81, in refresh
> stats.update(self.hosted_engine.collect_stats())
> File
>
"/usr/lib/python2.7/site-packages/ovirt_hosted_engine_ha/agent/hosted_engine.py",
> line 736, in collect_stats
> all_stats = self._broker.get_stats_from_storage()
> File
>
"/usr/lib/python2.7/site-packages/ovirt_hosted_engine_ha/lib/brokerlink.py",
> line 135, in get_stats_from_storage
> result = self._proxy.get_stats()
> File "/usr/lib64/python2.7/xmlrpclib.py", line 1233, in __call__
> return self.__send(self.__name, args)
> File "/usr/lib64/python2.7/xmlrpclib.py", line 1591, in __request
> verbose=self.__verbose
> File "/usr/lib64/python2.7/xmlrpclib.py", line 1273, in request
> return self.single_request(host, handler, request_body, verbose)
> File "/usr/lib64/python2.7/xmlrpclib.py", line 1301, in single_request
> self.send_content(h, request_body)
> File "/usr/lib64/python2.7/xmlrpclib.py", line 1448, in send_content
> connection.endheaders(request_body)
> File "/usr/lib64/python2.7/httplib.py", line 1037, in endheaders
> self._send_output(message_body)
> File "/usr/lib64/python2.7/httplib.py", line 881, in _send_output
> self.send(msg)
> File "/usr/lib64/python2.7/httplib.py", line 843, in send
> self.connect()
> File
> "/usr/lib/python2.7/site-packages/ovirt_hosted_engine_ha/lib/unixrpc.py",
> line 52, in connect
> self.sock.connect(base64.b16decode(self.host))
> File "/usr/lib64/python2.7/socket.py", line 224, in meth
> return getattr(self._sock,name)(*args)
> error: [Errno 2] No such file or directory
> MainThread::ERROR::2018-12-13
> 05:59:04,043::agent::144::ovirt_hosted_engine_ha.agent.agent.Agent::(_run_agent)
> Traceback (most recent call last):
> File
> "/usr/lib/python2.7/site-packages/ovirt_hosted_engine_ha/agent/agent.py",
> line 131, in _run_agent
> return action(he)
> File
> "/usr/lib/python2.7/site-packages/ovirt_hosted_engine_ha/agent/agent.py",
> line 55, in action_proper
> return he.start_monitoring()
> File
>
"/usr/lib/python2.7/site-packages/ovirt_hosted_engine_ha/agent/hosted_engine.py",
> line 435, in start_monitoring
> self.publish(stopped)
> File
>
"/usr/lib/python2.7/site-packages/ovirt_hosted_engine_ha/agent/hosted_engine.py",
> line 337, in publish
> self._push_to_storage(blocks)
> File
>
"/usr/lib/python2.7/site-packages/ovirt_hosted_engine_ha/agent/hosted_engine.py",
> line 707, in _push_to_storage
> self._broker.put_stats_on_storage(self.host_id, blocks)
> File
>
"/usr/lib/python2.7/site-packages/ovirt_hosted_engine_ha/lib/brokerlink.py",
> line 105, in put_stats_on_storage
> self._proxy.put_stats(host_id, xmlrpclib.Binary(data))
> File "/usr/lib64/python2.7/xmlrpclib.py", line 1233, in __call__
> return self.__send(self.__name, args)
> File "/usr/lib64/python2.7/xmlrpclib.py", line 1591, in __request
> verbose=self.__verbose
> File "/usr/lib64/python2.7/xmlrpclib.py", line 1273, in request
> return self.single_request(host, handler, request_body, verbose)
> File "/usr/lib64/python2.7/xmlrpclib.py", line 1301, in single_request
> self.send_content(h, request_body)
> File "/usr/lib64/python2.7/xmlrpclib.py", line 1448, in send_content
> connection.endheaders(request_body)
> File "/usr/lib64/python2.7/httplib.py", line 1037, in endheaders
> self._send_output(message_body)
> File "/usr/lib64/python2.7/httplib.py", line 881, in _send_output
> self.send(msg)
> File "/usr/lib64/python2.7/httplib.py", line 843, in send
> self.connect()
> File
> "/usr/lib/python2.7/site-packages/ovirt_hosted_engine_ha/lib/unixrpc.py",
> line 52, in connect
> self.sock.connect(base64.b16decode(self.host))
> File "/usr/lib64/python2.7/socket.py", line 224, in meth
> return getattr(self._sock,name)(*args)
> error: [Errno 2] No such file or directory
>
> MainThread::ERROR::2018-12-13
> 05:59:04,043::agent::145::ovirt_hosted_engine_ha.agent.agent.Agent::(_run_agent)
> Trying to restart agent
> MainThread::INFO::2018-12-13
> 05:59:04,044::agent::89::ovirt_hosted_engine_ha.agent.agent.Agent::(run)
> Agent shutting down
> MainThread::INFO::2018-12-13
> 05:59:14,923::agent::67::ovirt_hosted_engine_ha.agent.agent.Agent::(run)
> ovirt-hosted-engine-ha agent 2.2.16 started
>
>
>
> --
>
> Cordialement,
>
> *Frank Soyer *
> Mob. 06 72 28 38 53 - Fix. 05 49 50 52 34
> <http://www.systea.net>
>
> *Systea IG*
> Administration systèmes, réseaux et bases de données
> www.systea.net
> Membre du réseau Les Professionnels du Numérique <http://www.spn.asso.fr>
> <http://www.kogite.fr>
>
> *KoGite*
> Hébergement de proximité
> www.kogite.fr
>
>
>
> Le Vendredi, Décembre 14, 2018 12:11 CET, Martin Sivak <msivak(a)redhat.com>
> a écrit:
>
>
> Hi,
>
> no StartState is not common, it is only ever entered when the agent
> boots up. So something restarted or killed the agent process. Check
> the agent log in /var/log/ovirt-hosted-engine-ha for errors.
>
> Best regards
>
> Martin Sivak
>
> On Fri, Dec 14, 2018 at 12:05 PM fsoyer <fsoyer(a)systea.fr> wrote:
> >
> > Hi Martin,
> > my problem is that nobody restarted the agent. Do you mean that this is
> not a normal behavior ? Is it possible that it restarts itself ?
> >
> > Thanks
> > --
> >
> > Regards,
> >
> > Frank
> >
> >
> >
> > Le Jeudi, Décembre 13, 2018 15:25 CET, Martin Sivak <msivak(a)redhat.com>
> a écrit:
> >
> >
> > Hi,
> >
> > those are state change notifications from the hosted engine agent. It
> > basically means somebody restarted the ha-agent process and it found
> > out the VM is still running fine and returned to the proper state.
> >
> > Configuring it is possible using the broker.conf file in
> > /etc/ovirt-hosted-engine-ha (look for the notification section) or the
> > hosted-engine tool (search --help for set config) depending on the
> > version of hosted engine you are using.
> >
> > Best regards
> >
> > --
> > Martin Sivak
> >
> >
> > On Thu, Dec 13, 2018 at 3:10 PM fsoyer <fsoyer(a)systea.fr> wrote:
> > >
> > > Hi,
> > > I don't find revelant answer about this. Sorry il this was already
> asked.
> > > I receive randomly (one or two tims a week, differents hours) 3 mails
> with this subjects :
> > > first : ovirt-hosted-engine state transition
> StartState-ReinitializeFSM
> > > second : ovirt-hosted-engine state transition
> ReinitializeFSM-EngineStarting
> > > third : ovirt-hosted-engine state transition EngineStarting-EngineUp
> > > all at exactly the same time. The "events" in GUI doesn't
indicate
> anything about this. No impact on engine or VMs.
> > > So I wonder what this messages means ? And, if case of just
"info"
> messages, is there a way to disable them ?
> > >
> > > Thanks.
> > > --
> > >
> > > Reagrds,
> > >
> > > Frank
> > >
> > > _______________________________________________
> > > Users mailing list -- users(a)ovirt.org
> > > To unsubscribe send an email to users-leave(a)ovirt.org
> > > Privacy Statement: https://www.ovirt.org/site/privacy-policy/
> > > oVirt Code of Conduct:
> https://www.ovirt.org/community/about/community-guidelines/
> > > List Archives:
>
https://lists.ovirt.org/archives/list/users@ovirt.org/message/CVEHTWILWDE...
> >
> >
> >
> >
>
>
>
>