[ovirt-users] ha-agent and broker continually crashing after 4.2 update

[Jayme]

recently upgraded to 4.2 and had some problems with engine vm running, got
that cleared up now my only remaining issue is that now it seems
ovirt-ha-broker and ovirt-ha-agent are continually crashing on all three of
my hosts.  Everything is up and working fine otherwise, all VMs running and
hosted engine VM is running along with interface etc.

Jan 12 16:52:34 cultivar0 journal: vdsm storage.Dispatcher ERROR FINISH
prepareImage error=Volume does not exist:
(u'8582bdfc-ef54-47af-9f1e-f5b7ec1f1cf8',)
Jan 12 16:52:34 cultivar0 python: detected unhandled Python exception in
'/usr/share/ovirt-hosted-engine-ha/ovirt-ha-broker'
Jan 12 16:52:34 cultivar0 abrt-server: Not saving repeating crash in
'/usr/share/ovirt-hosted-engine-ha/ovirt-ha-broker'
Jan 12 16:52:34 cultivar0 systemd: ovirt-ha-broker.service: main process
exited, code=exited, status=1/FAILURE
Jan 12 16:52:34 cultivar0 systemd: Unit ovirt-ha-broker.service entered
failed state.
Jan 12 16:52:34 cultivar0 systemd: ovirt-ha-broker.service failed.
Jan 12 16:52:34 cultivar0 systemd: ovirt-ha-broker.service holdoff time
over, scheduling restart.
Jan 12 16:52:34 cultivar0 systemd: Cannot add dependency job for unit
lvm2-lvmetad.socket, ignoring: Unit is masked.
Jan 12 16:52:34 cultivar0 systemd: Started oVirt Hosted Engine High
Availability Communications Broker.
Jan 12 16:52:34 cultivar0 systemd: Starting oVirt Hosted Engine High
Availability Communications Broker...
Jan 12 16:52:36 cultivar0 journal: vdsm storage.TaskManager.Task ERROR
(Task='73141dec-9d8f-4164-9c4e-67c43a102eff') Unexpected error#012Traceback
(most recent call last):#012  File
"/usr/lib/python2.7/site-packages/vdsm/storage/task.py", line 882, in
_run#012    return fn(*args, **kargs)#012  File "<string>", line 2, in
prepareImage#012  File
"/usr/lib/python2.7/site-packages/vdsm/common/api.py", line 48, in
method#012    ret = func(*args, **kwargs)#012  File
"/usr/lib/python2.7/site-packages/vdsm/storage/hsm.py", line 3162, in
prepareImage#012    raise
se.VolumeDoesNotExist(leafUUID)#012VolumeDoesNotExist: Volume does not
exist: (u'8582bdfc-ef54-47af-9f1e-f5b7ec1f1cf8',)
Jan 12 16:52:36 cultivar0 journal: vdsm storage.Dispatcher ERROR FINISH
prepareImage error=Volume does not exist:
(u'8582bdfc-ef54-47af-9f1e-f5b7ec1f1cf8',)
Jan 12 16:52:36 cultivar0 python: detected unhandled Python exception in
'/usr/share/ovirt-hosted-engine-ha/ovirt-ha-broker'
Jan 12 16:52:36 cultivar0 abrt-server: Not saving repeating crash in
'/usr/share/ovirt-hosted-engine-ha/ovirt-ha-broker'
Jan 12 16:52:36 cultivar0 systemd: ovirt-ha-broker.service: main process
exited, code=exited, status=1/FAILURE
Jan 12 16:52:36 cultivar0 systemd: Unit ovirt-ha-broker.service entered
failed state.
Jan 12 16:52:36 cultivar0 systemd: ovirt-ha-broker.service failed.

Jan 12 16:52:36 cultivar0 systemd: ovirt-ha-broker.service holdoff time
over, scheduling restart.
Jan 12 16:52:36 cultivar0 systemd: Cannot add dependency job for unit
lvm2-lvmetad.socket, ignoring: Unit is masked.
Jan 12 16:52:36 cultivar0 systemd: Started oVirt Hosted Engine High
Availability Communications Broker.
Jan 12 16:52:36 cultivar0 systemd: Starting oVirt Hosted Engine High
Availability Communications Broker...
Jan 12 16:52:37 cultivar0 journal: vdsm storage.TaskManager.Task ERROR
(Task='bc7af1e2-0ab2-4164-ae88-d2bee03500f9') Unexpected error#012Traceback
(most recent call last):#012  File
"/usr/lib/python2.7/site-packages/vdsm/storage/task.py", line 882, in
_run#012    return fn(*args, **kargs)#012  File "<string>", line 2, in
prepareImage#012  File
"/usr/lib/python2.7/site-packages/vdsm/common/api.py", line 48, in
method#012    ret = func(*args, **kwargs)#012  File
"/usr/lib/python2.7/site-packages/vdsm/storage/hsm.py", line 3162, in
prepareImage#012    raise
se.VolumeDoesNotExist(leafUUID)#012VolumeDoesNotExist: Volume does not
exist: (u'8582bdfc-ef54-47af-9f1e-f5b7ec1f1cf8',)
Jan 12 16:52:37 cultivar0 journal: vdsm storage.Dispatcher ERROR FINISH
prepareImage error=Volume does not exist:
(u'8582bdfc-ef54-47af-9f1e-f5b7ec1f1cf8',)
Jan 12 16:52:37 cultivar0 python: detected unhandled Python exception in
'/usr/share/ovirt-hosted-engine-ha/ovirt-ha-broker'
Jan 12 16:52:38 cultivar0 abrt-server: Not saving repeating crash in
'/usr/share/ovirt-hosted-engine-ha/ovirt-ha-broker'
Jan 12 16:52:38 cultivar0 systemd: ovirt-ha-broker.service: main process
exited, code=exited, status=1/FAILURE
Jan 12 16:52:38 cultivar0 systemd: Unit ovirt-ha-broker.service entered
failed state.
Jan 12 16:52:38 cultivar0 systemd: ovirt-ha-broker.service failed.
Jan 12 16:52:38 cultivar0 systemd: ovirt-ha-broker.service holdoff time
over, scheduling restart.
Jan 12 16:52:38 cultivar0 systemd: Cannot add dependency job for unit
lvm2-lvmetad.socket, ignoring: Unit is masked.
Jan 12 16:52:38 cultivar0 systemd: start request repeated too quickly for
ovirt-ha-broker.service
Jan 12 16:52:38 cultivar0 systemd: Failed to start oVirt Hosted Engine High
Availability Communications Broker.
Jan 12 16:52:38 cultivar0 systemd: Unit ovirt-ha-broker.service entered
failed state.
Jan 12 16:52:38 cultivar0 systemd: ovirt-ha-broker.service failed.
Jan 12 16:52:40 cultivar0 systemd: ovirt-ha-agent.service holdoff time
over, scheduling restart.
Jan 12 16:52:40 cultivar0 systemd: Cannot add dependency job for unit
lvm2-lvmetad.socket, ignoring: Unit is masked.
Jan 12 16:52:40 cultivar0 systemd: Started oVirt Hosted Engine High
Availability Communications Broker.
Jan 12 16:52:40 cultivar0 systemd: Starting oVirt Hosted Engine High
Availability Communications Broker...
Jan 12 16:52:40 cultivar0 systemd: Started oVirt Hosted Engine High
Availability Monitoring Agent.
Jan 12 16:52:40 cultivar0 systemd: Starting oVirt Hosted Engine High
Availability Monitoring Agent...
Jan 12 16:52:41 cultivar0 journal: ovirt-ha-agent
ovirt_hosted_engine_ha.agent.hosted_engine.HostedEngine ERROR Failed to
start necessary monitors
Jan 12 16:52:41 cultivar0 journal: ovirt-ha-agent
ovirt_hosted_engine_ha.agent.agent.Agent ERROR Traceback (most recent call
last):#012  File
"/usr/lib/python2.7/site-packages/ovirt_hosted_engine_ha/agent/agent.py",
line 131, in _run_agent#012    return action(he)#012  File
"/usr/lib/python2.7/site-packages/ovirt_hosted_engine_ha/agent/agent.py",
line 55, in action_proper#012    return he.start_monitoring()#012  File
"/usr/lib/python2.7/site-packages/ovirt_hosted_engine_ha/agent/hosted_engine.py",
line 416, in start_monitoring#012    self._initialize_broker()#012  File
"/usr/lib/python2.7/site-packages/ovirt_hosted_engine_ha/agent/hosted_engine.py",
line 535, in _initialize_broker#012    m.get('options', {}))#012  File
"/usr/lib/python2.7/site-packages/ovirt_hosted_engine_ha/lib/brokerlink.py",
line 83, in start_monitor#012    .format(type, options,
e))#012RequestError: Failed to start monitor ping, options {'addr':
'192.168.0.1'}: [Errno 2] No such file or directory
Jan 12 16:52:41 cultivar0 journal: ovirt-ha-agent
ovirt_hosted_engine_ha.agent.agent.Agent ERROR Trying to restart agent
Jan 12 16:52:42 cultivar0 systemd: ovirt-ha-agent.service: main process
exited, code=exited, status=157/n/a
Jan 12 16:52:42 cultivar0 systemd: Unit ovirt-ha-agent.service entered
failed state.
Jan 12 16:52:42 cultivar0 systemd: ovirt-ha-agent.service failed.
-------------- next part --------------
An HTML attachment was scrubbed...
URL: <http://lists.ovirt.org/pipermail/users/attachments/20180112/e49f752b/attachment.html>