[ OST Failure Report ] [ oVirt Master (ovirt-engine-nodejs-modules) ] [ 23-03-2018] [ 002_bootstrap.check_update_host ]
1:02 p.m.
Hi,
we had a failure reported in CQ for change:
https://gerrit.ovirt.org/#/c/89338/ - Bump 1.5.2-1.
I don't think the failure is related to the change. it seems to be an issue
with mom failing to start.
*Link and headline of suspected patches:
https://gerrit.ovirt.org/#/c/89338/ <https://gerrit.ovirt.org/#/c/89338/> -
Bump 1.5.2-1Link to
Job:http://jenkins.ovirt.org/job/ovirt-master_change-queue-tester/6512/
<http://jenkins.ovirt.org/job/ovirt-master_change-queue-tester/6512/>Link
to all
logs:http://jenkins.ovirt.org/job/ovirt-master_change-queue-tester/6512/a...
<http://jenkins.ovirt.org/job/ovirt-master_change-queue-tester/6512/artifa...
error snippet from the log: <error>*
*engine: *018-03-23 07:12:33,435-04 ERROR
[org.ovirt.engine.core.bll.host.HostUpgradeManager]
(EE-ManagedThreadFactory-commandCoordinator-Thread-1)
[7a4f81e6-6ae6-4444-ace6-de757e78f41e] Failed to run check-update of host
'lago-basic-suite-master-
host-0'.
2018-03-23 07:12:33,436-04 ERROR
[org.ovirt.engine.core.bll.hostdeploy.HostUpdatesChecker]
(EE-ManagedThreadFactory-commandCoordinator-Thread-1)
[7a4f81e6-6ae6-4444-ace6-de757e78f41e] Failed to check if updates are
available for host 'lag
o-basic-suite-master-host-0' with error message 'Failed to run check-update
of host 'lago-basic-suite-master-host-0'.'
2018-03-23 07:12:33,441-04 ERROR
[org.ovirt.engine.core.dal.dbbroker.auditloghandling.AuditLogDirector]
(EE-ManagedThreadFactory-commandCoordinator-Thread-1)
[7a4f81e6-6ae6-4444-ace6-de757e78f41e] EVENT_ID:
HOST_AVAILABLE_UPDATES_FAILED(8
39), Failed to check for available updates on host
lago-basic-suite-master-host-0 with message 'Failed to run check-update of
host 'lago-basic-suite-master-host-0'.'.
2018-03-23 07:12:33,566-04 DEBUG
[org.ovirt.engine.core.vdsbroker.vdsbroker.GetAllVmStatsVDSCommand]
(EE-ManagedThreadFactory-engineScheduled-Thread-11) [] START,
GetAllVmStatsVDSCommand(HostName = lago-basic-suite-master-host-0, VdsIdVDS
CommandParametersBase:{hostId='dc339bcf-b769-41f6-8252-c61fffd56b17'}), log
id: 4eb92a5a
2018-03-23 07:12:33,567-04 DEBUG
[org.ovirt.vdsm.jsonrpc.client.reactors.stomp.impl.Message]
(EE-ManagedThreadFactory-engineScheduled-Thread-11) [] SEND
destination:jms.topic.vdsm_requests
reply-to:jms.topic.vdsm_responses
content-length:103
*host-0:*
vdsm log show: momStatus': 'inactive'
*Mom log: *
2018-03-23 07:09:48,849 - mom - INFO - MOM starting
2018-03-23 07:09:48,864 - mom.HostMonitor - INFO - Host Monitor starting
2018-03-23 07:09:48,867 - mom - INFO - hypervisor interface
vdsmjsonrpcclient
2018-03-23 07:09:48,983 - mom - ERROR - Failed to initialize MOM threads
Traceback (most recent call last):
File "/usr/lib/python2.7/site-packages/mom/__init__.py", line 29, in run
hypervisor_iface = self.get_hypervisor_interface()
File "/usr/lib/python2.7/site-packages/mom/__init__.py", line 217, in
get_hypervisor_interface
return module.instance(self.config)
File
"/usr/lib/python2.7/site-packages/mom/HypervisorInterfaces/vdsmjsonrpcclientInterface.py",
line 96, in instance
return JsonRpcVdsmClientInterface()
File
"/usr/lib/python2.7/site-packages/mom/HypervisorInterfaces/vdsmjsonrpcclientInterface.py",
line 31, in __init__
self._vdsm_api = client.connect(host="localhost")
File "/usr/lib/python2.7/site-packages/vdsm/client.py", line 139, in
connect
raise ConnectionError(host, port, use_tls, timeout, e)
ConnectionError: Connection to localhost:54321 with use_tls=True,
timeout=60 failed: [Errno 111] Connection refused
2018-03-23 07:09:54,475 - mom - INFO - MOM starting
2018-03-23 07:09:54,499 - mom.HostMonitor - INFO - Host Monitor starting
2018-03-23 07:09:54,500 - mom - INFO - hypervisor interface
vdsmjsonrpcclient
2018-03-23 07:09:54,725 - mom.HostMonitor - INFO - HostMonitor is ready
2018-03-23 07:09:54,813 - mom.GuestManager - INFO - Guest Manager starting:
multi-thread
2018-03-23 07:09:54,819 - mom.Policy - INFO - Loaded policy '00-defines'
2018-03-23 07:09:54,820 - mom.Policy - INFO - Loaded policy '01-parameters'
2018-03-23 07:09:54,841 - mom.Policy - INFO - Loaded policy '02-balloon'
2018-03-23 07:09:54,871 - mom.Policy - INFO - Loaded policy '03-ksm'
2018-03-23 07:09:54,909 - mom.Policy - INFO - Loaded policy '04-cputune'
2018-03-23 07:09:54,951 - mom.Policy - INFO - Loaded policy '05-iotune'
2018-03-23 07:09:54,956 - mom.PolicyEngine - INFO - Policy Engine starting
2018-03-23 07:09:54,957 - mom.RPCServer - INFO - Using unix socket
/var/run/vdsm/mom-vdsm.sock
2018-03-23 07:09:54,957 - mom.RPCServer - INFO - RPC Server starting
2018-03-23 07:10:10,020 - mom.Controllers.KSM - INFO - Updating KSM
configuration: pages_to_scan:0 merge_across_nodes:1 run:0 sleep_millisecs:0
lago-basic-suite-master-host-0/_var_log/vdsm/mom.log (END)
*</error>*
6:10 p.m.
On Fri, Mar 23, 2018 at 8:02 AM, Dafna Ron <dron(a)redhat.com> wrote:
Hi,
we had a failure reported in CQ for change: https://gerrit.ovirt.org/#/c/8
9338/ - Bump 1.5.2-1.
I don't think the failure is related to the change.
Definitely not related
it seems to be an issue with mom failing to start.
+Martin
*Link and headline of suspected patches:
https://gerrit.ovirt.org/#/c/89338/ <https://gerrit.ovirt.org/#/c/89338/> -
Bump 1.5.2-1Link to
Job:http://jenkins.ovirt.org/job/ovirt-master_change-queue-tester/6512/
<http://jenkins.ovirt.org/job/ovirt-master_change-queue-tester/6512/>Link
to all
logs:http://jenkins.ovirt.org/job/ovirt-master_change-queue-tester/6512/a...
<http://jenkins.ovirt.org/job/ovirt-master_change-queue-tester/6512/artifa...
error snippet from the log: <error>*
*engine: *018-03-23 07:12:33,435-04 ERROR
[org.ovirt.engine.core.bll.host.HostUpgradeManager]
(EE-ManagedThreadFactory-commandCoordinator-Thread-1)
[7a4f81e6-6ae6-4444-ace6-de757e78f41e] Failed to run check-update of host
'lago-basic-suite-master-
host-0'.
2018-03-23 07:12:33,436-04 ERROR
[org.ovirt.engine.core.bll.hostdeploy.HostUpdatesChecker]
(EE-ManagedThreadFactory-commandCoordinator-Thread-1)
[7a4f81e6-6ae6-4444-ace6-de757e78f41e] Failed to check if updates are
available for host 'lag
o-basic-suite-master-host-0' with error message 'Failed to run
check-update of host 'lago-basic-suite-master-host-0'.'
2018-03-23 07:12:33,441-04 ERROR [org.ovirt.engine.core.dal.dbb
roker.auditloghandling.AuditLogDirector]
(EE-ManagedThreadFactory-commandCoordinator-Thread-1)
[7a4f81e6-6ae6-4444-ace6-de757e78f41e] EVENT_ID:
HOST_AVAILABLE_UPDATES_FAILED(8
39), Failed to check for available updates on host
lago-basic-suite-master-host-0 with message 'Failed to run check-update of
host 'lago-basic-suite-master-host-0'.'.
2018-03-23 07:12:33,566-04 DEBUG [org.ovirt.engine.core.vdsbrok
er.vdsbroker.GetAllVmStatsVDSCommand]
(EE-ManagedThreadFactory-engineScheduled-Thread-11)
[] START, GetAllVmStatsVDSCommand(HostName =
lago-basic-suite-master-host-0, VdsIdVDS
CommandParametersBase:{hostId='dc339bcf-b769-41f6-8252-c61fffd56b17'}),
log id: 4eb92a5a
2018-03-23 07:12:33,567-04 DEBUG
[org.ovirt.vdsm.jsonrpc.client.reactors.stomp.impl.Message]
(EE-ManagedThreadFactory-engineScheduled-Thread-11) [] SEND
destination:jms.topic.vdsm_requests
reply-to:jms.topic.vdsm_responses
content-length:103
*host-0:*
vdsm log show: momStatus': 'inactive'
*Mom log: *
2018-03-23 07:09:48,849 - mom - INFO - MOM starting
2018-03-23 07:09:48,864 - mom.HostMonitor - INFO - Host Monitor starting
2018-03-23 07:09:48,867 - mom - INFO - hypervisor interface
vdsmjsonrpcclient
2018-03-23 07:09:48,983 - mom - ERROR - Failed to initialize MOM threads
Traceback (most recent call last):
File "/usr/lib/python2.7/site-packages/mom/__init__.py", line 29, in run
hypervisor_iface = self.get_hypervisor_interface()
File "/usr/lib/python2.7/site-packages/mom/__init__.py", line 217, in
get_hypervisor_interface
return module.instance(self.config)
File
"/usr/lib/python2.7/site-packages/mom/HypervisorInterfaces/vdsmjsonrpcclientInterface.py",
line 96, in instance
return JsonRpcVdsmClientInterface()
File
"/usr/lib/python2.7/site-packages/mom/HypervisorInterfaces/vdsmjsonrpcclientInterface.py",
line 31, in __init__
self._vdsm_api = client.connect(host="localhost")
File "/usr/lib/python2.7/site-packages/vdsm/client.py", line 139, in
connect
raise ConnectionError(host, port, use_tls, timeout, e)
ConnectionError: Connection to localhost:54321 with use_tls=True,
timeout=60 failed: [Errno 111] Connection refused
2018-03-23 07:09:54,475 - mom - INFO - MOM starting
2018-03-23 07:09:54,499 - mom.HostMonitor - INFO - Host Monitor starting
2018-03-23 07:09:54,500 - mom - INFO - hypervisor interface
vdsmjsonrpcclient
2018-03-23 07:09:54,725 - mom.HostMonitor - INFO - HostMonitor is ready
2018-03-23 07:09:54,813 - mom.GuestManager - INFO - Guest Manager
starting: multi-thread
2018-03-23 07:09:54,819 - mom.Policy - INFO - Loaded policy '00-defines'
2018-03-23 07:09:54,820 - mom.Policy - INFO - Loaded policy '01-parameters'
2018-03-23 07:09:54,841 - mom.Policy - INFO - Loaded policy '02-balloon'
2018-03-23 07:09:54,871 - mom.Policy - INFO - Loaded policy '03-ksm'
2018-03-23 07:09:54,909 - mom.Policy - INFO - Loaded policy '04-cputune'
2018-03-23 07:09:54,951 - mom.Policy - INFO - Loaded policy '05-iotune'
2018-03-23 07:09:54,956 - mom.PolicyEngine - INFO - Policy Engine starting
2018-03-23 07:09:54,957 - mom.RPCServer - INFO - Using unix socket
/var/run/vdsm/mom-vdsm.sock
2018-03-23 07:09:54,957 - mom.RPCServer - INFO - RPC Server starting
2018-03-23 07:10:10,020 - mom.Controllers.KSM - INFO - Updating KSM
configuration: pages_to_scan:0 merge_across_nodes:1 run:0 sleep_millisecs:0
lago-basic-suite-master-host-0/_var_log/vdsm/mom.log (END)
*</error>*
_______________________________________________
Infra mailing list
Infra(a)ovirt.org
http://lists.ovirt.org/mailman/listinfo/infra
--
GREG SHEREMETA
SENIOR SOFTWARE ENGINEER - TEAM LEAD - RHV UX
Red Hat NA
<https://www.redhat.com/>
gshereme(a)redhat.com IRC: gshereme
<https://red.ht/sig>
2218
days inactive
2219
days old
1 comments
2 participants
participants (2)
-
Dafna Ron -
Greg Sheremeta