10:23 p.m.
--_000_85068B30E9EA40D5883D6574AC1EC5FBbaicanadacom_
Content-Type: text/plain; charset="us-ascii"
Content-Transfer-Encoding: quoted-printable
OK after a host reboot I was able to get the Engine VM up again and into th=
e Web interface. Although continually when I try a second host to the data=
center within oVirt I run into the error:
"Host mennode2 cannot access the Storage Domain(s) hosted_storage attached =
to the Data Center Train1. Setting Host state to Non Operational."
Note: I can successfully read (and mount) the NFS exports oVirt is complai=
ning about:
[root@mennode2 ~]# showmount -e menmaster.traindemo.local
Export list for menmaster.traindemo.local:
/shares *
/shares/exports *
/shares/data *
/shares/isos *
[root@mennode2 ~]#
[root@mennode2 tmp]# mount -t nfs menmaster.traindemo.local:/shares test
[root@mennode2 tmp]# cd test
[root@mennode2 test]# ls
7d18ff24-57a3-4b4a-9934-0263191fe2e4 data __DIRECT_IO_TEST__ exports is=
os
[root@mennode2 test]#
One thing I DO notice is the path of my exports is the difference in the pa=
th for the hosted_storage domain. I wonder if the second host would have i=
ssues resolving this?
Data. =3D=3D> menmaster.traindemo.local:/shares/data
Export =3D=3D> menmaster.traindemo.local:/shares/exports
Hosted_storage. =3D=3D> localhost:/shares
ISO. =3D=3D> menmaster.traindemo.local:/shares/exports
Below I have copied exports from the VDSM and OVIRT-ENGINE log and have cop=
ied the output of the hosted-engine.conf file. Any help in pinpointing the=
source of the problem is greatly appreciated!!
VSDM logs:
2017-09-27 23:10:01,700-0400 INFO (jsonrpc/0) [jsonrpc.JsonRpcServer] RPC =
call Host.getStats succeeded in 0.06 seconds (__init__:539)
2017-09-27 23:10:03,104-0400 INFO (jsonrpc/5) [vdsm.api] START getSpmStatu=
s(spUUID=3Du'59c7f8f3-0063-00a8-02c7-0000000000f3', options=3DNone) from=3D=
::ffff:10.0.0.227,39748, flow_id=3D19e2dbb3, task_id=3D7a05a5bd-9b15-43be-8=
90d-c6f5d7650e5c (api:46)
2017-09-27 23:10:03,109-0400 INFO (jsonrpc/5) [vdsm.api] FINISH getSpmStat=
us return=3D{'spm_st': {'spmId': 1, 'spmStatus': 'SPM',
'spmLver': 8L}} fro=
m=3D::ffff:10.0.0.227,39748, flow_id=3D19e2dbb3, task_id=3D7a05a5bd-9b15-43=
be-890d-c6f5d7650e5c (api:52)
2017-09-27 23:10:03,110-0400 INFO (jsonrpc/5) [jsonrpc.JsonRpcServer] RPC =
call StoragePool.getSpmStatus succeeded in 0.01 seconds (__init__:539)
2017-09-27 23:10:03,189-0400 INFO (jsonrpc/7) [vdsm.api] START getStorageP=
oolInfo(spUUID=3Du'59c7f8f3-0063-00a8-02c7-0000000000f3', options=3DNone) f=
rom=3D::ffff:10.0.0.227,39866, flow_id=3D19e2dbb3, task_id=3D0f8b49e9-9d82-=
457e-a2a2-39dc7ed9f022 (api:46)
2017-09-27 23:10:03,196-0400 INFO (jsonrpc/7) [vdsm.api] FINISH getStorage=
PoolInfo return=3D{'info': {'name': 'No Description',
'isoprefix': u'/rhev/=
data-center/mnt/menmaster.traindemo.local:_shares_isos/da001a29-eca5-44d6-a=
097-129dd9be623f/images/11111111-1111-1111-1111-111111111111', 'pool_status=
': 'connected', 'lver': 8L, 'domains':
u'da001a29-eca5-44d6-a097-129dd9be62=
3f:Active,f36157cc-b25a-400a-ab0f-a071e8a8eea7:Active,7d18ff24-57a3-4b4a-99=
34-0263191fe2e4:Active,795d4a1d-3ceb-4773-99de-8e7cf05112f3:Active', 'maste=
r_uuid': u'f36157cc-b25a-400a-ab0f-a071e8a8eea7', 'version':
'4', 'spm_id':=
1, 'type': 'NFS', 'master_ver': 1}, 'dominfo':
{u'da001a29-eca5-44d6-a097-=
129dd9be623f': {'status': u'Active', 'diskfree':
'1044166737920', 'isoprefi=
x': u'/rhev/data-center/mnt/menmaster.traindemo.local:_shares_isos/da001a29=
-eca5-44d6-a097-129dd9be623f/images/11111111-1111-1111-1111-111111111111', =
'alerts': [], 'disktotal': '1049702170624', 'version': 0},
u'f36157cc-b25a-=
400a-ab0f-a071e8a8eea7': {'status': u'Active', 'diskfree':
'1044166737920',=
'isoprefix': '', 'alerts': [], 'disktotal':
'1049702170624', 'version': 4}=
, u'7d18ff24-57a3-4b4a-9934-0263191fe2e4': {'status': u'Active',
'diskfree'=
: '1044166737920', 'isoprefix': '', 'alerts': [],
'disktotal': '10497021706=
24', 'version': 4}, u'795d4a1d-3ceb-4773-99de-8e7cf05112f3':
{'status': u'A=
ctive', 'diskfree': '1044166737920', 'isoprefix': '',
'alerts': [], 'diskto=
tal': '1049702170624', 'version': 0}}} from=3D::ffff:10.0.0.227,39866,
flow=
_id=3D19e2dbb3, task_id=3D0f8b49e9-9d82-457e-a2a2-39dc7ed9f022 (api:52)
2017-09-27 23:10:03,198-0400 INFO (jsonrpc/7) [jsonrpc.JsonRpcServer] RPC =
call StoragePool.getInfo succeeded in 0.01 seconds (__init__:539)
2017-09-27 23:10:04,212-0400 INFO (Reactor thread) [ProtocolDetector.Accep=
torImpl] Accepted connection from ::1:35980 (protocoldetector:72)
2017-09-27 23:10:04,223-0400 INFO (Reactor thread) [ProtocolDetector.Detec=
tor] Detected protocol stomp from ::1:35980 (protocoldetector:127)
2017-09-27 23:10:04,224-0400 INFO (Reactor thread) [Broker.StompAdapter] P=
rocessing CONNECT request (stompreactor:103)
2017-09-27 23:10:04,225-0400 INFO (JsonRpc (StompReactor)) [Broker.StompAd=
apter] Subscribe command received (stompreactor:132)
2017-09-27 23:10:04,622-0400 INFO (monitor/7d18ff2) [storage.SANLock] Acqu=
iring host id for domain 7d18ff24-57a3-4b4a-9934-0263191fe2e4 (id=3D1, asyn=
c=3DTrue) (clusterlock:288)
2017-09-27 23:10:04,623-0400 ERROR (monitor/7d18ff2) [storage.Monitor] Erro=
r acquiring host id 1 for domain 7d18ff24-57a3-4b4a-9934-0263191fe2e4 (moni=
tor:570)
Traceback (most recent call last):
File "/usr/share/vdsm/storage/monitor.py", line 567, in _acquireHostId
self.domain.acquireHostId(self.hostId, async=3DTrue)
File "/usr/share/vdsm/storage/sd.py", line 785, in acquireHostId
self._manifest.acquireHostId(hostId, async)
File "/usr/share/vdsm/storage/sd.py", line 449, in acquireHostId
self._domainLock.acquireHostId(hostId, async)
File "/usr/lib/python2.7/site-packages/vdsm/storage/clusterlock.py", line=
319, in acquireHostId
raise se.AcquireHostIdFailure(self._sdUUID, e)
AcquireHostIdFailure: Cannot acquire host id: (u'7d18ff24-57a3-4b4a-9934-02=
63191fe2e4', SanlockException(22, 'Sanlock lockspace add failure',
'Invalid=
argument'))
2017-09-27 23:10:04,647-0400 INFO (jsonrpc/6) [jsonrpc.JsonRpcServer] RPC =
call Host.getHardwareInfo succeeded in 0.00 seconds (__init__:539)
2017-09-27 23:10:04,654-0400 INFO (jsonrpc/3) [vdsm.api] START repoStats(o=
ptions=3DNone) from=3D::1,35980, task_id=3D9732972d-9a76-4349-9c65-53e57b24=
72d6 (api:46)
2017-09-27 23:10:04,655-0400 INFO (jsonrpc/3) [vdsm.api] FINISH repoStats =
return=3D{u'da001a29-eca5-44d6-a097-129dd9be623f': {'code': 0,
'actual': Tr=
ue, 'version': 0, 'acquired': True, 'delay':
'0.000357054', 'lastCheck': '4=
.1', 'valid': True}, u'f36157cc-b25a-400a-ab0f-a071e8a8eea7':
{'code': 0, '=
actual': True, 'version': 4, 'acquired': True, 'delay':
'0.000531828', 'las=
tCheck': '4.0', 'valid': True},
u'7d18ff24-57a3-4b4a-9934-0263191fe2e4': {'=
code': 0, 'actual': True, 'version': 4, 'acquired': False,
'delay': '0.0002=
05228', 'lastCheck': '0.0', 'valid': True},
u'795d4a1d-3ceb-4773-99de-8e7cf=
05112f3': {'code': 0, 'actual': True, 'version': 0,
'acquired': True, 'dela=
y': '0.000352165', 'lastCheck': '4.1', 'valid': True}}
from=3D::1,35980, ta=
sk_id=3D9732972d-9a76-4349-9c65-53e57b2472d6 (api:52)
2017-09-27 23:10:04,656-0400 INFO (jsonrpc/3) [jsonrpc.JsonRpcServer] RPC =
call Host.getStorageRepoStats succeeded in 0.00 seconds (__init__:539)
2017-09-27 23:10:07,717-0400 INFO (Reactor thread) [ProtocolDetector.Accep=
torImpl] Accepted connection from ::1:35982 (protocoldetector:72)
2017-09-27 23:10:07,728-0400 INFO (Reactor thread) [ProtocolDetector.Detec=
tor] Detected protocol stomp from ::1:35982 (protocoldetector:127)
2017-09-27 23:10:07,729-0400 INFO (Reactor thread) [Broker.StompAdapter] P=
rocessing CONNECT request (stompreactor:103)
2017-09-27 23:10:07,730-0400 INFO (JsonRpc (StompReactor)) [Broker.StompAd=
apter] Subscribe command received (stompreactor:132)
2017-09-27 23:10:08,152-0400 INFO (jsonrpc/2) [jsonrpc.JsonRpcServer] RPC =
call Host.getHardwareInfo succeeded in 0.00 seconds (__init__:539)
2017-09-27 23:10:08,159-0400 INFO (jsonrpc/1) [vdsm.api] START repoStats(o=
ptions=3DNone) from=3D::1,35982, task_id=3D80eb282b-f297-4f98-a368-7d5627ac=
50f3 (api:46)
2017-09-27 23:10:08,160-0400 INFO (jsonrpc/1) [vdsm.api] FINISH repoStats =
return=3D{u'da001a29-eca5-44d6-a097-129dd9be623f': {'code': 0,
'actual': Tr=
ue, 'version': 0, 'acquired': True, 'delay':
'0.000357054', 'lastCheck': '7=
.6', 'valid': True}, u'f36157cc-b25a-400a-ab0f-a071e8a8eea7':
{'code': 0, '=
actual': True, 'version': 4, 'acquired': True, 'delay':
'0.000531828', 'las=
tCheck': '7.5', 'valid': True},
u'7d18ff24-57a3-4b4a-9934-0263191fe2e4': {'=
code': 0, 'actual': True, 'version': 4, 'acquired': False,
'delay': '0.0002=
05228', 'lastCheck': '3.5', 'valid': True},
u'795d4a1d-3ceb-4773-99de-8e7cf=
05112f3': {'code': 0, 'actual': True, 'version': 0,
'acquired': True, 'dela=
y': '0.000352165', 'lastCheck': '7.6', 'valid': True}}
from=3D::1,35982, ta=
sk_id=3D80eb282b-f297-4f98-a368-7d5627ac50f3 (api:52)
2017-09-27 23:10:08,171-0400 INFO (Reactor thread) [ProtocolDetector.Accep=
torImpl] Accepted connection from ::1:35984 (protocoldetector:72)
2017-09-27 23:10:08,198-0400 INFO (Reactor thread) [ProtocolDetector.Detec=
tor] Detected protocol stomp from ::1:35984 (protocoldetector:127)
2017-09-27 23:10:08,199-0400 INFO (Reactor thread) [Broker.StompAdapter] P=
rocessing CONNECT request (stompreactor:103)
2017-09-27 23:10:08,201-0400 INFO (JsonRpc (StompReactor)) [Broker.StompAd=
apter] Subscribe command received (stompreactor:132)
2017-09-27 23:10:08,374-0400 INFO (jsonrpc/1) [jsonrpc.JsonRpcServer] RPC =
call Host.getStats succeeded in 0.22 seconds (__init__:539)
2017-09-27 23:10:08,728-0400 INFO (Reactor thread) [ProtocolDetector.Accep=
torImpl] Accepted connection from ::1:35987 (protocoldetector:72)
2017-09-27 23:10:08,732-0400 INFO (jsonrpc/4) [jsonrpc.JsonRpcServer] RPC =
call Host.getHardwareInfo succeeded in 0.00 seconds (__init__:539)
2017-09-27 23:10:08,738-0400 INFO (jsonrpc/0) [jsonrpc.JsonRpcServer] RPC =
call VM.getStats succeeded in 0.00 seconds (__init__:539)
2017-09-27 23:10:08,756-0400 INFO (Reactor thread) [ProtocolDetector.Detec=
tor] Detected protocol stomp from ::1:35987 (protocoldetector:127)
2017-09-27 23:10:08,757-0400 INFO (Reactor thread) [Broker.StompAdapter] P=
rocessing CONNECT request (stompreactor:103)
2017-09-27 23:10:08,771-0400 INFO (JsonRpc (StompReactor)) [Broker.StompAd=
apter] Subscribe command received (stompreactor:132)
2017-09-27 23:10:09,286-0400 INFO (jsonrpc/5) [jsonrpc.JsonRpcServer] RPC =
call Host.getHardwareInfo succeeded in 0.00 seconds (__init__:539)
2017-09-27 23:10:09,395-0400 INFO (jsonrpc/7) [jsonrpc.JsonRpcServer] RPC =
call Host.getCapabilities succeeded in 0.10 seconds (__init__:539)
2017-09-27 23:10:10,590-0400 INFO (itmap/0) [IOProcessClient] Starting cli=
ent ioprocess-37 (__init__:330)
ENGINE LOGS:
2017-09-27 23:07:58,154-04 INFO [org.ovirt.engine.core.bll.hostdeploy.VdsD=
eployBase] (VdsDeploy) [3f898655] Retrieving installation logs to: '/var/lo=
g/ovirt-engine/host-deploy/ovirt-host-mgmt-20170927230758-10.0.0.222-3f8986=
55.log'
2017-09-27 23:07:58,316-04 INFO [org.ovirt.engine.core.bll.hostdeploy.VdsD=
eployBase] (VdsDeploy) [3f898655] Stage: Termination
2017-09-27 23:07:58,458-04 INFO [org.ovirt.engine.core.dal.dbbroker.auditl=
oghandling.AuditLogDirector] (pool-7-thread-1) [3f898655] EVENT_ID: HOST_AV=
AILABLE_UPDATES_FINISHED(885), Correlation ID: null, Call Stack: null, Cust=
om ID: null, Custom Event ID: -1, Message: Check for available updates on h=
ost mennode2 was completed successfully with message 'no updates found.'.
2017-09-27 23:07:58,874-04 INFO [org.ovirt.engine.core.bll.hostdeploy.VdsD=
eployBase] (VdsDeploy) [3f898655] Stage: Misc configuration
2017-09-27 23:07:58,888-04 INFO [org.ovirt.engine.core.bll.hostdeploy.VdsD=
eployBase] (VdsDeploy) [3f898655] Stage: Transaction commit
2017-09-27 23:07:59,029-04 INFO [org.ovirt.engine.core.bll.hostdeploy.VdsD=
eployBase] (VdsDeploy) [3f898655] Stage: Closing up
2017-09-27 23:07:59,039-04 INFO [org.ovirt.engine.core.bll.hostdeploy.VdsD=
eployBase] (VdsDeploy) [3f898655] Stage: Pre-termination
2017-09-27 23:07:59,092-04 INFO [org.ovirt.engine.core.bll.hostdeploy.VdsD=
eployBase] (VdsDeploy) [3f898655] Retrieving installation logs to: '/var/lo=
g/ovirt-engine/host-deploy/ovirt-host-mgmt-20170927230759-menmaster.trainde=
mo.local-3f898655.log'
2017-09-27 23:07:59,262-04 INFO [org.ovirt.engine.core.bll.hostdeploy.VdsD=
eployBase] (VdsDeploy) [3f898655] Stage: Termination
2017-09-27 23:07:59,375-04 INFO [org.ovirt.engine.core.dal.dbbroker.auditl=
oghandling.AuditLogDirector] (pool-7-thread-2) [3f898655] EVENT_ID: HOST_AV=
AILABLE_UPDATES_FINISHED(885), Correlation ID: null, Call Stack: null, Cust=
om ID: null, Custom Event ID: -1, Message: Check for available updates on h=
ost menmaster was completed successfully with message 'no updates found.'.
2017-09-27 23:08:38,705-04 INFO [org.ovirt.engine.core.bll.ActivateVdsComm=
and] (default task-13) [75de01a6-efd2-418f-a6b1-b4ef1f38923f] Lock Acquired=
to object 'EngineLock:{exclusiveLocks=3D'[0ef7ec10-af19-4afa-980b-77a772c4=
e014=3DVDS]', sharedLocks=3D''}'
2017-09-27 23:08:38,776-04 INFO [org.ovirt.engine.core.bll.ActivateVdsComm=
and] (org.ovirt.thread.pool-6-thread-2) [75de01a6-efd2-418f-a6b1-b4ef1f3892=
3f] Running command: ActivateVdsCommand internal: false. Entities affected =
: ID: 0ef7ec10-af19-4afa-980b-77a772c4e014 Type: VDSAction group MANIPULAT=
E_HOST with role type ADMIN
2017-09-27 23:08:38,777-04 INFO [org.ovirt.engine.core.bll.ActivateVdsComm=
and] (org.ovirt.thread.pool-6-thread-2) [75de01a6-efd2-418f-a6b1-b4ef1f3892=
3f] Before acquiring lock in order to prevent monitoring for host 'mennode2=
' from data-center 'Train1'
2017-09-27 23:08:38,778-04 INFO [org.ovirt.engine.core.bll.lock.InMemoryLo=
ckManager] (org.ovirt.thread.pool-6-thread-2) [75de01a6-efd2-418f-a6b1-b4ef=
1f38923f] Failed to acquire lock and wait lock 'HostEngineLock:{exclusiveLo=
cks=3D'[0ef7ec10-af19-4afa-980b-77a772c4e014=3DVDS_INIT]',
sharedLocks=3D''=
}'
2017-09-27 23:08:38,995-04 INFO [org.ovirt.engine.core.bll.ActivateVdsComm=
and] (org.ovirt.thread.pool-6-thread-2) [75de01a6-efd2-418f-a6b1-b4ef1f3892=
3f] Lock acquired, from now a monitoring of host will be skipped for host '=
mennode2' from data-center 'Train1'
2017-09-27 23:08:39,003-04 INFO [org.ovirt.engine.core.vdsbroker.SetVdsSta=
tusVDSCommand] (org.ovirt.thread.pool-6-thread-2) [75de01a6-efd2-418f-a6b1-=
b4ef1f38923f] START, SetVdsStatusVDSCommand(HostName =3D mennode2, SetVdsSt=
atusVDSCommandParameters:{runAsync=3D'true', hostId=3D'0ef7ec10-af19-4afa-9=
80b-77a772c4e014', status=3D'Unassigned',
nonOperationalReason=3D'NONE', st=
opSpmFailureLogged=3D'false', maintenanceReason=3D'null'}), log id:
78cc1ca=
f
2017-09-27 23:08:39,027-04 INFO [org.ovirt.engine.core.vdsbroker.SetVdsSta=
tusVDSCommand] (org.ovirt.thread.pool-6-thread-2) [75de01a6-efd2-418f-a6b1-=
b4ef1f38923f] FINISH, SetVdsStatusVDSCommand, log id: 78cc1caf
2017-09-27 23:08:39,040-04 INFO [org.ovirt.engine.core.bll.ActivateVdsComm=
and] (org.ovirt.thread.pool-6-thread-2) [75de01a6-efd2-418f-a6b1-b4ef1f3892=
3f] Activate host finished. Lock released. Monitoring can run now for host =
'mennode2' from data-center 'Train1'
2017-09-27 23:08:39,050-04 INFO [org.ovirt.engine.core.dal.dbbroker.auditl=
oghandling.AuditLogDirector] (org.ovirt.thread.pool-6-thread-2) [75de01a6-e=
fd2-418f-a6b1-b4ef1f38923f] EVENT_ID: VDS_ACTIVATE(16), Correlation ID: 75d=
e01a6-efd2-418f-a6b1-b4ef1f38923f, Job ID: 2e40097e-e991-45cc-bd49-3176822e=
bb5a, Call Stack: null, Custom ID: null, Custom Event ID: -1, Message: Acti=
vation of host mennode2 initiated by admin@internal-authz.
2017-09-27 23:08:39,059-04 INFO [org.ovirt.engine.core.bll.ActivateVdsComm=
and] (org.ovirt.thread.pool-6-thread-2) [75de01a6-efd2-418f-a6b1-b4ef1f3892=
3f] Lock freed to object 'EngineLock:{exclusiveLocks=3D'[0ef7ec10-af19-4afa=
-980b-77a772c4e014=3DVDS]', sharedLocks=3D''}'
2017-09-27 23:08:43,143-04 INFO [org.ovirt.engine.core.vdsbroker.vdsbroker=
.GetHardwareInfoVDSCommand] (DefaultQuartzScheduler7) [5437c1b7] START, Get=
HardwareInfoVDSCommand(HostName =3D mennode2, VdsIdAndVdsVDSCommandParamete=
rsBase:{runAsync=3D'true',
hostId=3D'0ef7ec10-af19-4afa-980b-77a772c4e014',=
vds=3D'Host[mennode2,0ef7ec10-af19-4afa-980b-77a772c4e014]'}), log id: 2bc=
be26e
2017-09-27 23:08:44,150-04 INFO [org.ovirt.engine.core.vdsbroker.vdsbroker=
.GetHardwareInfoVDSCommand] (DefaultQuartzScheduler7) [5437c1b7] FINISH, Ge=
tHardwareInfoVDSCommand, log id: 2bcbe26e
2017-09-27 23:08:44,186-04 INFO [org.ovirt.engine.core.bll.HandleVdsCpuFla=
gsOrClusterChangedCommand] (DefaultQuartzScheduler7) [12ef27d8] Running com=
mand: HandleVdsCpuFlagsOrClusterChangedCommand internal: true. Entities aff=
ected : ID: 0ef7ec10-af19-4afa-980b-77a772c4e014 Type: VDS
2017-09-27 23:08:44,214-04 INFO [org.ovirt.engine.core.bll.InitVdsOnUpComm=
and] (DefaultQuartzScheduler7) [63cdc728] Running command: InitVdsOnUpComma=
nd internal: true. Entities affected : ID: 59c7f8f3-0063-00a8-02c7-0000000=
000f3 Type: StoragePool
2017-09-27 23:08:44,234-04 INFO [org.ovirt.engine.core.bll.storage.pool.Co=
nnectHostToStoragePoolServersCommand] (DefaultQuartzScheduler7) [7670f89e] =
Running command: ConnectHostToStoragePoolServersCommand internal: true. Ent=
ities affected : ID: 59c7f8f3-0063-00a8-02c7-0000000000f3 Type: StoragePoo=
l
2017-09-27 23:08:44,257-04 INFO [org.ovirt.engine.core.vdsbroker.vdsbroker=
.ConnectStorageServerVDSCommand] (DefaultQuartzScheduler7) [7670f89e] START=
, ConnectStorageServerVDSCommand(HostName =3D mennode2, StorageServerConnec=
tionManagementVDSParameters:{runAsync=3D'true', hostId=3D'0ef7ec10-af19-4af=
a-980b-77a772c4e014', storagePoolId=3D'59c7f8f3-0063-00a8-02c7-0000000000f3=
', storageType=3D'NFS',
connectionList=3D'[StorageServerConnections:{id=3D'=
aac63433-88fb-4daf-8c83-871a85b3a49d', connection=3D'menmaster.traindemo.lo=
cal:/shares/data', iqn=3D'null', vfsType=3D'null',
mountOptions=3D'null', n=
fsVersion=3D'AUTO', nfsRetrans=3D'null', nfsTimeo=3D'null',
iface=3D'null',=
netIfaceName=3D'null'}, StorageServerConnections:{id=3D'5adb7a4b-ebbd-43a4=
-b5cb-8c8eb3d6f1ee', connection=3D'localhost:/shares', iqn=3D'null',
vfsTyp=
e=3D'null', mountOptions=3D'null', nfsVersion=3D'null',
nfsRetrans=3D'null'=
, nfsTimeo=3D'null', iface=3D'null', netIfaceName=3D'null'},
StorageServerC=
onnections:{id=3D'9eb3b1e2-4063-46cd-ba4e-440c4048e92a', connection=3D'menm=
aster.traindemo.local:/shares/isos', iqn=3D'null', vfsType=3D'null',
mountO=
ptions=3D'null', nfsVersion=3D'AUTO', nfsRetrans=3D'null',
nfsTimeo=3D'null=
', iface=3D'null', netIfaceName=3D'null'},
StorageServerConnections:{id=3D'=
1c9215a1-4154-4a78-b8ff-565c3a237c58', connection=3D'menmaster.traindemo.lo=
cal:/shares/exports', iqn=3D'null', vfsType=3D'null',
mountOptions=3D'null'=
, nfsVersion=3D'AUTO', nfsRetrans=3D'null', nfsTimeo=3D'null',
iface=3D'nul=
l', netIfaceName=3D'null'}]'}), log id: 5625b02f
2017-09-27 23:08:48,370-04 ERROR [org.ovirt.engine.core.dal.dbbroker.auditl=
oghandling.AuditLogDirector] (DefaultQuartzScheduler7) [7670f89e] EVENT_ID:=
VDS_STORAGES_CONNECTION_FAILED(188), Correlation ID: null, Call Stack: nul=
l, Custom ID: null, Custom Event ID: -1, Message: Failed to connect Host me=
nnode2 to the Storage Domains hosted_storage.
2017-09-27 23:08:48,370-04 INFO [org.ovirt.engine.core.vdsbroker.vdsbroker=
.ConnectStorageServerVDSCommand] (DefaultQuartzScheduler7) [7670f89e] FINIS=
H, ConnectStorageServerVDSCommand, return: {aac63433-88fb-4daf-8c83-871a85b=
3a49d=3D0, 5adb7a4b-ebbd-43a4-b5cb-8c8eb3d6f1ee=3D477, 9eb3b1e2-4063-46cd-b=
a4e-440c4048e92a=3D0, 1c9215a1-4154-4a78-b8ff-565c3a237c58=3D0}, log id: 56=
25b02f
2017-09-27 23:08:48,386-04 ERROR [org.ovirt.engine.core.dal.dbbroker.auditl=
oghandling.AuditLogDirector] (DefaultQuartzScheduler7) [7670f89e] EVENT_ID:=
STORAGE_DOMAIN_ERROR(996), Correlation ID: null, Call Stack: null, Custom =
ID: null, Custom Event ID: -1, Message: The error message for connection lo=
calhost:/shares returned by VDSM was: Problem while trying to mount target
2017-09-27 23:08:48,389-04 ERROR [org.ovirt.engine.core.bll.storage.connect=
ion.BaseFsStorageHelper] (DefaultQuartzScheduler7) [7670f89e] The connectio=
n with details 'localhost:/shares' failed because of error code '477' and
e=
rror message is: problem while trying to mount target
2017-09-27 23:08:48,401-04 INFO [org.ovirt.engine.core.bll.storage.pool.Co=
nnectHostToStoragePoolServersCommand] (DefaultQuartzScheduler7) [7670f89e] =
Host 'mennode2' storage connection was failed
2017-09-27 23:08:48,411-04 WARN [org.ovirt.engine.core.dal.dbbroker.auditl=
oghandling.AuditLogDirector] (DefaultQuartzScheduler7) [7670f89e] EVENT_ID:=
CONNECT_STORAGE_SERVERS_FAILED(994), Correlation ID: 7670f89e, Call Stack:=
null, Custom ID: null, Custom Event ID: -1, Message: Failed to connect Hos=
t mennode2 to Storage Servers
2017-09-27 23:08:48,423-04 INFO [org.ovirt.engine.core.vdsbroker.vdsbroker=
.ConnectStoragePoolVDSCommand] (org.ovirt.thread.pool-6-thread-8) [] START,=
ConnectStoragePoolVDSCommand(HostName =3D mennode2, ConnectStoragePoolVDSC=
ommandParameters:{runAsync=3D'true', hostId=3D'0ef7ec10-af19-4afa-980b-77a7=
72c4e014', vdsId=3D'0ef7ec10-af19-4afa-980b-77a772c4e014', storagePoolId=3D=
'59c7f8f3-0063-00a8-02c7-0000000000f3', masterVersion=3D'1'}), log id:
3d5c=
5d79
2017-09-27 23:08:48,423-04 INFO [org.ovirt.engine.core.vdsbroker.vdsbroker=
.ConnectStoragePoolVDSCommand] (org.ovirt.thread.pool-6-thread-8) [] Execut=
ing with domain map: {7d18ff24-57a3-4b4a-9934-0263191fe2e4=3Dactive, da001a=
29-eca5-44d6-a097-129dd9be623f=3Dactive, 795d4a1d-3ceb-4773-99de-8e7cf05112=
f3=3Dactive, f36157cc-b25a-400a-ab0f-a071e8a8eea7=3Dactive}
2017-09-27 23:08:49,656-04 INFO [org.ovirt.engine.core.vdsbroker.vdsbroker=
.ConnectStoragePoolVDSCommand] (org.ovirt.thread.pool-6-thread-8) [] FINISH=
, ConnectStoragePoolVDSCommand, log id: 3d5c5d79
2017-09-27 23:08:50,681-04 ERROR [org.ovirt.engine.core.vdsbroker.irsbroker=
.IrsProxy] (org.ovirt.thread.pool-6-thread-8) [] Domain '7d18ff24-57a3-4b4a=
-9934-0263191fe2e4:hosted_storage' was reported with error code '358'
2017-09-27 23:08:50,682-04 ERROR [org.ovirt.engine.core.bll.InitVdsOnUpComm=
and] (org.ovirt.thread.pool-6-thread-8) [] Storage Domain 'hosted_storage' =
of pool 'Train1' is in problem in host 'mennode2'
2017-09-27 23:08:50,690-04 ERROR [org.ovirt.engine.core.dal.dbbroker.auditl=
oghandling.AuditLogDirector] (org.ovirt.thread.pool-6-thread-8) [] EVENT_ID=
: VDS_STORAGE_VDS_STATS_FAILED(189), Correlation ID: 63cdc728, Call Stack: =
null, Custom ID: null, Custom Event ID: -1, Message: Host mennode2 reports =
about one of the Active Storage Domains as Problematic.
2017-09-27 23:08:50,726-04 INFO [org.ovirt.engine.core.bll.SetNonOperation=
alVdsCommand] (DefaultQuartzScheduler7) [748ef73f] Running command: SetNonO=
perationalVdsCommand internal: true. Entities affected : ID: 0ef7ec10-af19=
-4afa-980b-77a772c4e014 Type: VDS
2017-09-27 23:08:50,729-04 INFO [org.ovirt.engine.core.vdsbroker.SetVdsSta=
tusVDSCommand] (DefaultQuartzScheduler7) [748ef73f] START, SetVdsStatusVDSC=
ommand(HostName =3D mennode2, SetVdsStatusVDSCommandParameters:{runAsync=3D=
'true', hostId=3D'0ef7ec10-af19-4afa-980b-77a772c4e014',
status=3D'NonOpera=
tional', nonOperationalReason=3D'STORAGE_DOMAIN_UNREACHABLE', stopSpmFailur=
eLogged=3D'false', maintenanceReason=3D'null'}), log id: 2589acd2
2017-09-27 23:08:50,736-04 INFO [org.ovirt.engine.core.vdsbroker.SetVdsSta=
tusVDSCommand] (DefaultQuartzScheduler7) [748ef73f] FINISH, SetVdsStatusVDS=
Command, log id: 2589acd2
2017-09-27 23:08:50,807-04 WARN [org.ovirt.engine.core.dal.dbbroker.auditl=
oghandling.AuditLogDirector] (DefaultQuartzScheduler7) [748ef73f] EVENT_ID:=
VDS_SET_NONOPERATIONAL_DOMAIN(522), Correlation ID: 748ef73f, Job ID: e809=
41e4-3dbf-4159-8421-96d2d90160ca, Call Stack: null, Custom ID: null, Custom=
Event ID: -1, Message: Host mennode2 cannot access the Storage Domain(s) h=
osted_storage attached to the Data Center Train1. Setting Host state to Non=
-Operational.
2017-09-27 23:08:50,820-04 WARN [org.ovirt.engine.core.dal.dbbroker.auditl=
oghandling.AuditLogDirector] (DefaultQuartzScheduler7) [748ef73f] EVENT_ID:=
VDS_ALERT_FENCE_IS_NOT_CONFIGURED(9,000), Correlation ID: 63cdc728, Call S=
tack: null, Custom ID: null, Custom Event ID: -1, Message: Failed to verify=
Power Management configuration for Host mennode2.
2017-09-27 23:08:50,826-04 WARN [org.ovirt.engine.core.dal.dbbroker.auditl=
oghandling.AuditLogDirector] (DefaultQuartzScheduler7) [748ef73f] EVENT_ID:=
CONNECT_STORAGE_POOL_FAILED(995), Correlation ID: 63cdc728, Call Stack: nu=
ll, Custom ID: null, Custom Event ID: -1, Message: Failed to connect Host m=
ennode2 to Storage Pool Train1
2017-09-27 23:08:50,856-04 INFO [org.ovirt.engine.core.bll.HandleVdsVersio=
nCommand] (DefaultQuartzScheduler7) [42cbaee1] Running command: HandleVdsVe=
rsionCommand internal: true. Entities affected : ID: 0ef7ec10-af19-4afa-98=
0b-77a772c4e014 Type: VDS
2017-09-27 23:08:50,863-04 INFO [org.ovirt.engine.core.vdsbroker.monitorin=
g.HostMonitoring] (DefaultQuartzScheduler7) [42cbaee1] Host 'mennode2'(0ef7=
ec10-af19-4afa-980b-77a772c4e014) is already in NonOperational status for r=
eason 'STORAGE_DOMAIN_UNREACHABLE'. SetNonOperationalVds command is skipped=
.
/etc/ovirt-hosted-engine/hosted-engine.conf
fqdn=3Dengine.traindemo.local
vm_disk_id=3D0077cb58-7abe-4bdf-96b1-55cc85030d76
vm_disk_vol_id=3Dd5bd2690-7cfb-4aa2-b898-41579114ec08
vmid=3D564d290f-ae66-4352-9770-f2e062000358
storage=3Dmenmaster.traindemo.local:/shares
mnt_options=3D
conf=3D/var/run/ovirt-hosted-engine-ha/vm.conf
host_id=3D1
console=3Dvnc
domainType=3Dnfs3
spUUID=3D00000000-0000-0000-0000-000000000000
sdUUID=3D7d18ff24-57a3-4b4a-9934-0263191fe2e4
connectionUUID=3Daddfd5fc-8668-4d2a-9b0b-f801b21a98d8
ca_cert=3D/etc/pki/vdsm/libvirt-spice/ca-cert.pem
ca_subject=3D"C=3DEN, L=3DTest, O=3DTest, CN=3DTest"
vdsm_use_ssl=3Dtrue
gateway=3D10.0.0.1
bridge=3Dovirtmgmt
metadata_volume_UUID=3D156c7549-8e2e-4580-9825-81c3b13387ce
metadata_image_UUID=3Dc5db06ac-ea37-47bc-9065-b94eb230616a
lockspace_volume_UUID=3D5f4f4a63-5608-4daf-a034-2b8e83b5de42
lockspace_image_UUID=3D0d7f8661-3abd-44e8-87a7-3b85f2e29fda
conf_volume_UUID=3Dddffd499-294c-456f-bb0e-08e366380a22
conf_image_UUID=3D5f264ea1-4654-4e40-87aa-80fcaacc60d8
Thanks,
Alex Witte
On Sep 27, 2017, at 8:32 PM, Alexander Witte <alexander.witte(a)baicanada.com=
<mailto:alexander.witte@baicanada.com>> wrote:
/etc/ovirt-hosted-engine/hosted-engine.conf
--_000_85068B30E9EA40D5883D6574AC1EC5FBbaicanadacom_
Content-Type: text/html; charset="us-ascii"
Content-ID: <4C836429D3E8524F9E2744C6E02DEC5D(a)baicanada.local Content-Transfer-Encoding: quoted-printable
<html <head <meta http-equiv=3D"Content-Type"
content=3D"text/html; charset=3Dus-ascii"=
</head <body
style=3D"word-wrap: break-word; -webkit-nbsp-mode: space; -webkit-lin=
e-break: after-white-space;" class=3D"" <div
class=3D"">OK after a host reboot I was able to get the Engine VM up a=
gain and into the Web interface. Although continually when I try a se=
cond host to the datacenter within oVirt I run into the error:</div <div class=3D""><br class=3D"" </div <div
class=3D"">"Host mennode2 cannot access the Storage Domain(s) hos=
ted_storage attached to the Data Center Train1. Setting Host state to=
Non Operational."</div <div
class=3D""><br class=3D""
</div <div
class=3D"">Note: I can successfully read (and mount) the NFS exp=
orts oVirt is complaining about:</div <div
class=3D""><br class=3D""
</div <div class=3D"" <div style=3D"margin: 0px; line-height: normal;
font-family: 'Helvetica Neu=
e'; color: rgb(69, 69, 69);" class=3D"" <i
class=3D"">[root@mennode2 ~]# showmount -e
menmaster.traindemo.local</i>=
</div <div style=3D"margin: 0px;
line-height: normal; font-family: 'Helvetica Neu=
e'; color: rgb(69, 69, 69);" class=3D"" <i
class=3D"">Export list for menmaster.traindemo.local:</i></div <div style=3D"margin: 0px; line-height: normal;
font-family: 'Helvetica Neu=
e'; color: rgb(69, 69, 69);" class=3D"" <i
class=3D"">/shares
*</i></div <div style=3D"margin: 0px;
line-height: normal; font-family: 'Helvetica Neu=
e'; color: rgb(69, 69, 69);" class=3D"" <i
class=3D"">/shares/exports *</i></div <div
style=3D"margin: 0px; line-height: normal; font-family: 'Helvetica Neu=
e'; color: rgb(69, 69, 69);" class=3D"" <i
class=3D"">/shares/data *</i></div <div style=3D"margin: 0px; line-height: normal;
font-family: 'Helvetica Neu=
e'; color: rgb(69, 69, 69);" class=3D"" <i
class=3D"">/shares/isos *</i></div <div style=3D"margin: 0px; line-height: normal;
font-family: 'Helvetica Neu=
e'; color: rgb(69, 69, 69);" class=3D"" <i
class=3D"">[root@mennode2 ~]# </i></div </div <div
class=3D""><br class=3D""
</div <div class=3D"" <div style=3D"margin: 0px; line-height: normal;
font-family: 'Helvetica Neu=
e'; color: rgb(69, 69, 69);" class=3D""
[root@mennode2 tmp]# mount -t nfs menmaster.traindemo.local:/shares test</d=
iv <div style=3D"margin: 0px; line-height: normal;
font-family: 'Helvetica Neu=
e'; color: rgb(69, 69, 69);" class=3D""
[root@mennode2 tmp]# cd test</div <div
style=3D"margin: 0px; line-height: normal; font-family: 'Helvetica Neu=
e'; color: rgb(69, 69, 69);" class=3D""
[root@mennode2 test]# ls</div <div
style=3D"margin: 0px; line-height: normal; font-family: 'Helvetica Neu=
e'; color: rgb(69, 69, 69);" class=3D""
7d18ff24-57a3-4b4a-9934-0263191fe2e4 data
__DIRECT_IO_TEST__&nb=
sp; exports isos</div <div style=3D"margin: 0px;
line-height: normal; font-family: 'Helvetica Neu=
e'; color: rgb(69, 69, 69);" class=3D""
[root@mennode2 test]# </div
</div <div style=3D"margin: 0px;
line-height: normal; font-family: 'Helvetica Neu=
e'; color: rgb(69, 69, 69);" class=3D"" <br
class=3D"" </div <div style=3D"margin: 0px; line-height: normal;
font-family: 'Helvetica Neu=
e'; color: rgb(69, 69, 69);" class=3D"" One thing
I DO notice is the path of my exports is the difference in the pa=
th for the hosted_storage domain. I wonder if the second host would h=
ave issues resolving this?</div <div style=3D"margin: 0px;
line-height: normal; font-family: 'Helvetica Neu=
e'; color: rgb(69, 69, 69);" class=3D"" <br
class=3D"" </div <div style=3D"margin: 0px; line-height: normal;
font-family: 'Helvetica Neu=
e'; color: rgb(69, 69, 69);" class=3D"" Data.
=3D=3D> <span style=3D"color: rgb(119, 119, 119); font-family=
: Arial, sans-serif; font-size: small; background-color: rgb(255, 255, 255)=
;"
class=3D"">menmaster.traindemo.local:/shares/data</span></div <div style=3D"margin: 0px; line-height: normal;
font-family: 'Helvetica Neu=
e'; color: rgb(69, 69, 69);" class=3D"" Export
=3D=3D> <span style=3D"color: rgb(119, 119, 119); font=
-family: Arial, sans-serif; font-size: small; background-color: rgb(255, 25=
5, 255);"
class=3D"">menmaster.traindemo.local:/shares/exports</span></div <div style=3D"margin: 0px; line-height: normal;
font-family: 'Helvetica Neu=
e'; color: rgb(69, 69, 69);" class=3D""
Hosted_storage. =3D=3D> <span style=3D"color: rgb(119, 119,
119); f=
ont-family: Arial, sans-serif; font-size: small; background-color: rgb(255,=
255, 255);" class=3D"">localhost:/shares</span></div <div style=3D"margin: 0px; line-height: normal;
font-family: 'Helvetica Neu=
e'; color: rgb(69, 69, 69);" class=3D"" ISO.
=3D=3D> <span style=3D"color: rgb(119, 119, 119); font-family:=
Arial, sans-serif; font-size: small; background-color: rgb(255, 255, 255);=
"
class=3D"">menmaster.traindemo.local:/shares/exports</span></div <div style=3D"margin: 0px; line-height: normal;
font-family: 'Helvetica Neu=
e'; color: rgb(69, 69, 69);" class=3D"" <br
class=3D"" </div <div class=3D""><br class=3D"" </div <div
class=3D"">Below I have copied exports from the VDSM and OVIRT-ENGINE =
log and have copied the output of the hosted-engine.conf file. Any he=
lp in pinpointing the source of the problem is greatly appreciated!!</div <div class=3D""><br class=3D"" </div <div
class=3D""><br class=3D""
</div <div
class=3D""><b class=3D"">VSDM logs:</b></div <div class=3D""><br class=3D"" </div <div
class=3D"" <div style=3D"margin: 0px;
line-height: normal; font-family: 'Helvetica Neu=
e'; color: rgb(69, 69, 69);" class=3D"" <div
style=3D"margin: 0px; line-height: normal; font-family: 'Andale Mono';=
color: rgb(40, 254, 20); background-color: rgba(0, 0, 0, 0.901961);" class=
=3D"" <span
style=3D"font-variant-ligatures: no-common-ligatures"
class=3D"">2017=
-09-27 23:10:01,700-0400 INFO (jsonrpc/0) [jsonrpc.JsonRpcServer] RPC=
call Host.getStats succeeded in 0.06 seconds (__init__:539)</span></div <div style=3D"margin: 0px; line-height: normal;
font-family: 'Andale Mono';=
color: rgb(40, 254, 20); background-color: rgba(0, 0, 0, 0.901961);" class=
=3D"" <span
style=3D"font-variant-ligatures: no-common-ligatures"
class=3D"">2017=
-09-27 23:10:03,104-0400 INFO (jsonrpc/5) [vdsm.api] START getSpmStat=
us(spUUID=3Du'59c7f8f3-0063-00a8-02c7-0000000000f3', options=3DNone) from=
=3D::ffff:10.0.0.227,39748, flow_id=3D19e2dbb3, task_id=3D7a05a5bd-9b15-43b=
e-890d-c6f5d7650e5c
(api:46)</span></div <div style=3D"margin: 0px;
line-height: normal; font-family: 'Andale Mono';=
color: rgb(40, 254, 20); background-color: rgba(0, 0, 0, 0.901961);" class=
=3D"" <span
style=3D"font-variant-ligatures: no-common-ligatures"
class=3D"">2017=
-09-27 23:10:03,109-0400 INFO (jsonrpc/5) [vdsm.api] FINISH getSpmSta=
tus return=3D{'spm_st': {'spmId': 1, 'spmStatus': 'SPM',
'spmLver': 8L}} fr=
om=3D::ffff:10.0.0.227,39748, flow_id=3D19e2dbb3,
task_id=3D7a05a5bd-9b15-43be-890d-c6f5d7650e5c (api:52)</span></div <div style=3D"margin: 0px; line-height: normal;
font-family: 'Andale Mono';=
color: rgb(40, 254, 20); background-color: rgba(0, 0, 0, 0.901961);" class=
=3D"" <span
style=3D"font-variant-ligatures: no-common-ligatures"
class=3D"">2017=
-09-27 23:10:03,110-0400 INFO (jsonrpc/5) [jsonrpc.JsonRpcServer] RPC=
call StoragePool.getSpmStatus succeeded in 0.01 seconds (__init__:539)</sp=
an></div <div style=3D"margin: 0px;
line-height: normal; font-family: 'Andale Mono';=
color: rgb(40, 254, 20); background-color: rgba(0, 0, 0, 0.901961);" class=
=3D"" <span
style=3D"font-variant-ligatures: no-common-ligatures"
class=3D"">2017=
-09-27 23:10:03,189-0400 INFO (jsonrpc/7) [vdsm.api] START getStorage=
PoolInfo(spUUID=3Du'59c7f8f3-0063-00a8-02c7-0000000000f3', options=3DNone) =
from=3D::ffff:10.0.0.227,39866, flow_id=3D19e2dbb3,
task_id=3D0f8b49e9-9d82-457e-a2a2-39dc7ed9f022 (api:46)</span></div <div style=3D"margin: 0px; line-height: normal;
font-family: 'Andale Mono';=
color: rgb(40, 254, 20); background-color: rgba(0, 0, 0, 0.901961);" class=
=3D"" <span
style=3D"font-variant-ligatures: no-common-ligatures"
class=3D"">2017=
-09-27 23:10:03,196-0400 INFO (jsonrpc/7) [vdsm.api] FINISH getStorag=
ePoolInfo return=3D{'info': {'name': 'No Description',
'isoprefix': u'/rhev=
/data-center/mnt/menmaster.traindemo.local:_shares_isos/da001a29-eca5-44d6-=
a097-129dd9be623f/images/11111111-1111-1111-1111-111111111111',
'pool_status': 'connected', 'lver': 8L, 'domains':
u'da001a29-eca5-44d6-a0=
97-129dd9be623f:Active,f36157cc-b25a-400a-ab0f-a071e8a8eea7:Active,7d18ff24=
-57a3-4b4a-9934-0263191fe2e4:Active,795d4a1d-3ceb-4773-99de-8e7cf05112f3:Ac=
tive', 'master_uuid': u'f36157cc-b25a-400a-ab0f-a071e8a8eea7',
'version': '4', 'spm_id': 1, 'type': 'NFS',
'master_ver': 1}, 'dominfo': {=
u'da001a29-eca5-44d6-a097-129dd9be623f': {'status': u'Active',
'diskfree': =
'1044166737920', 'isoprefix':
u'/rhev/data-center/mnt/menmaster.traindemo.l=
ocal:_shares_isos/da001a29-eca5-44d6-a097-129dd9be623f/images/11111111-1111=
-1111-1111-111111111111',
'alerts': [], 'disktotal': '1049702170624', 'version':
0}, u'f36157cc-b25a=
-400a-ab0f-a071e8a8eea7': {'status': u'Active', 'diskfree':
'1044166737920'=
, 'isoprefix': '', 'alerts': [], 'disktotal':
'1049702170624', 'version': 4=
}, u'7d18ff24-57a3-4b4a-9934-0263191fe2e4':
{'status': u'Active', 'diskfree': '1044166737920',
'isoprefix': '', 'alert=
s': [], 'disktotal': '1049702170624', 'version': 4},
u'795d4a1d-3ceb-4773-9=
9de-8e7cf05112f3': {'status': u'Active', 'diskfree':
'1044166737920', 'isop=
refix': '', 'alerts': [], 'disktotal':
'1049702170624', 'version': 0}}} from=3D::ffff:10.0.0.227,39866,
flow_id=
=3D19e2dbb3, task_id=3D0f8b49e9-9d82-457e-a2a2-39dc7ed9f022 (api:52)</span>=
</div <div style=3D"margin: 0px;
line-height: normal; font-family: 'Andale Mono';=
color: rgb(40, 254, 20); background-color: rgba(0, 0, 0, 0.901961);" class=
=3D"" <span
style=3D"font-variant-ligatures: no-common-ligatures"
class=3D"">2017=
-09-27 23:10:03,198-0400 INFO (jsonrpc/7) [jsonrpc.JsonRpcServer] RPC=
call StoragePool.getInfo succeeded in 0.01 seconds (__init__:539)</span></=
div <div style=3D"margin: 0px; line-height: normal;
font-family: 'Andale Mono';=
color: rgb(40, 254, 20); background-color: rgba(0, 0, 0, 0.901961);" class=
=3D"" <span
style=3D"font-variant-ligatures: no-common-ligatures"
class=3D"">2017=
-09-27 23:10:04,212-0400 INFO (Reactor thread) [ProtocolDetector.Acce=
ptorImpl] Accepted connection from ::1:35980 (protocoldetector:72)</span></=
div <div style=3D"margin: 0px; line-height: normal;
font-family: 'Andale Mono';=
color: rgb(40, 254, 20); background-color: rgba(0, 0, 0, 0.901961);" class=
=3D"" <span
style=3D"font-variant-ligatures: no-common-ligatures"
class=3D"">2017=
-09-27 23:10:04,223-0400 INFO (Reactor thread) [ProtocolDetector.Dete=
ctor] Detected protocol stomp from ::1:35980 (protocoldetector:127)</span><=
/div <div style=3D"margin: 0px; line-height: normal;
font-family: 'Andale Mono';=
color: rgb(40, 254, 20); background-color: rgba(0, 0, 0, 0.901961);" class=
=3D"" <span
style=3D"font-variant-ligatures: no-common-ligatures"
class=3D"">2017=
-09-27 23:10:04,224-0400 INFO (Reactor thread) [Broker.StompAdapter] =
Processing CONNECT request (stompreactor:103)</span></div <div style=3D"margin: 0px; line-height: normal;
font-family: 'Andale Mono';=
color: rgb(40, 254, 20); background-color: rgba(0, 0, 0, 0.901961);" class=
=3D"" <span
style=3D"font-variant-ligatures: no-common-ligatures"
class=3D"">2017=
-09-27 23:10:04,225-0400 INFO (JsonRpc (StompReactor)) [Broker.StompA=
dapter] Subscribe command received (stompreactor:132)</span></div <div style=3D"margin: 0px; line-height: normal;
font-family: 'Andale Mono';=
color: rgb(40, 254, 20); background-color: rgba(0, 0, 0, 0.901961);" class=
=3D"" <span
style=3D"font-variant-ligatures: no-common-ligatures"
class=3D"">2017=
-09-27 23:10:04,622-0400 INFO (monitor/7d18ff2) [storage.SANLock] Acq=
uiring host id for domain 7d18ff24-57a3-4b4a-9934-0263191fe2e4 (id=3D1, asy=
nc=3DTrue) (clusterlock:288)</span></div <div
style=3D"margin: 0px; line-height: normal; font-family: 'Andale Mono';=
color: rgb(40, 254, 20); background-color: rgba(0, 0, 0, 0.901961);" class=
=3D"" <span
style=3D"font-variant-ligatures: no-common-ligatures"
class=3D"">2017=
-09-27 23:10:04,623-0400 ERROR (monitor/7d18ff2) [storage.Monitor] Error ac=
quiring host id 1 for domain 7d18ff24-57a3-4b4a-9934-0263191fe2e4 (monitor:=
570)</span></div <div style=3D"margin: 0px;
line-height: normal; font-family: 'Andale Mono';=
color: rgb(40, 254, 20); background-color: rgba(0, 0, 0, 0.901961);" class=
=3D"" <span
style=3D"font-variant-ligatures: no-common-ligatures"
class=3D"">Trac=
eback (most recent call last):</span></div <div
style=3D"margin: 0px; line-height: normal; font-family: 'Andale Mono';=
color: rgb(40, 254, 20); background-color: rgba(0, 0, 0, 0.901961);" class=
=3D"" <span
style=3D"font-variant-ligatures: no-common-ligatures"
class=3D"">&nbs=
p; File "/usr/share/vdsm/storage/monitor.py", line 567, in _acqui=
reHostId</span></div <div style=3D"margin: 0px;
line-height: normal; font-family: 'Andale Mono';=
color: rgb(40, 254, 20); background-color: rgba(0, 0, 0, 0.901961);" class=
=3D"" <span
style=3D"font-variant-ligatures: no-common-ligatures"
class=3D"">&nbs=
p; self.domain.acquireHostId(self.hostId,
async=3DTrue)</span></div <div style=3D"margin: 0px;
line-height: normal; font-family: 'Andale Mono';=
color: rgb(40, 254, 20); background-color: rgba(0, 0, 0, 0.901961);" class=
=3D"" <span
style=3D"font-variant-ligatures: no-common-ligatures"
class=3D"">&nbs=
p; File "/usr/share/vdsm/storage/sd.py", line 785, in acquireHost=
Id</span></div <div style=3D"margin: 0px;
line-height: normal; font-family: 'Andale Mono';=
color: rgb(40, 254, 20); background-color: rgba(0, 0, 0, 0.901961);" class=
=3D"" <span
style=3D"font-variant-ligatures: no-common-ligatures"
class=3D"">&nbs=
p; self._manifest.acquireHostId(hostId, async)</span></div <div style=3D"margin: 0px; line-height: normal;
font-family: 'Andale Mono';=
color: rgb(40, 254, 20); background-color: rgba(0, 0, 0, 0.901961);" class=
=3D"" <span
style=3D"font-variant-ligatures: no-common-ligatures"
class=3D"">&nbs=
p; File "/usr/share/vdsm/storage/sd.py", line 449, in acquireHost=
Id</span></div <div style=3D"margin: 0px;
line-height: normal; font-family: 'Andale Mono';=
color: rgb(40, 254, 20); background-color: rgba(0, 0, 0, 0.901961);" class=
=3D"" <span
style=3D"font-variant-ligatures: no-common-ligatures"
class=3D"">&nbs=
p; self._domainLock.acquireHostId(hostId, async)</span></div <div style=3D"margin: 0px; line-height: normal;
font-family: 'Andale Mono';=
color: rgb(40, 254, 20); background-color: rgba(0, 0, 0, 0.901961);" class=
=3D"" <span
style=3D"font-variant-ligatures: no-common-ligatures"
class=3D"">&nbs=
p; File "/usr/lib/python2.7/site-packages/vdsm/storage/clusterlock.py&=
quot;, line 319, in acquireHostId</span></div <div
style=3D"margin: 0px; line-height: normal; font-family: 'Andale Mono';=
color: rgb(40, 254, 20); background-color: rgba(0, 0, 0, 0.901961);" class=
=3D"" <span
style=3D"font-variant-ligatures: no-common-ligatures"
class=3D"">&nbs=
p; raise se.AcquireHostIdFailure(self._sdUUID, e)</span></div <div style=3D"margin: 0px; line-height: normal;
font-family: 'Andale Mono';=
color: rgb(40, 254, 20); background-color: rgba(0, 0, 0, 0.901961);" class=
=3D"" <span
style=3D"font-variant-ligatures: no-common-ligatures"
class=3D"">Acqu=
ireHostIdFailure: Cannot acquire host id: (u'7d18ff24-57a3-4b4a-9934-026319=
1fe2e4', SanlockException(22, 'Sanlock lockspace add failure', 'Invalid
arg=
ument'))</span></div <div style=3D"margin: 0px;
line-height: normal; font-family: 'Andale Mono';=
color: rgb(40, 254, 20); background-color: rgba(0, 0, 0, 0.901961);" class=
=3D"" <span
style=3D"font-variant-ligatures: no-common-ligatures"
class=3D"">2017=
-09-27 23:10:04,647-0400 INFO (jsonrpc/6) [jsonrpc.JsonRpcServer] RPC=
call Host.getHardwareInfo succeeded in 0.00 seconds (__init__:539)</span><=
/div <div style=3D"margin: 0px; line-height: normal;
font-family: 'Andale Mono';=
color: rgb(40, 254, 20); background-color: rgba(0, 0, 0, 0.901961);" class=
=3D"" <span
style=3D"font-variant-ligatures: no-common-ligatures"
class=3D"">2017=
-09-27 23:10:04,654-0400 INFO (jsonrpc/3) [vdsm.api] START repoStats(=
options=3DNone) from=3D::1,35980, task_id=3D9732972d-9a76-4349-9c65-53e57b2=
472d6 (api:46)</span></div <div
style=3D"margin: 0px; line-height: normal; font-family: 'Andale Mono';=
color: rgb(40, 254, 20); background-color: rgba(0, 0, 0, 0.901961);" class=
=3D"" <span
style=3D"font-variant-ligatures: no-common-ligatures"
class=3D"">2017=
-09-27 23:10:04,655-0400 INFO (jsonrpc/3) [vdsm.api] FINISH repoStats=
return=3D{u'da001a29-eca5-44d6-a097-129dd9be623f': {'code': 0,
'actual': T=
rue, 'version': 0, 'acquired': True, 'delay':
'0.000357054', 'lastCheck': '4.1', 'valid': True},
u'f36157cc-b25a-400a-ab=
0f-a071e8a8eea7': {'code': 0, 'actual': True, 'version': 4,
'acquired': Tru=
e, 'delay': '0.000531828', 'lastCheck': '4.0',
'valid': True}, u'7d18ff24-5=
7a3-4b4a-9934-0263191fe2e4': {'code':
0, 'actual': True, 'version': 4, 'acquired': False,
'delay': '0.000205228'=
, 'lastCheck': '0.0', 'valid': True},
u'795d4a1d-3ceb-4773-99de-8e7cf05112f=
3': {'code': 0, 'actual': True, 'version': 0,
'acquired': True, 'delay': '0=
.000352165', 'lastCheck': '4.1',
'valid': True}} from=3D::1,35980, task_id=3D9732972d-9a76-4349-9c65-53e57b=
2472d6 (api:52)</span></div <div
style=3D"margin: 0px; line-height: normal; font-family: 'Andale Mono';=
color: rgb(40, 254, 20); background-color: rgba(0, 0, 0, 0.901961);" class=
=3D"" <span
style=3D"font-variant-ligatures: no-common-ligatures"
class=3D"">2017=
-09-27 23:10:04,656-0400 INFO (jsonrpc/3) [jsonrpc.JsonRpcServer] RPC=
call Host.getStorageRepoStats succeeded in 0.00 seconds (__init__:539)</sp=
an></div <div style=3D"margin: 0px;
line-height: normal; font-family: 'Andale Mono';=
color: rgb(40, 254, 20); background-color: rgba(0, 0, 0, 0.901961);" class=
=3D"" <span
style=3D"font-variant-ligatures: no-common-ligatures"
class=3D"">2017=
-09-27 23:10:07,717-0400 INFO (Reactor thread) [ProtocolDetector.Acce=
ptorImpl] Accepted connection from ::1:35982 (protocoldetector:72)</span></=
div <div style=3D"margin: 0px; line-height: normal;
font-family: 'Andale Mono';=
color: rgb(40, 254, 20); background-color: rgba(0, 0, 0, 0.901961);" class=
=3D"" <span
style=3D"font-variant-ligatures: no-common-ligatures"
class=3D"">2017=
-09-27 23:10:07,728-0400 INFO (Reactor thread) [ProtocolDetector.Dete=
ctor] Detected protocol stomp from ::1:35982 (protocoldetector:127)</span><=
/div <div style=3D"margin: 0px; line-height: normal;
font-family: 'Andale Mono';=
color: rgb(40, 254, 20); background-color: rgba(0, 0, 0, 0.901961);" class=
=3D"" <span
style=3D"font-variant-ligatures: no-common-ligatures"
class=3D"">2017=
-09-27 23:10:07,729-0400 INFO (Reactor thread) [Broker.StompAdapter] =
Processing CONNECT request (stompreactor:103)</span></div <div style=3D"margin: 0px; line-height: normal;
font-family: 'Andale Mono';=
color: rgb(40, 254, 20); background-color: rgba(0, 0, 0, 0.901961);" class=
=3D"" <span
style=3D"font-variant-ligatures: no-common-ligatures"
class=3D"">2017=
-09-27 23:10:07,730-0400 INFO (JsonRpc (StompReactor)) [Broker.StompA=
dapter] Subscribe command received (stompreactor:132)</span></div <div style=3D"margin: 0px; line-height: normal;
font-family: 'Andale Mono';=
color: rgb(40, 254, 20); background-color: rgba(0, 0, 0, 0.901961);" class=
=3D"" <span
style=3D"font-variant-ligatures: no-common-ligatures"
class=3D"">2017=
-09-27 23:10:08,152-0400 INFO (jsonrpc/2) [jsonrpc.JsonRpcServer] RPC=
call Host.getHardwareInfo succeeded in 0.00 seconds (__init__:539)</span><=
/div <div style=3D"margin: 0px; line-height: normal;
font-family: 'Andale Mono';=
color: rgb(40, 254, 20); background-color: rgba(0, 0, 0, 0.901961);" class=
=3D"" <span
style=3D"font-variant-ligatures: no-common-ligatures"
class=3D"">2017=
-09-27 23:10:08,159-0400 INFO (jsonrpc/1) [vdsm.api] START repoStats(=
options=3DNone) from=3D::1,35982, task_id=3D80eb282b-f297-4f98-a368-7d5627a=
c50f3 (api:46)</span></div <div
style=3D"margin: 0px; line-height: normal; font-family: 'Andale Mono';=
color: rgb(40, 254, 20); background-color: rgba(0, 0, 0, 0.901961);" class=
=3D"" <span
style=3D"font-variant-ligatures: no-common-ligatures"
class=3D"">2017=
-09-27 23:10:08,160-0400 INFO (jsonrpc/1) [vdsm.api] FINISH repoStats=
return=3D{u'da001a29-eca5-44d6-a097-129dd9be623f': {'code': 0,
'actual': T=
rue, 'version': 0, 'acquired': True, 'delay':
'0.000357054', 'lastCheck': '7.6', 'valid': True},
u'f36157cc-b25a-400a-ab=
0f-a071e8a8eea7': {'code': 0, 'actual': True, 'version': 4,
'acquired': Tru=
e, 'delay': '0.000531828', 'lastCheck': '7.5',
'valid': True}, u'7d18ff24-5=
7a3-4b4a-9934-0263191fe2e4': {'code':
0, 'actual': True, 'version': 4, 'acquired': False,
'delay': '0.000205228'=
, 'lastCheck': '3.5', 'valid': True},
u'795d4a1d-3ceb-4773-99de-8e7cf05112f=
3': {'code': 0, 'actual': True, 'version': 0,
'acquired': True, 'delay': '0=
.000352165', 'lastCheck': '7.6',
'valid': True}} from=3D::1,35982, task_id=3D80eb282b-f297-4f98-a368-7d5627=
ac50f3 (api:52)</span></div <div
style=3D"margin: 0px; line-height: normal; font-family: 'Andale Mono';=
color: rgb(40, 254, 20); background-color: rgba(0, 0, 0, 0.901961);" class=
=3D"" <span
style=3D"font-variant-ligatures: no-common-ligatures"
class=3D"">2017=
-09-27 23:10:08,171-0400 INFO (Reactor thread) [ProtocolDetector.Acce=
ptorImpl] Accepted connection from ::1:35984 (protocoldetector:72)</span></=
div <div style=3D"margin: 0px; line-height: normal;
font-family: 'Andale Mono';=
color: rgb(40, 254, 20); background-color: rgba(0, 0, 0, 0.901961);" class=
=3D"" <span
style=3D"font-variant-ligatures: no-common-ligatures"
class=3D"">2017=
-09-27 23:10:08,198-0400 INFO (Reactor thread) [ProtocolDetector.Dete=
ctor] Detected protocol stomp from ::1:35984 (protocoldetector:127)</span><=
/div <div style=3D"margin: 0px; line-height: normal;
font-family: 'Andale Mono';=
color: rgb(40, 254, 20); background-color: rgba(0, 0, 0, 0.901961);" class=
=3D"" <span
style=3D"font-variant-ligatures: no-common-ligatures"
class=3D"">2017=
-09-27 23:10:08,199-0400 INFO (Reactor thread) [Broker.StompAdapter] =
Processing CONNECT request (stompreactor:103)</span></div <div style=3D"margin: 0px; line-height: normal;
font-family: 'Andale Mono';=
color: rgb(40, 254, 20); background-color: rgba(0, 0, 0, 0.901961);" class=
=3D"" <span
style=3D"font-variant-ligatures: no-common-ligatures"
class=3D"">2017=
-09-27 23:10:08,201-0400 INFO (JsonRpc (StompReactor)) [Broker.StompA=
dapter] Subscribe command received (stompreactor:132)</span></div <div style=3D"margin: 0px; line-height: normal;
font-family: 'Andale Mono';=
color: rgb(40, 254, 20); background-color: rgba(0, 0, 0, 0.901961);" class=
=3D"" <span
style=3D"font-variant-ligatures: no-common-ligatures"
class=3D"">2017=
-09-27 23:10:08,374-0400 INFO (jsonrpc/1) [jsonrpc.JsonRpcServer] RPC=
call Host.getStats succeeded in 0.22 seconds (__init__:539)</span></div <div style=3D"margin: 0px; line-height: normal;
font-family: 'Andale Mono';=
color: rgb(40, 254, 20); background-color: rgba(0, 0, 0, 0.901961);" class=
=3D"" <span
style=3D"font-variant-ligatures: no-common-ligatures"
class=3D"">2017=
-09-27 23:10:08,728-0400 INFO (Reactor thread) [ProtocolDetector.Acce=
ptorImpl] Accepted connection from ::1:35987 (protocoldetector:72)</span></=
div <div style=3D"margin: 0px; line-height: normal;
font-family: 'Andale Mono';=
color: rgb(40, 254, 20); background-color: rgba(0, 0, 0, 0.901961);" class=
=3D"" <span
style=3D"font-variant-ligatures: no-common-ligatures"
class=3D"">2017=
-09-27 23:10:08,732-0400 INFO (jsonrpc/4) [jsonrpc.JsonRpcServer] RPC=
call Host.getHardwareInfo succeeded in 0.00 seconds (__init__:539)</span><=
/div <div style=3D"margin: 0px; line-height: normal;
font-family: 'Andale Mono';=
color: rgb(40, 254, 20); background-color: rgba(0, 0, 0, 0.901961);" class=
=3D"" <span
style=3D"font-variant-ligatures: no-common-ligatures"
class=3D"">2017=
-09-27 23:10:08,738-0400 INFO (jsonrpc/0) [jsonrpc.JsonRpcServer] RPC=
call VM.getStats succeeded in 0.00 seconds (__init__:539)</span></div <div style=3D"margin: 0px; line-height: normal;
font-family: 'Andale Mono';=
color: rgb(40, 254, 20); background-color: rgba(0, 0, 0, 0.901961);" class=
=3D"" <span
style=3D"font-variant-ligatures: no-common-ligatures"
class=3D"">2017=
-09-27 23:10:08,756-0400 INFO (Reactor thread) [ProtocolDetector.Dete=
ctor] Detected protocol stomp from ::1:35987 (protocoldetector:127)</span><=
/div <div style=3D"margin: 0px; line-height: normal;
font-family: 'Andale Mono';=
color: rgb(40, 254, 20); background-color: rgba(0, 0, 0, 0.901961);" class=
=3D"" <span
style=3D"font-variant-ligatures: no-common-ligatures"
class=3D"">2017=
-09-27 23:10:08,757-0400 INFO (Reactor thread) [Broker.StompAdapter] =
Processing CONNECT request (stompreactor:103)</span></div <div style=3D"margin: 0px; line-height: normal;
font-family: 'Andale Mono';=
color: rgb(40, 254, 20); background-color: rgba(0, 0, 0, 0.901961);" class=
=3D"" <span
style=3D"font-variant-ligatures: no-common-ligatures"
class=3D"">2017=
-09-27 23:10:08,771-0400 INFO (JsonRpc (StompReactor)) [Broker.StompA=
dapter] Subscribe command received (stompreactor:132)</span></div <div style=3D"margin: 0px; line-height: normal;
font-family: 'Andale Mono';=
color: rgb(40, 254, 20); background-color: rgba(0, 0, 0, 0.901961);" class=
=3D"" <span
style=3D"font-variant-ligatures: no-common-ligatures"
class=3D"">2017=
-09-27 23:10:09,286-0400 INFO (jsonrpc/5) [jsonrpc.JsonRpcServer] RPC=
call Host.getHardwareInfo succeeded in 0.00 seconds (__init__:539)</span><=
/div <div style=3D"margin: 0px; line-height: normal;
font-family: 'Andale Mono';=
color: rgb(40, 254, 20); background-color: rgba(0, 0, 0, 0.901961);" class=
=3D"" <span
style=3D"font-variant-ligatures: no-common-ligatures"
class=3D"">2017=
-09-27 23:10:09,395-0400 INFO (jsonrpc/7) [jsonrpc.JsonRpcServer] RPC=
call Host.getCapabilities succeeded in 0.10 seconds (__init__:539)</span><=
/div <div style=3D"margin: 0px; line-height: normal;
font-family: 'Andale Mono';=
color: rgb(40, 254, 20); background-color: rgba(0, 0, 0, 0.901961);" class=
=3D"" <span
style=3D"font-variant-ligatures: no-common-ligatures"
class=3D"">2017=
-09-27 23:10:10,590-0400 INFO (itmap/0) [IOProcessClient] Starting cl=
ient ioprocess-37 (__init__:330)</span></div
</div <div style=3D"margin: 0px;
line-height: normal; font-family: 'Helvetica Neu=
e'; color: rgb(69, 69, 69);" class=3D"" <br
class=3D"" </div </div <div
style=3D"margin: 0px; line-height: normal; font-family: 'Helvetica Neu=
e'; color: rgb(69, 69, 69);" class=3D"" <br
class=3D"" </div <div style=3D"margin: 0px; line-height: normal;
font-family: 'Helvetica Neu=
e'; color: rgb(69, 69, 69);" class=3D"" <b
class=3D"">ENGINE LOGS:</b></div <div
style=3D"margin: 0px; line-height: normal; font-family: 'Helvetica Neu=
e'; color: rgb(69, 69, 69);" class=3D"" <br
class=3D"" </div <div style=3D"margin: 0px; line-height: normal;
font-family: 'Helvetica Neu=
e'; color: rgb(69, 69, 69);" class=3D"" <div
style=3D"margin: 0px; line-height: normal; font-family: 'Andale Mono';=
color: rgb(40, 254, 20); background-color: rgba(0, 0, 0, 0.901961);" class=
=3D"" <span
style=3D"font-variant-ligatures: no-common-ligatures"
class=3D"">2017=
-09-27 23:07:58,154-04 INFO [org.ovirt.engine.core.bll.hostdeploy.Vds=
DeployBase] (VdsDeploy) [3f898655] Retrieving installation logs to: '/var/l=
og/ovirt-engine/host-deploy/ovirt-host-mgmt-20170927230758-10.0.0.222-3f898=
655.log'</span></div <div style=3D"margin: 0px;
line-height: normal; font-family: 'Andale Mono';=
color: rgb(40, 254, 20); background-color: rgba(0, 0, 0, 0.901961);" class=
=3D"" <span
style=3D"font-variant-ligatures: no-common-ligatures"
class=3D"">2017=
-09-27 23:07:58,316-04 INFO [org.ovirt.engine.core.bll.hostdeploy.Vds=
DeployBase] (VdsDeploy) [3f898655] Stage: Termination</span></div <div style=3D"margin: 0px; line-height: normal;
font-family: 'Andale Mono';=
color: rgb(40, 254, 20); background-color: rgba(0, 0, 0, 0.901961);" class=
=3D"" <span
style=3D"font-variant-ligatures: no-common-ligatures"
class=3D"">2017=
-09-27 23:07:58,458-04 INFO [org.ovirt.engine.core.dal.dbbroker.audit=
loghandling.AuditLogDirector] (pool-7-thread-1) [3f898655] EVENT_ID: HOST_A=
VAILABLE_UPDATES_FINISHED(885), Correlation
ID: null, Call Stack: null, Custom ID: null, Custom Event ID: -1, Message:=
Check for available updates on host mennode2 was completed successfully wi=
th message 'no updates found.'.</span></div <div
style=3D"margin: 0px; line-height: normal; font-family: 'Andale Mono';=
color: rgb(40, 254, 20); background-color: rgba(0, 0, 0, 0.901961);" class=
=3D"" <span
style=3D"font-variant-ligatures: no-common-ligatures"
class=3D"">2017=
-09-27 23:07:58,874-04 INFO [org.ovirt.engine.core.bll.hostdeploy.Vds=
DeployBase] (VdsDeploy) [3f898655] Stage: Misc configuration</span></div <div style=3D"margin: 0px; line-height: normal;
font-family: 'Andale Mono';=
color: rgb(40, 254, 20); background-color: rgba(0, 0, 0, 0.901961);" class=
=3D"" <span
style=3D"font-variant-ligatures: no-common-ligatures"
class=3D"">2017=
-09-27 23:07:58,888-04 INFO [org.ovirt.engine.core.bll.hostdeploy.Vds=
DeployBase] (VdsDeploy) [3f898655] Stage: Transaction commit</span></div <div style=3D"margin: 0px; line-height: normal;
font-family: 'Andale Mono';=
color: rgb(40, 254, 20); background-color: rgba(0, 0, 0, 0.901961);" class=
=3D"" <span
style=3D"font-variant-ligatures: no-common-ligatures"
class=3D"">2017=
-09-27 23:07:59,029-04 INFO [org.ovirt.engine.core.bll.hostdeploy.Vds=
DeployBase] (VdsDeploy) [3f898655] Stage: Closing up</span></div <div style=3D"margin: 0px; line-height: normal;
font-family: 'Andale Mono';=
color: rgb(40, 254, 20); background-color: rgba(0, 0, 0, 0.901961);" class=
=3D"" <span
style=3D"font-variant-ligatures: no-common-ligatures"
class=3D"">2017=
-09-27 23:07:59,039-04 INFO [org.ovirt.engine.core.bll.hostdeploy.Vds=
DeployBase] (VdsDeploy) [3f898655] Stage: Pre-termination</span></div <div style=3D"margin: 0px; line-height: normal;
font-family: 'Andale Mono';=
color: rgb(40, 254, 20); background-color: rgba(0, 0, 0, 0.901961);" class=
=3D"" <span
style=3D"font-variant-ligatures: no-common-ligatures"
class=3D"">2017=
-09-27 23:07:59,092-04 INFO [org.ovirt.engine.core.bll.hostdeploy.Vds=
DeployBase] (VdsDeploy) [3f898655] Retrieving installation logs to: '/var/l=
og/ovirt-engine/host-deploy/ovirt-host-mgmt-20170927230759-menmaster.traind=
emo.local-3f898655.log'</span></div <div
style=3D"margin: 0px; line-height: normal; font-family: 'Andale Mono';=
color: rgb(40, 254, 20); background-color: rgba(0, 0, 0, 0.901961);" class=
=3D"" <span
style=3D"font-variant-ligatures: no-common-ligatures"
class=3D"">2017=
-09-27 23:07:59,262-04 INFO [org.ovirt.engine.core.bll.hostdeploy.Vds=
DeployBase] (VdsDeploy) [3f898655] Stage: Termination</span></div <div style=3D"margin: 0px; line-height: normal;
font-family: 'Andale Mono';=
color: rgb(40, 254, 20); background-color: rgba(0, 0, 0, 0.901961);" class=
=3D"" <span
style=3D"font-variant-ligatures: no-common-ligatures"
class=3D"">2017=
-09-27 23:07:59,375-04 INFO [org.ovirt.engine.core.dal.dbbroker.audit=
loghandling.AuditLogDirector] (pool-7-thread-2) [3f898655] EVENT_ID: HOST_A=
VAILABLE_UPDATES_FINISHED(885), Correlation
ID: null, Call Stack: null, Custom ID: null, Custom Event ID: -1, Message:=
Check for available updates on host menmaster was completed successfully w=
ith message 'no updates found.'.</span></div <div
style=3D"margin: 0px; line-height: normal; font-family: 'Andale Mono';=
color: rgb(40, 254, 20); background-color: rgba(0, 0, 0, 0.901961);" class=
=3D"" <span
style=3D"font-variant-ligatures: no-common-ligatures"
class=3D"">2017=
-09-27 23:08:38,705-04 INFO [org.ovirt.engine.core.bll.ActivateVdsCom=
mand] (default task-13) [75de01a6-efd2-418f-a6b1-b4ef1f38923f] Lock Acquire=
d to object 'EngineLock:{exclusiveLocks=3D'[0ef7ec10-af19-4afa-980b-77a772c=
4e014=3DVDS]',
sharedLocks=3D''}'</span></div <div
style=3D"margin: 0px; line-height: normal; font-family: 'Andale Mono';=
color: rgb(40, 254, 20); background-color: rgba(0, 0, 0, 0.901961);" class=
=3D"" <span
style=3D"font-variant-ligatures: no-common-ligatures"
class=3D"">2017=
-09-27 23:08:38,776-04 INFO [org.ovirt.engine.core.bll.ActivateVdsCom=
mand] (org.ovirt.thread.pool-6-thread-2) [75de01a6-efd2-418f-a6b1-b4ef1f389=
23f] Running command: ActivateVdsCommand
internal: false. Entities affected : ID: 0ef7ec10-af19-4afa-980b-77a=
772c4e014 Type: VDSAction group MANIPULATE_HOST with role type ADMIN</span>=
</div <div style=3D"margin: 0px;
line-height: normal; font-family: 'Andale Mono';=
color: rgb(40, 254, 20); background-color: rgba(0, 0, 0, 0.901961);" class=
=3D"" <span
style=3D"font-variant-ligatures: no-common-ligatures"
class=3D"">2017=
-09-27 23:08:38,777-04 INFO [org.ovirt.engine.core.bll.ActivateVdsCom=
mand] (org.ovirt.thread.pool-6-thread-2) [75de01a6-efd2-418f-a6b1-b4ef1f389=
23f] Before acquiring lock in order to prevent
monitoring for host 'mennode2' from data-center
'Train1'</span></div <div
style=3D"margin: 0px; line-height: normal; font-family: 'Andale Mono';=
color: rgb(40, 254, 20); background-color: rgba(0, 0, 0, 0.901961);" class=
=3D"" <span
style=3D"font-variant-ligatures: no-common-ligatures"
class=3D"">2017=
-09-27 23:08:38,778-04 INFO [org.ovirt.engine.core.bll.lock.InMemoryL=
ockManager] (org.ovirt.thread.pool-6-thread-2) [75de01a6-efd2-418f-a6b1-b4e=
f1f38923f] Failed to acquire lock and wait
lock 'HostEngineLock:{exclusiveLocks=3D'[0ef7ec10-af19-4afa-980b-77a772c4e=
014=3DVDS_INIT]', sharedLocks=3D''}'</span></div <div style=3D"margin: 0px; line-height: normal;
font-family: 'Andale Mono';=
color: rgb(40, 254, 20); background-color: rgba(0, 0, 0, 0.901961);" class=
=3D"" <span
style=3D"font-variant-ligatures: no-common-ligatures"
class=3D"">2017=
-09-27 23:08:38,995-04 INFO [org.ovirt.engine.core.bll.ActivateVdsCom=
mand] (org.ovirt.thread.pool-6-thread-2) [75de01a6-efd2-418f-a6b1-b4ef1f389=
23f] Lock acquired, from now a monitoring
of host will be skipped for host 'mennode2' from data-center
'Train1'</spa=
n></div <div style=3D"margin: 0px;
line-height: normal; font-family: 'Andale Mono';=
color: rgb(40, 254, 20); background-color: rgba(0, 0, 0, 0.901961);" class=
=3D"" <span
style=3D"font-variant-ligatures: no-common-ligatures"
class=3D"">2017=
-09-27 23:08:39,003-04 INFO [org.ovirt.engine.core.vdsbroker.SetVdsSt=
atusVDSCommand] (org.ovirt.thread.pool-6-thread-2) [75de01a6-efd2-418f-a6b1=
-b4ef1f38923f] START, SetVdsStatusVDSCommand(HostName
=3D mennode2, SetVdsStatusVDSCommandParameters:{runAsync=3D'true', hostId=
=3D'0ef7ec10-af19-4afa-980b-77a772c4e014', status=3D'Unassigned',
nonOperat=
ionalReason=3D'NONE', stopSpmFailureLogged=3D'false',
maintenanceReason=3D'=
null'}), log id: 78cc1caf</span></div <div
style=3D"margin: 0px; line-height: normal; font-family: 'Andale Mono';=
color: rgb(40, 254, 20); background-color: rgba(0, 0, 0, 0.901961);" class=
=3D"" <span
style=3D"font-variant-ligatures: no-common-ligatures"
class=3D"">2017=
-09-27 23:08:39,027-04 INFO [org.ovirt.engine.core.vdsbroker.SetVdsSt=
atusVDSCommand] (org.ovirt.thread.pool-6-thread-2) [75de01a6-efd2-418f-a6b1=
-b4ef1f38923f] FINISH, SetVdsStatusVDSCommand,
log id: 78cc1caf</span></div <div
style=3D"margin: 0px; line-height: normal; font-family: 'Andale Mono';=
color: rgb(40, 254, 20); background-color: rgba(0, 0, 0, 0.901961);" class=
=3D"" <span
style=3D"font-variant-ligatures: no-common-ligatures"
class=3D"">2017=
-09-27 23:08:39,040-04 INFO [org.ovirt.engine.core.bll.ActivateVdsCom=
mand] (org.ovirt.thread.pool-6-thread-2) [75de01a6-efd2-418f-a6b1-b4ef1f389=
23f] Activate host finished. Lock released.
Monitoring can run now for host 'mennode2' from data-center
'Train1'</span=
></div <div style=3D"margin: 0px;
line-height: normal; font-family: 'Andale Mono';=
color: rgb(40, 254, 20); background-color: rgba(0, 0, 0, 0.901961);" class=
=3D"" <span
style=3D"font-variant-ligatures: no-common-ligatures"
class=3D"">2017=
-09-27 23:08:39,050-04 INFO [org.ovirt.engine.core.dal.dbbroker.audit=
loghandling.AuditLogDirector] (org.ovirt.thread.pool-6-thread-2) [75de01a6-=
efd2-418f-a6b1-b4ef1f38923f] EVENT_ID: VDS_ACTIVATE(16),
Correlation ID: 75de01a6-efd2-418f-a6b1-b4ef1f38923f, Job ID: 2e40097e-e99=
1-45cc-bd49-3176822ebb5a, Call Stack: null, Custom ID: null, Custom Event I=
D: -1, Message: Activation of host mennode2 initiated by admin@internal-aut=
hz.</span></div <div style=3D"margin: 0px;
line-height: normal; font-family: 'Andale Mono';=
color: rgb(40, 254, 20); background-color: rgba(0, 0, 0, 0.901961);" class=
=3D"" <span
style=3D"font-variant-ligatures: no-common-ligatures"
class=3D"">2017=
-09-27 23:08:39,059-04 INFO [org.ovirt.engine.core.bll.ActivateVdsCom=
mand] (org.ovirt.thread.pool-6-thread-2) [75de01a6-efd2-418f-a6b1-b4ef1f389=
23f] Lock freed to object 'EngineLock:{exclusiveLocks=3D'[0ef7ec10-af19-4af=
a-980b-77a772c4e014=3DVDS]',
sharedLocks=3D''}'</span></div <div
style=3D"margin: 0px; line-height: normal; font-family: 'Andale Mono';=
color: rgb(40, 254, 20); background-color: rgba(0, 0, 0, 0.901961);" class=
=3D"" <span
style=3D"font-variant-ligatures: no-common-ligatures"
class=3D"">2017=
-09-27 23:08:43,143-04 INFO [org.ovirt.engine.core.vdsbroker.vdsbroke=
r.GetHardwareInfoVDSCommand] (DefaultQuartzScheduler7) [5437c1b7] START, Ge=
tHardwareInfoVDSCommand(HostName =3D mennode2,
VdsIdAndVdsVDSCommandParametersBase:{runAsync=3D'true', hostId=3D'0ef7ec10=
-af19-4afa-980b-77a772c4e014', vds=3D'Host[mennode2,0ef7ec10-af19-4afa-980b=
-77a772c4e014]'}), log id: 2bcbe26e</span></div <div
style=3D"margin: 0px; line-height: normal; font-family: 'Andale Mono';=
color: rgb(40, 254, 20); background-color: rgba(0, 0, 0, 0.901961);" class=
=3D"" <span
style=3D"font-variant-ligatures: no-common-ligatures"
class=3D"">2017=
-09-27 23:08:44,150-04 INFO [org.ovirt.engine.core.vdsbroker.vdsbroke=
r.GetHardwareInfoVDSCommand] (DefaultQuartzScheduler7) [5437c1b7] FINISH, G=
etHardwareInfoVDSCommand, log id: 2bcbe26e</span></div <div style=3D"margin: 0px; line-height: normal;
font-family: 'Andale Mono';=
color: rgb(40, 254, 20); background-color: rgba(0, 0, 0, 0.901961);" class=
=3D"" <span
style=3D"font-variant-ligatures: no-common-ligatures"
class=3D"">2017=
-09-27 23:08:44,186-04 INFO [org.ovirt.engine.core.bll.HandleVdsCpuFl=
agsOrClusterChangedCommand] (DefaultQuartzScheduler7) [12ef27d8] Running co=
mmand: HandleVdsCpuFlagsOrClusterChangedCommand
internal: true. Entities affected : ID: 0ef7ec10-af19-4afa-980b-77a7=
72c4e014 Type: VDS</span></div <div
style=3D"margin: 0px; line-height: normal; font-family: 'Andale Mono';=
color: rgb(40, 254, 20); background-color: rgba(0, 0, 0, 0.901961);" class=
=3D"" <span
style=3D"font-variant-ligatures: no-common-ligatures"
class=3D"">2017=
-09-27 23:08:44,214-04 INFO [org.ovirt.engine.core.bll.InitVdsOnUpCom=
mand] (DefaultQuartzScheduler7) [63cdc728] Running command: InitVdsOnUpComm=
and internal: true. Entities affected :
ID: 59c7f8f3-0063-00a8-02c7-0000000000f3 Type: StoragePool</span></div <div style=3D"margin: 0px; line-height: normal;
font-family: 'Andale Mono';=
color: rgb(40, 254, 20); background-color: rgba(0, 0, 0, 0.901961);" class=
=3D"" <span
style=3D"font-variant-ligatures: no-common-ligatures"
class=3D"">2017=
-09-27 23:08:44,234-04 INFO [org.ovirt.engine.core.bll.storage.pool.C=
onnectHostToStoragePoolServersCommand] (DefaultQuartzScheduler7) [7670f89e]=
Running command: ConnectHostToStoragePoolServersCommand
internal: true. Entities affected : ID: 59c7f8f3-0063-00a8-02c7-0000=
000000f3 Type: StoragePool</span></div <div
style=3D"margin: 0px; line-height: normal; font-family: 'Andale Mono';=
color: rgb(40, 254, 20); background-color: rgba(0, 0, 0, 0.901961);" class=
=3D"" <span
style=3D"font-variant-ligatures: no-common-ligatures"
class=3D"">2017=
-09-27 23:08:44,257-04 INFO [org.ovirt.engine.core.vdsbroker.vdsbroke=
r.ConnectStorageServerVDSCommand] (DefaultQuartzScheduler7) [7670f89e] STAR=
T, ConnectStorageServerVDSCommand(HostName
=3D mennode2, StorageServerConnectionManagementVDSParameters:{runAsync=3D'=
true', hostId=3D'0ef7ec10-af19-4afa-980b-77a772c4e014',
storagePoolId=3D'59=
c7f8f3-0063-00a8-02c7-0000000000f3', storageType=3D'NFS', connectionList=3D=
'[StorageServerConnections:{id=3D'aac63433-88fb-4daf-8c83-871a85b3a49d',
connection=3D'menmaster.traindemo.local:/shares/data', iqn=3D'null',
vfsTy=
pe=3D'null', mountOptions=3D'null', nfsVersion=3D'AUTO',
nfsRetrans=3D'null=
', nfsTimeo=3D'null', iface=3D'null', netIfaceName=3D'null'},
StorageServer=
Connections:{id=3D'5adb7a4b-ebbd-43a4-b5cb-8c8eb3d6f1ee',
connection=3D'localhost:/shares', iqn=3D'null', vfsType=3D'null',
mountOpt=
ions=3D'null', nfsVersion=3D'null', nfsRetrans=3D'null',
nfsTimeo=3D'null',=
iface=3D'null', netIfaceName=3D'null'},
StorageServerConnections:{id=3D'9e=
b3b1e2-4063-46cd-ba4e-440c4048e92a', connection=3D'menmaster.traindemo.loca=
l:/shares/isos',
iqn=3D'null', vfsType=3D'null', mountOptions=3D'null',
nfsVersion=3D'AUTO'=
, nfsRetrans=3D'null', nfsTimeo=3D'null', iface=3D'null',
netIfaceName=3D'n=
ull'}, StorageServerConnections:{id=3D'1c9215a1-4154-4a78-b8ff-565c3a237c58=
', connection=3D'menmaster.traindemo.local:/shares/exports',
iqn=3D'null', vfsType=3D'null', mountOptions=3D'null',
nfsVersion=3D'AUTO'=
, nfsRetrans=3D'null', nfsTimeo=3D'null', iface=3D'null',
netIfaceName=3D'n=
ull'}]'}), log id: 5625b02f</span></div <div
style=3D"margin: 0px; line-height: normal; font-family: 'Andale Mono';=
color: rgb(40, 254, 20); background-color: rgba(0, 0, 0, 0.901961);" class=
=3D"" <span
style=3D"font-variant-ligatures: no-common-ligatures"
class=3D"">2017=
-09-27 23:08:48,370-04 ERROR [org.ovirt.engine.core.dal.dbbroker.auditlogha=
ndling.AuditLogDirector] (DefaultQuartzScheduler7) [7670f89e] EVENT_ID: VDS=
_STORAGES_CONNECTION_FAILED(188), Correlation
ID: null, Call Stack: null, Custom ID: null, Custom Event ID: -1, Message:=
Failed to connect Host mennode2 to the Storage Domains hosted_storage.</sp=
an></div <div style=3D"margin: 0px;
line-height: normal; font-family: 'Andale Mono';=
color: rgb(40, 254, 20); background-color: rgba(0, 0, 0, 0.901961);" class=
=3D"" <span
style=3D"font-variant-ligatures: no-common-ligatures"
class=3D"">2017=
-09-27 23:08:48,370-04 INFO [org.ovirt.engine.core.vdsbroker.vdsbroke=
r.ConnectStorageServerVDSCommand] (DefaultQuartzScheduler7) [7670f89e] FINI=
SH, ConnectStorageServerVDSCommand, return:
{aac63433-88fb-4daf-8c83-871a85b3a49d=3D0, 5adb7a4b-ebbd-43a4-b5cb-8c8eb3d=
6f1ee=3D477, 9eb3b1e2-4063-46cd-ba4e-440c4048e92a=3D0, 1c9215a1-4154-4a78-b=
8ff-565c3a237c58=3D0}, log id: 5625b02f</span></div <div
style=3D"margin: 0px; line-height: normal; font-family: 'Andale Mono';=
color: rgb(40, 254, 20); background-color: rgba(0, 0, 0, 0.901961);" class=
=3D"" <span
style=3D"font-variant-ligatures: no-common-ligatures"
class=3D"">2017=
-09-27 23:08:48,386-04 ERROR [org.ovirt.engine.core.dal.dbbroker.auditlogha=
ndling.AuditLogDirector] (DefaultQuartzScheduler7) [7670f89e] EVENT_ID: STO=
RAGE_DOMAIN_ERROR(996), Correlation
ID: null, Call Stack: null, Custom ID: null, Custom Event ID: -1, Message:=
The error message for connection localhost:/shares returned by VDSM was: P=
roblem while trying to mount target</span></div <div
style=3D"margin: 0px; line-height: normal; font-family: 'Andale Mono';=
color: rgb(40, 254, 20); background-color: rgba(0, 0, 0, 0.901961);" class=
=3D"" <span
style=3D"font-variant-ligatures: no-common-ligatures"
class=3D"">2017=
-09-27 23:08:48,389-04 ERROR [org.ovirt.engine.core.bll.storage.connection.=
BaseFsStorageHelper] (DefaultQuartzScheduler7) [7670f89e] The connection wi=
th details 'localhost:/shares' failed
because of error code '477' and error message is: problem while trying to =
mount target</span></div <div style=3D"margin: 0px;
line-height: normal; font-family: 'Andale Mono';=
color: rgb(40, 254, 20); background-color: rgba(0, 0, 0, 0.901961);" class=
=3D"" <span
style=3D"font-variant-ligatures: no-common-ligatures"
class=3D"">2017=
-09-27 23:08:48,401-04 INFO [org.ovirt.engine.core.bll.storage.pool.C=
onnectHostToStoragePoolServersCommand] (DefaultQuartzScheduler7) [7670f89e]=
Host 'mennode2' storage connection was failed </span></div <div style=3D"margin: 0px; line-height: normal;
font-family: 'Andale Mono';=
color: rgb(40, 254, 20); background-color: rgba(0, 0, 0, 0.901961);" class=
=3D"" <span
style=3D"font-variant-ligatures: no-common-ligatures"
class=3D"">2017=
-09-27 23:08:48,411-04 WARN [org.ovirt.engine.core.dal.dbbroker.audit=
loghandling.AuditLogDirector] (DefaultQuartzScheduler7) [7670f89e] EVENT_ID=
: CONNECT_STORAGE_SERVERS_FAILED(994), Correlation
ID: 7670f89e, Call Stack: null, Custom ID: null, Custom Event ID: -1, Mess=
age: Failed to connect Host mennode2 to Storage Servers</span></div <div style=3D"margin: 0px; line-height: normal;
font-family: 'Andale Mono';=
color: rgb(40, 254, 20); background-color: rgba(0, 0, 0, 0.901961);" class=
=3D"" <span
style=3D"font-variant-ligatures: no-common-ligatures"
class=3D"">2017=
-09-27 23:08:48,423-04 INFO [org.ovirt.engine.core.vdsbroker.vdsbroke=
r.ConnectStoragePoolVDSCommand] (org.ovirt.thread.pool-6-thread-8) [] START=
, ConnectStoragePoolVDSCommand(HostName =3D
mennode2, ConnectStoragePoolVDSCommandParameters:{runAsync=3D'true', hostI=
d=3D'0ef7ec10-af19-4afa-980b-77a772c4e014', vdsId=3D'0ef7ec10-af19-4afa-980=
b-77a772c4e014', storagePoolId=3D'59c7f8f3-0063-00a8-02c7-0000000000f3', ma=
sterVersion=3D'1'}), log id: 3d5c5d79</span></div <div style=3D"margin: 0px; line-height: normal;
font-family: 'Andale Mono';=
color: rgb(40, 254, 20); background-color: rgba(0, 0, 0, 0.901961);" class=
=3D"" <span
style=3D"font-variant-ligatures: no-common-ligatures"
class=3D"">2017=
-09-27 23:08:48,423-04 INFO [org.ovirt.engine.core.vdsbroker.vdsbroke=
r.ConnectStoragePoolVDSCommand] (org.ovirt.thread.pool-6-thread-8) [] Execu=
ting with domain map: {7d18ff24-57a3-4b4a-9934-0263191fe2e4=3Dactive,
da001a29-eca5-44d6-a097-129dd9be623f=3Dactive, 795d4a1d-3ceb-4773-99de-8e7=
cf05112f3=3Dactive, f36157cc-b25a-400a-ab0f-a071e8a8eea7=3Dactive}</span></=
div <div style=3D"margin: 0px; line-height: normal;
font-family: 'Andale Mono';=
color: rgb(40, 254, 20); background-color: rgba(0, 0, 0, 0.901961);" class=
=3D"" <span
style=3D"font-variant-ligatures: no-common-ligatures"
class=3D"">2017=
-09-27 23:08:49,656-04 INFO [org.ovirt.engine.core.vdsbroker.vdsbroke=
r.ConnectStoragePoolVDSCommand] (org.ovirt.thread.pool-6-thread-8) [] FINIS=
H, ConnectStoragePoolVDSCommand, log id:
3d5c5d79</span></div <div style=3D"margin: 0px;
line-height: normal; font-family: 'Andale Mono';=
color: rgb(40, 254, 20); background-color: rgba(0, 0, 0, 0.901961);" class=
=3D"" <span
style=3D"font-variant-ligatures: no-common-ligatures"
class=3D"">2017=
-09-27 23:08:50,681-04 ERROR [org.ovirt.engine.core.vdsbroker.irsbroker.Irs=
Proxy] (org.ovirt.thread.pool-6-thread-8) [] Domain '7d18ff24-57a3-4b4a-993=
4-0263191fe2e4:hosted_storage' was reported
with error code '358'</span></div <div
style=3D"margin: 0px; line-height: normal; font-family: 'Andale Mono';=
color: rgb(40, 254, 20); background-color: rgba(0, 0, 0, 0.901961);" class=
=3D"" <span
style=3D"font-variant-ligatures: no-common-ligatures"
class=3D"">2017=
-09-27 23:08:50,682-04 ERROR [org.ovirt.engine.core.bll.InitVdsOnUpCommand]=
(org.ovirt.thread.pool-6-thread-8) [] Storage Domain 'hosted_storage' of p=
ool 'Train1' is in problem in host 'mennode2'</span></div <div style=3D"margin: 0px; line-height: normal;
font-family: 'Andale Mono';=
color: rgb(40, 254, 20); background-color: rgba(0, 0, 0, 0.901961);" class=
=3D"" <span
style=3D"font-variant-ligatures: no-common-ligatures"
class=3D"">2017=
-09-27 23:08:50,690-04 ERROR [org.ovirt.engine.core.dal.dbbroker.auditlogha=
ndling.AuditLogDirector] (org.ovirt.thread.pool-6-thread-8) [] EVENT_ID: VD=
S_STORAGE_VDS_STATS_FAILED(189), Correlation
ID: 63cdc728, Call Stack: null, Custom ID: null, Custom Event ID: -1, Mess=
age: Host mennode2 reports about one of the Active Storage Domains as Probl=
ematic.</span></div <div style=3D"margin: 0px;
line-height: normal; font-family: 'Andale Mono';=
color: rgb(40, 254, 20); background-color: rgba(0, 0, 0, 0.901961);" class=
=3D"" <span
style=3D"font-variant-ligatures: no-common-ligatures"
class=3D"">2017=
-09-27 23:08:50,726-04 INFO [org.ovirt.engine.core.bll.SetNonOperatio=
nalVdsCommand] (DefaultQuartzScheduler7) [748ef73f] Running command: SetNon=
OperationalVdsCommand internal: true. Entities
affected : ID: 0ef7ec10-af19-4afa-980b-77a772c4e014 Type: VDS</span>=
</div <div style=3D"margin: 0px;
line-height: normal; font-family: 'Andale Mono';=
color: rgb(40, 254, 20); background-color: rgba(0, 0, 0, 0.901961);" class=
=3D"" <span
style=3D"font-variant-ligatures: no-common-ligatures"
class=3D"">2017=
-09-27 23:08:50,729-04 INFO [org.ovirt.engine.core.vdsbroker.SetVdsSt=
atusVDSCommand] (DefaultQuartzScheduler7) [748ef73f] START, SetVdsStatusVDS=
Command(HostName =3D mennode2, SetVdsStatusVDSCommandParameters:{runAsync=
=3D'true',
hostId=3D'0ef7ec10-af19-4afa-980b-77a772c4e014',
status=3D'NonOperational'=
, nonOperationalReason=3D'STORAGE_DOMAIN_UNREACHABLE', stopSpmFailureLogged=
=3D'false', maintenanceReason=3D'null'}), log id:
2589acd2</span></div <div style=3D"margin: 0px;
line-height: normal; font-family: 'Andale Mono';=
color: rgb(40, 254, 20); background-color: rgba(0, 0, 0, 0.901961);" class=
=3D"" <span
style=3D"font-variant-ligatures: no-common-ligatures"
class=3D"">2017=
-09-27 23:08:50,736-04 INFO [org.ovirt.engine.core.vdsbroker.SetVdsSt=
atusVDSCommand] (DefaultQuartzScheduler7) [748ef73f] FINISH, SetVdsStatusVD=
SCommand, log id: 2589acd2</span></div <div
style=3D"margin: 0px; line-height: normal; font-family: 'Andale Mono';=
color: rgb(40, 254, 20); background-color: rgba(0, 0, 0, 0.901961);" class=
=3D"" <span
style=3D"font-variant-ligatures: no-common-ligatures"
class=3D"">2017=
-09-27 23:08:50,807-04 WARN [org.ovirt.engine.core.dal.dbbroker.audit=
loghandling.AuditLogDirector] (DefaultQuartzScheduler7) [748ef73f] EVENT_ID=
: VDS_SET_NONOPERATIONAL_DOMAIN(522), Correlation
ID: 748ef73f, Job ID: e80941e4-3dbf-4159-8421-96d2d90160ca, Call Stack: nu=
ll, Custom ID: null, Custom Event ID: -1, Message: Host mennode2 cannot acc=
ess the Storage Domain(s) hosted_storage attached to the Data Center Train1=
. Setting Host state to Non-Operational.</span></div <div
style=3D"margin: 0px; line-height: normal; font-family: 'Andale Mono';=
color: rgb(40, 254, 20); background-color: rgba(0, 0, 0, 0.901961);" class=
=3D"" <span
style=3D"font-variant-ligatures: no-common-ligatures"
class=3D"">2017=
-09-27 23:08:50,820-04 WARN [org.ovirt.engine.core.dal.dbbroker.audit=
loghandling.AuditLogDirector] (DefaultQuartzScheduler7) [748ef73f] EVENT_ID=
: VDS_ALERT_FENCE_IS_NOT_CONFIGURED(9,000),
Correlation ID: 63cdc728, Call Stack: null, Custom ID: null, Custom Event =
ID: -1, Message: Failed to verify Power Management configuration for Host m=
ennode2.</span></div <div style=3D"margin: 0px;
line-height: normal; font-family: 'Andale Mono';=
color: rgb(40, 254, 20); background-color: rgba(0, 0, 0, 0.901961);" class=
=3D"" <span
style=3D"font-variant-ligatures: no-common-ligatures"
class=3D"">2017=
-09-27 23:08:50,826-04 WARN [org.ovirt.engine.core.dal.dbbroker.audit=
loghandling.AuditLogDirector] (DefaultQuartzScheduler7) [748ef73f] EVENT_ID=
: CONNECT_STORAGE_POOL_FAILED(995), Correlation
ID: 63cdc728, Call Stack: null, Custom ID: null, Custom Event ID: -1, Mess=
age: Failed to connect Host mennode2 to Storage Pool Train1</span></div <div style=3D"margin: 0px; line-height: normal;
font-family: 'Andale Mono';=
color: rgb(40, 254, 20); background-color: rgba(0, 0, 0, 0.901961);" class=
=3D"" <span
style=3D"font-variant-ligatures: no-common-ligatures"
class=3D"">2017=
-09-27 23:08:50,856-04 INFO [org.ovirt.engine.core.bll.HandleVdsVersi=
onCommand] (DefaultQuartzScheduler7) [42cbaee1] Running command: HandleVdsV=
ersionCommand internal: true. Entities affected
: ID: 0ef7ec10-af19-4afa-980b-77a772c4e014 Type: VDS</span></div <div style=3D"margin: 0px; line-height: normal;
font-family: 'Andale Mono';=
color: rgb(40, 254, 20); background-color: rgba(0, 0, 0, 0.901961);" class=
=3D"" <span
style=3D"font-variant-ligatures: no-common-ligatures"
class=3D"">2017=
-09-27 23:08:50,863-04 INFO [org.ovirt.engine.core.vdsbroker.monitori=
ng.HostMonitoring] (DefaultQuartzScheduler7) [42cbaee1] Host 'mennode2'(0ef=
7ec10-af19-4afa-980b-77a772c4e014) is already
in NonOperational status for reason 'STORAGE_DOMAIN_UNREACHABLE'. SetNonOp=
erationalVds command is skipped.</span></div
</div <div
class=3D""><br class=3D""
</div <div
class=3D""><br class=3D""
</div <div class=3D"" <blockquote type=3D"cite"
class=3D""><b class=3D""><span class=3D""
style=
=3D"float: none; display: inline !important;">/</span><span
class=3D"" styl=
e=3D"color: rgb(69, 69, 69); font-family: 'Helvetica
Neue';">etc/ovirt-host=
ed-engine/hosted-engine.conf </span></b></blockquote </div <div
class=3D""><br class=3D""
</div <div class=3D"" <div style=3D"margin: 0px; line-height: normal;
font-family: 'Andale Mono';=
color: rgb(40, 254, 20); background-color: rgba(0, 0, 0, 0.901961);" class=
=3D"" <span
style=3D"font-variant-ligatures: no-common-ligatures"
class=3D"">fqdn=
=3Dengine.traindemo.local</span></div <div
style=3D"margin: 0px; line-height: normal; font-family: 'Andale Mono';=
color: rgb(40, 254, 20); background-color: rgba(0, 0, 0, 0.901961);" class=
=3D"" <span
style=3D"font-variant-ligatures: no-common-ligatures"
class=3D"">vm_d=
isk_id=3D0077cb58-7abe-4bdf-96b1-55cc85030d76</span></div <div style=3D"margin: 0px; line-height: normal;
font-family: 'Andale Mono';=
color: rgb(40, 254, 20); background-color: rgba(0, 0, 0, 0.901961);" class=
=3D"" <span
style=3D"font-variant-ligatures: no-common-ligatures"
class=3D"">vm_d=
isk_vol_id=3Dd5bd2690-7cfb-4aa2-b898-41579114ec08</span></div <div style=3D"margin: 0px; line-height: normal;
font-family: 'Andale Mono';=
color: rgb(40, 254, 20); background-color: rgba(0, 0, 0, 0.901961);" class=
=3D"" <span
style=3D"font-variant-ligatures: no-common-ligatures"
class=3D"">vmid=
=3D564d290f-ae66-4352-9770-f2e062000358</span></div <div
style=3D"margin: 0px; line-height: normal; font-family: 'Andale Mono';=
color: rgb(40, 254, 20); background-color: rgba(0, 0, 0, 0.901961);" class=
=3D"" <span
style=3D"font-variant-ligatures: no-common-ligatures"
class=3D"">stor=
age=3Dmenmaster.traindemo.local:/shares</span></div <div
style=3D"margin: 0px; line-height: normal; font-family: 'Andale Mono';=
color: rgb(40, 254, 20); background-color: rgba(0, 0, 0, 0.901961);" class=
=3D"" <span
style=3D"font-variant-ligatures: no-common-ligatures"
class=3D"">mnt_=
options=3D</span></div <div style=3D"margin: 0px;
line-height: normal; font-family: 'Andale Mono';=
color: rgb(40, 254, 20); background-color: rgba(0, 0, 0, 0.901961);" class=
=3D"" <span
style=3D"font-variant-ligatures: no-common-ligatures"
class=3D"">conf=
=3D/var/run/ovirt-hosted-engine-ha/vm.conf</span></div <div style=3D"margin: 0px; line-height: normal;
font-family: 'Andale Mono';=
color: rgb(40, 254, 20); background-color: rgba(0, 0, 0, 0.901961);" class=
=3D"" <span
style=3D"font-variant-ligatures: no-common-ligatures"
class=3D"">host=
_id=3D1</span></div <div style=3D"margin: 0px;
line-height: normal; font-family: 'Andale Mono';=
color: rgb(40, 254, 20); background-color: rgba(0, 0, 0, 0.901961);" class=
=3D"" <span
style=3D"font-variant-ligatures: no-common-ligatures"
class=3D"">cons=
ole=3Dvnc</span></div <div style=3D"margin: 0px;
line-height: normal; font-family: 'Andale Mono';=
color: rgb(40, 254, 20); background-color: rgba(0, 0, 0, 0.901961);" class=
=3D"" <span
style=3D"font-variant-ligatures: no-common-ligatures"
class=3D"">doma=
inType=3Dnfs3</span></div <div style=3D"margin: 0px;
line-height: normal; font-family: 'Andale Mono';=
color: rgb(40, 254, 20); background-color: rgba(0, 0, 0, 0.901961);" class=
=3D"" <span
style=3D"font-variant-ligatures: no-common-ligatures"
class=3D"">spUU=
ID=3D00000000-0000-0000-0000-000000000000</span></div <div style=3D"margin: 0px; line-height: normal;
font-family: 'Andale Mono';=
color: rgb(40, 254, 20); background-color: rgba(0, 0, 0, 0.901961);" class=
=3D"" <span
style=3D"font-variant-ligatures: no-common-ligatures"
class=3D"">sdUU=
ID=3D7d18ff24-57a3-4b4a-9934-0263191fe2e4</span></div <div style=3D"margin: 0px; line-height: normal;
font-family: 'Andale Mono';=
color: rgb(40, 254, 20); background-color: rgba(0, 0, 0, 0.901961);" class=
=3D"" <span
style=3D"font-variant-ligatures: no-common-ligatures"
class=3D"">conn=
ectionUUID=3Daddfd5fc-8668-4d2a-9b0b-f801b21a98d8</span></div <div style=3D"margin: 0px; line-height: normal;
font-family: 'Andale Mono';=
color: rgb(40, 254, 20); background-color: rgba(0, 0, 0, 0.901961);" class=
=3D"" <span
style=3D"font-variant-ligatures: no-common-ligatures"
class=3D"">ca_c=
ert=3D/etc/pki/vdsm/libvirt-spice/ca-cert.pem</span></div <div style=3D"margin: 0px; line-height: normal;
font-family: 'Andale Mono';=
color: rgb(40, 254, 20); background-color: rgba(0, 0, 0, 0.901961);" class=
=3D"" <span
style=3D"font-variant-ligatures: no-common-ligatures"
class=3D"">ca_s=
ubject=3D"C=3DEN, L=3DTest, O=3DTest,
CN=3DTest"</span></div <div
style=3D"margin: 0px; line-height: normal; font-family: 'Andale Mono';=
color: rgb(40, 254, 20); background-color: rgba(0, 0, 0, 0.901961);" class=
=3D"" <span
style=3D"font-variant-ligatures: no-common-ligatures"
class=3D"">vdsm=
_use_ssl=3Dtrue</span></div <div
style=3D"margin: 0px; line-height: normal; font-family: 'Andale Mono';=
color: rgb(40, 254, 20); background-color: rgba(0, 0, 0, 0.901961);" class=
=3D"" <span
style=3D"font-variant-ligatures: no-common-ligatures"
class=3D"">gate=
way=3D10.0.0.1</span></div <div
style=3D"margin: 0px; line-height: normal; font-family: 'Andale Mono';=
color: rgb(40, 254, 20); background-color: rgba(0, 0, 0, 0.901961);" class=
=3D"" <span
style=3D"font-variant-ligatures: no-common-ligatures"
class=3D"">brid=
ge=3Dovirtmgmt</span></div <div
style=3D"margin: 0px; line-height: normal; font-family: 'Andale Mono';=
color: rgb(40, 254, 20); background-color: rgba(0, 0, 0, 0.901961);" class=
=3D"" <span
style=3D"font-variant-ligatures: no-common-ligatures"
class=3D"">meta=
data_volume_UUID=3D156c7549-8e2e-4580-9825-81c3b13387ce</span></div <div style=3D"margin: 0px; line-height: normal;
font-family: 'Andale Mono';=
color: rgb(40, 254, 20); background-color: rgba(0, 0, 0, 0.901961);" class=
=3D"" <span
style=3D"font-variant-ligatures: no-common-ligatures"
class=3D"">meta=
data_image_UUID=3Dc5db06ac-ea37-47bc-9065-b94eb230616a</span></div <div style=3D"margin: 0px; line-height: normal;
font-family: 'Andale Mono';=
color: rgb(40, 254, 20); background-color: rgba(0, 0, 0, 0.901961);" class=
=3D"" <span
style=3D"font-variant-ligatures: no-common-ligatures"
class=3D"">lock=
space_volume_UUID=3D5f4f4a63-5608-4daf-a034-2b8e83b5de42</span></div <div style=3D"margin: 0px; line-height: normal;
font-family: 'Andale Mono';=
color: rgb(40, 254, 20); background-color: rgba(0, 0, 0, 0.901961);" class=
=3D"" <span
style=3D"font-variant-ligatures: no-common-ligatures"
class=3D"">lock=
space_image_UUID=3D0d7f8661-3abd-44e8-87a7-3b85f2e29fda</span></div <div style=3D"margin: 0px; line-height: normal;
font-family: 'Andale Mono';=
color: rgb(40, 254, 20); background-color: rgba(0, 0, 0, 0.901961);" class=
=3D"" <span
style=3D"font-variant-ligatures: no-common-ligatures"
class=3D"">conf=
_volume_UUID=3Dddffd499-294c-456f-bb0e-08e366380a22</span></div <div style=3D"margin: 0px; line-height: normal;
font-family: 'Andale Mono';=
color: rgb(40, 254, 20); background-color: rgba(0, 0, 0, 0.901961);" class=
=3D"" <span
style=3D"font-variant-ligatures: no-common-ligatures"
class=3D"">conf=
_image_UUID=3D5f264ea1-4654-4e40-87aa-80fcaacc60d8</span></div </div <div
class=3D"">Thanks,</div <div
class=3D""><br class=3D""
</div <br class=3D"" <div class=3D"" <div
style=3D"color: rgb(0, 0, 0); font-family: Helvetica; font-size: 12px;=
font-style: normal; font-variant-caps: normal; font-weight: normal; letter=
-spacing: normal; orphans: auto; text-align: start; text-indent: 0px; text-=
transform: none; white-space: normal; widows: auto; word-spacing: 0px; -web=
kit-text-size-adjust: auto; -webkit-text-stroke-width: 0px; word-wrap: brea=
k-word; -webkit-nbsp-mode: space; -webkit-line-break: after-white-space;" c=
lass=3D"" <div style=3D"color: rgb(0,
0, 0); font-family: Helvetica; font-size: 12px;=
font-style: normal; font-variant-caps: normal; font-weight: normal; letter=
-spacing: normal; text-align: start; text-indent: 0px; text-transform: none=
; white-space: normal; word-spacing: 0px; -webkit-text-stroke-width: 0px;" Alex Witte<br class=3D"" <br class=3D""
</div </div </div <br
class=3D"" <div
<blockquote type=3D"cite" class=3D"" <div class=3D"">On Sep 27, 2017, at 8:32 PM,
Alexander Witte <<a href=3D=
"mailto:alexander.witte@baicanada.com"
class=3D"">alexander.witte@baicanada=
.com</a>> wrote:</div <br
class=3D"Apple-interchange-newline" <div
class=3D""><span style=3D"font-family: Helvetica; font-size: 12px;
fon=
t-style: normal; font-variant-caps: normal; font-weight: normal; letter-spa=
cing: normal; text-align: start; text-indent: 0px; text-transform: none; wh=
ite-space: normal; word-spacing: 0px; -webkit-text-stroke-width: 0px; float=
: none; display: inline !important;" class=3D"">/</span><span
class=3D"" st=
yle=3D"font-size: 12px; font-style: normal; font-variant-caps: normal; font=
-weight: normal; letter-spacing: normal; orphans: auto; text-align: start; =
text-indent: 0px; text-transform: none; white-space: normal; widows: auto; =
word-spacing: 0px; -webkit-text-size-adjust: auto; -webkit-text-stroke-widt=
h: 0px; color: rgb(69, 69, 69); font-family: 'Helvetica
Neue';">etc/ovirt-h=
osted-engine/hosted-engine.conf<span
class=3D"Apple-converted-space"> =
</span></span></div </blockquote </div <br
class=3D"" </body </html
--_000_85068B30E9EA40D5883D6574AC1EC5FBbaicanadacom_--