What about the hosts status?
Can you please send the hosts screenshot and the datacenter screenshot?
On Tue, Feb 20, 2018 at 9:03 AM, michael pagdanganan <mhke_aj5566(a)yahoo.com>
wrote:
Hi,
Screenshot
only 1 data center but 2 master domain and got event warning
[image: Inline image]
On Tuesday, February 20, 2018 2:56 PM, Eyal Shenitzky <eshenitz(a)redhat.com>
wrote:
Also, can you please describe the current setup of the environment?
I'm not sure I understand, do you have 2 data-centers?
Please attach some screenshots of the current situation.
On Tue, Feb 20, 2018 at 8:43 AM, michael pagdanganan <
mhke_aj5566(a)yahoo.com> wrote:
Sorry can't attached log file it's too big file
VDSM.log for node 1
2766', 'lastCheck': '4.9', 'valid': True}} from=internal,
task_id=645d456e-f59f-4b1c- 9e97-fc82d19a36b1 (api:52)
2018-02-20 14:38:47,222+0800 INFO (jsonrpc/3) [vdsm.api] START
repoStats(options=None) from=::ffff:10.10.43.1,60554, flow_id=3b2e802e,
task_id=28c795d1-1639-4e68- a9fe-a00006be268f (api:46)
2018-02-20 14:38:47,222+0800 INFO (jsonrpc/3) [vdsm.api] FINISH repoStats
return={'f3e372e3-1251-4195- a4b9-1027e40059df': {'code': 0,
'actual':
True, 'version': 4, 'acquired': True, 'delay':
'0.000119109', 'lastCheck':
'6.0', 'valid': True}, 'e83d0d46-6ea6-4aa3-80bf- 6e95c66b0454':
{'code': 0,
'actual': True, 'version': 0, 'acquired': True, 'delay':
'0.000195263',
'lastCheck': '6.0', 'valid': True},
'65ca2e2d-b472-4bee-85b4-
09a161464b20': {'code': 0, 'actual': True, 'version': 4,
'acquired': True,
'delay': '0.00022988', 'lastCheck': '6.0',
'valid': True},
'4bf2ba2f-f57a-4d9f-b42a- fb78f440a358': {'code': 0, 'actual':
True,
'version': 0, 'acquired': True, 'delay': '0.000270206',
'lastCheck': '5.9',
'valid': True}, '42e591b7-f86c-4b67-a3d2- 40cc007f7662': {'code':
0,
'actual': True, 'version': 4, 'acquired': True, 'delay':
'0.000272766',
'lastCheck': '6.0', 'valid': True}}
from=::ffff:10.10.43.1,60554,
flow_id=3b2e802e, task_id=28c795d1-1639-4e68- a9fe-a00006be268f (api:52)
2018-02-20 14:38:47,226+0800 INFO (jsonrpc/3) [jsonrpc.JsonRpcServer] RPC
call Host.getStats succeeded in 0.00 seconds (__init__:539)
2018-02-20 14:38:55,252+0800 INFO (jsonrpc/4) [jsonrpc.JsonRpcServer] RPC
call Host.getAllVmStats succeeded in 0.00 seconds (__init__:539)
2018-02-20 14:38:58,566+0800 INFO (jsonrpc/2) [jsonrpc.JsonRpcServer] RPC
call Host.getAllVmStats succeeded in 0.00 seconds (__init__:539)
2018-02-20 14:39:01,093+0800 INFO (periodic/1) [vdsm.api] START
repoStats(options=None) from=internal, task_id=04f55ded-5841-44ae-
a376-4f6e723e4b10 (api:46)
2018-02-20 14:39:01,093+0800 INFO (periodic/1) [vdsm.api] FINISH
repoStats return={'f3e372e3-1251-4195- a4b9-1027e40059df': {'code': 0,
'actual': True, 'version': 4, 'acquired': True, 'delay':
'9.3888e-05',
'lastCheck': '9.9', 'valid': True},
'e83d0d46-6ea6-4aa3-80bf-
6e95c66b0454': {'code': 0, 'actual': True, 'version': 0,
'acquired': True,
'delay': '0.000197535', 'lastCheck': '9.8',
'valid': True},
'65ca2e2d-b472-4bee-85b4- 09a161464b20': {'code': 0, 'actual':
True,
'version': 4, 'acquired': True, 'delay': '0.000191456',
'lastCheck': '9.8',
'valid': True}, '4bf2ba2f-f57a-4d9f-b42a- fb78f440a358': {'code':
0,
'actual': True, 'version': 0, 'acquired': True, 'delay':
'0.00026365',
'lastCheck': '9.8', 'valid': True},
'42e591b7-f86c-4b67-a3d2-
40cc007f7662': {'code': 0, 'actual': True, 'version': 4,
'acquired': True,
'delay': '0.000243658', 'lastCheck': '0.0',
'valid': True}} from=internal,
task_id=04f55ded-5841-44ae- a376-4f6e723e4b10 (api:52)
2018-02-20 14:39:02,295+0800 INFO (jsonrpc/6) [vdsm.api] START
repoStats(options=None) from=::ffff:10.10.43.1,60554, flow_id=13149d6e,
task_id=5f19d76a-d343-4dc4- ad09-024ec27f7443 (api:46)
2018-02-20 14:39:02,295+0800 INFO (jsonrpc/6) [vdsm.api] FINISH repoStats
return={'f3e372e3-1251-4195- a4b9-1027e40059df': {'code': 0,
'actual':
True, 'version': 4, 'acquired': True, 'delay':
'9.4759e-05', 'lastCheck':
'1.1', 'valid': True}, 'e83d0d46-6ea6-4aa3-80bf- 6e95c66b0454':
{'code': 0,
'actual': True, 'version': 0, 'acquired': True, 'delay':
'0.000183158',
'lastCheck': '1.0', 'valid': True},
'65ca2e2d-b472-4bee-85b4-
09a161464b20': {'code': 0, 'actual': True, 'version': 4,
'acquired': True,
'delay': '0.000222609', 'lastCheck': '1.0',
'valid': True},
'4bf2ba2f-f57a-4d9f-b42a- fb78f440a358': {'code': 0, 'actual':
True,
'version': 0, 'acquired': True, 'delay': '0.000211253',
'lastCheck': '1.0',
'valid': True}, '42e591b7-f86c-4b67-a3d2- 40cc007f7662': {'code':
0,
'actual': True, 'version': 4, 'acquired': True, 'delay':
'0.000243658',
'lastCheck': '1.1', 'valid': True}}
from=::ffff:10.10.43.1,60554,
flow_id=13149d6e, task_id=5f19d76a-d343-4dc4- ad09-024ec27f7443 (api:52)
2018-02-20 14:39:02,300+0800 INFO (jsonrpc/6) [jsonrpc.JsonRpcServer] RPC
call Host.getStats succeeded in 0.01 seconds (__init__:539)
2018-02-20 14:39:10,270+0800 INFO (jsonrpc/1) [jsonrpc.JsonRpcServer] RPC
call Host.getAllVmStats succeeded in 0.00 seconds (__init__:539)
2018-02-20 14:39:13,631+0800 INFO (jsonrpc/0) [vdsm.api] START
connectStorageServer(domType= 1, spUUID=u'00000000-0000-0000-
0000-000000000000', conList=[{u'id': u'5e4c94f1-f3b9-4fbd-a6c7-
e732d0fe3123', u'connection': u'dev2node1.lares.com.ph:/run/
media/root/Slave1Data/ dataNode1', u'iqn': u'', u'user':
u'', u'tpgt':
u'1', u'protocol_version': u'auto', u'password':
'********', u'port':
u''}], options=None) from=::ffff:10.10.43.1,60554, flow_id=15b57417,
task_id=bfce1e70-ef4a-4e13- aaaa-7a66aaf44429 (api:46)
2018-02-20 14:39:13,633+0800 INFO (jsonrpc/0) [vdsm.api] FINISH
connectStorageServer return={'statuslist': [{'status': 0, 'id':
u'5e4c94f1-f3b9-4fbd-a6c7- e732d0fe3123'}]} from=::ffff:10.10.43.1,60554,
flow_id=15b57417, task_id=bfce1e70-ef4a-4e13- aaaa-7a66aaf44429 (api:52)
2018-02-20 14:39:13,634+0800 INFO (jsonrpc/0) [jsonrpc.JsonRpcServer] RPC
call StoragePool. connectStorageServer succeeded in 0.00 seconds
(__init__:539)
2018-02-20 14:39:13,806+0800 INFO (jsonrpc/7) [vdsm.api] START
connectStoragePool(spUUID=u' 5a865884-0366-0330-02b8- 0000000002d4',
hostID=1, msdUUID=u'f3e372e3-1251-4195- a4b9-1027e40059df',
masterVersion=65, domainsMap={u'e83d0d46-6ea6- 4aa3-80bf-6e95c66b0454':
u'active', u'4bf2ba2f-f57a-4d9f-b42a- fb78f440a358': u'active',
u'225e1975-8121-4370-b317- 86e964ae326f': u'attached',
u'f3e372e3-1251-4195-a4b9- 1027e40059df': u'active',
u'65ca2e2d-b472-4bee-85b4- 09a161464b20': u'active',
u'42e591b7-f86c-4b67-a3d2- 40cc007f7662': u'active'}, options=None)
from=::ffff:10.10.43.1,60554, flow_id=15b57417, task_id=927a5d9a-4304-4776-
b5c9-22ba5d0cb853 (api:46)
2018-02-20 14:39:13,807+0800 INFO (jsonrpc/7) [storage.
StoragePoolMemoryBackend] new storage pool master version 65 and domains
map {u'e83d0d46-6ea6-4aa3-80bf- 6e95c66b0454': u'Active',
u'4bf2ba2f-f57a-4d9f-b42a- fb78f440a358': u'Active',
u'225e1975-8121-4370-b317- 86e964ae326f': u'Attached',
u'f3e372e3-1251-4195-a4b9- 1027e40059df': u'Active',
u'65ca2e2d-b472-4bee-85b4- 09a161464b20': u'Active',
u'42e591b7-f86c-4b67-a3d2- 40cc007f7662': u'Active'} (spbackends:450)
VDSM.log 2
2018-02-20 14:41:14,598+0800 INFO (periodic/3) [vdsm.api] FINISH
repoStats return={'f3e372e3-1251-4195- a4b9-1027e40059df': {'code': 0,
'actual': True, 'version': 4, 'acquired': True, 'delay':
'0.000252423',
'lastCheck': '6.0', 'valid': True},
'e83d0d46-6ea6-4aa3-80bf-
6e95c66b0454': {'code': 0, 'actual': True, 'version': 0,
'acquired': True,
'delay': '7.5567e-05', 'lastCheck': '6.0',
'valid': True},
'65ca2e2d-b472-4bee-85b4- 09a161464b20': {'code': 0, 'actual':
True,
'version': 4, 'acquired': True, 'delay': '8.2433e-05',
'lastCheck': '6.0',
'valid': True}, '4bf2ba2f-f57a-4d9f-b42a- fb78f440a358': {'code':
0,
'actual': True, 'version': 0, 'acquired': True, 'delay':
'0.000237247',
'lastCheck': '5.9', 'valid': True},
'42e591b7-f86c-4b67-a3d2-
40cc007f7662': {'code': 0, 'actual': True, 'version': 4,
'acquired': True,
'delay': '8.4376e-05', 'lastCheck': '6.0',
'valid': True}} from=internal,
task_id=6213712b-9903-4db8- 9836-3baf85cd63e4 (api:52)
2018-02-20 14:41:18,074+0800 INFO (jsonrpc/6) [jsonrpc.JsonRpcServer] RPC
call Host.getAllVmStats succeeded in 0.00 seconds (__init__:539)
2018-02-20 14:41:24,333+0800 INFO (jsonrpc/4) [vdsm.api] START
repoStats(options=None) from=::ffff:10.10.43.1,56540, flow_id=73f86113,
task_id=e8231ecb-3543-4d8f- af54-4cf2b06ee98a (api:46)
2018-02-20 14:41:24,334+0800 INFO (jsonrpc/4) [vdsm.api] FINISH repoStats
return={'f3e372e3-1251-4195- a4b9-1027e40059df': {'code': 0,
'actual':
True, 'version': 4, 'acquired': True, 'delay':
'0.000172953', 'lastCheck':
'5.7', 'valid': True}, 'e83d0d46-6ea6-4aa3-80bf- 6e95c66b0454':
{'code': 0,
'actual': True, 'version': 0, 'acquired': True, 'delay':
'7.701e-05',
'lastCheck': '5.7', 'valid': True},
'65ca2e2d-b472-4bee-85b4-
09a161464b20': {'code': 0, 'actual': True, 'version': 4,
'acquired': True,
'delay': '0.000117677', 'lastCheck': '5.7',
'valid': True},
'4bf2ba2f-f57a-4d9f-b42a- fb78f440a358': {'code': 0, 'actual':
True,
'version': 0, 'acquired': True, 'delay': '0.000185146',
'lastCheck': '5.7',
'valid': True}, '42e591b7-f86c-4b67-a3d2- 40cc007f7662': {'code':
0,
'actual': True, 'version': 4, 'acquired': True, 'delay':
'9.1467e-05',
'lastCheck': '5.8', 'valid': True}}
from=::ffff:10.10.43.1,56540,
flow_id=73f86113, task_id=e8231ecb-3543-4d8f- af54-4cf2b06ee98a (api:52)
2018-02-20 14:41:24,338+0800 INFO (jsonrpc/4) [jsonrpc.JsonRpcServer] RPC
call Host.getStats succeeded in 0.00 seconds (__init__:539)
2018-02-20 14:41:27,087+0800 INFO (jsonrpc/0) [jsonrpc.JsonRpcServer] RPC
call Host.getAllVmStats succeeded in 0.00 seconds (__init__:539)
2018-02-20 14:41:29,607+0800 INFO (periodic/1) [vdsm.api] START
repoStats(options=None) from=internal, task_id=f14b7aab-b64a-4903-
9368-d665e39b49d1 (api:46)
2018-02-20 14:41:29,608+0800 INFO (periodic/1) [vdsm.api] FINISH
repoStats return={'f3e372e3-1251-4195- a4b9-1027e40059df': {'code': 0,
'actual': True, 'version': 4, 'acquired': True, 'delay':
'0.000246237',
'lastCheck': '1.0', 'valid': True},
'e83d0d46-6ea6-4aa3-80bf-
6e95c66b0454': {'code': 0, 'actual': True, 'version': 0,
'acquired': True,
'delay': '8.8773e-05', 'lastCheck': '1.0',
'valid': True},
'65ca2e2d-b472-4bee-85b4- 09a161464b20': {'code': 0, 'actual':
True,
'version': 4, 'acquired': True, 'delay': '9.7145e-05',
'lastCheck': '1.0',
'valid': True}, '4bf2ba2f-f57a-4d9f-b42a- fb78f440a358': {'code':
0,
'actual': True, 'version': 0, 'acquired': True, 'delay':
'0.000218729',
'lastCheck': '0.9', 'valid': True},
'42e591b7-f86c-4b67-a3d2-
40cc007f7662': {'code': 0, 'actual': True, 'version': 4,
'acquired': True,
'delay': '9.1035e-05', 'lastCheck': '1.0',
'valid': True}} from=internal,
task_id=f14b7aab-b64a-4903- 9368-d665e39b49d1 (api:52)
2018-02-20 14:41:33,079+0800 INFO (jsonrpc/7) [jsonrpc.JsonRpcServer] RPC
call Host.getAllVmStats succeeded in 0.00 seconds (__init__:539)
2018-02-20 14:41:40,456+0800 INFO (jsonrpc/2) [vdsm.api] START
repoStats(options=None) from=::ffff:10.10.43.1,56540, flow_id=3be8150c,
task_id=1b0c86a8-6fd8-4882- a742-fbd56ccb4037 (api:46)
2018-02-20 14:41:40,457+0800 INFO (jsonrpc/2) [vdsm.api] FINISH repoStats
return={'f3e372e3-1251-4195- a4b9-1027e40059df': {'code': 0,
'actual':
True, 'version': 4, 'acquired': True, 'delay':
'0.000174027', 'lastCheck':
'1.8', 'valid': True}, 'e83d0d46-6ea6-4aa3-80bf- 6e95c66b0454':
{'code': 0,
'actual': True, 'version': 0, 'acquired': True, 'delay':
'0.00011454',
'lastCheck': '1.8', 'valid': True},
'65ca2e2d-b472-4bee-85b4-
09a161464b20': {'code': 0, 'actual': True, 'version': 4,
'acquired': True,
'delay': '0.000145955', 'lastCheck': '1.8',
'valid': True},
'4bf2ba2f-f57a-4d9f-b42a- fb78f440a358': {'code': 0, 'actual':
True,
'version': 0, 'acquired': True, 'delay': '0.000209871',
'lastCheck': '1.8',
'valid': True}, '42e591b7-f86c-4b67-a3d2- 40cc007f7662': {'code':
0,
'actual': True, 'version': 4, 'acquired': True, 'delay':
'9.5744e-05',
'lastCheck': '1.9', 'valid': True}}
from=::ffff:10.10.43.1,56540,
flow_id=3be8150c, task_id=1b0c86a8-6fd8-4882- a742-fbd56ccb4037 (api:52)
2018-02-20 14:41:40,461+0800 INFO (jsonrpc/2) [jsonrpc.JsonRpcServer] RPC
call Host.getStats succeeded in 0.01 seconds (__init__:539)
2018-02-20 14:41:42,106+0800 INFO (jsonrpc/3) [jsonrpc.JsonRpcServer] RPC
call Host.getAllVmStats succeeded in 0.00 seconds (__init__:539)
2018-02-20 14:41:44,622+0800 INFO (periodic/3) [vdsm.api] START
repoStats(options=None) from=internal, task_id=20e0b29b-d3cd-4e44-
b92c-213f9c984ab2 (api:46)
2018-02-20 14:41:44,622+0800 INFO (periodic/3) [vdsm.api] FINISH
repoStats return={'f3e372e3-1251-4195- a4b9-1027e40059df': {'code': 0,
'actual': True, 'version': 4, 'acquired': True, 'delay':
'0.000174027',
'lastCheck': '6.0', 'valid': True},
'e83d0d46-6ea6-4aa3-80bf-
6e95c66b0454': {'code': 0, 'actual': True, 'version': 0,
'acquired': True,
'delay': '0.00011454', 'lastCheck': '6.0',
'valid': True},
'65ca2e2d-b472-4bee-85b4- 09a161464b20': {'code': 0, 'actual':
True,
'version': 4, 'acquired': True, 'delay': '0.000145955',
'lastCheck': '6.0',
'valid': True}, '4bf2ba2f-f57a-4d9f-b42a- fb78f440a358': {'code':
0,
'actual': True, 'version': 0, 'acquired': True, 'delay':
'0.000209871',
'lastCheck': '5.9', 'valid': True},
'42e591b7-f86c-4b67-a3d2-
40cc007f7662': {'code': 0, 'actual': True, 'version': 4,
'acquired': True,
'delay': '9.5744e-05', 'lastCheck': '6.0',
'valid': True}} from=internal,
task_id=20e0b29b-d3cd-4e44- b92c-213f9c984ab2 (api:52)
2018-02-20 14:41:49,083+0800 INFO (jsonrpc/1) [jsonrpc.JsonRpcServer] RPC
call Host.getAllVmStats succeeded in 0.00 seconds (__init__:539)
Engine log
skId '404ccecc-aa7f-45ea-89e4- 726956269bc9' task status 'finished'
2018-02-20 14:42:21,966+08 INFO [org.ovirt.engine.core.
vdsbroker.vdsbroker. SpmStartVDSCommand] (org.ovirt.thread.pool-7-
thread-38) [29528f9] spmStart polling ended, spm status: SPM
2018-02-20 14:42:21,967+08 INFO [org.ovirt.engine.core.
vdsbroker.vdsbroker. HSMClearTaskVDSCommand] (org.ovirt.thread.pool-7-
thread-38) [29528f9] START, HSMClearTaskVDSCommand( HostName = Node1,
HSMTaskGuidBaseVDSCommandParam eters:{runAsync='true',
hostId='7dee35bb-8c97-4f6a- b6cd-abc4258540e4', taskId='404ccecc-aa7f-45ea-
89e4-726956269bc9'}), log id: 71688f70
2018-02-20 14:42:22,922+08 INFO [org.ovirt.engine.core.
vdsbroker.vdsbroker. HSMClearTaskVDSCommand] (org.ovirt.thread.pool-7-
thread-38) [29528f9] FINISH, HSMClearTaskVDSCommand, log id: 71688f70
2018-02-20 14:42:22,923+08 INFO [org.ovirt.engine.core.
vdsbroker.vdsbroker. SpmStartVDSCommand] (org.ovirt.thread.pool-7-
thread-38) [29528f9] FINISH, SpmStartVDSCommand, return:
org.ovirt.engine.core.common. businessentities. SpmStatusResult@78332453,
log id: 3ea35d5
2018-02-20 14:42:22,935+08 INFO [org.ovirt.engine.core.
vdsbroker.irsbroker.IrsProxy] (org.ovirt.thread.pool-7- thread-38)
[29528f9] Initialize Irs proxy from vds: dev2node1.lares.com.ph
2018-02-20 14:42:22,951+08 INFO [org.ovirt.engine.core.dal.
dbbroker.auditloghandling. AuditLogDirector] (org.ovirt.thread.pool-7-
thread-38) [29528f9] EVENT_ID: IRS_HOSTED_ON_VDS(204), Correlation ID:
null, Call Stack: null, Custom ID: null, Custom Event ID: -1, Message:
Storage Pool Manager runs on Host Node1 (Address: dev2node1.lares.com.ph).
2018-02-20 14:42:22,952+08 INFO [org.ovirt.engine.core.
vdsbroker.irsbroker. SPMGetAllTasksInfoVDSCommand]
(org.ovirt.thread.pool-7- thread-38) [29528f9] -- executeIrsBrokerCommand:
Attempting on storage pool '5a865884-0366-0330-02b8- 0000000002d4'
2018-02-20 14:42:22,952+08 INFO [org.ovirt.engine.core.
vdsbroker.vdsbroker. HSMGetAllTasksInfoVDSCommand]
(org.ovirt.thread.pool-7- thread-38) [29528f9] START,
HSMGetAllTasksInfoVDSCommand( HostName = Node1,
VdsIdVDSCommandParametersBase: {runAsync='true',
hostId='7dee35bb-8c97-4f6a- b6cd-abc4258540e4'}), log id: 1bdbea9d
2018-02-20 14:42:22,955+08 INFO [org.ovirt.engine.core.
vdsbroker.irsbroker. SPMGetAllTasksInfoVDSCommand]
(org.ovirt.thread.pool-7- thread-7) [29528f9] START,
SPMGetAllTasksInfoVDSCommand( IrsBaseVDSCommandParameters:{
runAsync='true', storagePoolId='5a865884-0366- 0330-02b8-0000000002d4',
ignoreFailoverLimit='false'}), log id: 5c2422d6
2018-02-20 14:42:23,956+08 INFO [org.ovirt.engine.core.
vdsbroker.vdsbroker. HSMGetAllTasksInfoVDSCommand]
(org.ovirt.thread.pool-7- thread-38) [29528f9] FINISH,
HSMGetAllTasksInfoVDSCommand, return: [], log id: 1bdbea9d
2018-02-20 14:42:23,956+08 INFO [org.ovirt.engine.core.
vdsbroker.irsbroker. SPMGetAllTasksInfoVDSCommand]
(org.ovirt.thread.pool-7- thread-38) [29528f9] FINISH,
SPMGetAllTasksInfoVDSCommand, return: [], log id: 58cbe7b7
2018-02-20 14:42:23,956+08 INFO [org.ovirt.engine.core.bll.
tasks.AsyncTaskManager] (org.ovirt.thread.pool-7- thread-38) [29528f9]
Discovered no tasks on Storage Pool 'UnsecuredEnv'
2018-02-20 14:42:24,936+08 INFO [org.ovirt.vdsm.jsonrpc.
client.reactors.ReactorClient] (SSL Stomp Reactor) [] Connecting to
dev2node1.lares.com.ph/10.10.
43.2 <
http://dev2node1.lares.com.ph/10.10.43.2>
2018-02-20 14:42:27,012+08 WARN [org.ovirt.engine.core.
vdsbroker.irsbroker.IrsProxy] (org.ovirt.thread.pool-7- thread-43) []
Master domain is not in sync between DB and VDSM. Domain Node1Container
marked as master in DB and not in the storage
2018-02-20 14:42:27,026+08 WARN [org.ovirt.engine.core.dal.
dbbroker.auditloghandling. AuditLogDirector] (org.ovirt.thread.pool-7-
thread-43) [] EVENT_ID: SYSTEM_MASTER_DOMAIN_NOT_IN_ SYNC(990), Correlation
ID: null, Call Stack: null, Custom ID: null, Custom Event ID: -1, Message:
Sync Error on Master Domain between Host Node1 and oVirt Engine. Domain:
Node1Container is marked as Master in oVirt Engine database but not on the
Storage side. Please consult with Support on how to fix this issue.
2018-02-20 14:42:27,103+08 INFO [org.ovirt.engine.core.bll. storage.pool.
ReconstructMasterDomainCommand ] (org.ovirt.thread.pool-7- thread-43)
[3e5965ca] Running command: ReconstructMasterDomainCommand internal: true.
Entities affected : ID: f3e372e3-1251-4195-a4b9- 1027e40059df Type: Storage
2018-02-20 14:42:27,137+08 INFO [org.ovirt.engine.core.
vdsbroker.irsbroker. ResetIrsVDSCommand] (org.ovirt.thread.pool-7-
thread-43) [3e5965ca] START, ResetIrsVDSCommand(
ResetIrsVDSCommandParameters:{ runAsync='true',
storagePoolId='5a865884-0366- 0330-02b8-0000000002d4',
ignoreFailoverLimit='false', vdsId='7dee35bb-8c97-4f6a-
b6cd-abc4258540e4',
ignoreStopFailed='true'}), log id: 3e0a239d
2018-02-20 14:42:27,140+08 INFO [org.ovirt.engine.core.
vdsbroker.vdsbroker. SpmStopVDSCommand] (org.ovirt.thread.pool-7-
thread-43) [3e5965ca] START, SpmStopVDSCommand(HostName = Node1,
SpmStopVDSCommandParameters:{ runAsync='true', hostId='7dee35bb-8c97-4f6a-
b6cd-abc4258540e4', storagePoolId='5a865884-0366-
0330-02b8-0000000002d4'}), log id: 7c67bf06
2018-02-20 14:42:28,144+08 INFO [org.ovirt.engine.core.
vdsbroker.vdsbroker. SpmStopVDSCommand] (org.ovirt.thread.pool-7-
thread-43) [3e5965ca] SpmStopVDSCommand::Stopping SPM on vds 'Node1', pool
id '5a865884-0366-0330-02b8- 0000000002d4'
On Tuesday, February 20, 2018 2:33 PM, michael pagdanganan <
mhke_aj5566(a)yahoo.com> wrote:
Thanks for quick response,
see attachment.
On Tuesday, February 20, 2018 2:10 PM, Eyal Shenitzky <eshenitz(a)redhat.com>
wrote:
Hi,
Can you please attach full Engine and VDSM logs?
Thanks,
On Tue, Feb 20, 2018 at 2:59 AM, michael pagdanganan <
mhke_aj5566(a)yahoo.com> wrote:
My storage pool has 2 Master Domain(Stored2 on Node2,Node1Container on
Node1). and my old master domain (DATANd01 on Node1 ) hung on preparing
for maintenance. When I tried to activate my old master domain (DATANd01
on Node1 ) all storage domain goes down and up master keep on rotating.
Ovirt Version (oVirt Engine Version: 4.1.9.1.el7.centos)
Event Error:
Sync Error on Master Domain between Host Node2 and oVirt Engine. Domain
Stored2 is marked as master in ovirt engine Database but not on storage
side Please consult with support
VDSM Node2 command ConnectStoragePoolVDS failed: Wrong Master domain or
it's version: u'SD=f3e372e3-1251-4195-a4b9- 1027e40059df,
pool=5a865884-0366-0330-02b8- 0000
VDSM Node2 command HSMGetAllTastsStatusesVDS failed: Not SPM: ()
Failed to deactivate Storage Domain DATANd01 (Data Center UnsecuredEnv)
Here's logs from engine:
------------------------------ ------------------------------
------------------------------ -------------
[root@dev2engine ~]# tail /var/log/messages
Feb 20 07:01:01 dev2engine systemd: Starting Session 20 of user root.
Feb 20 07:01:01 dev2engine systemd: Removed slice User Slice of root.
Feb 20 07:01:01 dev2engine systemd: Stopping User Slice of root.
Feb 20 07:58:52 dev2engine systemd: Created slice User Slice of root.
Feb 20 07:58:52 dev2engine systemd: Starting User Slice of root.
Feb 20 07:58:52 dev2engine systemd-logind: New session 21 of user root.
Feb 20 07:58:52 dev2engine systemd: Started Session 21 of user root.
Feb 20 07:58:52 dev2engine systemd: Starting Session 21 of user root.
Feb 20 08:01:01 dev2engine systemd: Started Session 22 of user root.
Feb 20 08:01:01 dev2engine systemd: Starting Session 22 of user root.
------------------------------ ------------------------------
------------------------------ -------------
[root@dev2engine ~]# tail /var/log/ovirt-engine/engine. log
2018-02-20 08:01:16,062+08 INFO [org.ovirt.engine.core.bll.
eventqueue.EventQueueMonitor] (org.ovirt.thread.pool-7- thread-32)
[102e9d3c] Finished reconstruct for pool '5a865884-0366-0330-02b8-
0000000002d4'. Clearing event queue
2018-02-20 08:01:27,825+08 WARN [org.ovirt.engine.core.
vdsbroker.irsbroker.IrsProxy] (org.ovirt.thread.pool-7- thread-23) []
Master domain is not in sync between DB and VDSM. Domain Stored2 marked as
master in DB and not in the storage
2018-02-20 08:01:27,862+08 WARN [org.ovirt.engine.core.bll. storage.pool.
ReconstructMasterDomainCommand ] (org.ovirt.thread.pool-7- thread-23)
[213f42b9] Validation of action 'ReconstructMasterDomain' failed for user
SYSTEM. Reasons: VAR__ACTION__RECONSTRUCT_ MASTER,VAR__TYPE__STORAGE__
DOMAIN,ACTION_TYPE_FAILED_ STORAGE_DOMAIN_STATUS_ ILLEGAL2,$status
PreparingForMaintenance
2018-02-20 08:01:27,882+08 INFO [org.ovirt.engine.core.bll.
eventqueue.EventQueueMonitor] (org.ovirt.thread.pool-7- thread-20)
[929330e] Finished reconstruct for pool '5a865884-0366-0330-02b8-
0000000002d4'. Clearing event queue
2018-02-20 08:01:40,106+08 WARN [org.ovirt.engine.core.
vdsbroker.irsbroker.IrsProxy] (org.ovirt.thread.pool-7- thread-17) []
Master domain is not in sync between DB and VDSM. Domain Stored2 marked as
master in DB and not in the storage
2018-02-20 08:01:40,197+08 WARN [org.ovirt.engine.core.bll. storage.pool.
ReconstructMasterDomainCommand ] (org.ovirt.thread.pool-7- thread-17)
[7af552c1] Validation of action 'ReconstructMasterDomain' failed for user
SYSTEM. Reasons: VAR__ACTION__RECONSTRUCT_ MASTER,VAR__TYPE__STORAGE__
DOMAIN,ACTION_TYPE_FAILED_ STORAGE_DOMAIN_STATUS_ ILLEGAL2,$status
PreparingForMaintenance
2018-02-20 08:01:40,246+08 INFO [org.ovirt.engine.core.bll.
eventqueue.EventQueueMonitor] (org.ovirt.thread.pool-7- thread-22)
[73673040] Finished reconstruct for pool '5a865884-0366-0330-02b8-
0000000002d4'. Clearing event queue
2018-02-20 08:01:51,809+08 WARN [org.ovirt.engine.core.
vdsbroker.irsbroker.IrsProxy] (org.ovirt.thread.pool-7- thread-26) []
Master domain is not in sync between DB and VDSM. Domain Stored2 marked as
master in DB and not in the storage
2018-02-20 08:01:51,846+08 WARN [org.ovirt.engine.core.bll. storage.pool.
ReconstructMasterDomainCommand ] (org.ovirt.thread.pool-7- thread-26)
[20307cbe] Validation of action 'ReconstructMasterDomain' failed for user
SYSTEM. Reasons: VAR__ACTION__RECONSTRUCT_ MASTER,VAR__TYPE__STORAGE__
DOMAIN,ACTION_TYPE_FAILED_ STORAGE_DOMAIN_STATUS_ ILLEGAL2,$status
PreparingForMaintenance
2018-02-20 08:01:51,866+08 INFO [org.ovirt.engine.core.bll.
eventqueue.EventQueueMonitor] (org.ovirt.thread.pool-7- thread-49)
[2c11a866] Finished reconstruct for pool '5a865884-0366-0330-02b8-
0000000002d4'. Clearing event queue
______________________________ _________________
Users mailing list
Users(a)ovirt.org
http://lists.ovirt.org/ mailman/listinfo/users
<
http://lists.ovirt.org/mailman/listinfo/users>
--
Regards,
Eyal Shenitzky
--
Regards,
Eyal Shenitzky