Re: [ovirt-users] 2 Master on Storage Pool [Event Error]

Tuesday, 20 February 2018

Also, can you please describe the current setup of the environment?

I'm not sure I understand, do you have 2 data-centers?

Please attach some screenshots of the current situation.

On Tue, Feb 20, 2018 at 8:43 AM, michael pagdanganan <mhke_aj5566(a)yahoo.com&gt;
wrote:

...
 Sorry can't attached log file it's too big file

 VDSM.log for node 1

 2766', 'lastCheck': '4.9', 'valid': True}} from=internal,
 task_id=645d456e-f59f-4b1c-9e97-fc82d19a36b1 (api:52)
 2018-02-20 14:38:47,222+0800 INFO  (jsonrpc/3) [vdsm.api] START
 repoStats(options=None) from=::ffff:10.10.43.1,60554, flow_id=3b2e802e,
 task_id=28c795d1-1639-4e68-a9fe-a00006be268f (api:46)
 2018-02-20 14:38:47,222+0800 INFO  (jsonrpc/3) [vdsm.api] FINISH repoStats
 return={'f3e372e3-1251-4195-a4b9-1027e40059df': {'code': 0,
'actual':
 True, 'version': 4, 'acquired': True, 'delay':
'0.000119109', 'lastCheck':
 '6.0', 'valid': True}, 'e83d0d46-6ea6-4aa3-80bf-6e95c66b0454':
{'code':
 0, 'actual': True, 'version': 0, 'acquired': True,
'delay': '0.000195263',
 'lastCheck': '6.0', 'valid': True},
'65ca2e2d-b472-4bee-85b4-09a161464b20':
 {'code': 0, 'actual': True, 'version': 4, 'acquired':
True, 'delay':
 '0.00022988', 'lastCheck': '6.0', 'valid': True},
'4bf2ba2f-f57a-4d9f-b42a-fb78f440a358':
 {'code': 0, 'actual': True, 'version': 0, 'acquired':
True, 'delay':
 '0.000270206', 'lastCheck': '5.9', 'valid': True},
'42e591b7-f86c-4b67-a3d2-40cc007f7662':
 {'code': 0, 'actual': True, 'version': 4, 'acquired':
True, 'delay':
 '0.000272766', 'lastCheck': '6.0', 'valid': True}}
 from=::ffff:10.10.43.1,60554, flow_id=3b2e802e,
task_id=28c795d1-1639-4e68-a9fe-a00006be268f
 (api:52)
 2018-02-20 14:38:47,226+0800 INFO  (jsonrpc/3) [jsonrpc.JsonRpcServer] RPC
 call Host.getStats succeeded in 0.00 seconds (__init__:539)
 2018-02-20 14:38:55,252+0800 INFO  (jsonrpc/4) [jsonrpc.JsonRpcServer] RPC
 call Host.getAllVmStats succeeded in 0.00 seconds (__init__:539)
 2018-02-20 14:38:58,566+0800 INFO  (jsonrpc/2) [jsonrpc.JsonRpcServer] RPC
 call Host.getAllVmStats succeeded in 0.00 seconds (__init__:539)
 2018-02-20 14:39:01,093+0800 INFO  (periodic/1) [vdsm.api] START
 repoStats(options=None) from=internal, task_id=04f55ded-5841-44ae-a376-4f6e723e4b10
 (api:46)
 2018-02-20 14:39:01,093+0800 INFO  (periodic/1) [vdsm.api] FINISH
 repoStats return={'f3e372e3-1251-4195-a4b9-1027e40059df': {'code': 0,
 'actual': True, 'version': 4, 'acquired': True, 'delay':
'9.3888e-05',
 'lastCheck': '9.9', 'valid': True},
'e83d0d46-6ea6-4aa3-80bf-6e95c66b0454':
 {'code': 0, 'actual': True, 'version': 0, 'acquired':
True, 'delay':
 '0.000197535', 'lastCheck': '9.8', 'valid': True},
'65ca2e2d-b472-4bee-85b4-09a161464b20':
 {'code': 0, 'actual': True, 'version': 4, 'acquired':
True, 'delay':
 '0.000191456', 'lastCheck': '9.8', 'valid': True},
'4bf2ba2f-f57a-4d9f-b42a-fb78f440a358':
 {'code': 0, 'actual': True, 'version': 0, 'acquired':
True, 'delay':
 '0.00026365', 'lastCheck': '9.8', 'valid': True},
'42e591b7-f86c-4b67-a3d2-40cc007f7662':
 {'code': 0, 'actual': True, 'version': 4, 'acquired':
True, 'delay':
 '0.000243658', 'lastCheck': '0.0', 'valid': True}}
from=internal,
 task_id=04f55ded-5841-44ae-a376-4f6e723e4b10 (api:52)
 2018-02-20 14:39:02,295+0800 INFO  (jsonrpc/6) [vdsm.api] START
 repoStats(options=None) from=::ffff:10.10.43.1,60554, flow_id=13149d6e,
 task_id=5f19d76a-d343-4dc4-ad09-024ec27f7443 (api:46)
 2018-02-20 14:39:02,295+0800 INFO  (jsonrpc/6) [vdsm.api] FINISH repoStats
 return={'f3e372e3-1251-4195-a4b9-1027e40059df': {'code': 0,
'actual':
 True, 'version': 4, 'acquired': True, 'delay':
'9.4759e-05', 'lastCheck':
 '1.1', 'valid': True}, 'e83d0d46-6ea6-4aa3-80bf-6e95c66b0454':
{'code':
 0, 'actual': True, 'version': 0, 'acquired': True,
'delay': '0.000183158',
 'lastCheck': '1.0', 'valid': True},
'65ca2e2d-b472-4bee-85b4-09a161464b20':
 {'code': 0, 'actual': True, 'version': 4, 'acquired':
True, 'delay':
 '0.000222609', 'lastCheck': '1.0', 'valid': True},
'4bf2ba2f-f57a-4d9f-b42a-fb78f440a358':
 {'code': 0, 'actual': True, 'version': 0, 'acquired':
True, 'delay':
 '0.000211253', 'lastCheck': '1.0', 'valid': True},
'42e591b7-f86c-4b67-a3d2-40cc007f7662':
 {'code': 0, 'actual': True, 'version': 4, 'acquired':
True, 'delay':
 '0.000243658', 'lastCheck': '1.1', 'valid': True}}
 from=::ffff:10.10.43.1,60554, flow_id=13149d6e,
task_id=5f19d76a-d343-4dc4-ad09-024ec27f7443
 (api:52)
 2018-02-20 14:39:02,300+0800 INFO  (jsonrpc/6) [jsonrpc.JsonRpcServer] RPC
 call Host.getStats succeeded in 0.01 seconds (__init__:539)
 2018-02-20 14:39:10,270+0800 INFO  (jsonrpc/1) [jsonrpc.JsonRpcServer] RPC
 call Host.getAllVmStats succeeded in 0.00 seconds (__init__:539)
 2018-02-20 14:39:13,631+0800 INFO  (jsonrpc/0) [vdsm.api] START
 connectStorageServer(domType=1, spUUID=u'00000000-0000-0000-0000-000000000000',
 conList=[{u'id': u'5e4c94f1-f3b9-4fbd-a6c7-e732d0fe3123',
u'connection':
 u'dev2node1.lares.com.ph:/run/media/root/Slave1Data/dataNode1', u'iqn':
 u'', u'user': u'', u'tpgt': u'1',
u'protocol_version': u'auto',
 u'password': '********', u'port': u''}], options=None)
 from=::ffff:10.10.43.1,60554, flow_id=15b57417,
task_id=bfce1e70-ef4a-4e13-aaaa-7a66aaf44429
 (api:46)
 2018-02-20 14:39:13,633+0800 INFO  (jsonrpc/0) [vdsm.api] FINISH
 connectStorageServer return={'statuslist': [{'status': 0, 'id':
 u'5e4c94f1-f3b9-4fbd-a6c7-e732d0fe3123'}]} from=::ffff:10.10.43.1,60554,
 flow_id=15b57417, task_id=bfce1e70-ef4a-4e13-aaaa-7a66aaf44429 (api:52)
 2018-02-20 14:39:13,634+0800 INFO  (jsonrpc/0) [jsonrpc.JsonRpcServer] RPC
 call StoragePool.connectStorageServer succeeded in 0.00 seconds
 (__init__:539)
 2018-02-20 14:39:13,806+0800 INFO  (jsonrpc/7) [vdsm.api] START
 connectStoragePool(spUUID=u'5a865884-0366-0330-02b8-0000000002d4',
 hostID=1, msdUUID=u'f3e372e3-1251-4195-a4b9-1027e40059df',
 masterVersion=65, domainsMap={u'e83d0d46-6ea6-4aa3-80bf-6e95c66b0454':
 u'active', u'4bf2ba2f-f57a-4d9f-b42a-fb78f440a358': u'active',
 u'225e1975-8121-4370-b317-86e964ae326f': u'attached',
 u'f3e372e3-1251-4195-a4b9-1027e40059df': u'active',
 u'65ca2e2d-b472-4bee-85b4-09a161464b20': u'active',
 u'42e591b7-f86c-4b67-a3d2-40cc007f7662': u'active'}, options=None)
 from=::ffff:10.10.43.1,60554, flow_id=15b57417,
task_id=927a5d9a-4304-4776-b5c9-22ba5d0cb853
 (api:46)
 2018-02-20 14:39:13,807+0800 INFO  (jsonrpc/7) [storage.StoragePoolMemoryBackend]
 new storage pool master version 65 and domains map
 {u'e83d0d46-6ea6-4aa3-80bf-6e95c66b0454': u'Active',
 u'4bf2ba2f-f57a-4d9f-b42a-fb78f440a358': u'Active',
 u'225e1975-8121-4370-b317-86e964ae326f': u'Attached',
 u'f3e372e3-1251-4195-a4b9-1027e40059df': u'Active',
 u'65ca2e2d-b472-4bee-85b4-09a161464b20': u'Active',
 u'42e591b7-f86c-4b67-a3d2-40cc007f7662': u'Active'} (spbackends:450)

 VDSM.log 2
 2018-02-20 14:41:14,598+0800 INFO  (periodic/3) [vdsm.api] FINISH
 repoStats return={'f3e372e3-1251-4195-a4b9-1027e40059df': {'code': 0,
 'actual': True, 'version': 4, 'acquired': True, 'delay':
'0.000252423',
 'lastCheck': '6.0', 'valid': True},
'e83d0d46-6ea6-4aa3-80bf-6e95c66b0454':
 {'code': 0, 'actual': True, 'version': 0, 'acquired':
True, 'delay':
 '7.5567e-05', 'lastCheck': '6.0', 'valid': True},
'65ca2e2d-b472-4bee-85b4-09a161464b20':
 {'code': 0, 'actual': True, 'version': 4, 'acquired':
True, 'delay':
 '8.2433e-05', 'lastCheck': '6.0', 'valid': True},
'4bf2ba2f-f57a-4d9f-b42a-fb78f440a358':
 {'code': 0, 'actual': True, 'version': 0, 'acquired':
True, 'delay':
 '0.000237247', 'lastCheck': '5.9', 'valid': True},
'42e591b7-f86c-4b67-a3d2-40cc007f7662':
 {'code': 0, 'actual': True, 'version': 4, 'acquired':
True, 'delay':
 '8.4376e-05', 'lastCheck': '6.0', 'valid': True}}
from=internal,
 task_id=6213712b-9903-4db8-9836-3baf85cd63e4 (api:52)
 2018-02-20 14:41:18,074+0800 INFO  (jsonrpc/6) [jsonrpc.JsonRpcServer] RPC
 call Host.getAllVmStats succeeded in 0.00 seconds (__init__:539)
 2018-02-20 14:41:24,333+0800 INFO  (jsonrpc/4) [vdsm.api] START
 repoStats(options=None) from=::ffff:10.10.43.1,56540, flow_id=73f86113,
 task_id=e8231ecb-3543-4d8f-af54-4cf2b06ee98a (api:46)
 2018-02-20 14:41:24,334+0800 INFO  (jsonrpc/4) [vdsm.api] FINISH repoStats
 return={'f3e372e3-1251-4195-a4b9-1027e40059df': {'code': 0,
'actual':
 True, 'version': 4, 'acquired': True, 'delay':
'0.000172953', 'lastCheck':
 '5.7', 'valid': True}, 'e83d0d46-6ea6-4aa3-80bf-6e95c66b0454':
{'code':
 0, 'actual': True, 'version': 0, 'acquired': True,
'delay': '7.701e-05',
 'lastCheck': '5.7', 'valid': True},
'65ca2e2d-b472-4bee-85b4-09a161464b20':
 {'code': 0, 'actual': True, 'version': 4, 'acquired':
True, 'delay':
 '0.000117677', 'lastCheck': '5.7', 'valid': True},
'4bf2ba2f-f57a-4d9f-b42a-fb78f440a358':
 {'code': 0, 'actual': True, 'version': 0, 'acquired':
True, 'delay':
 '0.000185146', 'lastCheck': '5.7', 'valid': True},
'42e591b7-f86c-4b67-a3d2-40cc007f7662':
 {'code': 0, 'actual': True, 'version': 4, 'acquired':
True, 'delay':
 '9.1467e-05', 'lastCheck': '5.8', 'valid': True}}
 from=::ffff:10.10.43.1,56540, flow_id=73f86113,
task_id=e8231ecb-3543-4d8f-af54-4cf2b06ee98a
 (api:52)
 2018-02-20 14:41:24,338+0800 INFO  (jsonrpc/4) [jsonrpc.JsonRpcServer] RPC
 call Host.getStats succeeded in 0.00 seconds (__init__:539)
 2018-02-20 14:41:27,087+0800 INFO  (jsonrpc/0) [jsonrpc.JsonRpcServer] RPC
 call Host.getAllVmStats succeeded in 0.00 seconds (__init__:539)
 2018-02-20 14:41:29,607+0800 INFO  (periodic/1) [vdsm.api] START
 repoStats(options=None) from=internal, task_id=f14b7aab-b64a-4903-9368-d665e39b49d1
 (api:46)
 2018-02-20 14:41:29,608+0800 INFO  (periodic/1) [vdsm.api] FINISH
 repoStats return={'f3e372e3-1251-4195-a4b9-1027e40059df': {'code': 0,
 'actual': True, 'version': 4, 'acquired': True, 'delay':
'0.000246237',
 'lastCheck': '1.0', 'valid': True},
'e83d0d46-6ea6-4aa3-80bf-6e95c66b0454':
 {'code': 0, 'actual': True, 'version': 0, 'acquired':
True, 'delay':
 '8.8773e-05', 'lastCheck': '1.0', 'valid': True},
'65ca2e2d-b472-4bee-85b4-09a161464b20':
 {'code': 0, 'actual': True, 'version': 4, 'acquired':
True, 'delay':
 '9.7145e-05', 'lastCheck': '1.0', 'valid': True},
'4bf2ba2f-f57a-4d9f-b42a-fb78f440a358':
 {'code': 0, 'actual': True, 'version': 0, 'acquired':
True, 'delay':
 '0.000218729', 'lastCheck': '0.9', 'valid': True},
'42e591b7-f86c-4b67-a3d2-40cc007f7662':
 {'code': 0, 'actual': True, 'version': 4, 'acquired':
True, 'delay':
 '9.1035e-05', 'lastCheck': '1.0', 'valid': True}}
from=internal,
 task_id=f14b7aab-b64a-4903-9368-d665e39b49d1 (api:52)
 2018-02-20 14:41:33,079+0800 INFO  (jsonrpc/7) [jsonrpc.JsonRpcServer] RPC
 call Host.getAllVmStats succeeded in 0.00 seconds (__init__:539)
 2018-02-20 14:41:40,456+0800 INFO  (jsonrpc/2) [vdsm.api] START
 repoStats(options=None) from=::ffff:10.10.43.1,56540, flow_id=3be8150c,
 task_id=1b0c86a8-6fd8-4882-a742-fbd56ccb4037 (api:46)
 2018-02-20 14:41:40,457+0800 INFO  (jsonrpc/2) [vdsm.api] FINISH repoStats
 return={'f3e372e3-1251-4195-a4b9-1027e40059df': {'code': 0,
'actual':
 True, 'version': 4, 'acquired': True, 'delay':
'0.000174027', 'lastCheck':
 '1.8', 'valid': True}, 'e83d0d46-6ea6-4aa3-80bf-6e95c66b0454':
{'code':
 0, 'actual': True, 'version': 0, 'acquired': True,
'delay': '0.00011454',
 'lastCheck': '1.8', 'valid': True},
'65ca2e2d-b472-4bee-85b4-09a161464b20':
 {'code': 0, 'actual': True, 'version': 4, 'acquired':
True, 'delay':
 '0.000145955', 'lastCheck': '1.8', 'valid': True},
'4bf2ba2f-f57a-4d9f-b42a-fb78f440a358':
 {'code': 0, 'actual': True, 'version': 0, 'acquired':
True, 'delay':
 '0.000209871', 'lastCheck': '1.8', 'valid': True},
'42e591b7-f86c-4b67-a3d2-40cc007f7662':
 {'code': 0, 'actual': True, 'version': 4, 'acquired':
True, 'delay':
 '9.5744e-05', 'lastCheck': '1.9', 'valid': True}}
 from=::ffff:10.10.43.1,56540, flow_id=3be8150c,
task_id=1b0c86a8-6fd8-4882-a742-fbd56ccb4037
 (api:52)
 2018-02-20 14:41:40,461+0800 INFO  (jsonrpc/2) [jsonrpc.JsonRpcServer] RPC
 call Host.getStats succeeded in 0.01 seconds (__init__:539)
 2018-02-20 14:41:42,106+0800 INFO  (jsonrpc/3) [jsonrpc.JsonRpcServer] RPC
 call Host.getAllVmStats succeeded in 0.00 seconds (__init__:539)
 2018-02-20 14:41:44,622+0800 INFO  (periodic/3) [vdsm.api] START
 repoStats(options=None) from=internal, task_id=20e0b29b-d3cd-4e44-b92c-213f9c984ab2
 (api:46)
 2018-02-20 14:41:44,622+0800 INFO  (periodic/3) [vdsm.api] FINISH
 repoStats return={'f3e372e3-1251-4195-a4b9-1027e40059df': {'code': 0,
 'actual': True, 'version': 4, 'acquired': True, 'delay':
'0.000174027',
 'lastCheck': '6.0', 'valid': True},
'e83d0d46-6ea6-4aa3-80bf-6e95c66b0454':
 {'code': 0, 'actual': True, 'version': 0, 'acquired':
True, 'delay':
 '0.00011454', 'lastCheck': '6.0', 'valid': True},
'65ca2e2d-b472-4bee-85b4-09a161464b20':
 {'code': 0, 'actual': True, 'version': 4, 'acquired':
True, 'delay':
 '0.000145955', 'lastCheck': '6.0', 'valid': True},
'4bf2ba2f-f57a-4d9f-b42a-fb78f440a358':
 {'code': 0, 'actual': True, 'version': 0, 'acquired':
True, 'delay':
 '0.000209871', 'lastCheck': '5.9', 'valid': True},
'42e591b7-f86c-4b67-a3d2-40cc007f7662':
 {'code': 0, 'actual': True, 'version': 4, 'acquired':
True, 'delay':
 '9.5744e-05', 'lastCheck': '6.0', 'valid': True}}
from=internal,
 task_id=20e0b29b-d3cd-4e44-b92c-213f9c984ab2 (api:52)
 2018-02-20 14:41:49,083+0800 INFO  (jsonrpc/1) [jsonrpc.JsonRpcServer] RPC
 call Host.getAllVmStats succeeded in 0.00 seconds (__init__:539)

 Engine log

 skId '404ccecc-aa7f-45ea-89e4-726956269bc9' task status 'finished'
 2018-02-20 14:42:21,966+08 INFO  [org.ovirt.engine.core.
 vdsbroker.vdsbroker.SpmStartVDSCommand] (org.ovirt.thread.pool-7-thread-38)
 [29528f9] spmStart polling ended, spm status: SPM
 2018-02-20 14:42:21,967+08 INFO  [org.ovirt.engine.core.
 vdsbroker.vdsbroker.HSMClearTaskVDSCommand] (org.ovirt.thread.pool-7-thread-38)
 [29528f9] START, HSMClearTaskVDSCommand(HostName = Node1,
 HSMTaskGuidBaseVDSCommandParameters:{runAsync='true',
 hostId='7dee35bb-8c97-4f6a-b6cd-abc4258540e4',
taskId='404ccecc-aa7f-45ea-89e4-726956269bc9'}),
 log id: 71688f70
 2018-02-20 14:42:22,922+08 INFO  [org.ovirt.engine.core.
 vdsbroker.vdsbroker.HSMClearTaskVDSCommand] (org.ovirt.thread.pool-7-thread-38)
 [29528f9] FINISH, HSMClearTaskVDSCommand, log id: 71688f70
 2018-02-20 14:42:22,923+08 INFO  [org.ovirt.engine.core.
 vdsbroker.vdsbroker.SpmStartVDSCommand] (org.ovirt.thread.pool-7-thread-38)
 [29528f9] FINISH, SpmStartVDSCommand, return: org.ovirt.engine.core.common.
 businessentities.SpmStatusResult@78332453, log id: 3ea35d5
 2018-02-20 14:42:22,935+08 INFO  [org.ovirt.engine.core.vdsbroker.irsbroker.IrsProxy]
 (org.ovirt.thread.pool-7-thread-38) [29528f9] Initialize Irs proxy from
 vds: dev2node1.lares.com.ph
 2018-02-20 14:42:22,951+08 INFO  [org.ovirt.engine.core.dal.
 dbbroker.auditloghandling.AuditLogDirector] (org.ovirt.thread.pool-7-thread-38)
 [29528f9] EVENT_ID: IRS_HOSTED_ON_VDS(204), Correlation ID: null, Call
 Stack: null, Custom ID: null, Custom Event ID: -1, Message: Storage Pool
 Manager runs on Host Node1 (Address: dev2node1.lares.com.ph).
 2018-02-20 14:42:22,952+08 INFO  [org.ovirt.engine.core.
 vdsbroker.irsbroker.SPMGetAllTasksInfoVDSCommand]
 (org.ovirt.thread.pool-7-thread-38) [29528f9] -- executeIrsBrokerCommand:
 Attempting on storage pool '5a865884-0366-0330-02b8-0000000002d4'
 2018-02-20 14:42:22,952+08 INFO  [org.ovirt.engine.core.
 vdsbroker.vdsbroker.HSMGetAllTasksInfoVDSCommand]
 (org.ovirt.thread.pool-7-thread-38) [29528f9] START,
 HSMGetAllTasksInfoVDSCommand(HostName = Node1,
 VdsIdVDSCommandParametersBase:{runAsync='true',
 hostId='7dee35bb-8c97-4f6a-b6cd-abc4258540e4'}), log id: 1bdbea9d
 2018-02-20 14:42:22,955+08 INFO  [org.ovirt.engine.core.
 vdsbroker.irsbroker.SPMGetAllTasksInfoVDSCommand]
 (org.ovirt.thread.pool-7-thread-7) [29528f9] START,
 SPMGetAllTasksInfoVDSCommand( IrsBaseVDSCommandParameters:{runAsync='true',
 storagePoolId='5a865884-0366-0330-02b8-0000000002d4',
 ignoreFailoverLimit='false'}), log id: 5c2422d6
 2018-02-20 14:42:23,956+08 INFO  [org.ovirt.engine.core.
 vdsbroker.vdsbroker.HSMGetAllTasksInfoVDSCommand]
 (org.ovirt.thread.pool-7-thread-38) [29528f9] FINISH,
 HSMGetAllTasksInfoVDSCommand, return: [], log id: 1bdbea9d
 2018-02-20 14:42:23,956+08 INFO  [org.ovirt.engine.core.
 vdsbroker.irsbroker.SPMGetAllTasksInfoVDSCommand]
 (org.ovirt.thread.pool-7-thread-38) [29528f9] FINISH,
 SPMGetAllTasksInfoVDSCommand, return: [], log id: 58cbe7b7
 2018-02-20 14:42:23,956+08 INFO  [org.ovirt.engine.core.bll.tasks.AsyncTaskManager]
 (org.ovirt.thread.pool-7-thread-38) [29528f9] Discovered no tasks on
 Storage Pool 'UnsecuredEnv'
 2018-02-20 14:42:24,936+08 INFO  [org.ovirt.vdsm.jsonrpc.client.reactors.ReactorClient]
 (SSL Stomp Reactor) [] Connecting to dev2node1.lares.com.ph/10.10.43.2
 2018-02-20 14:42:27,012+08 WARN  [org.ovirt.engine.core.vdsbroker.irsbroker.IrsProxy]
 (org.ovirt.thread.pool-7-thread-43) [] Master domain is not in sync
 between DB and VDSM. Domain Node1Container marked as master in DB and not
 in the storage
 2018-02-20 14:42:27,026+08 WARN  [org.ovirt.engine.core.dal.
 dbbroker.auditloghandling.AuditLogDirector] (org.ovirt.thread.pool-7-thread-43)
 [] EVENT_ID: SYSTEM_MASTER_DOMAIN_NOT_IN_SYNC(990), Correlation ID: null,
 Call Stack: null, Custom ID: null, Custom Event ID: -1, Message: Sync Error
 on Master Domain between Host Node1 and oVirt Engine. Domain:
 Node1Container is marked as Master in oVirt Engine database but not on the
 Storage side. Please consult with Support on how to fix this issue.
 2018-02-20 14:42:27,103+08 INFO  [org.ovirt.engine.core.bll.storage.pool.
 ReconstructMasterDomainCommand] (org.ovirt.thread.pool-7-thread-43)
 [3e5965ca] Running command: ReconstructMasterDomainCommand internal: true.
 Entities affected :  ID: f3e372e3-1251-4195-a4b9-1027e40059df Type:
 Storage
 2018-02-20 14:42:27,137+08 INFO  [org.ovirt.engine.core.
 vdsbroker.irsbroker.ResetIrsVDSCommand] (org.ovirt.thread.pool-7-thread-43)
 [3e5965ca] START, ResetIrsVDSCommand(
ResetIrsVDSCommandParameters:{runAsync='true',
 storagePoolId='5a865884-0366-0330-02b8-0000000002d4',
 ignoreFailoverLimit='false',
vdsId='7dee35bb-8c97-4f6a-b6cd-abc4258540e4',
 ignoreStopFailed='true'}), log id: 3e0a239d
 2018-02-20 14:42:27,140+08 INFO  [org.ovirt.engine.core.
 vdsbroker.vdsbroker.SpmStopVDSCommand] (org.ovirt.thread.pool-7-thread-43)
 [3e5965ca] START, SpmStopVDSCommand(HostName = Node1,
 SpmStopVDSCommandParameters:{runAsync='true',
hostId='7dee35bb-8c97-4f6a-b6cd-abc4258540e4',
 storagePoolId='5a865884-0366-0330-02b8-0000000002d4'}), log id: 7c67bf06
 2018-02-20 14:42:28,144+08 INFO  [org.ovirt.engine.core.
 vdsbroker.vdsbroker.SpmStopVDSCommand] (org.ovirt.thread.pool-7-thread-43)
 [3e5965ca] SpmStopVDSCommand::Stopping SPM on vds 'Node1', pool id
 '5a865884-0366-0330-02b8-0000000002d4'

 On Tuesday, February 20, 2018 2:33 PM, michael pagdanganan <
 mhke_aj5566(a)yahoo.com&gt; wrote:

 Thanks for quick response,

 see attachment.

 On Tuesday, February 20, 2018 2:10 PM, Eyal Shenitzky <eshenitz(a)redhat.com&gt;
 wrote:

 Hi,

 Can you please attach full Engine and VDSM logs?

 Thanks,

 On Tue, Feb 20, 2018 at 2:59 AM, michael pagdanganan <
 mhke_aj5566(a)yahoo.com&gt; wrote:

 My storage pool has 2 Master Domain(Stored2 on Node2,Node1Container on
 Node1). and my old master domain (DATANd01 on Node1 ) hung on preparing
 for maintenance. When I tried to activate my old master domain (DATANd01
 on Node1 ) all  storage domain goes down and up master keep on rotating.

 Ovirt Version (oVirt Engine Version: 4.1.9.1.el7.centos)

 Event Error:
 Sync Error on Master Domain between Host Node2 and oVirt Engine. Domain
 Stored2 is marked as master in ovirt engine Database but not on storage
 side Please consult with support

 VDSM Node2 command ConnectStoragePoolVDS failed: Wrong Master domain or
 it's version: u'SD=f3e372e3-1251-4195-a4b9- 1027e40059df,
 pool=5a865884-0366-0330-02b8- 0000

 VDSM Node2 command HSMGetAllTastsStatusesVDS failed: Not SPM: ()

 Failed to deactivate Storage Domain DATANd01 (Data Center UnsecuredEnv)

 Here's logs from engine:

 ------------------------------ ------------------------------
 ------------------------------ -------------
 [root@dev2engine ~]# tail /var/log/messages
 Feb 20 07:01:01 dev2engine systemd: Starting Session 20 of user root.
 Feb 20 07:01:01 dev2engine systemd: Removed slice User Slice of root.
 Feb 20 07:01:01 dev2engine systemd: Stopping User Slice of root.
 Feb 20 07:58:52 dev2engine systemd: Created slice User Slice of root.
 Feb 20 07:58:52 dev2engine systemd: Starting User Slice of root.
 Feb 20 07:58:52 dev2engine systemd-logind: New session 21 of user root.
 Feb 20 07:58:52 dev2engine systemd: Started Session 21 of user root.
 Feb 20 07:58:52 dev2engine systemd: Starting Session 21 of user root.
 Feb 20 08:01:01 dev2engine systemd: Started Session 22 of user root.
 Feb 20 08:01:01 dev2engine systemd: Starting Session 22 of user root.

 ------------------------------ ------------------------------
 ------------------------------ -------------
 [root@dev2engine ~]# tail /var/log/ovirt-engine/engine. log
 2018-02-20 08:01:16,062+08 INFO  [org.ovirt.engine.core.bll.
 eventqueue.EventQueueMonitor] (org.ovirt.thread.pool-7- thread-32)
 [102e9d3c] Finished reconstruct for pool '5a865884-0366-0330-02b8-
 0000000002d4'. Clearing event queue
 2018-02-20 08:01:27,825+08 WARN  [org.ovirt.engine.core.
 vdsbroker.irsbroker.IrsProxy] (org.ovirt.thread.pool-7- thread-23) []
 Master domain is not in sync between DB and VDSM. Domain Stored2 marked as
 master in DB and not in the storage
 2018-02-20 08:01:27,862+08 WARN  [org.ovirt.engine.core.bll. storage.pool.
 ReconstructMasterDomainCommand ] (org.ovirt.thread.pool-7- thread-23)
 [213f42b9] Validation of action 'ReconstructMasterDomain' failed for user
 SYSTEM. Reasons: VAR__ACTION__RECONSTRUCT_ MASTER,VAR__TYPE__STORAGE__
 DOMAIN,ACTION_TYPE_FAILED_ STORAGE_DOMAIN_STATUS_ ILLEGAL2,$status
 PreparingForMaintenance
 2018-02-20 08:01:27,882+08 INFO  [org.ovirt.engine.core.bll.
 eventqueue.EventQueueMonitor] (org.ovirt.thread.pool-7- thread-20)
 [929330e] Finished reconstruct for pool '5a865884-0366-0330-02b8-
 0000000002d4'. Clearing event queue
 2018-02-20 08:01:40,106+08 WARN  [org.ovirt.engine.core.
 vdsbroker.irsbroker.IrsProxy] (org.ovirt.thread.pool-7- thread-17) []
 Master domain is not in sync between DB and VDSM. Domain Stored2 marked as
 master in DB and not in the storage
 2018-02-20 08:01:40,197+08 WARN  [org.ovirt.engine.core.bll. storage.pool.
 ReconstructMasterDomainCommand ] (org.ovirt.thread.pool-7- thread-17)
 [7af552c1] Validation of action 'ReconstructMasterDomain' failed for user
 SYSTEM. Reasons: VAR__ACTION__RECONSTRUCT_ MASTER,VAR__TYPE__STORAGE__
 DOMAIN,ACTION_TYPE_FAILED_ STORAGE_DOMAIN_STATUS_ ILLEGAL2,$status
 PreparingForMaintenance
 2018-02-20 08:01:40,246+08 INFO  [org.ovirt.engine.core.bll.
 eventqueue.EventQueueMonitor] (org.ovirt.thread.pool-7- thread-22)
 [73673040] Finished reconstruct for pool '5a865884-0366-0330-02b8-
 0000000002d4'. Clearing event queue
 2018-02-20 08:01:51,809+08 WARN  [org.ovirt.engine.core.
 vdsbroker.irsbroker.IrsProxy] (org.ovirt.thread.pool-7- thread-26) []
 Master domain is not in sync between DB and VDSM. Domain Stored2 marked as
 master in DB and not in the storage
 2018-02-20 08:01:51,846+08 WARN  [org.ovirt.engine.core.bll. storage.pool.
 ReconstructMasterDomainCommand ] (org.ovirt.thread.pool-7- thread-26)
 [20307cbe] Validation of action 'ReconstructMasterDomain' failed for user
 SYSTEM. Reasons: VAR__ACTION__RECONSTRUCT_ MASTER,VAR__TYPE__STORAGE__
 DOMAIN,ACTION_TYPE_FAILED_ STORAGE_DOMAIN_STATUS_ ILLEGAL2,$status
 PreparingForMaintenance
 2018-02-20 08:01:51,866+08 INFO  [org.ovirt.engine.core.bll.
 eventqueue.EventQueueMonitor] (org.ovirt.thread.pool-7- thread-49)
 [2c11a866] Finished reconstruct for pool '5a865884-0366-0330-02b8-
 0000000002d4'. Clearing event queue

 ______________________________ _________________
 Users mailing list
 Users(a)ovirt.org
 http://lists.ovirt.org/ mailman/listinfo/users
 <http://lists.ovirt.org/mailman/listinfo/users>

 --
 Regards,
 Eyal Shenitzky

-- 
Regards,
Eyal Shenitzky

2024

2023

2022

2021

2020

2019

2018

2017

2016

2015

2014

2013

2012

2011

Re: [ovirt-users] 2 Master on Storage Pool [Event Error]