[ovirt-users] 2 Master on Storage Pool [Event Error]
Eyal Shenitzky
eshenitz at redhat.com
Tue Feb 20 06:56:02 UTC 2018
Also, can you please describe the current setup of the environment?
I'm not sure I understand, do you have 2 data-centers?
Please attach some screenshots of the current situation.
On Tue, Feb 20, 2018 at 8:43 AM, michael pagdanganan <mhke_aj5566 at yahoo.com>
wrote:
> Sorry can't attached log file it's too big file
>
>
> VDSM.log for node 1
>
> 2766', 'lastCheck': '4.9', 'valid': True}} from=internal,
> task_id=645d456e-f59f-4b1c-9e97-fc82d19a36b1 (api:52)
> 2018-02-20 14:38:47,222+0800 INFO (jsonrpc/3) [vdsm.api] START
> repoStats(options=None) from=::ffff:10.10.43.1,60554, flow_id=3b2e802e,
> task_id=28c795d1-1639-4e68-a9fe-a00006be268f (api:46)
> 2018-02-20 14:38:47,222+0800 INFO (jsonrpc/3) [vdsm.api] FINISH repoStats
> return={'f3e372e3-1251-4195-a4b9-1027e40059df': {'code': 0, 'actual':
> True, 'version': 4, 'acquired': True, 'delay': '0.000119109', 'lastCheck':
> '6.0', 'valid': True}, 'e83d0d46-6ea6-4aa3-80bf-6e95c66b0454': {'code':
> 0, 'actual': True, 'version': 0, 'acquired': True, 'delay': '0.000195263',
> 'lastCheck': '6.0', 'valid': True}, '65ca2e2d-b472-4bee-85b4-09a161464b20':
> {'code': 0, 'actual': True, 'version': 4, 'acquired': True, 'delay':
> '0.00022988', 'lastCheck': '6.0', 'valid': True}, '4bf2ba2f-f57a-4d9f-b42a-fb78f440a358':
> {'code': 0, 'actual': True, 'version': 0, 'acquired': True, 'delay':
> '0.000270206', 'lastCheck': '5.9', 'valid': True}, '42e591b7-f86c-4b67-a3d2-40cc007f7662':
> {'code': 0, 'actual': True, 'version': 4, 'acquired': True, 'delay':
> '0.000272766', 'lastCheck': '6.0', 'valid': True}}
> from=::ffff:10.10.43.1,60554, flow_id=3b2e802e, task_id=28c795d1-1639-4e68-a9fe-a00006be268f
> (api:52)
> 2018-02-20 14:38:47,226+0800 INFO (jsonrpc/3) [jsonrpc.JsonRpcServer] RPC
> call Host.getStats succeeded in 0.00 seconds (__init__:539)
> 2018-02-20 14:38:55,252+0800 INFO (jsonrpc/4) [jsonrpc.JsonRpcServer] RPC
> call Host.getAllVmStats succeeded in 0.00 seconds (__init__:539)
> 2018-02-20 14:38:58,566+0800 INFO (jsonrpc/2) [jsonrpc.JsonRpcServer] RPC
> call Host.getAllVmStats succeeded in 0.00 seconds (__init__:539)
> 2018-02-20 14:39:01,093+0800 INFO (periodic/1) [vdsm.api] START
> repoStats(options=None) from=internal, task_id=04f55ded-5841-44ae-a376-4f6e723e4b10
> (api:46)
> 2018-02-20 14:39:01,093+0800 INFO (periodic/1) [vdsm.api] FINISH
> repoStats return={'f3e372e3-1251-4195-a4b9-1027e40059df': {'code': 0,
> 'actual': True, 'version': 4, 'acquired': True, 'delay': '9.3888e-05',
> 'lastCheck': '9.9', 'valid': True}, 'e83d0d46-6ea6-4aa3-80bf-6e95c66b0454':
> {'code': 0, 'actual': True, 'version': 0, 'acquired': True, 'delay':
> '0.000197535', 'lastCheck': '9.8', 'valid': True}, '65ca2e2d-b472-4bee-85b4-09a161464b20':
> {'code': 0, 'actual': True, 'version': 4, 'acquired': True, 'delay':
> '0.000191456', 'lastCheck': '9.8', 'valid': True}, '4bf2ba2f-f57a-4d9f-b42a-fb78f440a358':
> {'code': 0, 'actual': True, 'version': 0, 'acquired': True, 'delay':
> '0.00026365', 'lastCheck': '9.8', 'valid': True}, '42e591b7-f86c-4b67-a3d2-40cc007f7662':
> {'code': 0, 'actual': True, 'version': 4, 'acquired': True, 'delay':
> '0.000243658', 'lastCheck': '0.0', 'valid': True}} from=internal,
> task_id=04f55ded-5841-44ae-a376-4f6e723e4b10 (api:52)
> 2018-02-20 14:39:02,295+0800 INFO (jsonrpc/6) [vdsm.api] START
> repoStats(options=None) from=::ffff:10.10.43.1,60554, flow_id=13149d6e,
> task_id=5f19d76a-d343-4dc4-ad09-024ec27f7443 (api:46)
> 2018-02-20 14:39:02,295+0800 INFO (jsonrpc/6) [vdsm.api] FINISH repoStats
> return={'f3e372e3-1251-4195-a4b9-1027e40059df': {'code': 0, 'actual':
> True, 'version': 4, 'acquired': True, 'delay': '9.4759e-05', 'lastCheck':
> '1.1', 'valid': True}, 'e83d0d46-6ea6-4aa3-80bf-6e95c66b0454': {'code':
> 0, 'actual': True, 'version': 0, 'acquired': True, 'delay': '0.000183158',
> 'lastCheck': '1.0', 'valid': True}, '65ca2e2d-b472-4bee-85b4-09a161464b20':
> {'code': 0, 'actual': True, 'version': 4, 'acquired': True, 'delay':
> '0.000222609', 'lastCheck': '1.0', 'valid': True}, '4bf2ba2f-f57a-4d9f-b42a-fb78f440a358':
> {'code': 0, 'actual': True, 'version': 0, 'acquired': True, 'delay':
> '0.000211253', 'lastCheck': '1.0', 'valid': True}, '42e591b7-f86c-4b67-a3d2-40cc007f7662':
> {'code': 0, 'actual': True, 'version': 4, 'acquired': True, 'delay':
> '0.000243658', 'lastCheck': '1.1', 'valid': True}}
> from=::ffff:10.10.43.1,60554, flow_id=13149d6e, task_id=5f19d76a-d343-4dc4-ad09-024ec27f7443
> (api:52)
> 2018-02-20 14:39:02,300+0800 INFO (jsonrpc/6) [jsonrpc.JsonRpcServer] RPC
> call Host.getStats succeeded in 0.01 seconds (__init__:539)
> 2018-02-20 14:39:10,270+0800 INFO (jsonrpc/1) [jsonrpc.JsonRpcServer] RPC
> call Host.getAllVmStats succeeded in 0.00 seconds (__init__:539)
> 2018-02-20 14:39:13,631+0800 INFO (jsonrpc/0) [vdsm.api] START
> connectStorageServer(domType=1, spUUID=u'00000000-0000-0000-0000-000000000000',
> conList=[{u'id': u'5e4c94f1-f3b9-4fbd-a6c7-e732d0fe3123', u'connection':
> u'dev2node1.lares.com.ph:/run/media/root/Slave1Data/dataNode1', u'iqn':
> u'', u'user': u'', u'tpgt': u'1', u'protocol_version': u'auto',
> u'password': '********', u'port': u''}], options=None)
> from=::ffff:10.10.43.1,60554, flow_id=15b57417, task_id=bfce1e70-ef4a-4e13-aaaa-7a66aaf44429
> (api:46)
> 2018-02-20 14:39:13,633+0800 INFO (jsonrpc/0) [vdsm.api] FINISH
> connectStorageServer return={'statuslist': [{'status': 0, 'id':
> u'5e4c94f1-f3b9-4fbd-a6c7-e732d0fe3123'}]} from=::ffff:10.10.43.1,60554,
> flow_id=15b57417, task_id=bfce1e70-ef4a-4e13-aaaa-7a66aaf44429 (api:52)
> 2018-02-20 14:39:13,634+0800 INFO (jsonrpc/0) [jsonrpc.JsonRpcServer] RPC
> call StoragePool.connectStorageServer succeeded in 0.00 seconds
> (__init__:539)
> 2018-02-20 14:39:13,806+0800 INFO (jsonrpc/7) [vdsm.api] START
> connectStoragePool(spUUID=u'5a865884-0366-0330-02b8-0000000002d4',
> hostID=1, msdUUID=u'f3e372e3-1251-4195-a4b9-1027e40059df',
> masterVersion=65, domainsMap={u'e83d0d46-6ea6-4aa3-80bf-6e95c66b0454':
> u'active', u'4bf2ba2f-f57a-4d9f-b42a-fb78f440a358': u'active',
> u'225e1975-8121-4370-b317-86e964ae326f': u'attached',
> u'f3e372e3-1251-4195-a4b9-1027e40059df': u'active',
> u'65ca2e2d-b472-4bee-85b4-09a161464b20': u'active',
> u'42e591b7-f86c-4b67-a3d2-40cc007f7662': u'active'}, options=None)
> from=::ffff:10.10.43.1,60554, flow_id=15b57417, task_id=927a5d9a-4304-4776-b5c9-22ba5d0cb853
> (api:46)
> 2018-02-20 14:39:13,807+0800 INFO (jsonrpc/7) [storage.StoragePoolMemoryBackend]
> new storage pool master version 65 and domains map
> {u'e83d0d46-6ea6-4aa3-80bf-6e95c66b0454': u'Active',
> u'4bf2ba2f-f57a-4d9f-b42a-fb78f440a358': u'Active',
> u'225e1975-8121-4370-b317-86e964ae326f': u'Attached',
> u'f3e372e3-1251-4195-a4b9-1027e40059df': u'Active',
> u'65ca2e2d-b472-4bee-85b4-09a161464b20': u'Active',
> u'42e591b7-f86c-4b67-a3d2-40cc007f7662': u'Active'} (spbackends:450)
>
> VDSM.log 2
> 2018-02-20 14:41:14,598+0800 INFO (periodic/3) [vdsm.api] FINISH
> repoStats return={'f3e372e3-1251-4195-a4b9-1027e40059df': {'code': 0,
> 'actual': True, 'version': 4, 'acquired': True, 'delay': '0.000252423',
> 'lastCheck': '6.0', 'valid': True}, 'e83d0d46-6ea6-4aa3-80bf-6e95c66b0454':
> {'code': 0, 'actual': True, 'version': 0, 'acquired': True, 'delay':
> '7.5567e-05', 'lastCheck': '6.0', 'valid': True}, '65ca2e2d-b472-4bee-85b4-09a161464b20':
> {'code': 0, 'actual': True, 'version': 4, 'acquired': True, 'delay':
> '8.2433e-05', 'lastCheck': '6.0', 'valid': True}, '4bf2ba2f-f57a-4d9f-b42a-fb78f440a358':
> {'code': 0, 'actual': True, 'version': 0, 'acquired': True, 'delay':
> '0.000237247', 'lastCheck': '5.9', 'valid': True}, '42e591b7-f86c-4b67-a3d2-40cc007f7662':
> {'code': 0, 'actual': True, 'version': 4, 'acquired': True, 'delay':
> '8.4376e-05', 'lastCheck': '6.0', 'valid': True}} from=internal,
> task_id=6213712b-9903-4db8-9836-3baf85cd63e4 (api:52)
> 2018-02-20 14:41:18,074+0800 INFO (jsonrpc/6) [jsonrpc.JsonRpcServer] RPC
> call Host.getAllVmStats succeeded in 0.00 seconds (__init__:539)
> 2018-02-20 14:41:24,333+0800 INFO (jsonrpc/4) [vdsm.api] START
> repoStats(options=None) from=::ffff:10.10.43.1,56540, flow_id=73f86113,
> task_id=e8231ecb-3543-4d8f-af54-4cf2b06ee98a (api:46)
> 2018-02-20 14:41:24,334+0800 INFO (jsonrpc/4) [vdsm.api] FINISH repoStats
> return={'f3e372e3-1251-4195-a4b9-1027e40059df': {'code': 0, 'actual':
> True, 'version': 4, 'acquired': True, 'delay': '0.000172953', 'lastCheck':
> '5.7', 'valid': True}, 'e83d0d46-6ea6-4aa3-80bf-6e95c66b0454': {'code':
> 0, 'actual': True, 'version': 0, 'acquired': True, 'delay': '7.701e-05',
> 'lastCheck': '5.7', 'valid': True}, '65ca2e2d-b472-4bee-85b4-09a161464b20':
> {'code': 0, 'actual': True, 'version': 4, 'acquired': True, 'delay':
> '0.000117677', 'lastCheck': '5.7', 'valid': True}, '4bf2ba2f-f57a-4d9f-b42a-fb78f440a358':
> {'code': 0, 'actual': True, 'version': 0, 'acquired': True, 'delay':
> '0.000185146', 'lastCheck': '5.7', 'valid': True}, '42e591b7-f86c-4b67-a3d2-40cc007f7662':
> {'code': 0, 'actual': True, 'version': 4, 'acquired': True, 'delay':
> '9.1467e-05', 'lastCheck': '5.8', 'valid': True}}
> from=::ffff:10.10.43.1,56540, flow_id=73f86113, task_id=e8231ecb-3543-4d8f-af54-4cf2b06ee98a
> (api:52)
> 2018-02-20 14:41:24,338+0800 INFO (jsonrpc/4) [jsonrpc.JsonRpcServer] RPC
> call Host.getStats succeeded in 0.00 seconds (__init__:539)
> 2018-02-20 14:41:27,087+0800 INFO (jsonrpc/0) [jsonrpc.JsonRpcServer] RPC
> call Host.getAllVmStats succeeded in 0.00 seconds (__init__:539)
> 2018-02-20 14:41:29,607+0800 INFO (periodic/1) [vdsm.api] START
> repoStats(options=None) from=internal, task_id=f14b7aab-b64a-4903-9368-d665e39b49d1
> (api:46)
> 2018-02-20 14:41:29,608+0800 INFO (periodic/1) [vdsm.api] FINISH
> repoStats return={'f3e372e3-1251-4195-a4b9-1027e40059df': {'code': 0,
> 'actual': True, 'version': 4, 'acquired': True, 'delay': '0.000246237',
> 'lastCheck': '1.0', 'valid': True}, 'e83d0d46-6ea6-4aa3-80bf-6e95c66b0454':
> {'code': 0, 'actual': True, 'version': 0, 'acquired': True, 'delay':
> '8.8773e-05', 'lastCheck': '1.0', 'valid': True}, '65ca2e2d-b472-4bee-85b4-09a161464b20':
> {'code': 0, 'actual': True, 'version': 4, 'acquired': True, 'delay':
> '9.7145e-05', 'lastCheck': '1.0', 'valid': True}, '4bf2ba2f-f57a-4d9f-b42a-fb78f440a358':
> {'code': 0, 'actual': True, 'version': 0, 'acquired': True, 'delay':
> '0.000218729', 'lastCheck': '0.9', 'valid': True}, '42e591b7-f86c-4b67-a3d2-40cc007f7662':
> {'code': 0, 'actual': True, 'version': 4, 'acquired': True, 'delay':
> '9.1035e-05', 'lastCheck': '1.0', 'valid': True}} from=internal,
> task_id=f14b7aab-b64a-4903-9368-d665e39b49d1 (api:52)
> 2018-02-20 14:41:33,079+0800 INFO (jsonrpc/7) [jsonrpc.JsonRpcServer] RPC
> call Host.getAllVmStats succeeded in 0.00 seconds (__init__:539)
> 2018-02-20 14:41:40,456+0800 INFO (jsonrpc/2) [vdsm.api] START
> repoStats(options=None) from=::ffff:10.10.43.1,56540, flow_id=3be8150c,
> task_id=1b0c86a8-6fd8-4882-a742-fbd56ccb4037 (api:46)
> 2018-02-20 14:41:40,457+0800 INFO (jsonrpc/2) [vdsm.api] FINISH repoStats
> return={'f3e372e3-1251-4195-a4b9-1027e40059df': {'code': 0, 'actual':
> True, 'version': 4, 'acquired': True, 'delay': '0.000174027', 'lastCheck':
> '1.8', 'valid': True}, 'e83d0d46-6ea6-4aa3-80bf-6e95c66b0454': {'code':
> 0, 'actual': True, 'version': 0, 'acquired': True, 'delay': '0.00011454',
> 'lastCheck': '1.8', 'valid': True}, '65ca2e2d-b472-4bee-85b4-09a161464b20':
> {'code': 0, 'actual': True, 'version': 4, 'acquired': True, 'delay':
> '0.000145955', 'lastCheck': '1.8', 'valid': True}, '4bf2ba2f-f57a-4d9f-b42a-fb78f440a358':
> {'code': 0, 'actual': True, 'version': 0, 'acquired': True, 'delay':
> '0.000209871', 'lastCheck': '1.8', 'valid': True}, '42e591b7-f86c-4b67-a3d2-40cc007f7662':
> {'code': 0, 'actual': True, 'version': 4, 'acquired': True, 'delay':
> '9.5744e-05', 'lastCheck': '1.9', 'valid': True}}
> from=::ffff:10.10.43.1,56540, flow_id=3be8150c, task_id=1b0c86a8-6fd8-4882-a742-fbd56ccb4037
> (api:52)
> 2018-02-20 14:41:40,461+0800 INFO (jsonrpc/2) [jsonrpc.JsonRpcServer] RPC
> call Host.getStats succeeded in 0.01 seconds (__init__:539)
> 2018-02-20 14:41:42,106+0800 INFO (jsonrpc/3) [jsonrpc.JsonRpcServer] RPC
> call Host.getAllVmStats succeeded in 0.00 seconds (__init__:539)
> 2018-02-20 14:41:44,622+0800 INFO (periodic/3) [vdsm.api] START
> repoStats(options=None) from=internal, task_id=20e0b29b-d3cd-4e44-b92c-213f9c984ab2
> (api:46)
> 2018-02-20 14:41:44,622+0800 INFO (periodic/3) [vdsm.api] FINISH
> repoStats return={'f3e372e3-1251-4195-a4b9-1027e40059df': {'code': 0,
> 'actual': True, 'version': 4, 'acquired': True, 'delay': '0.000174027',
> 'lastCheck': '6.0', 'valid': True}, 'e83d0d46-6ea6-4aa3-80bf-6e95c66b0454':
> {'code': 0, 'actual': True, 'version': 0, 'acquired': True, 'delay':
> '0.00011454', 'lastCheck': '6.0', 'valid': True}, '65ca2e2d-b472-4bee-85b4-09a161464b20':
> {'code': 0, 'actual': True, 'version': 4, 'acquired': True, 'delay':
> '0.000145955', 'lastCheck': '6.0', 'valid': True}, '4bf2ba2f-f57a-4d9f-b42a-fb78f440a358':
> {'code': 0, 'actual': True, 'version': 0, 'acquired': True, 'delay':
> '0.000209871', 'lastCheck': '5.9', 'valid': True}, '42e591b7-f86c-4b67-a3d2-40cc007f7662':
> {'code': 0, 'actual': True, 'version': 4, 'acquired': True, 'delay':
> '9.5744e-05', 'lastCheck': '6.0', 'valid': True}} from=internal,
> task_id=20e0b29b-d3cd-4e44-b92c-213f9c984ab2 (api:52)
> 2018-02-20 14:41:49,083+0800 INFO (jsonrpc/1) [jsonrpc.JsonRpcServer] RPC
> call Host.getAllVmStats succeeded in 0.00 seconds (__init__:539)
>
>
> Engine log
>
> skId '404ccecc-aa7f-45ea-89e4-726956269bc9' task status 'finished'
> 2018-02-20 14:42:21,966+08 INFO [org.ovirt.engine.core.
> vdsbroker.vdsbroker.SpmStartVDSCommand] (org.ovirt.thread.pool-7-thread-38)
> [29528f9] spmStart polling ended, spm status: SPM
> 2018-02-20 14:42:21,967+08 INFO [org.ovirt.engine.core.
> vdsbroker.vdsbroker.HSMClearTaskVDSCommand] (org.ovirt.thread.pool-7-thread-38)
> [29528f9] START, HSMClearTaskVDSCommand(HostName = Node1,
> HSMTaskGuidBaseVDSCommandParameters:{runAsync='true',
> hostId='7dee35bb-8c97-4f6a-b6cd-abc4258540e4', taskId='404ccecc-aa7f-45ea-89e4-726956269bc9'}),
> log id: 71688f70
> 2018-02-20 14:42:22,922+08 INFO [org.ovirt.engine.core.
> vdsbroker.vdsbroker.HSMClearTaskVDSCommand] (org.ovirt.thread.pool-7-thread-38)
> [29528f9] FINISH, HSMClearTaskVDSCommand, log id: 71688f70
> 2018-02-20 14:42:22,923+08 INFO [org.ovirt.engine.core.
> vdsbroker.vdsbroker.SpmStartVDSCommand] (org.ovirt.thread.pool-7-thread-38)
> [29528f9] FINISH, SpmStartVDSCommand, return: org.ovirt.engine.core.common.
> businessentities.SpmStatusResult at 78332453, log id: 3ea35d5
> 2018-02-20 14:42:22,935+08 INFO [org.ovirt.engine.core.vdsbroker.irsbroker.IrsProxy]
> (org.ovirt.thread.pool-7-thread-38) [29528f9] Initialize Irs proxy from
> vds: dev2node1.lares.com.ph
> 2018-02-20 14:42:22,951+08 INFO [org.ovirt.engine.core.dal.
> dbbroker.auditloghandling.AuditLogDirector] (org.ovirt.thread.pool-7-thread-38)
> [29528f9] EVENT_ID: IRS_HOSTED_ON_VDS(204), Correlation ID: null, Call
> Stack: null, Custom ID: null, Custom Event ID: -1, Message: Storage Pool
> Manager runs on Host Node1 (Address: dev2node1.lares.com.ph).
> 2018-02-20 14:42:22,952+08 INFO [org.ovirt.engine.core.
> vdsbroker.irsbroker.SPMGetAllTasksInfoVDSCommand]
> (org.ovirt.thread.pool-7-thread-38) [29528f9] -- executeIrsBrokerCommand:
> Attempting on storage pool '5a865884-0366-0330-02b8-0000000002d4'
> 2018-02-20 14:42:22,952+08 INFO [org.ovirt.engine.core.
> vdsbroker.vdsbroker.HSMGetAllTasksInfoVDSCommand]
> (org.ovirt.thread.pool-7-thread-38) [29528f9] START,
> HSMGetAllTasksInfoVDSCommand(HostName = Node1,
> VdsIdVDSCommandParametersBase:{runAsync='true',
> hostId='7dee35bb-8c97-4f6a-b6cd-abc4258540e4'}), log id: 1bdbea9d
> 2018-02-20 14:42:22,955+08 INFO [org.ovirt.engine.core.
> vdsbroker.irsbroker.SPMGetAllTasksInfoVDSCommand]
> (org.ovirt.thread.pool-7-thread-7) [29528f9] START,
> SPMGetAllTasksInfoVDSCommand( IrsBaseVDSCommandParameters:{runAsync='true',
> storagePoolId='5a865884-0366-0330-02b8-0000000002d4',
> ignoreFailoverLimit='false'}), log id: 5c2422d6
> 2018-02-20 14:42:23,956+08 INFO [org.ovirt.engine.core.
> vdsbroker.vdsbroker.HSMGetAllTasksInfoVDSCommand]
> (org.ovirt.thread.pool-7-thread-38) [29528f9] FINISH,
> HSMGetAllTasksInfoVDSCommand, return: [], log id: 1bdbea9d
> 2018-02-20 14:42:23,956+08 INFO [org.ovirt.engine.core.
> vdsbroker.irsbroker.SPMGetAllTasksInfoVDSCommand]
> (org.ovirt.thread.pool-7-thread-38) [29528f9] FINISH,
> SPMGetAllTasksInfoVDSCommand, return: [], log id: 58cbe7b7
> 2018-02-20 14:42:23,956+08 INFO [org.ovirt.engine.core.bll.tasks.AsyncTaskManager]
> (org.ovirt.thread.pool-7-thread-38) [29528f9] Discovered no tasks on
> Storage Pool 'UnsecuredEnv'
> 2018-02-20 14:42:24,936+08 INFO [org.ovirt.vdsm.jsonrpc.client.reactors.ReactorClient]
> (SSL Stomp Reactor) [] Connecting to dev2node1.lares.com.ph/10.10.43.2
> 2018-02-20 14:42:27,012+08 WARN [org.ovirt.engine.core.vdsbroker.irsbroker.IrsProxy]
> (org.ovirt.thread.pool-7-thread-43) [] Master domain is not in sync
> between DB and VDSM. Domain Node1Container marked as master in DB and not
> in the storage
> 2018-02-20 14:42:27,026+08 WARN [org.ovirt.engine.core.dal.
> dbbroker.auditloghandling.AuditLogDirector] (org.ovirt.thread.pool-7-thread-43)
> [] EVENT_ID: SYSTEM_MASTER_DOMAIN_NOT_IN_SYNC(990), Correlation ID: null,
> Call Stack: null, Custom ID: null, Custom Event ID: -1, Message: Sync Error
> on Master Domain between Host Node1 and oVirt Engine. Domain:
> Node1Container is marked as Master in oVirt Engine database but not on the
> Storage side. Please consult with Support on how to fix this issue.
> 2018-02-20 14:42:27,103+08 INFO [org.ovirt.engine.core.bll.storage.pool.
> ReconstructMasterDomainCommand] (org.ovirt.thread.pool-7-thread-43)
> [3e5965ca] Running command: ReconstructMasterDomainCommand internal: true.
> Entities affected : ID: f3e372e3-1251-4195-a4b9-1027e40059df Type:
> Storage
> 2018-02-20 14:42:27,137+08 INFO [org.ovirt.engine.core.
> vdsbroker.irsbroker.ResetIrsVDSCommand] (org.ovirt.thread.pool-7-thread-43)
> [3e5965ca] START, ResetIrsVDSCommand( ResetIrsVDSCommandParameters:{runAsync='true',
> storagePoolId='5a865884-0366-0330-02b8-0000000002d4',
> ignoreFailoverLimit='false', vdsId='7dee35bb-8c97-4f6a-b6cd-abc4258540e4',
> ignoreStopFailed='true'}), log id: 3e0a239d
> 2018-02-20 14:42:27,140+08 INFO [org.ovirt.engine.core.
> vdsbroker.vdsbroker.SpmStopVDSCommand] (org.ovirt.thread.pool-7-thread-43)
> [3e5965ca] START, SpmStopVDSCommand(HostName = Node1,
> SpmStopVDSCommandParameters:{runAsync='true', hostId='7dee35bb-8c97-4f6a-b6cd-abc4258540e4',
> storagePoolId='5a865884-0366-0330-02b8-0000000002d4'}), log id: 7c67bf06
> 2018-02-20 14:42:28,144+08 INFO [org.ovirt.engine.core.
> vdsbroker.vdsbroker.SpmStopVDSCommand] (org.ovirt.thread.pool-7-thread-43)
> [3e5965ca] SpmStopVDSCommand::Stopping SPM on vds 'Node1', pool id
> '5a865884-0366-0330-02b8-0000000002d4'
>
>
> On Tuesday, February 20, 2018 2:33 PM, michael pagdanganan <
> mhke_aj5566 at yahoo.com> wrote:
>
>
> Thanks for quick response,
>
> see attachment.
>
>
> On Tuesday, February 20, 2018 2:10 PM, Eyal Shenitzky <eshenitz at redhat.com>
> wrote:
>
>
> Hi,
>
> Can you please attach full Engine and VDSM logs?
>
> Thanks,
>
> On Tue, Feb 20, 2018 at 2:59 AM, michael pagdanganan <
> mhke_aj5566 at yahoo.com> wrote:
>
> My storage pool has 2 Master Domain(Stored2 on Node2,Node1Container on
> Node1). and my old master domain (DATANd01 on Node1 ) hung on preparing
> for maintenance. When I tried to activate my old master domain (DATANd01
> on Node1 ) all storage domain goes down and up master keep on rotating.
>
>
> Ovirt Version (oVirt Engine Version: 4.1.9.1.el7.centos)
>
>
> Event Error:
> Sync Error on Master Domain between Host Node2 and oVirt Engine. Domain
> Stored2 is marked as master in ovirt engine Database but not on storage
> side Please consult with support
>
> VDSM Node2 command ConnectStoragePoolVDS failed: Wrong Master domain or
> it's version: u'SD=f3e372e3-1251-4195-a4b9- 1027e40059df,
> pool=5a865884-0366-0330-02b8- 0000
>
> VDSM Node2 command HSMGetAllTastsStatusesVDS failed: Not SPM: ()
>
> Failed to deactivate Storage Domain DATANd01 (Data Center UnsecuredEnv)
>
> Here's logs from engine:
>
> ------------------------------ ------------------------------
> ------------------------------ -------------
> [root at dev2engine ~]# tail /var/log/messages
> Feb 20 07:01:01 dev2engine systemd: Starting Session 20 of user root.
> Feb 20 07:01:01 dev2engine systemd: Removed slice User Slice of root.
> Feb 20 07:01:01 dev2engine systemd: Stopping User Slice of root.
> Feb 20 07:58:52 dev2engine systemd: Created slice User Slice of root.
> Feb 20 07:58:52 dev2engine systemd: Starting User Slice of root.
> Feb 20 07:58:52 dev2engine systemd-logind: New session 21 of user root.
> Feb 20 07:58:52 dev2engine systemd: Started Session 21 of user root.
> Feb 20 07:58:52 dev2engine systemd: Starting Session 21 of user root.
> Feb 20 08:01:01 dev2engine systemd: Started Session 22 of user root.
> Feb 20 08:01:01 dev2engine systemd: Starting Session 22 of user root.
>
>
> ------------------------------ ------------------------------
> ------------------------------ -------------
> [root at dev2engine ~]# tail /var/log/ovirt-engine/engine. log
> 2018-02-20 08:01:16,062+08 INFO [org.ovirt.engine.core.bll.
> eventqueue.EventQueueMonitor] (org.ovirt.thread.pool-7- thread-32)
> [102e9d3c] Finished reconstruct for pool '5a865884-0366-0330-02b8-
> 0000000002d4'. Clearing event queue
> 2018-02-20 08:01:27,825+08 WARN [org.ovirt.engine.core.
> vdsbroker.irsbroker.IrsProxy] (org.ovirt.thread.pool-7- thread-23) []
> Master domain is not in sync between DB and VDSM. Domain Stored2 marked as
> master in DB and not in the storage
> 2018-02-20 08:01:27,862+08 WARN [org.ovirt.engine.core.bll. storage.pool.
> ReconstructMasterDomainCommand ] (org.ovirt.thread.pool-7- thread-23)
> [213f42b9] Validation of action 'ReconstructMasterDomain' failed for user
> SYSTEM. Reasons: VAR__ACTION__RECONSTRUCT_ MASTER,VAR__TYPE__STORAGE__
> DOMAIN,ACTION_TYPE_FAILED_ STORAGE_DOMAIN_STATUS_ ILLEGAL2,$status
> PreparingForMaintenance
> 2018-02-20 08:01:27,882+08 INFO [org.ovirt.engine.core.bll.
> eventqueue.EventQueueMonitor] (org.ovirt.thread.pool-7- thread-20)
> [929330e] Finished reconstruct for pool '5a865884-0366-0330-02b8-
> 0000000002d4'. Clearing event queue
> 2018-02-20 08:01:40,106+08 WARN [org.ovirt.engine.core.
> vdsbroker.irsbroker.IrsProxy] (org.ovirt.thread.pool-7- thread-17) []
> Master domain is not in sync between DB and VDSM. Domain Stored2 marked as
> master in DB and not in the storage
> 2018-02-20 08:01:40,197+08 WARN [org.ovirt.engine.core.bll. storage.pool.
> ReconstructMasterDomainCommand ] (org.ovirt.thread.pool-7- thread-17)
> [7af552c1] Validation of action 'ReconstructMasterDomain' failed for user
> SYSTEM. Reasons: VAR__ACTION__RECONSTRUCT_ MASTER,VAR__TYPE__STORAGE__
> DOMAIN,ACTION_TYPE_FAILED_ STORAGE_DOMAIN_STATUS_ ILLEGAL2,$status
> PreparingForMaintenance
> 2018-02-20 08:01:40,246+08 INFO [org.ovirt.engine.core.bll.
> eventqueue.EventQueueMonitor] (org.ovirt.thread.pool-7- thread-22)
> [73673040] Finished reconstruct for pool '5a865884-0366-0330-02b8-
> 0000000002d4'. Clearing event queue
> 2018-02-20 08:01:51,809+08 WARN [org.ovirt.engine.core.
> vdsbroker.irsbroker.IrsProxy] (org.ovirt.thread.pool-7- thread-26) []
> Master domain is not in sync between DB and VDSM. Domain Stored2 marked as
> master in DB and not in the storage
> 2018-02-20 08:01:51,846+08 WARN [org.ovirt.engine.core.bll. storage.pool.
> ReconstructMasterDomainCommand ] (org.ovirt.thread.pool-7- thread-26)
> [20307cbe] Validation of action 'ReconstructMasterDomain' failed for user
> SYSTEM. Reasons: VAR__ACTION__RECONSTRUCT_ MASTER,VAR__TYPE__STORAGE__
> DOMAIN,ACTION_TYPE_FAILED_ STORAGE_DOMAIN_STATUS_ ILLEGAL2,$status
> PreparingForMaintenance
> 2018-02-20 08:01:51,866+08 INFO [org.ovirt.engine.core.bll.
> eventqueue.EventQueueMonitor] (org.ovirt.thread.pool-7- thread-49)
> [2c11a866] Finished reconstruct for pool '5a865884-0366-0330-02b8-
> 0000000002d4'. Clearing event queue
>
>
> ______________________________ _________________
> Users mailing list
> Users at ovirt.org
> http://lists.ovirt.org/ mailman/listinfo/users
> <http://lists.ovirt.org/mailman/listinfo/users>
>
>
>
>
> --
> Regards,
> Eyal Shenitzky
>
>
>
>
>
--
Regards,
Eyal Shenitzky
-------------- next part --------------
An HTML attachment was scrubbed...
URL: <http://lists.ovirt.org/pipermail/users/attachments/20180220/812467cb/attachment.html>
More information about the Users
mailing list