Please try to compress the logs maybe it will help. On Tue, Feb 20, 2018 at 8:43 AM, michael pagdanganan <mhke_aj5566@yahoo.com> wrote:
Sorry can't attached log file it's too big file
VDSM.log for node 1
2766', 'lastCheck': '4.9', 'valid': True}} from=internal, task_id=645d456e-f59f-4b1c-9e97-fc82d19a36b1 (api:52) 2018-02-20 14:38:47,222+0800 INFO (jsonrpc/3) [vdsm.api] START repoStats(options=None) from=::ffff:10.10.43.1,60554, flow_id=3b2e802e, task_id=28c795d1-1639-4e68-a9fe-a00006be268f (api:46) 2018-02-20 14:38:47,222+0800 INFO (jsonrpc/3) [vdsm.api] FINISH repoStats return={'f3e372e3-1251-4195-a4b9-1027e40059df': {'code': 0, 'actual': True, 'version': 4, 'acquired': True, 'delay': '0.000119109', 'lastCheck': '6.0', 'valid': True}, 'e83d0d46-6ea6-4aa3-80bf-6e95c66b0454': {'code': 0, 'actual': True, 'version': 0, 'acquired': True, 'delay': '0.000195263', 'lastCheck': '6.0', 'valid': True}, '65ca2e2d-b472-4bee-85b4-09a161464b20': {'code': 0, 'actual': True, 'version': 4, 'acquired': True, 'delay': '0.00022988', 'lastCheck': '6.0', 'valid': True}, '4bf2ba2f-f57a-4d9f-b42a-fb78f440a358': {'code': 0, 'actual': True, 'version': 0, 'acquired': True, 'delay': '0.000270206', 'lastCheck': '5.9', 'valid': True}, '42e591b7-f86c-4b67-a3d2-40cc007f7662': {'code': 0, 'actual': True, 'version': 4, 'acquired': True, 'delay': '0.000272766', 'lastCheck': '6.0', 'valid': True}} from=::ffff:10.10.43.1,60554, flow_id=3b2e802e, task_id=28c795d1-1639-4e68-a9fe-a00006be268f (api:52) 2018-02-20 14:38:47,226+0800 INFO (jsonrpc/3) [jsonrpc.JsonRpcServer] RPC call Host.getStats succeeded in 0.00 seconds (__init__:539) 2018-02-20 14:38:55,252+0800 INFO (jsonrpc/4) [jsonrpc.JsonRpcServer] RPC call Host.getAllVmStats succeeded in 0.00 seconds (__init__:539) 2018-02-20 14:38:58,566+0800 INFO (jsonrpc/2) [jsonrpc.JsonRpcServer] RPC call Host.getAllVmStats succeeded in 0.00 seconds (__init__:539) 2018-02-20 14:39:01,093+0800 INFO (periodic/1) [vdsm.api] START repoStats(options=None) from=internal, task_id=04f55ded-5841-44ae-a376-4f6e723e4b10 (api:46) 2018-02-20 14:39:01,093+0800 INFO (periodic/1) [vdsm.api] FINISH repoStats return={'f3e372e3-1251-4195-a4b9-1027e40059df': {'code': 0, 'actual': True, 'version': 4, 'acquired': True, 'delay': '9.3888e-05', 'lastCheck': '9.9', 'valid': True}, 'e83d0d46-6ea6-4aa3-80bf-6e95c66b0454': {'code': 0, 'actual': True, 'version': 0, 'acquired': True, 'delay': '0.000197535', 'lastCheck': '9.8', 'valid': True}, '65ca2e2d-b472-4bee-85b4-09a161464b20': {'code': 0, 'actual': True, 'version': 4, 'acquired': True, 'delay': '0.000191456', 'lastCheck': '9.8', 'valid': True}, '4bf2ba2f-f57a-4d9f-b42a-fb78f440a358': {'code': 0, 'actual': True, 'version': 0, 'acquired': True, 'delay': '0.00026365', 'lastCheck': '9.8', 'valid': True}, '42e591b7-f86c-4b67-a3d2-40cc007f7662': {'code': 0, 'actual': True, 'version': 4, 'acquired': True, 'delay': '0.000243658', 'lastCheck': '0.0', 'valid': True}} from=internal, task_id=04f55ded-5841-44ae-a376-4f6e723e4b10 (api:52) 2018-02-20 14:39:02,295+0800 INFO (jsonrpc/6) [vdsm.api] START repoStats(options=None) from=::ffff:10.10.43.1,60554, flow_id=13149d6e, task_id=5f19d76a-d343-4dc4-ad09-024ec27f7443 (api:46) 2018-02-20 14:39:02,295+0800 INFO (jsonrpc/6) [vdsm.api] FINISH repoStats return={'f3e372e3-1251-4195-a4b9-1027e40059df': {'code': 0, 'actual': True, 'version': 4, 'acquired': True, 'delay': '9.4759e-05', 'lastCheck': '1.1', 'valid': True}, 'e83d0d46-6ea6-4aa3-80bf-6e95c66b0454': {'code': 0, 'actual': True, 'version': 0, 'acquired': True, 'delay': '0.000183158', 'lastCheck': '1.0', 'valid': True}, '65ca2e2d-b472-4bee-85b4-09a161464b20': {'code': 0, 'actual': True, 'version': 4, 'acquired': True, 'delay': '0.000222609', 'lastCheck': '1.0', 'valid': True}, '4bf2ba2f-f57a-4d9f-b42a-fb78f440a358': {'code': 0, 'actual': True, 'version': 0, 'acquired': True, 'delay': '0.000211253', 'lastCheck': '1.0', 'valid': True}, '42e591b7-f86c-4b67-a3d2-40cc007f7662': {'code': 0, 'actual': True, 'version': 4, 'acquired': True, 'delay': '0.000243658', 'lastCheck': '1.1', 'valid': True}} from=::ffff:10.10.43.1,60554, flow_id=13149d6e, task_id=5f19d76a-d343-4dc4-ad09-024ec27f7443 (api:52) 2018-02-20 14:39:02,300+0800 INFO (jsonrpc/6) [jsonrpc.JsonRpcServer] RPC call Host.getStats succeeded in 0.01 seconds (__init__:539) 2018-02-20 14:39:10,270+0800 INFO (jsonrpc/1) [jsonrpc.JsonRpcServer] RPC call Host.getAllVmStats succeeded in 0.00 seconds (__init__:539) 2018-02-20 14:39:13,631+0800 INFO (jsonrpc/0) [vdsm.api] START connectStorageServer(domType=1, spUUID=u'00000000-0000-0000-0000-000000000000', conList=[{u'id': u'5e4c94f1-f3b9-4fbd-a6c7-e732d0fe3123', u'connection': u'dev2node1.lares.com.ph:/run/media/root/Slave1Data/dataNode1', u'iqn': u'', u'user': u'', u'tpgt': u'1', u'protocol_version': u'auto', u'password': '********', u'port': u''}], options=None) from=::ffff:10.10.43.1,60554, flow_id=15b57417, task_id=bfce1e70-ef4a-4e13-aaaa-7a66aaf44429 (api:46) 2018-02-20 14:39:13,633+0800 INFO (jsonrpc/0) [vdsm.api] FINISH connectStorageServer return={'statuslist': [{'status': 0, 'id': u'5e4c94f1-f3b9-4fbd-a6c7-e732d0fe3123'}]} from=::ffff:10.10.43.1,60554, flow_id=15b57417, task_id=bfce1e70-ef4a-4e13-aaaa-7a66aaf44429 (api:52) 2018-02-20 14:39:13,634+0800 INFO (jsonrpc/0) [jsonrpc.JsonRpcServer] RPC call StoragePool.connectStorageServer succeeded in 0.00 seconds (__init__:539) 2018-02-20 14:39:13,806+0800 INFO (jsonrpc/7) [vdsm.api] START connectStoragePool(spUUID=u'5a865884-0366-0330-02b8-0000000002d4', hostID=1, msdUUID=u'f3e372e3-1251-4195-a4b9-1027e40059df', masterVersion=65, domainsMap={u'e83d0d46-6ea6-4aa3-80bf-6e95c66b0454': u'active', u'4bf2ba2f-f57a-4d9f-b42a-fb78f440a358': u'active', u'225e1975-8121-4370-b317-86e964ae326f': u'attached', u'f3e372e3-1251-4195-a4b9-1027e40059df': u'active', u'65ca2e2d-b472-4bee-85b4-09a161464b20': u'active', u'42e591b7-f86c-4b67-a3d2-40cc007f7662': u'active'}, options=None) from=::ffff:10.10.43.1,60554, flow_id=15b57417, task_id=927a5d9a-4304-4776-b5c9-22ba5d0cb853 (api:46) 2018-02-20 14:39:13,807+0800 INFO (jsonrpc/7) [storage.StoragePoolMemoryBackend] new storage pool master version 65 and domains map {u'e83d0d46-6ea6-4aa3-80bf-6e95c66b0454': u'Active', u'4bf2ba2f-f57a-4d9f-b42a-fb78f440a358': u'Active', u'225e1975-8121-4370-b317-86e964ae326f': u'Attached', u'f3e372e3-1251-4195-a4b9-1027e40059df': u'Active', u'65ca2e2d-b472-4bee-85b4-09a161464b20': u'Active', u'42e591b7-f86c-4b67-a3d2-40cc007f7662': u'Active'} (spbackends:450)
VDSM.log 2 2018-02-20 14:41:14,598+0800 INFO (periodic/3) [vdsm.api] FINISH repoStats return={'f3e372e3-1251-4195-a4b9-1027e40059df': {'code': 0, 'actual': True, 'version': 4, 'acquired': True, 'delay': '0.000252423', 'lastCheck': '6.0', 'valid': True}, 'e83d0d46-6ea6-4aa3-80bf-6e95c66b0454': {'code': 0, 'actual': True, 'version': 0, 'acquired': True, 'delay': '7.5567e-05', 'lastCheck': '6.0', 'valid': True}, '65ca2e2d-b472-4bee-85b4-09a161464b20': {'code': 0, 'actual': True, 'version': 4, 'acquired': True, 'delay': '8.2433e-05', 'lastCheck': '6.0', 'valid': True}, '4bf2ba2f-f57a-4d9f-b42a-fb78f440a358': {'code': 0, 'actual': True, 'version': 0, 'acquired': True, 'delay': '0.000237247', 'lastCheck': '5.9', 'valid': True}, '42e591b7-f86c-4b67-a3d2-40cc007f7662': {'code': 0, 'actual': True, 'version': 4, 'acquired': True, 'delay': '8.4376e-05', 'lastCheck': '6.0', 'valid': True}} from=internal, task_id=6213712b-9903-4db8-9836-3baf85cd63e4 (api:52) 2018-02-20 14:41:18,074+0800 INFO (jsonrpc/6) [jsonrpc.JsonRpcServer] RPC call Host.getAllVmStats succeeded in 0.00 seconds (__init__:539) 2018-02-20 14:41:24,333+0800 INFO (jsonrpc/4) [vdsm.api] START repoStats(options=None) from=::ffff:10.10.43.1,56540, flow_id=73f86113, task_id=e8231ecb-3543-4d8f-af54-4cf2b06ee98a (api:46) 2018-02-20 14:41:24,334+0800 INFO (jsonrpc/4) [vdsm.api] FINISH repoStats return={'f3e372e3-1251-4195-a4b9-1027e40059df': {'code': 0, 'actual': True, 'version': 4, 'acquired': True, 'delay': '0.000172953', 'lastCheck': '5.7', 'valid': True}, 'e83d0d46-6ea6-4aa3-80bf-6e95c66b0454': {'code': 0, 'actual': True, 'version': 0, 'acquired': True, 'delay': '7.701e-05', 'lastCheck': '5.7', 'valid': True}, '65ca2e2d-b472-4bee-85b4-09a161464b20': {'code': 0, 'actual': True, 'version': 4, 'acquired': True, 'delay': '0.000117677', 'lastCheck': '5.7', 'valid': True}, '4bf2ba2f-f57a-4d9f-b42a-fb78f440a358': {'code': 0, 'actual': True, 'version': 0, 'acquired': True, 'delay': '0.000185146', 'lastCheck': '5.7', 'valid': True}, '42e591b7-f86c-4b67-a3d2-40cc007f7662': {'code': 0, 'actual': True, 'version': 4, 'acquired': True, 'delay': '9.1467e-05', 'lastCheck': '5.8', 'valid': True}} from=::ffff:10.10.43.1,56540, flow_id=73f86113, task_id=e8231ecb-3543-4d8f-af54-4cf2b06ee98a (api:52) 2018-02-20 14:41:24,338+0800 INFO (jsonrpc/4) [jsonrpc.JsonRpcServer] RPC call Host.getStats succeeded in 0.00 seconds (__init__:539) 2018-02-20 14:41:27,087+0800 INFO (jsonrpc/0) [jsonrpc.JsonRpcServer] RPC call Host.getAllVmStats succeeded in 0.00 seconds (__init__:539) 2018-02-20 14:41:29,607+0800 INFO (periodic/1) [vdsm.api] START repoStats(options=None) from=internal, task_id=f14b7aab-b64a-4903-9368-d665e39b49d1 (api:46) 2018-02-20 14:41:29,608+0800 INFO (periodic/1) [vdsm.api] FINISH repoStats return={'f3e372e3-1251-4195-a4b9-1027e40059df': {'code': 0, 'actual': True, 'version': 4, 'acquired': True, 'delay': '0.000246237', 'lastCheck': '1.0', 'valid': True}, 'e83d0d46-6ea6-4aa3-80bf-6e95c66b0454': {'code': 0, 'actual': True, 'version': 0, 'acquired': True, 'delay': '8.8773e-05', 'lastCheck': '1.0', 'valid': True}, '65ca2e2d-b472-4bee-85b4-09a161464b20': {'code': 0, 'actual': True, 'version': 4, 'acquired': True, 'delay': '9.7145e-05', 'lastCheck': '1.0', 'valid': True}, '4bf2ba2f-f57a-4d9f-b42a-fb78f440a358': {'code': 0, 'actual': True, 'version': 0, 'acquired': True, 'delay': '0.000218729', 'lastCheck': '0.9', 'valid': True}, '42e591b7-f86c-4b67-a3d2-40cc007f7662': {'code': 0, 'actual': True, 'version': 4, 'acquired': True, 'delay': '9.1035e-05', 'lastCheck': '1.0', 'valid': True}} from=internal, task_id=f14b7aab-b64a-4903-9368-d665e39b49d1 (api:52) 2018-02-20 14:41:33,079+0800 INFO (jsonrpc/7) [jsonrpc.JsonRpcServer] RPC call Host.getAllVmStats succeeded in 0.00 seconds (__init__:539) 2018-02-20 14:41:40,456+0800 INFO (jsonrpc/2) [vdsm.api] START repoStats(options=None) from=::ffff:10.10.43.1,56540, flow_id=3be8150c, task_id=1b0c86a8-6fd8-4882-a742-fbd56ccb4037 (api:46) 2018-02-20 14:41:40,457+0800 INFO (jsonrpc/2) [vdsm.api] FINISH repoStats return={'f3e372e3-1251-4195-a4b9-1027e40059df': {'code': 0, 'actual': True, 'version': 4, 'acquired': True, 'delay': '0.000174027', 'lastCheck': '1.8', 'valid': True}, 'e83d0d46-6ea6-4aa3-80bf-6e95c66b0454': {'code': 0, 'actual': True, 'version': 0, 'acquired': True, 'delay': '0.00011454', 'lastCheck': '1.8', 'valid': True}, '65ca2e2d-b472-4bee-85b4-09a161464b20': {'code': 0, 'actual': True, 'version': 4, 'acquired': True, 'delay': '0.000145955', 'lastCheck': '1.8', 'valid': True}, '4bf2ba2f-f57a-4d9f-b42a-fb78f440a358': {'code': 0, 'actual': True, 'version': 0, 'acquired': True, 'delay': '0.000209871', 'lastCheck': '1.8', 'valid': True}, '42e591b7-f86c-4b67-a3d2-40cc007f7662': {'code': 0, 'actual': True, 'version': 4, 'acquired': True, 'delay': '9.5744e-05', 'lastCheck': '1.9', 'valid': True}} from=::ffff:10.10.43.1,56540, flow_id=3be8150c, task_id=1b0c86a8-6fd8-4882-a742-fbd56ccb4037 (api:52) 2018-02-20 14:41:40,461+0800 INFO (jsonrpc/2) [jsonrpc.JsonRpcServer] RPC call Host.getStats succeeded in 0.01 seconds (__init__:539) 2018-02-20 14:41:42,106+0800 INFO (jsonrpc/3) [jsonrpc.JsonRpcServer] RPC call Host.getAllVmStats succeeded in 0.00 seconds (__init__:539) 2018-02-20 14:41:44,622+0800 INFO (periodic/3) [vdsm.api] START repoStats(options=None) from=internal, task_id=20e0b29b-d3cd-4e44-b92c-213f9c984ab2 (api:46) 2018-02-20 14:41:44,622+0800 INFO (periodic/3) [vdsm.api] FINISH repoStats return={'f3e372e3-1251-4195-a4b9-1027e40059df': {'code': 0, 'actual': True, 'version': 4, 'acquired': True, 'delay': '0.000174027', 'lastCheck': '6.0', 'valid': True}, 'e83d0d46-6ea6-4aa3-80bf-6e95c66b0454': {'code': 0, 'actual': True, 'version': 0, 'acquired': True, 'delay': '0.00011454', 'lastCheck': '6.0', 'valid': True}, '65ca2e2d-b472-4bee-85b4-09a161464b20': {'code': 0, 'actual': True, 'version': 4, 'acquired': True, 'delay': '0.000145955', 'lastCheck': '6.0', 'valid': True}, '4bf2ba2f-f57a-4d9f-b42a-fb78f440a358': {'code': 0, 'actual': True, 'version': 0, 'acquired': True, 'delay': '0.000209871', 'lastCheck': '5.9', 'valid': True}, '42e591b7-f86c-4b67-a3d2-40cc007f7662': {'code': 0, 'actual': True, 'version': 4, 'acquired': True, 'delay': '9.5744e-05', 'lastCheck': '6.0', 'valid': True}} from=internal, task_id=20e0b29b-d3cd-4e44-b92c-213f9c984ab2 (api:52) 2018-02-20 14:41:49,083+0800 INFO (jsonrpc/1) [jsonrpc.JsonRpcServer] RPC call Host.getAllVmStats succeeded in 0.00 seconds (__init__:539)
Engine log
skId '404ccecc-aa7f-45ea-89e4-726956269bc9' task status 'finished' 2018-02-20 14:42:21,966+08 INFO [org.ovirt.engine.core. vdsbroker.vdsbroker.SpmStartVDSCommand] (org.ovirt.thread.pool-7-thread-38) [29528f9] spmStart polling ended, spm status: SPM 2018-02-20 14:42:21,967+08 INFO [org.ovirt.engine.core. vdsbroker.vdsbroker.HSMClearTaskVDSCommand] (org.ovirt.thread.pool-7-thread-38) [29528f9] START, HSMClearTaskVDSCommand(HostName = Node1, HSMTaskGuidBaseVDSCommandParameters:{runAsync='true', hostId='7dee35bb-8c97-4f6a-b6cd-abc4258540e4', taskId='404ccecc-aa7f-45ea-89e4-726956269bc9'}), log id: 71688f70 2018-02-20 14:42:22,922+08 INFO [org.ovirt.engine.core. vdsbroker.vdsbroker.HSMClearTaskVDSCommand] (org.ovirt.thread.pool-7-thread-38) [29528f9] FINISH, HSMClearTaskVDSCommand, log id: 71688f70 2018-02-20 14:42:22,923+08 INFO [org.ovirt.engine.core. vdsbroker.vdsbroker.SpmStartVDSCommand] (org.ovirt.thread.pool-7-thread-38) [29528f9] FINISH, SpmStartVDSCommand, return: org.ovirt.engine.core.common. businessentities.SpmStatusResult@78332453, log id: 3ea35d5 2018-02-20 14:42:22,935+08 INFO [org.ovirt.engine.core.vdsbroker.irsbroker.IrsProxy] (org.ovirt.thread.pool-7-thread-38) [29528f9] Initialize Irs proxy from vds: dev2node1.lares.com.ph 2018-02-20 14:42:22,951+08 INFO [org.ovirt.engine.core.dal. dbbroker.auditloghandling.AuditLogDirector] (org.ovirt.thread.pool-7-thread-38) [29528f9] EVENT_ID: IRS_HOSTED_ON_VDS(204), Correlation ID: null, Call Stack: null, Custom ID: null, Custom Event ID: -1, Message: Storage Pool Manager runs on Host Node1 (Address: dev2node1.lares.com.ph). 2018-02-20 14:42:22,952+08 INFO [org.ovirt.engine.core. vdsbroker.irsbroker.SPMGetAllTasksInfoVDSCommand] (org.ovirt.thread.pool-7-thread-38) [29528f9] -- executeIrsBrokerCommand: Attempting on storage pool '5a865884-0366-0330-02b8-0000000002d4' 2018-02-20 14:42:22,952+08 INFO [org.ovirt.engine.core. vdsbroker.vdsbroker.HSMGetAllTasksInfoVDSCommand] (org.ovirt.thread.pool-7-thread-38) [29528f9] START, HSMGetAllTasksInfoVDSCommand(HostName = Node1, VdsIdVDSCommandParametersBase:{runAsync='true', hostId='7dee35bb-8c97-4f6a-b6cd-abc4258540e4'}), log id: 1bdbea9d 2018-02-20 14:42:22,955+08 INFO [org.ovirt.engine.core. vdsbroker.irsbroker.SPMGetAllTasksInfoVDSCommand] (org.ovirt.thread.pool-7-thread-7) [29528f9] START, SPMGetAllTasksInfoVDSCommand( IrsBaseVDSCommandParameters:{runAsync='true', storagePoolId='5a865884-0366-0330-02b8-0000000002d4', ignoreFailoverLimit='false'}), log id: 5c2422d6 2018-02-20 14:42:23,956+08 INFO [org.ovirt.engine.core. vdsbroker.vdsbroker.HSMGetAllTasksInfoVDSCommand] (org.ovirt.thread.pool-7-thread-38) [29528f9] FINISH, HSMGetAllTasksInfoVDSCommand, return: [], log id: 1bdbea9d 2018-02-20 14:42:23,956+08 INFO [org.ovirt.engine.core. vdsbroker.irsbroker.SPMGetAllTasksInfoVDSCommand] (org.ovirt.thread.pool-7-thread-38) [29528f9] FINISH, SPMGetAllTasksInfoVDSCommand, return: [], log id: 58cbe7b7 2018-02-20 14:42:23,956+08 INFO [org.ovirt.engine.core.bll.tasks.AsyncTaskManager] (org.ovirt.thread.pool-7-thread-38) [29528f9] Discovered no tasks on Storage Pool 'UnsecuredEnv' 2018-02-20 14:42:24,936+08 INFO [org.ovirt.vdsm.jsonrpc.client.reactors.ReactorClient] (SSL Stomp Reactor) [] Connecting to dev2node1.lares.com.ph/10.10.43.2 2018-02-20 14:42:27,012+08 WARN [org.ovirt.engine.core.vdsbroker.irsbroker.IrsProxy] (org.ovirt.thread.pool-7-thread-43) [] Master domain is not in sync between DB and VDSM. Domain Node1Container marked as master in DB and not in the storage 2018-02-20 14:42:27,026+08 WARN [org.ovirt.engine.core.dal. dbbroker.auditloghandling.AuditLogDirector] (org.ovirt.thread.pool-7-thread-43) [] EVENT_ID: SYSTEM_MASTER_DOMAIN_NOT_IN_SYNC(990), Correlation ID: null, Call Stack: null, Custom ID: null, Custom Event ID: -1, Message: Sync Error on Master Domain between Host Node1 and oVirt Engine. Domain: Node1Container is marked as Master in oVirt Engine database but not on the Storage side. Please consult with Support on how to fix this issue. 2018-02-20 14:42:27,103+08 INFO [org.ovirt.engine.core.bll.storage.pool. ReconstructMasterDomainCommand] (org.ovirt.thread.pool-7-thread-43) [3e5965ca] Running command: ReconstructMasterDomainCommand internal: true. Entities affected : ID: f3e372e3-1251-4195-a4b9-1027e40059df Type: Storage 2018-02-20 14:42:27,137+08 INFO [org.ovirt.engine.core. vdsbroker.irsbroker.ResetIrsVDSCommand] (org.ovirt.thread.pool-7-thread-43) [3e5965ca] START, ResetIrsVDSCommand( ResetIrsVDSCommandParameters:{runAsync='true', storagePoolId='5a865884-0366-0330-02b8-0000000002d4', ignoreFailoverLimit='false', vdsId='7dee35bb-8c97-4f6a-b6cd-abc4258540e4', ignoreStopFailed='true'}), log id: 3e0a239d 2018-02-20 14:42:27,140+08 INFO [org.ovirt.engine.core. vdsbroker.vdsbroker.SpmStopVDSCommand] (org.ovirt.thread.pool-7-thread-43) [3e5965ca] START, SpmStopVDSCommand(HostName = Node1, SpmStopVDSCommandParameters:{runAsync='true', hostId='7dee35bb-8c97-4f6a-b6cd-abc4258540e4', storagePoolId='5a865884-0366-0330-02b8-0000000002d4'}), log id: 7c67bf06 2018-02-20 14:42:28,144+08 INFO [org.ovirt.engine.core. vdsbroker.vdsbroker.SpmStopVDSCommand] (org.ovirt.thread.pool-7-thread-43) [3e5965ca] SpmStopVDSCommand::Stopping SPM on vds 'Node1', pool id '5a865884-0366-0330-02b8-0000000002d4'
On Tuesday, February 20, 2018 2:33 PM, michael pagdanganan < mhke_aj5566@yahoo.com> wrote:
Thanks for quick response,
see attachment.
On Tuesday, February 20, 2018 2:10 PM, Eyal Shenitzky <eshenitz@redhat.com> wrote:
Hi,
Can you please attach full Engine and VDSM logs?
Thanks,
On Tue, Feb 20, 2018 at 2:59 AM, michael pagdanganan < mhke_aj5566@yahoo.com> wrote:
My storage pool has 2 Master Domain(Stored2 on Node2,Node1Container on Node1). and my old master domain (DATANd01 on Node1 ) hung on preparing for maintenance. When I tried to activate my old master domain (DATANd01 on Node1 ) all storage domain goes down and up master keep on rotating.
Ovirt Version (oVirt Engine Version: 4.1.9.1.el7.centos)
Event Error: Sync Error on Master Domain between Host Node2 and oVirt Engine. Domain Stored2 is marked as master in ovirt engine Database but not on storage side Please consult with support
VDSM Node2 command ConnectStoragePoolVDS failed: Wrong Master domain or it's version: u'SD=f3e372e3-1251-4195-a4b9- 1027e40059df, pool=5a865884-0366-0330-02b8- 0000
VDSM Node2 command HSMGetAllTastsStatusesVDS failed: Not SPM: ()
Failed to deactivate Storage Domain DATANd01 (Data Center UnsecuredEnv)
Here's logs from engine:
------------------------------ ------------------------------ ------------------------------ ------------- [root@dev2engine ~]# tail /var/log/messages Feb 20 07:01:01 dev2engine systemd: Starting Session 20 of user root. Feb 20 07:01:01 dev2engine systemd: Removed slice User Slice of root. Feb 20 07:01:01 dev2engine systemd: Stopping User Slice of root. Feb 20 07:58:52 dev2engine systemd: Created slice User Slice of root. Feb 20 07:58:52 dev2engine systemd: Starting User Slice of root. Feb 20 07:58:52 dev2engine systemd-logind: New session 21 of user root. Feb 20 07:58:52 dev2engine systemd: Started Session 21 of user root. Feb 20 07:58:52 dev2engine systemd: Starting Session 21 of user root. Feb 20 08:01:01 dev2engine systemd: Started Session 22 of user root. Feb 20 08:01:01 dev2engine systemd: Starting Session 22 of user root.
------------------------------ ------------------------------ ------------------------------ ------------- [root@dev2engine ~]# tail /var/log/ovirt-engine/engine. log 2018-02-20 08:01:16,062+08 INFO [org.ovirt.engine.core.bll. eventqueue.EventQueueMonitor] (org.ovirt.thread.pool-7- thread-32) [102e9d3c] Finished reconstruct for pool '5a865884-0366-0330-02b8- 0000000002d4'. Clearing event queue 2018-02-20 08:01:27,825+08 WARN [org.ovirt.engine.core. vdsbroker.irsbroker.IrsProxy] (org.ovirt.thread.pool-7- thread-23) [] Master domain is not in sync between DB and VDSM. Domain Stored2 marked as master in DB and not in the storage 2018-02-20 08:01:27,862+08 WARN [org.ovirt.engine.core.bll. storage.pool. ReconstructMasterDomainCommand ] (org.ovirt.thread.pool-7- thread-23) [213f42b9] Validation of action 'ReconstructMasterDomain' failed for user SYSTEM. Reasons: VAR__ACTION__RECONSTRUCT_ MASTER,VAR__TYPE__STORAGE__ DOMAIN,ACTION_TYPE_FAILED_ STORAGE_DOMAIN_STATUS_ ILLEGAL2,$status PreparingForMaintenance 2018-02-20 08:01:27,882+08 INFO [org.ovirt.engine.core.bll. eventqueue.EventQueueMonitor] (org.ovirt.thread.pool-7- thread-20) [929330e] Finished reconstruct for pool '5a865884-0366-0330-02b8- 0000000002d4'. Clearing event queue 2018-02-20 08:01:40,106+08 WARN [org.ovirt.engine.core. vdsbroker.irsbroker.IrsProxy] (org.ovirt.thread.pool-7- thread-17) [] Master domain is not in sync between DB and VDSM. Domain Stored2 marked as master in DB and not in the storage 2018-02-20 08:01:40,197+08 WARN [org.ovirt.engine.core.bll. storage.pool. ReconstructMasterDomainCommand ] (org.ovirt.thread.pool-7- thread-17) [7af552c1] Validation of action 'ReconstructMasterDomain' failed for user SYSTEM. Reasons: VAR__ACTION__RECONSTRUCT_ MASTER,VAR__TYPE__STORAGE__ DOMAIN,ACTION_TYPE_FAILED_ STORAGE_DOMAIN_STATUS_ ILLEGAL2,$status PreparingForMaintenance 2018-02-20 08:01:40,246+08 INFO [org.ovirt.engine.core.bll. eventqueue.EventQueueMonitor] (org.ovirt.thread.pool-7- thread-22) [73673040] Finished reconstruct for pool '5a865884-0366-0330-02b8- 0000000002d4'. Clearing event queue 2018-02-20 08:01:51,809+08 WARN [org.ovirt.engine.core. vdsbroker.irsbroker.IrsProxy] (org.ovirt.thread.pool-7- thread-26) [] Master domain is not in sync between DB and VDSM. Domain Stored2 marked as master in DB and not in the storage 2018-02-20 08:01:51,846+08 WARN [org.ovirt.engine.core.bll. storage.pool. ReconstructMasterDomainCommand ] (org.ovirt.thread.pool-7- thread-26) [20307cbe] Validation of action 'ReconstructMasterDomain' failed for user SYSTEM. Reasons: VAR__ACTION__RECONSTRUCT_ MASTER,VAR__TYPE__STORAGE__ DOMAIN,ACTION_TYPE_FAILED_ STORAGE_DOMAIN_STATUS_ ILLEGAL2,$status PreparingForMaintenance 2018-02-20 08:01:51,866+08 INFO [org.ovirt.engine.core.bll. eventqueue.EventQueueMonitor] (org.ovirt.thread.pool-7- thread-49) [2c11a866] Finished reconstruct for pool '5a865884-0366-0330-02b8- 0000000002d4'. Clearing event queue
______________________________ _________________ Users mailing list Users@ovirt.org http://lists.ovirt.org/ mailman/listinfo/users <http://lists.ovirt.org/mailman/listinfo/users>
-- Regards, Eyal Shenitzky
-- Regards, Eyal Shenitzky