<html><head></head><body><div style="color:#000; background-color:#fff; font-family:Helvetica Neue, Helvetica, Arial, Lucida Grande, sans-serif;font-size:10px"><div id="yui_3_16_0_ym19_1_1519110083785_25183">Host</div><div id="yui_3_16_0_ym19_1_1519110083785_25181"><br></div><img class="ymail-preserve-class inline-image-guid-a3c3e3e2-8bc3-25f4-e933-6a74256a29a8 img-overlay-selected rte-inline-saved-image" src="cid:a6b8fe93-fd8c-52fd-6914-98681e6b93a2@yahoo.com" data-id="inline-image-guid-a3c3e3e2-8bc3-25f4-e933-6a74256a29a8" alt="Inline image" style="width:100%;max-width:800px;max-height:260px;" id="yui_3_16_0_ym19_1_1519110083785_25089"><br><div id="yui_3_16_0_ym19_1_1519110083785_25090"><span></span></div> <div class="qtdSeparateBR"><br><br></div><div class="yahoo_quoted" style="display: block;"> <div style="font-family: Helvetica Neue, Helvetica, Arial, Lucida Grande, sans-serif; font-size: 10px;"> <div style="font-family: HelveticaNeue, Helvetica Neue, Helvetica, Arial, Lucida Grande, sans-serif; font-size: 16px;"> <div dir="ltr"><font size="2" face="Arial"> On Tuesday, February 20, 2018 4:20 PM, michael pagdanganan <mhke_aj5566@yahoo.com> wrote:<br></font></div> <br><br> <div class="y_msg_container"><div id="yiv2961678542"><div><div style="color:#000;background-color:#fff;font-family:Helvetica Neue, Helvetica, Arial, Lucida Grande, sans-serif;font-size:10px;"><div>Datacenter</div><div id="yiv2961678542yui_3_16_0_ym19_1_1519110083785_23852"><br clear="none"></div><div id="yiv2961678542yui_3_16_0_ym19_1_1519110083785_23963"><img class="yiv2961678542ymail-preserve-class yiv2961678542inline-image-guid-d96babae-1066-e3aa-16a3-deb23b9ab2ab yiv2961678542rte-inline-saved-image" id="yiv2961678542yui_3_16_0_ym19_1_1519110083785_23905" src="cid:4f3dd916-a5fc-8ed7-36c6-c1f20ce72b3c@yahoo.com" yahoo_partid="3" alt="Inline image" style="width:100%;max-width:800px;max-height:356px;"></div><div id="yiv2961678542yui_3_16_0_ym19_1_1519110083785_23966"><br clear="none"></div><div id="yiv2961678542yui_3_16_0_ym19_1_1519110083785_23972">Host</div><div id="yiv2961678542yui_3_16_0_ym19_1_1519110083785_23999"><br clear="none"></div><div id="yiv2961678542yui_3_16_0_ym19_1_1519110083785_23850"><span></span></div> <div class="yiv2961678542qtdSeparateBR"><br clear="none"><br clear="none"></div><div class="yiv2961678542yqt7581189573" id="yiv2961678542yqt40720"><div class="yiv2961678542yahoo_quoted" style="display:block;"> <div style="font-family:Helvetica Neue, Helvetica, Arial, Lucida Grande, sans-serif;font-size:10px;"> <div style="font-family:HelveticaNeue, Helvetica Neue, Helvetica, Arial, Lucida Grande, sans-serif;font-size:16px;"> <div dir="ltr"><font size="2" face="Arial"> On Tuesday, February 20, 2018 3:35 PM, Eyal Shenitzky <eshenitz@redhat.com> wrote:<br clear="none"></font></div> <br clear="none"><br clear="none"> <div class="yiv2961678542y_msg_container"><div id="yiv2961678542"><div><div dir="ltr">What about the hosts status?<div><br clear="none"></div><div>Can you please send the hosts screenshot and the datacenter screenshot?</div></div><div class="yiv2961678542gmail_extra"><br clear="none"><div class="yiv2961678542yqt3303519494" id="yiv2961678542yqt45001"><div class="yiv2961678542gmail_quote">On Tue, Feb 20, 2018 at 9:03 AM, michael pagdanganan <span dir="ltr"><<a rel="nofollow" shape="rect" ymailto="mailto:mhke_aj5566@yahoo.com" target="_blank" href="mailto:mhke_aj5566@yahoo.com">mhke_aj5566@yahoo.com</a>></span> wrote:<br clear="none"><blockquote class="yiv2961678542gmail_quote" style="margin:0 0 0 .8ex;border-left:1px #ccc solid;padding-left:1ex;"><div><div style="color:#000;background-color:#fff;font-family:Helvetica Neue, Helvetica, Arial, Lucida Grande, sans-serif;font-size:10px;"><div><font size="3">Hi,</font></div><div id="yiv2961678542m_1856169086348407128yui_3_16_0_ym19_1_1519110083785_3194"><font size="3"><br clear="none"></font></div><div id="yiv2961678542m_1856169086348407128yui_3_16_0_ym19_1_1519110083785_3274"><font id="yiv2961678542m_1856169086348407128yui_3_16_0_ym19_1_1519110083785_3273" size="3">Screenshot</font></div><div id="yiv2961678542m_1856169086348407128yui_3_16_0_ym19_1_1519110083785_3276"><font id="yiv2961678542m_1856169086348407128yui_3_16_0_ym19_1_1519110083785_3273" size="3"><br clear="none"></font></div><div dir="ltr" id="yiv2961678542m_1856169086348407128yui_3_16_0_ym19_1_1519110083785_3277"><font id="yiv2961678542m_1856169086348407128yui_3_16_0_ym19_1_1519110083785_3273" size="3">only 1 data center but 2 master domain and got event warning<br clear="none"></font></div><div id="yiv2961678542m_1856169086348407128yui_3_16_0_ym19_1_1519110083785_3272"><font size="3"><br clear="none"></font></div><img class="yiv2961678542m_1856169086348407128ymail-preserve-class yiv2961678542m_1856169086348407128inline-image-guid-abe2d8b4-6e36-0d75-dc72-1b1b98af7fe7 yiv2961678542m_1856169086348407128rte-inline-saved-image yiv2961678542m_1856169086348407128img-overlay-selected" id="yiv2961678542m_1856169086348407128yui_3_16_0_ym19_1_1519110083785_3198" src="cid:4ce96275-93d6-d17e-214d-b1674fc574f1@yahoo.com" yahoo_partid="2" alt="Inline image" style="width:100%;max-width:800px;max-height:291px;"><br id="yiv2961678542m_1856169086348407128yui_3_16_0_ym19_1_1519110083785_3199" clear="none"><div id="yiv2961678542m_1856169086348407128yui_3_16_0_ym19_1_1519110083785_3064"><span></span></div> <div class="yiv2961678542m_1856169086348407128qtdSeparateBR"><br clear="none"><br clear="none"></div><div class="yiv2961678542m_1856169086348407128yahoo_quoted" style="display:block;"> <div style="font-family:Helvetica Neue, Helvetica, Arial, Lucida Grande, sans-serif;font-size:10px;"> <div style="font-family:HelveticaNeue, Helvetica Neue, Helvetica, Arial, Lucida Grande, sans-serif;font-size:16px;"> <div dir="ltr"><font size="2" face="Arial"> On Tuesday, February 20, 2018 2:56 PM, Eyal Shenitzky <<a rel="nofollow" shape="rect" ymailto="mailto:eshenitz@redhat.com" target="_blank" href="mailto:eshenitz@redhat.com">eshenitz@redhat.com</a>> wrote:<br clear="none"></font></div> <br clear="none"><br clear="none"> <div class="yiv2961678542m_1856169086348407128y_msg_container"><div id="yiv2961678542m_1856169086348407128yiv5069007669"><div><div dir="ltr">Also, can you please describe the current setup of the environment?<div><br clear="none"></div><div>I'm not sure I understand, do you have 2 data-centers?</div><div><br clear="none"></div><div>Please attach some screenshots of the current situation.</div></div><div class="yiv2961678542m_1856169086348407128yiv5069007669gmail_extra"><br clear="none"><div class="yiv2961678542m_1856169086348407128yiv5069007669yqt2982502881" id="yiv2961678542m_1856169086348407128yiv5069007669yqt87411"><div class="yiv2961678542m_1856169086348407128yiv5069007669gmail_quote">On Tue, Feb 20, 2018 at 8:43 AM, michael pagdanganan <span dir="ltr"><<a rel="nofollow" shape="rect" ymailto="mailto:mhke_aj5566@yahoo.com" target="_blank" href="mailto:mhke_aj5566@yahoo.com">mhke_aj5566@yahoo.com</a>></span> wrote:<br clear="none"><blockquote class="yiv2961678542m_1856169086348407128yiv5069007669gmail_quote" style="margin:0 0 0 .8ex;border-left:1px #ccc solid;padding-left:1ex;"><div><div style="color:#000;background-color:#fff;font-family:Helvetica Neue, Helvetica, Arial, Lucida Grande, sans-serif;font-size:10px;"><div id="yiv2961678542m_1856169086348407128yiv5069007669m_-8653308167440808248yui_3_16_0_ym19_1_1519085807369_70843">Sorry can't attached log file it's too big file<br clear="none"></div><div id="yiv2961678542m_1856169086348407128yiv5069007669m_-8653308167440808248yui_3_16_0_ym19_1_1519085807369_70846"><br clear="none"></div><div><br clear="none"></div><div>VDSM.log for node 1</div><div id="yiv2961678542m_1856169086348407128yiv5069007669m_-8653308167440808248yui_3_16_0_ym19_1_1519085807369_70844"><br clear="none"></div><div dir="ltr" id="yiv2961678542m_1856169086348407128yiv5069007669m_-8653308167440808248yui_3_16_0_ym19_1_1519085807369_70530">2766', 'lastCheck': '4.9', 'valid': True}} from=internal, task_id=645d456e-f59f-4b1c- 9e97-fc82d19a36b1 (api:52)<br id="yiv2961678542m_1856169086348407128yiv5069007669m_-8653308167440808248yui_3_16_0_ym19_1_1519085807369_70565" clear="none">2018-02-20 14:38:47,222+0800 INFO (jsonrpc/3) [vdsm.api] START repoStats(options=None) from=::ffff:10.10.43.1,60554, flow_id=3b2e802e, task_id=28c795d1-1639-4e68- a9fe-a00006be268f (api:46)<br id="yiv2961678542m_1856169086348407128yiv5069007669m_-8653308167440808248yui_3_16_0_ym19_1_1519085807369_70566" clear="none">2018-02-20 14:38:47,222+0800 INFO (jsonrpc/3) [vdsm.api] FINISH repoStats return={'f3e372e3-1251-4195- a4b9-1027e40059df': {'code': 0, 'actual': True, 'version': 4, 'acquired': True, 'delay': '0.000119109', 'lastCheck': '6.0', 'valid': True}, 'e83d0d46-6ea6-4aa3-80bf- 6e95c66b0454': {'code': 0, 'actual': True, 'version': 0, 'acquired': True, 'delay': '0.000195263', 'lastCheck': '6.0', 'valid': True}, '65ca2e2d-b472-4bee-85b4- 09a161464b20': {'code': 0, 'actual': True, 'version': 4, 'acquired': True, 'delay': '0.00022988', 'lastCheck': '6.0', 'valid': True}, '4bf2ba2f-f57a-4d9f-b42a- fb78f440a358': {'code': 0, 'actual': True, 'version': 0, 'acquired': True, 'delay': '0.000270206', 'lastCheck': '5.9', 'valid': True}, '42e591b7-f86c-4b67-a3d2- 40cc007f7662': {'code': 0, 'actual': True, 'version': 4, 'acquired': True, 'delay': '0.000272766', 'lastCheck': '6.0', 'valid': True}} from=::ffff:10.10.43.1,60554, flow_id=3b2e802e, task_id=28c795d1-1639-4e68- a9fe-a00006be268f (api:52)<br id="yiv2961678542m_1856169086348407128yiv5069007669m_-8653308167440808248yui_3_16_0_ym19_1_1519085807369_70567" clear="none">2018-02-20 14:38:47,226+0800 INFO (jsonrpc/3) [jsonrpc.JsonRpcServer] RPC call Host.getStats succeeded in 0.00 seconds (__init__:539)<br id="yiv2961678542m_1856169086348407128yiv5069007669m_-8653308167440808248yui_3_16_0_ym19_1_1519085807369_70568" clear="none">2018-02-20 14:38:55,252+0800 INFO (jsonrpc/4) [jsonrpc.JsonRpcServer] RPC call Host.getAllVmStats succeeded in 0.00 seconds (__init__:539)<br id="yiv2961678542m_1856169086348407128yiv5069007669m_-8653308167440808248yui_3_16_0_ym19_1_1519085807369_70569" clear="none">2018-02-20 14:38:58,566+0800 INFO (jsonrpc/2) [jsonrpc.JsonRpcServer] RPC call Host.getAllVmStats succeeded in 0.00 seconds (__init__:539)<br id="yiv2961678542m_1856169086348407128yiv5069007669m_-8653308167440808248yui_3_16_0_ym19_1_1519085807369_70570" clear="none">2018-02-20 14:39:01,093+0800 INFO (periodic/1) [vdsm.api] START repoStats(options=None) from=internal, task_id=04f55ded-5841-44ae- a376-4f6e723e4b10 (api:46)<br id="yiv2961678542m_1856169086348407128yiv5069007669m_-8653308167440808248yui_3_16_0_ym19_1_1519085807369_70571" clear="none">2018-02-20 14:39:01,093+0800 INFO (periodic/1) [vdsm.api] FINISH repoStats return={'f3e372e3-1251-4195- a4b9-1027e40059df': {'code': 0, 'actual': True, 'version': 4, 'acquired': True, 'delay': '9.3888e-05', 'lastCheck': '9.9', 'valid': True}, 'e83d0d46-6ea6-4aa3-80bf- 6e95c66b0454': {'code': 0, 'actual': True, 'version': 0, 'acquired': True, 'delay': '0.000197535', 'lastCheck': '9.8', 'valid': True}, '65ca2e2d-b472-4bee-85b4- 09a161464b20': {'code': 0, 'actual': True, 'version': 4, 'acquired': True, 'delay': '0.000191456', 'lastCheck': '9.8', 'valid': True}, '4bf2ba2f-f57a-4d9f-b42a- fb78f440a358': {'code': 0, 'actual': True, 'version': 0, 'acquired': True, 'delay': '0.00026365', 'lastCheck': '9.8', 'valid': True}, '42e591b7-f86c-4b67-a3d2- 40cc007f7662': {'code': 0, 'actual': True, 'version': 4, 'acquired': True, 'delay': '0.000243658', 'lastCheck': '0.0', 'valid': True}} from=internal, task_id=04f55ded-5841-44ae- a376-4f6e723e4b10 (api:52)<br id="yiv2961678542m_1856169086348407128yiv5069007669m_-8653308167440808248yui_3_16_0_ym19_1_1519085807369_70572" clear="none">2018-02-20 14:39:02,295+0800 INFO (jsonrpc/6) [vdsm.api] START repoStats(options=None) from=::ffff:10.10.43.1,60554, flow_id=13149d6e, task_id=5f19d76a-d343-4dc4- ad09-024ec27f7443 (api:46)<br id="yiv2961678542m_1856169086348407128yiv5069007669m_-8653308167440808248yui_3_16_0_ym19_1_1519085807369_70573" clear="none">2018-02-20 14:39:02,295+0800 INFO (jsonrpc/6) [vdsm.api] FINISH repoStats return={'f3e372e3-1251-4195- a4b9-1027e40059df': {'code': 0, 'actual': True, 'version': 4, 'acquired': True, 'delay': '9.4759e-05', 'lastCheck': '1.1', 'valid': True}, 'e83d0d46-6ea6-4aa3-80bf- 6e95c66b0454': {'code': 0, 'actual': True, 'version': 0, 'acquired': True, 'delay': '0.000183158', 'lastCheck': '1.0', 'valid': True}, '65ca2e2d-b472-4bee-85b4- 09a161464b20': {'code': 0, 'actual': True, 'version': 4, 'acquired': True, 'delay': '0.000222609', 'lastCheck': '1.0', 'valid': True}, '4bf2ba2f-f57a-4d9f-b42a- fb78f440a358': {'code': 0, 'actual': True, 'version': 0, 'acquired': True, 'delay': '0.000211253', 'lastCheck': '1.0', 'valid': True}, '42e591b7-f86c-4b67-a3d2- 40cc007f7662': {'code': 0, 'actual': True, 'version': 4, 'acquired': True, 'delay': '0.000243658', 'lastCheck': '1.1', 'valid': True}} from=::ffff:10.10.43.1,60554, flow_id=13149d6e, task_id=5f19d76a-d343-4dc4- ad09-024ec27f7443 (api:52)<br id="yiv2961678542m_1856169086348407128yiv5069007669m_-8653308167440808248yui_3_16_0_ym19_1_1519085807369_70574" clear="none">2018-02-20 14:39:02,300+0800 INFO (jsonrpc/6) [jsonrpc.JsonRpcServer] RPC call Host.getStats succeeded in 0.01 seconds (__init__:539)<br id="yiv2961678542m_1856169086348407128yiv5069007669m_-8653308167440808248yui_3_16_0_ym19_1_1519085807369_70575" clear="none">2018-02-20 14:39:10,270+0800 INFO (jsonrpc/1) [jsonrpc.JsonRpcServer] RPC call Host.getAllVmStats succeeded in 0.00 seconds (__init__:539)<br id="yiv2961678542m_1856169086348407128yiv5069007669m_-8653308167440808248yui_3_16_0_ym19_1_1519085807369_70576" clear="none">2018-02-20 14:39:13,631+0800 INFO (jsonrpc/0) [vdsm.api] START connectStorageServer(domType= 1, spUUID=u'00000000-0000-0000- 0000-000000000000', conList=[{u'id': u'5e4c94f1-f3b9-4fbd-a6c7- e732d0fe3123', u'connection': u'dev2node1.lares.com.ph:/run/ media/root/Slave1Data/ dataNode1', u'iqn': u'', u'user': u'', u'tpgt': u'1', u'protocol_version': u'auto', u'password': '********', u'port': u''}], options=None) from=::ffff:10.10.43.1,60554, flow_id=15b57417, task_id=bfce1e70-ef4a-4e13- aaaa-7a66aaf44429 (api:46)<br id="yiv2961678542m_1856169086348407128yiv5069007669m_-8653308167440808248yui_3_16_0_ym19_1_1519085807369_70577" clear="none">2018-02-20 14:39:13,633+0800 INFO (jsonrpc/0) [vdsm.api] FINISH connectStorageServer return={'statuslist': [{'status': 0, 'id': u'5e4c94f1-f3b9-4fbd-a6c7- e732d0fe3123'}]} from=::ffff:10.10.43.1,60554, flow_id=15b57417, task_id=bfce1e70-ef4a-4e13- aaaa-7a66aaf44429 (api:52)<br id="yiv2961678542m_1856169086348407128yiv5069007669m_-8653308167440808248yui_3_16_0_ym19_1_1519085807369_70578" clear="none">2018-02-20 14:39:13,634+0800 INFO (jsonrpc/0) [jsonrpc.JsonRpcServer] RPC call StoragePool. connectStorageServer succeeded in 0.00 seconds (__init__:539)<br id="yiv2961678542m_1856169086348407128yiv5069007669m_-8653308167440808248yui_3_16_0_ym19_1_1519085807369_70579" clear="none">2018-02-20 14:39:13,806+0800 INFO (jsonrpc/7) [vdsm.api] START connectStoragePool(spUUID=u' 5a865884-0366-0330-02b8- 0000000002d4', hostID=1, msdUUID=u'f3e372e3-1251-4195- a4b9-1027e40059df', masterVersion=65, domainsMap={u'e83d0d46-6ea6- 4aa3-80bf-6e95c66b0454': u'active', u'4bf2ba2f-f57a-4d9f-b42a- fb78f440a358': u'active', u'225e1975-8121-4370-b317- 86e964ae326f': u'attached', u'f3e372e3-1251-4195-a4b9- 1027e40059df': u'active', u'65ca2e2d-b472-4bee-85b4- 09a161464b20': u'active', u'42e591b7-f86c-4b67-a3d2- 40cc007f7662': u'active'}, options=None) from=::ffff:10.10.43.1,60554, flow_id=15b57417, task_id=927a5d9a-4304-4776- b5c9-22ba5d0cb853 (api:46)<br id="yiv2961678542m_1856169086348407128yiv5069007669m_-8653308167440808248yui_3_16_0_ym19_1_1519085807369_70580" clear="none">2018-02-20 14:39:13,807+0800 INFO (jsonrpc/7) [storage. StoragePoolMemoryBackend] new storage pool master version 65 and domains map {u'e83d0d46-6ea6-4aa3-80bf- 6e95c66b0454': u'Active', u'4bf2ba2f-f57a-4d9f-b42a- fb78f440a358': u'Active', u'225e1975-8121-4370-b317- 86e964ae326f': u'Attached', u'f3e372e3-1251-4195-a4b9- 1027e40059df': u'Active', u'65ca2e2d-b472-4bee-85b4- 09a161464b20': u'Active', u'42e591b7-f86c-4b67-a3d2- 40cc007f7662': u'Active'} (spbackends:450)<br id="yiv2961678542m_1856169086348407128yiv5069007669m_-8653308167440808248yui_3_16_0_ym19_1_1519085807369_70581" clear="none"></div><div dir="ltr" id="yiv2961678542m_1856169086348407128yiv5069007669m_-8653308167440808248yui_3_16_0_ym19_1_1519085807369_70649"><br clear="none"></div><div dir="ltr" id="yiv2961678542m_1856169086348407128yiv5069007669m_-8653308167440808248yui_3_16_0_ym19_1_1519085807369_70648">VDSM.log 2</div><div dir="ltr" id="yiv2961678542m_1856169086348407128yiv5069007669m_-8653308167440808248yui_3_16_0_ym19_1_1519085807369_70650">2018-02-20 14:41:14,598+0800 INFO (periodic/3) [vdsm.api] FINISH repoStats return={'f3e372e3-1251-4195- a4b9-1027e40059df': {'code': 0, 'actual': True, 'version': 4, 'acquired': True, 'delay': '0.000252423', 'lastCheck': '6.0', 'valid': True}, 'e83d0d46-6ea6-4aa3-80bf- 6e95c66b0454': {'code': 0, 'actual': True, 'version': 0, 'acquired': True, 'delay': '7.5567e-05', 'lastCheck': '6.0', 'valid': True}, '65ca2e2d-b472-4bee-85b4- 09a161464b20': {'code': 0, 'actual': True, 'version': 4, 'acquired': True, 'delay': '8.2433e-05', 'lastCheck': '6.0', 'valid': True}, '4bf2ba2f-f57a-4d9f-b42a- fb78f440a358': {'code': 0, 'actual': True, 'version': 0, 'acquired': True, 'delay': '0.000237247', 'lastCheck': '5.9', 'valid': True}, '42e591b7-f86c-4b67-a3d2- 40cc007f7662': {'code': 0, 'actual': True, 'version': 4, 'acquired': True, 'delay': '8.4376e-05', 'lastCheck': '6.0', 'valid': True}} from=internal, task_id=6213712b-9903-4db8- 9836-3baf85cd63e4 (api:52)<br id="yiv2961678542m_1856169086348407128yiv5069007669m_-8653308167440808248yui_3_16_0_ym19_1_1519085807369_70683" clear="none">2018-02-20 14:41:18,074+0800 INFO (jsonrpc/6) [jsonrpc.JsonRpcServer] RPC call Host.getAllVmStats succeeded in 0.00 seconds (__init__:539)<br id="yiv2961678542m_1856169086348407128yiv5069007669m_-8653308167440808248yui_3_16_0_ym19_1_1519085807369_70684" clear="none">2018-02-20 14:41:24,333+0800 INFO (jsonrpc/4) [vdsm.api] START repoStats(options=None) from=::ffff:10.10.43.1,56540, flow_id=73f86113, task_id=e8231ecb-3543-4d8f- af54-4cf2b06ee98a (api:46)<br id="yiv2961678542m_1856169086348407128yiv5069007669m_-8653308167440808248yui_3_16_0_ym19_1_1519085807369_70685" clear="none">2018-02-20 14:41:24,334+0800 INFO (jsonrpc/4) [vdsm.api] FINISH repoStats return={'f3e372e3-1251-4195- a4b9-1027e40059df': {'code': 0, 'actual': True, 'version': 4, 'acquired': True, 'delay': '0.000172953', 'lastCheck': '5.7', 'valid': True}, 'e83d0d46-6ea6-4aa3-80bf- 6e95c66b0454': {'code': 0, 'actual': True, 'version': 0, 'acquired': True, 'delay': '7.701e-05', 'lastCheck': '5.7', 'valid': True}, '65ca2e2d-b472-4bee-85b4- 09a161464b20': {'code': 0, 'actual': True, 'version': 4, 'acquired': True, 'delay': '0.000117677', 'lastCheck': '5.7', 'valid': True}, '4bf2ba2f-f57a-4d9f-b42a- fb78f440a358': {'code': 0, 'actual': True, 'version': 0, 'acquired': True, 'delay': '0.000185146', 'lastCheck': '5.7', 'valid': True}, '42e591b7-f86c-4b67-a3d2- 40cc007f7662': {'code': 0, 'actual': True, 'version': 4, 'acquired': True, 'delay': '9.1467e-05', 'lastCheck': '5.8', 'valid': True}} from=::ffff:10.10.43.1,56540, flow_id=73f86113, task_id=e8231ecb-3543-4d8f- af54-4cf2b06ee98a (api:52)<br id="yiv2961678542m_1856169086348407128yiv5069007669m_-8653308167440808248yui_3_16_0_ym19_1_1519085807369_70686" clear="none">2018-02-20 14:41:24,338+0800 INFO (jsonrpc/4) [jsonrpc.JsonRpcServer] RPC call Host.getStats succeeded in 0.00 seconds (__init__:539)<br id="yiv2961678542m_1856169086348407128yiv5069007669m_-8653308167440808248yui_3_16_0_ym19_1_1519085807369_70687" clear="none">2018-02-20 14:41:27,087+0800 INFO (jsonrpc/0) [jsonrpc.JsonRpcServer] RPC call Host.getAllVmStats succeeded in 0.00 seconds (__init__:539)<br id="yiv2961678542m_1856169086348407128yiv5069007669m_-8653308167440808248yui_3_16_0_ym19_1_1519085807369_70688" clear="none">2018-02-20 14:41:29,607+0800 INFO (periodic/1) [vdsm.api] START repoStats(options=None) from=internal, task_id=f14b7aab-b64a-4903- 9368-d665e39b49d1 (api:46)<br id="yiv2961678542m_1856169086348407128yiv5069007669m_-8653308167440808248yui_3_16_0_ym19_1_1519085807369_70689" clear="none">2018-02-20 14:41:29,608+0800 INFO (periodic/1) [vdsm.api] FINISH repoStats return={'f3e372e3-1251-4195- a4b9-1027e40059df': {'code': 0, 'actual': True, 'version': 4, 'acquired': True, 'delay': '0.000246237', 'lastCheck': '1.0', 'valid': True}, 'e83d0d46-6ea6-4aa3-80bf- 6e95c66b0454': {'code': 0, 'actual': True, 'version': 0, 'acquired': True, 'delay': '8.8773e-05', 'lastCheck': '1.0', 'valid': True}, '65ca2e2d-b472-4bee-85b4- 09a161464b20': {'code': 0, 'actual': True, 'version': 4, 'acquired': True, 'delay': '9.7145e-05', 'lastCheck': '1.0', 'valid': True}, '4bf2ba2f-f57a-4d9f-b42a- fb78f440a358': {'code': 0, 'actual': True, 'version': 0, 'acquired': True, 'delay': '0.000218729', 'lastCheck': '0.9', 'valid': True}, '42e591b7-f86c-4b67-a3d2- 40cc007f7662': {'code': 0, 'actual': True, 'version': 4, 'acquired': True, 'delay': '9.1035e-05', 'lastCheck': '1.0', 'valid': True}} from=internal, task_id=f14b7aab-b64a-4903- 9368-d665e39b49d1 (api:52)<br id="yiv2961678542m_1856169086348407128yiv5069007669m_-8653308167440808248yui_3_16_0_ym19_1_1519085807369_70690" clear="none">2018-02-20 14:41:33,079+0800 INFO (jsonrpc/7) [jsonrpc.JsonRpcServer] RPC call Host.getAllVmStats succeeded in 0.00 seconds (__init__:539)<br id="yiv2961678542m_1856169086348407128yiv5069007669m_-8653308167440808248yui_3_16_0_ym19_1_1519085807369_70691" clear="none">2018-02-20 14:41:40,456+0800 INFO (jsonrpc/2) [vdsm.api] START repoStats(options=None) from=::ffff:10.10.43.1,56540, flow_id=3be8150c, task_id=1b0c86a8-6fd8-4882- a742-fbd56ccb4037 (api:46)<br id="yiv2961678542m_1856169086348407128yiv5069007669m_-8653308167440808248yui_3_16_0_ym19_1_1519085807369_70692" clear="none">2018-02-20 14:41:40,457+0800 INFO (jsonrpc/2) [vdsm.api] FINISH repoStats return={'f3e372e3-1251-4195- a4b9-1027e40059df': {'code': 0, 'actual': True, 'version': 4, 'acquired': True, 'delay': '0.000174027', 'lastCheck': '1.8', 'valid': True}, 'e83d0d46-6ea6-4aa3-80bf- 6e95c66b0454': {'code': 0, 'actual': True, 'version': 0, 'acquired': True, 'delay': '0.00011454', 'lastCheck': '1.8', 'valid': True}, '65ca2e2d-b472-4bee-85b4- 09a161464b20': {'code': 0, 'actual': True, 'version': 4, 'acquired': True, 'delay': '0.000145955', 'lastCheck': '1.8', 'valid': True}, '4bf2ba2f-f57a-4d9f-b42a- fb78f440a358': {'code': 0, 'actual': True, 'version': 0, 'acquired': True, 'delay': '0.000209871', 'lastCheck': '1.8', 'valid': True}, '42e591b7-f86c-4b67-a3d2- 40cc007f7662': {'code': 0, 'actual': True, 'version': 4, 'acquired': True, 'delay': '9.5744e-05', 'lastCheck': '1.9', 'valid': True}} from=::ffff:10.10.43.1,56540, flow_id=3be8150c, task_id=1b0c86a8-6fd8-4882- a742-fbd56ccb4037 (api:52)<br id="yiv2961678542m_1856169086348407128yiv5069007669m_-8653308167440808248yui_3_16_0_ym19_1_1519085807369_70693" clear="none">2018-02-20 14:41:40,461+0800 INFO (jsonrpc/2) [jsonrpc.JsonRpcServer] RPC call Host.getStats succeeded in 0.01 seconds (__init__:539)<br id="yiv2961678542m_1856169086348407128yiv5069007669m_-8653308167440808248yui_3_16_0_ym19_1_1519085807369_70694" clear="none">2018-02-20 14:41:42,106+0800 INFO (jsonrpc/3) [jsonrpc.JsonRpcServer] RPC call Host.getAllVmStats succeeded in 0.00 seconds (__init__:539)<br id="yiv2961678542m_1856169086348407128yiv5069007669m_-8653308167440808248yui_3_16_0_ym19_1_1519085807369_70695" clear="none">2018-02-20 14:41:44,622+0800 INFO (periodic/3) [vdsm.api] START repoStats(options=None) from=internal, task_id=20e0b29b-d3cd-4e44- b92c-213f9c984ab2 (api:46)<br id="yiv2961678542m_1856169086348407128yiv5069007669m_-8653308167440808248yui_3_16_0_ym19_1_1519085807369_70696" clear="none">2018-02-20 14:41:44,622+0800 INFO (periodic/3) [vdsm.api] FINISH repoStats return={'f3e372e3-1251-4195- a4b9-1027e40059df': {'code': 0, 'actual': True, 'version': 4, 'acquired': True, 'delay': '0.000174027', 'lastCheck': '6.0', 'valid': True}, 'e83d0d46-6ea6-4aa3-80bf- 6e95c66b0454': {'code': 0, 'actual': True, 'version': 0, 'acquired': True, 'delay': '0.00011454', 'lastCheck': '6.0', 'valid': True}, '65ca2e2d-b472-4bee-85b4- 09a161464b20': {'code': 0, 'actual': True, 'version': 4, 'acquired': True, 'delay': '0.000145955', 'lastCheck': '6.0', 'valid': True}, '4bf2ba2f-f57a-4d9f-b42a- fb78f440a358': {'code': 0, 'actual': True, 'version': 0, 'acquired': True, 'delay': '0.000209871', 'lastCheck': '5.9', 'valid': True}, '42e591b7-f86c-4b67-a3d2- 40cc007f7662': {'code': 0, 'actual': True, 'version': 4, 'acquired': True, 'delay': '9.5744e-05', 'lastCheck': '6.0', 'valid': True}} from=internal, task_id=20e0b29b-d3cd-4e44- b92c-213f9c984ab2 (api:52)<br id="yiv2961678542m_1856169086348407128yiv5069007669m_-8653308167440808248yui_3_16_0_ym19_1_1519085807369_70697" clear="none">2018-02-20 14:41:49,083+0800 INFO (jsonrpc/1) [jsonrpc.JsonRpcServer] RPC call Host.getAllVmStats succeeded in 0.00 seconds (__init__:539)<br id="yiv2961678542m_1856169086348407128yiv5069007669m_-8653308167440808248yui_3_16_0_ym19_1_1519085807369_70698" clear="none"></div><div dir="ltr"><br clear="none"></div><div dir="ltr" id="yiv2961678542m_1856169086348407128yiv5069007669m_-8653308167440808248yui_3_16_0_ym19_1_1519085807369_70767"><br clear="none"></div><div dir="ltr" id="yiv2961678542m_1856169086348407128yiv5069007669m_-8653308167440808248yui_3_16_0_ym19_1_1519085807369_70768">Engine log</div><div dir="ltr"><br clear="none"></div><div dir="ltr" id="yiv2961678542m_1856169086348407128yiv5069007669m_-8653308167440808248yui_3_16_0_ym19_1_1519085807369_70769">skId '404ccecc-aa7f-45ea-89e4- 726956269bc9' task status 'finished'<br id="yiv2961678542m_1856169086348407128yiv5069007669m_-8653308167440808248yui_3_16_0_ym19_1_1519085807369_70808" clear="none">2018-02-20 14:42:21,966+08 INFO [org.ovirt.engine.core. vdsbroker.vdsbroker. SpmStartVDSCommand] (org.ovirt.thread.pool-7- thread-38) [29528f9] spmStart polling ended, spm status: SPM<br id="yiv2961678542m_1856169086348407128yiv5069007669m_-8653308167440808248yui_3_16_0_ym19_1_1519085807369_70809" clear="none">2018-02-20 14:42:21,967+08 INFO [org.ovirt.engine.core. vdsbroker.vdsbroker. HSMClearTaskVDSCommand] (org.ovirt.thread.pool-7- thread-38) [29528f9] START, HSMClearTaskVDSCommand( HostName = Node1, HSMTaskGuidBaseVDSCommandParam eters:{runAsync='true', hostId='7dee35bb-8c97-4f6a- b6cd-abc4258540e4', taskId='404ccecc-aa7f-45ea- 89e4-726956269bc9'}), log id: 71688f70<br id="yiv2961678542m_1856169086348407128yiv5069007669m_-8653308167440808248yui_3_16_0_ym19_1_1519085807369_70810" clear="none">2018-02-20 14:42:22,922+08 INFO [org.ovirt.engine.core. vdsbroker.vdsbroker. HSMClearTaskVDSCommand] (org.ovirt.thread.pool-7- thread-38) [29528f9] FINISH, HSMClearTaskVDSCommand, log id: 71688f70<br id="yiv2961678542m_1856169086348407128yiv5069007669m_-8653308167440808248yui_3_16_0_ym19_1_1519085807369_70811" clear="none">2018-02-20 14:42:22,923+08 INFO [org.ovirt.engine.core. vdsbroker.vdsbroker. SpmStartVDSCommand] (org.ovirt.thread.pool-7- thread-38) [29528f9] FINISH, SpmStartVDSCommand, return: org.ovirt.engine.core.common. businessentities. SpmStatusResult@78332453, log id: 3ea35d5<br id="yiv2961678542m_1856169086348407128yiv5069007669m_-8653308167440808248yui_3_16_0_ym19_1_1519085807369_70812" clear="none">2018-02-20 14:42:22,935+08 INFO [org.ovirt.engine.core. vdsbroker.irsbroker.IrsProxy] (org.ovirt.thread.pool-7- thread-38) [29528f9] Initialize Irs proxy from vds: <a rel="nofollow" shape="rect" target="_blank" href="http://dev2node1.lares.com.ph/">dev2node1.lares.com.ph</a><br id="yiv2961678542m_1856169086348407128yiv5069007669m_-8653308167440808248yui_3_16_0_ym19_1_1519085807369_70813" clear="none">2018-02-20 14:42:22,951+08 INFO [org.ovirt.engine.core.dal. dbbroker.auditloghandling. AuditLogDirector] (org.ovirt.thread.pool-7- thread-38) [29528f9] EVENT_ID: IRS_HOSTED_ON_VDS(204), Correlation ID: null, Call Stack: null, Custom ID: null, Custom Event ID: -1, Message: Storage Pool Manager runs on Host Node1 (Address: <a rel="nofollow" shape="rect" target="_blank" href="http://dev2node1.lares.com.ph/">dev2node1.lares.com.ph</a>).<br id="yiv2961678542m_1856169086348407128yiv5069007669m_-8653308167440808248yui_3_16_0_ym19_1_1519085807369_70814" clear="none">2018-02-20 14:42:22,952+08 INFO [org.ovirt.engine.core. vdsbroker.irsbroker. SPMGetAllTasksInfoVDSCommand] (org.ovirt.thread.pool-7- thread-38) [29528f9] -- executeIrsBrokerCommand: Attempting on storage pool '5a865884-0366-0330-02b8- 0000000002d4'<br id="yiv2961678542m_1856169086348407128yiv5069007669m_-8653308167440808248yui_3_16_0_ym19_1_1519085807369_70815" clear="none">2018-02-20 14:42:22,952+08 INFO [org.ovirt.engine.core. vdsbroker.vdsbroker. HSMGetAllTasksInfoVDSCommand] (org.ovirt.thread.pool-7- thread-38) [29528f9] START, HSMGetAllTasksInfoVDSCommand( HostName = Node1, VdsIdVDSCommandParametersBase: {runAsync='true', hostId='7dee35bb-8c97-4f6a- b6cd-abc4258540e4'}), log id: 1bdbea9d<br id="yiv2961678542m_1856169086348407128yiv5069007669m_-8653308167440808248yui_3_16_0_ym19_1_1519085807369_70816" clear="none">2018-02-20 14:42:22,955+08 INFO [org.ovirt.engine.core. vdsbroker.irsbroker. SPMGetAllTasksInfoVDSCommand] (org.ovirt.thread.pool-7- thread-7) [29528f9] START, SPMGetAllTasksInfoVDSCommand( IrsBaseVDSCommandParameters:{ runAsync='true', storagePoolId='5a865884-0366- 0330-02b8-0000000002d4', ignoreFailoverLimit='false'}), log id: 5c2422d6<br id="yiv2961678542m_1856169086348407128yiv5069007669m_-8653308167440808248yui_3_16_0_ym19_1_1519085807369_70817" clear="none">2018-02-20 14:42:23,956+08 INFO [org.ovirt.engine.core. vdsbroker.vdsbroker. HSMGetAllTasksInfoVDSCommand] (org.ovirt.thread.pool-7- thread-38) [29528f9] FINISH, HSMGetAllTasksInfoVDSCommand, return: [], log id: 1bdbea9d<br id="yiv2961678542m_1856169086348407128yiv5069007669m_-8653308167440808248yui_3_16_0_ym19_1_1519085807369_70818" clear="none">2018-02-20 14:42:23,956+08 INFO [org.ovirt.engine.core. vdsbroker.irsbroker. SPMGetAllTasksInfoVDSCommand] (org.ovirt.thread.pool-7- thread-38) [29528f9] FINISH, SPMGetAllTasksInfoVDSCommand, return: [], log id: 58cbe7b7<br id="yiv2961678542m_1856169086348407128yiv5069007669m_-8653308167440808248yui_3_16_0_ym19_1_1519085807369_70819" clear="none">2018-02-20 14:42:23,956+08 INFO [org.ovirt.engine.core.bll. tasks.AsyncTaskManager] (org.ovirt.thread.pool-7- thread-38) [29528f9] Discovered no tasks on Storage Pool 'UnsecuredEnv'<br id="yiv2961678542m_1856169086348407128yiv5069007669m_-8653308167440808248yui_3_16_0_ym19_1_1519085807369_70820" clear="none">2018-02-20 14:42:24,936+08 INFO [org.ovirt.vdsm.jsonrpc. client.reactors.ReactorClient] (SSL Stomp Reactor) [] Connecting to <a rel="nofollow" shape="rect" target="_blank" href="http://dev2node1.lares.com.ph/10.10.43.2">dev2node1.lares.com.ph/10.10. 43.2</a><br id="yiv2961678542m_1856169086348407128yiv5069007669m_-8653308167440808248yui_3_16_0_ym19_1_1519085807369_70821" clear="none">2018-02-20 14:42:27,012+08 WARN [org.ovirt.engine.core. vdsbroker.irsbroker.IrsProxy] (org.ovirt.thread.pool-7- thread-43) [] Master domain is not in sync between DB and VDSM. Domain Node1Container marked as master in DB and not in the storage<br id="yiv2961678542m_1856169086348407128yiv5069007669m_-8653308167440808248yui_3_16_0_ym19_1_1519085807369_70822" clear="none">2018-02-20 14:42:27,026+08 WARN [org.ovirt.engine.core.dal. dbbroker.auditloghandling. AuditLogDirector] (org.ovirt.thread.pool-7- thread-43) [] EVENT_ID: SYSTEM_MASTER_DOMAIN_NOT_IN_ SYNC(990), Correlation ID: null, Call Stack: null, Custom ID: null, Custom Event ID: -1, Message: Sync Error on Master Domain between Host Node1 and oVirt Engine. Domain: Node1Container is marked as Master in oVirt Engine database but not on the Storage side. Please consult with Support on how to fix this issue.<br id="yiv2961678542m_1856169086348407128yiv5069007669m_-8653308167440808248yui_3_16_0_ym19_1_1519085807369_70823" clear="none">2018-02-20 14:42:27,103+08 INFO [org.ovirt.engine.core.bll. storage.pool. ReconstructMasterDomainCommand ] (org.ovirt.thread.pool-7- thread-43) [3e5965ca] Running command: ReconstructMasterDomainCommand internal: true. Entities affected : ID: f3e372e3-1251-4195-a4b9- 1027e40059df Type: Storage<br id="yiv2961678542m_1856169086348407128yiv5069007669m_-8653308167440808248yui_3_16_0_ym19_1_1519085807369_70824" clear="none">2018-02-20 14:42:27,137+08 INFO [org.ovirt.engine.core. vdsbroker.irsbroker. ResetIrsVDSCommand] (org.ovirt.thread.pool-7- thread-43) [3e5965ca] START, ResetIrsVDSCommand( ResetIrsVDSCommandParameters:{ runAsync='true', storagePoolId='5a865884-0366- 0330-02b8-0000000002d4', ignoreFailoverLimit='false', vdsId='7dee35bb-8c97-4f6a- b6cd-abc4258540e4', ignoreStopFailed='true'}), log id: 3e0a239d<br id="yiv2961678542m_1856169086348407128yiv5069007669m_-8653308167440808248yui_3_16_0_ym19_1_1519085807369_70825" clear="none">2018-02-20 14:42:27,140+08 INFO [org.ovirt.engine.core. vdsbroker.vdsbroker. SpmStopVDSCommand] (org.ovirt.thread.pool-7- thread-43) [3e5965ca] START, SpmStopVDSCommand(HostName = Node1, SpmStopVDSCommandParameters:{ runAsync='true', hostId='7dee35bb-8c97-4f6a- b6cd-abc4258540e4', storagePoolId='5a865884-0366- 0330-02b8-0000000002d4'}), log id: 7c67bf06<br id="yiv2961678542m_1856169086348407128yiv5069007669m_-8653308167440808248yui_3_16_0_ym19_1_1519085807369_70826" clear="none">2018-02-20 14:42:28,144+08 INFO [org.ovirt.engine.core. vdsbroker.vdsbroker. SpmStopVDSCommand] (org.ovirt.thread.pool-7- thread-43) [3e5965ca] SpmStopVDSCommand::Stopping SPM on vds 'Node1', pool id '5a865884-0366-0330-02b8- 0000000002d4'<br clear="none"></div><div id="yiv2961678542m_1856169086348407128yiv5069007669m_-8653308167440808248yui_3_16_0_ym19_1_1519085807369_70529"><span></span></div> <div class="yiv2961678542m_1856169086348407128yiv5069007669m_-8653308167440808248qtdSeparateBR"><br clear="none"><br clear="none"></div><div class="yiv2961678542m_1856169086348407128yiv5069007669m_-8653308167440808248yahoo_quoted" style="display:block;"> <div style="font-family:Helvetica Neue, Helvetica, Arial, Lucida Grande, sans-serif;font-size:10px;"> <div style="font-family:HelveticaNeue, Helvetica Neue, Helvetica, Arial, Lucida Grande, sans-serif;font-size:16px;"> <div dir="ltr"><font size="2" face="Arial"> On Tuesday, February 20, 2018 2:33 PM, michael pagdanganan <<a rel="nofollow" shape="rect" ymailto="mailto:mhke_aj5566@yahoo.com" target="_blank" href="mailto:mhke_aj5566@yahoo.com">mhke_aj5566@yahoo.com</a>> wrote:<br clear="none"></font></div> <br clear="none"><br clear="none"> <div class="yiv2961678542m_1856169086348407128yiv5069007669m_-8653308167440808248y_msg_container"><div id="yiv2961678542m_1856169086348407128yiv5069007669m_-8653308167440808248yiv0115319588"><div><div style="color:#000;background-color:#fff;font-family:Helvetica Neue, Helvetica, Arial, Lucida Grande, sans-serif;font-size:10px;"><div id="yiv2961678542m_1856169086348407128yiv5069007669m_-8653308167440808248yiv0115319588yui_3_16_0_ym19_1_1519085807369_60672"><font size="3">Thanks for quick response,</font></div><div id="yiv2961678542m_1856169086348407128yiv5069007669m_-8653308167440808248yiv0115319588yui_3_16_0_ym19_1_1519085807369_62803"><font size="3"><br clear="none"></font></div><div id="yiv2961678542m_1856169086348407128yiv5069007669m_-8653308167440808248yiv0115319588yui_3_16_0_ym19_1_1519085807369_64423"><font size="3">see attachment.<br clear="none"></font></div><div id="yiv2961678542m_1856169086348407128yiv5069007669m_-8653308167440808248yiv0115319588yui_3_16_0_ym19_1_1519085807369_60552"><span></span></div> <div class="yiv2961678542m_1856169086348407128yiv5069007669m_-8653308167440808248yiv0115319588qtdSeparateBR"><br clear="none"><br clear="none"></div><div class="yiv2961678542m_1856169086348407128yiv5069007669m_-8653308167440808248yiv0115319588yahoo_quoted" style="display:block;"> <div style="font-family:Helvetica Neue, Helvetica, Arial, Lucida Grande, sans-serif;font-size:10px;"> <div style="font-family:HelveticaNeue, Helvetica Neue, Helvetica, Arial, Lucida Grande, sans-serif;font-size:16px;"> <div class="yiv2961678542m_1856169086348407128yiv5069007669m_-8653308167440808248yiv0115319588yqt2980525560" id="yiv2961678542m_1856169086348407128yiv5069007669m_-8653308167440808248yiv0115319588yqtfd88886"><div dir="ltr"><font size="2" face="Arial"> On Tuesday, February 20, 2018 2:10 PM, Eyal Shenitzky <<a rel="nofollow" shape="rect" ymailto="mailto:eshenitz@redhat.com" target="_blank" href="mailto:eshenitz@redhat.com">eshenitz@redhat.com</a>> wrote:<br clear="none"></font></div> <br clear="none"><br clear="none"> <div class="yiv2961678542m_1856169086348407128yiv5069007669m_-8653308167440808248yiv0115319588y_msg_container"><div id="yiv2961678542m_1856169086348407128yiv5069007669m_-8653308167440808248yiv0115319588"><div><div dir="ltr">Hi, <div><br clear="none"></div><div>Can you please attach full Engine and VDSM logs?</div><div><br clear="none"></div><div>Thanks,</div></div><div class="yiv2961678542m_1856169086348407128yiv5069007669m_-8653308167440808248yiv0115319588gmail_extra"><br clear="none"><div class="yiv2961678542m_1856169086348407128yiv5069007669m_-8653308167440808248yiv0115319588gmail_quote">On Tue, Feb 20, 2018 at 2:59 AM, michael pagdanganan <span dir="ltr"><<a rel="nofollow" shape="rect" ymailto="mailto:mhke_aj5566@yahoo.com" target="_blank" href="mailto:mhke_aj5566@yahoo.com">mhke_aj5566@yahoo.com</a>></span> wrote:<br clear="none"><blockquote class="yiv2961678542m_1856169086348407128yiv5069007669m_-8653308167440808248yiv0115319588gmail_quote" style="margin:0 0 0 .8ex;border-left:1px #ccc solid;padding-left:1ex;"><div class="yiv2961678542m_1856169086348407128yiv5069007669m_-8653308167440808248yiv0115319588yqt7135196070" id="yiv2961678542m_1856169086348407128yiv5069007669m_-8653308167440808248yiv0115319588yqt29383"><div><div style="color:#000;background-color:#fff;font-family:Helvetica Neue, Helvetica, Arial, Lucida Grande, sans-serif;font-size:10px;"><div dir="ltr" id="yiv2961678542m_1856169086348407128yiv5069007669m_-8653308167440808248yiv0115319588m_-4605301492274769828yui_3_16_0_ym19_1_1519085807369_10542"><font id="yiv2961678542m_1856169086348407128yiv5069007669m_-8653308167440808248yiv0115319588m_-4605301492274769828yui_3_16_0_ym19_1_1519085807369_15967" size="3">My storage pool has 2 Master Domain(</font><font id="yiv2961678542m_1856169086348407128yiv5069007669m_-8653308167440808248yiv0115319588m_-4605301492274769828yui_3_16_0_ym19_1_1519085807369_16063" size="3"><font id="yiv2961678542m_1856169086348407128yiv5069007669m_-8653308167440808248yiv0115319588m_-4605301492274769828yui_3_16_0_ym19_1_1519085807369_15968" size="3">Stored2 on Node2,Node1Container on Node1)</font>. and my old master domain </font><font id="yiv2961678542m_1856169086348407128yiv5069007669m_-8653308167440808248yiv0115319588m_-4605301492274769828yui_3_16_0_ym19_1_1519085807369_15934" size="3"><font id="yiv2961678542m_1856169086348407128yiv5069007669m_-8653308167440808248yiv0115319588m_-4605301492274769828yui_3_16_0_ym19_1_1519085807369_16064" size="3">(</font><font id="yiv2961678542m_1856169086348407128yiv5069007669m_-8653308167440808248yiv0115319588m_-4605301492274769828yui_3_16_0_ym19_1_1519085807369_16065" size="3"><font id="yiv2961678542m_1856169086348407128yiv5069007669m_-8653308167440808248yiv0115319588m_-4605301492274769828yui_3_16_0_ym19_1_1519085807369_16066" size="3">DATANd01 on Node1 )</font></font> hung on preparing for maintenance. When I tried to activate my old master domain (</font><font id="yiv2961678542m_1856169086348407128yiv5069007669m_-8653308167440808248yiv0115319588m_-4605301492274769828yui_3_16_0_ym19_1_1519085807369_10517" size="3"><font id="yiv2961678542m_1856169086348407128yiv5069007669m_-8653308167440808248yiv0115319588m_-4605301492274769828yui_3_16_0_ym19_1_1519085807369_15935" size="3">DATANd01 on Node1 )</font> all storage domain goes down and up master keep on rotating.</font></div><div id="yiv2961678542m_1856169086348407128yiv5069007669m_-8653308167440808248yiv0115319588m_-4605301492274769828yui_3_16_0_ym19_1_1519085807369_10905"><font id="yiv2961678542m_1856169086348407128yiv5069007669m_-8653308167440808248yiv0115319588m_-4605301492274769828yui_3_16_0_ym19_1_1519085807369_10517" size="3"><br clear="none"></font></div><div id="yiv2961678542m_1856169086348407128yiv5069007669m_-8653308167440808248yiv0115319588m_-4605301492274769828yui_3_16_0_ym19_1_1519085807369_10692"><font id="yiv2961678542m_1856169086348407128yiv5069007669m_-8653308167440808248yiv0115319588m_-4605301492274769828yui_3_16_0_ym19_1_1519085807369_10517" size="3"><br clear="none"></font></div><div dir="ltr" id="yiv2961678542m_1856169086348407128yiv5069007669m_-8653308167440808248yiv0115319588m_-4605301492274769828yui_3_16_0_ym19_1_1519085807369_10710"><font id="yiv2961678542m_1856169086348407128yiv5069007669m_-8653308167440808248yiv0115319588m_-4605301492274769828yui_3_16_0_ym19_1_1519085807369_10517" size="3">Ovirt Version (oVirt Engine Version: 4.1.9.1.el7.centos)<br clear="none"></font></div><div dir="ltr" id="yiv2961678542m_1856169086348407128yiv5069007669m_-8653308167440808248yiv0115319588m_-4605301492274769828yui_3_16_0_ym19_1_1519085807369_14202"><font id="yiv2961678542m_1856169086348407128yiv5069007669m_-8653308167440808248yiv0115319588m_-4605301492274769828yui_3_16_0_ym19_1_1519085807369_10517" size="3"><br clear="none"></font></div><div dir="ltr" id="yiv2961678542m_1856169086348407128yiv5069007669m_-8653308167440808248yiv0115319588m_-4605301492274769828yui_3_16_0_ym19_1_1519085807369_12604"><font id="yiv2961678542m_1856169086348407128yiv5069007669m_-8653308167440808248yiv0115319588m_-4605301492274769828yui_3_16_0_ym19_1_1519085807369_10517" size="3"><br clear="none"></font></div><div dir="ltr" id="yiv2961678542m_1856169086348407128yiv5069007669m_-8653308167440808248yiv0115319588m_-4605301492274769828yui_3_16_0_ym19_1_1519085807369_14203"><font id="yiv2961678542m_1856169086348407128yiv5069007669m_-8653308167440808248yiv0115319588m_-4605301492274769828yui_3_16_0_ym19_1_1519085807369_10517" size="3">Event Error:</font></div><div dir="ltr" id="yiv2961678542m_1856169086348407128yiv5069007669m_-8653308167440808248yiv0115319588m_-4605301492274769828yui_3_16_0_ym19_1_1519085807369_14215"><font id="yiv2961678542m_1856169086348407128yiv5069007669m_-8653308167440808248yiv0115319588m_-4605301492274769828yui_3_16_0_ym19_1_1519085807369_10517" size="3">Sync Error on Master Domain between Host Node2 and oVirt Engine. Domain Stored2 is marked as master in ovirt engine Database but not on storage side Please consult with support<br id="yiv2961678542m_1856169086348407128yiv5069007669m_-8653308167440808248yiv0115319588m_-4605301492274769828yui_3_16_0_ym19_1_1519085807369_12607" clear="none"></font></div><div dir="ltr" id="yiv2961678542m_1856169086348407128yiv5069007669m_-8653308167440808248yiv0115319588m_-4605301492274769828yui_3_16_0_ym19_1_1519085807369_14204"><font id="yiv2961678542m_1856169086348407128yiv5069007669m_-8653308167440808248yiv0115319588m_-4605301492274769828yui_3_16_0_ym19_1_1519085807369_10517" size="3"><br clear="none"></font></div><div dir="ltr" id="yiv2961678542m_1856169086348407128yiv5069007669m_-8653308167440808248yiv0115319588m_-4605301492274769828yui_3_16_0_ym19_1_1519085807369_15891"><font id="yiv2961678542m_1856169086348407128yiv5069007669m_-8653308167440808248yiv0115319588m_-4605301492274769828yui_3_16_0_ym19_1_1519085807369_10517" size="3">VDSM Node2 command ConnectStoragePoolVDS failed: Wrong Master domain or it's version: u'SD=f3e372e3-1251-4195-a4b9- 1027e40059df, pool=5a865884-0366-0330-02b8- 0000</font></div><div dir="ltr" id="yiv2961678542m_1856169086348407128yiv5069007669m_-8653308167440808248yiv0115319588m_-4605301492274769828yui_3_16_0_ym19_1_1519085807369_15895"><font id="yiv2961678542m_1856169086348407128yiv5069007669m_-8653308167440808248yiv0115319588m_-4605301492274769828yui_3_16_0_ym19_1_1519085807369_10517" size="3"><br clear="none"></font></div><div dir="ltr" id="yiv2961678542m_1856169086348407128yiv5069007669m_-8653308167440808248yiv0115319588m_-4605301492274769828yui_3_16_0_ym19_1_1519085807369_15896"><font id="yiv2961678542m_1856169086348407128yiv5069007669m_-8653308167440808248yiv0115319588m_-4605301492274769828yui_3_16_0_ym19_1_1519085807369_10517" size="3">VDSM Node2 command HSMGetAllTastsStatusesVDS failed: Not SPM: ()<br id="yiv2961678542m_1856169086348407128yiv5069007669m_-8653308167440808248yiv0115319588m_-4605301492274769828yui_3_16_0_ym19_1_1519085807369_15901" clear="none"><br id="yiv2961678542m_1856169086348407128yiv5069007669m_-8653308167440808248yiv0115319588m_-4605301492274769828yui_3_16_0_ym19_1_1519085807369_15902" clear="none">Failed to deactivate Storage Domain DATANd01 (Data Center UnsecuredEnv)<br clear="none"></font></div><div id="yiv2961678542m_1856169086348407128yiv5069007669m_-8653308167440808248yiv0115319588m_-4605301492274769828yui_3_16_0_ym19_1_1519085807369_10545"><font id="yiv2961678542m_1856169086348407128yiv5069007669m_-8653308167440808248yiv0115319588m_-4605301492274769828yui_3_16_0_ym19_1_1519085807369_10517" size="3"><br clear="none"></font></div><div id="yiv2961678542m_1856169086348407128yiv5069007669m_-8653308167440808248yiv0115319588m_-4605301492274769828yui_3_16_0_ym19_1_1519085807369_10551"><font id="yiv2961678542m_1856169086348407128yiv5069007669m_-8653308167440808248yiv0115319588m_-4605301492274769828yui_3_16_0_ym19_1_1519085807369_10517" size="3">Here's logs from engine:</font></div><div id="yiv2961678542m_1856169086348407128yiv5069007669m_-8653308167440808248yiv0115319588m_-4605301492274769828yui_3_16_0_ym19_1_1519085807369_10620"><font id="yiv2961678542m_1856169086348407128yiv5069007669m_-8653308167440808248yiv0115319588m_-4605301492274769828yui_3_16_0_ym19_1_1519085807369_10517" size="3"><br clear="none"></font></div><div id="yiv2961678542m_1856169086348407128yiv5069007669m_-8653308167440808248yiv0115319588m_-4605301492274769828yui_3_16_0_ym19_1_1519085807369_10573"><font id="yiv2961678542m_1856169086348407128yiv5069007669m_-8653308167440808248yiv0115319588m_-4605301492274769828yui_3_16_0_ym19_1_1519085807369_10517" size="3">------------------------------ ------------------------------ ------------------------------ -------------</font></div><div dir="ltr" id="yiv2961678542m_1856169086348407128yiv5069007669m_-8653308167440808248yiv0115319588m_-4605301492274769828yui_3_16_0_ym19_1_1519085807369_10588"><font id="yiv2961678542m_1856169086348407128yiv5069007669m_-8653308167440808248yiv0115319588m_-4605301492274769828yui_3_16_0_ym19_1_1519085807369_10517" size="3">[root@dev2engine ~]# tail /var/log/messages<br id="yiv2961678542m_1856169086348407128yiv5069007669m_-8653308167440808248yiv0115319588m_-4605301492274769828yui_3_16_0_ym19_1_1519085807369_10609" clear="none">Feb 20 07:01:01 dev2engine systemd: Starting Session 20 of user root.<br id="yiv2961678542m_1856169086348407128yiv5069007669m_-8653308167440808248yiv0115319588m_-4605301492274769828yui_3_16_0_ym19_1_1519085807369_10610" clear="none">Feb 20 07:01:01 dev2engine systemd: Removed slice User Slice of root.<br id="yiv2961678542m_1856169086348407128yiv5069007669m_-8653308167440808248yiv0115319588m_-4605301492274769828yui_3_16_0_ym19_1_1519085807369_10611" clear="none">Feb 20 07:01:01 dev2engine systemd: Stopping User Slice of root.<br id="yiv2961678542m_1856169086348407128yiv5069007669m_-8653308167440808248yiv0115319588m_-4605301492274769828yui_3_16_0_ym19_1_1519085807369_10612" clear="none">Feb 20 07:58:52 dev2engine systemd: Created slice User Slice of root.<br id="yiv2961678542m_1856169086348407128yiv5069007669m_-8653308167440808248yiv0115319588m_-4605301492274769828yui_3_16_0_ym19_1_1519085807369_10613" clear="none">Feb 20 07:58:52 dev2engine systemd: Starting User Slice of root.<br id="yiv2961678542m_1856169086348407128yiv5069007669m_-8653308167440808248yiv0115319588m_-4605301492274769828yui_3_16_0_ym19_1_1519085807369_10614" clear="none">Feb 20 07:58:52 dev2engine systemd-logind: New session 21 of user root.<br id="yiv2961678542m_1856169086348407128yiv5069007669m_-8653308167440808248yiv0115319588m_-4605301492274769828yui_3_16_0_ym19_1_1519085807369_10615" clear="none">Feb 20 07:58:52 dev2engine systemd: Started Session 21 of user root.<br id="yiv2961678542m_1856169086348407128yiv5069007669m_-8653308167440808248yiv0115319588m_-4605301492274769828yui_3_16_0_ym19_1_1519085807369_10616" clear="none">Feb 20 07:58:52 dev2engine systemd: Starting Session 21 of user root.<br id="yiv2961678542m_1856169086348407128yiv5069007669m_-8653308167440808248yiv0115319588m_-4605301492274769828yui_3_16_0_ym19_1_1519085807369_10617" clear="none">Feb 20 08:01:01 dev2engine systemd: Started Session 22 of user root.<br id="yiv2961678542m_1856169086348407128yiv5069007669m_-8653308167440808248yiv0115319588m_-4605301492274769828yui_3_16_0_ym19_1_1519085807369_10618" clear="none">Feb 20 08:01:01 dev2engine systemd: Starting Session 22 of user root.</font></div><div dir="ltr" id="yiv2961678542m_1856169086348407128yiv5069007669m_-8653308167440808248yiv0115319588m_-4605301492274769828yui_3_16_0_ym19_1_1519085807369_10619"><font id="yiv2961678542m_1856169086348407128yiv5069007669m_-8653308167440808248yiv0115319588m_-4605301492274769828yui_3_16_0_ym19_1_1519085807369_10517" size="3"><br clear="none"></font></div><div dir="ltr" id="yiv2961678542m_1856169086348407128yiv5069007669m_-8653308167440808248yiv0115319588m_-4605301492274769828yui_3_16_0_ym19_1_1519085807369_10623"><br clear="none"><font id="yiv2961678542m_1856169086348407128yiv5069007669m_-8653308167440808248yiv0115319588m_-4605301492274769828yui_3_16_0_ym19_1_1519085807369_10517" size="3"><font id="yiv2961678542m_1856169086348407128yiv5069007669m_-8653308167440808248yiv0115319588m_-4605301492274769828yui_3_16_0_ym19_1_1519085807369_10626" size="3">------------------------------ ------------------------------ ------------------------------ -------------</font></font></div><div dir="ltr" id="yiv2961678542m_1856169086348407128yiv5069007669m_-8653308167440808248yiv0115319588m_-4605301492274769828yui_3_16_0_ym19_1_1519085807369_10654"><font id="yiv2961678542m_1856169086348407128yiv5069007669m_-8653308167440808248yiv0115319588m_-4605301492274769828yui_3_16_0_ym19_1_1519085807369_10517" size="3"><font id="yiv2961678542m_1856169086348407128yiv5069007669m_-8653308167440808248yiv0115319588m_-4605301492274769828yui_3_16_0_ym19_1_1519085807369_10626" size="3">[root@dev2engine ~]# tail /var/log/ovirt-engine/engine. log<br id="yiv2961678542m_1856169086348407128yiv5069007669m_-8653308167440808248yiv0115319588m_-4605301492274769828yui_3_16_0_ym19_1_1519085807369_10677" clear="none">2018-02-20 08:01:16,062+08 INFO [org.ovirt.engine.core.bll. eventqueue.EventQueueMonitor] (org.ovirt.thread.pool-7- thread-32) [102e9d3c] Finished reconstruct for pool '5a865884-0366-0330-02b8- 0000000002d4'. Clearing event queue<br id="yiv2961678542m_1856169086348407128yiv5069007669m_-8653308167440808248yiv0115319588m_-4605301492274769828yui_3_16_0_ym19_1_1519085807369_10678" clear="none">2018-02-20 08:01:27,825+08 WARN [org.ovirt.engine.core. vdsbroker.irsbroker.IrsProxy] (org.ovirt.thread.pool-7- thread-23) [] Master domain is not in sync between DB and VDSM. Domain Stored2 marked as master in DB and not in the storage<br id="yiv2961678542m_1856169086348407128yiv5069007669m_-8653308167440808248yiv0115319588m_-4605301492274769828yui_3_16_0_ym19_1_1519085807369_10679" clear="none">2018-02-20 08:01:27,862+08 WARN [org.ovirt.engine.core.bll. storage.pool. ReconstructMasterDomainCommand ] (org.ovirt.thread.pool-7- thread-23) [213f42b9] Validation of action 'ReconstructMasterDomain' failed for user SYSTEM. Reasons: VAR__ACTION__RECONSTRUCT_ MASTER,VAR__TYPE__STORAGE__ DOMAIN,ACTION_TYPE_FAILED_ STORAGE_DOMAIN_STATUS_ ILLEGAL2,$status PreparingForMaintenance<br id="yiv2961678542m_1856169086348407128yiv5069007669m_-8653308167440808248yiv0115319588m_-4605301492274769828yui_3_16_0_ym19_1_1519085807369_10680" clear="none">2018-02-20 08:01:27,882+08 INFO [org.ovirt.engine.core.bll. eventqueue.EventQueueMonitor] (org.ovirt.thread.pool-7- thread-20) [929330e] Finished reconstruct for pool '5a865884-0366-0330-02b8- 0000000002d4'. Clearing event queue<br id="yiv2961678542m_1856169086348407128yiv5069007669m_-8653308167440808248yiv0115319588m_-4605301492274769828yui_3_16_0_ym19_1_1519085807369_10681" clear="none">2018-02-20 08:01:40,106+08 WARN [org.ovirt.engine.core. vdsbroker.irsbroker.IrsProxy] (org.ovirt.thread.pool-7- thread-17) [] Master domain is not in sync between DB and VDSM. Domain Stored2 marked as master in DB and not in the storage<br id="yiv2961678542m_1856169086348407128yiv5069007669m_-8653308167440808248yiv0115319588m_-4605301492274769828yui_3_16_0_ym19_1_1519085807369_10682" clear="none">2018-02-20 08:01:40,197+08 WARN [org.ovirt.engine.core.bll. storage.pool. ReconstructMasterDomainCommand ] (org.ovirt.thread.pool-7- thread-17) [7af552c1] Validation of action 'ReconstructMasterDomain' failed for user SYSTEM. Reasons: VAR__ACTION__RECONSTRUCT_ MASTER,VAR__TYPE__STORAGE__ DOMAIN,ACTION_TYPE_FAILED_ STORAGE_DOMAIN_STATUS_ ILLEGAL2,$status PreparingForMaintenance<br id="yiv2961678542m_1856169086348407128yiv5069007669m_-8653308167440808248yiv0115319588m_-4605301492274769828yui_3_16_0_ym19_1_1519085807369_10683" clear="none">2018-02-20 08:01:40,246+08 INFO [org.ovirt.engine.core.bll. eventqueue.EventQueueMonitor] (org.ovirt.thread.pool-7- thread-22) [73673040] Finished reconstruct for pool '5a865884-0366-0330-02b8- 0000000002d4'. Clearing event queue<br id="yiv2961678542m_1856169086348407128yiv5069007669m_-8653308167440808248yiv0115319588m_-4605301492274769828yui_3_16_0_ym19_1_1519085807369_10684" clear="none">2018-02-20 08:01:51,809+08 WARN [org.ovirt.engine.core. vdsbroker.irsbroker.IrsProxy] (org.ovirt.thread.pool-7- thread-26) [] Master domain is not in sync between DB and VDSM. Domain Stored2 marked as master in DB and not in the storage<br id="yiv2961678542m_1856169086348407128yiv5069007669m_-8653308167440808248yiv0115319588m_-4605301492274769828yui_3_16_0_ym19_1_1519085807369_10685" clear="none">2018-02-20 08:01:51,846+08 WARN [org.ovirt.engine.core.bll. storage.pool. ReconstructMasterDomainCommand ] (org.ovirt.thread.pool-7- thread-26) [20307cbe] Validation of action 'ReconstructMasterDomain' failed for user SYSTEM. Reasons: VAR__ACTION__RECONSTRUCT_ MASTER,VAR__TYPE__STORAGE__ DOMAIN,ACTION_TYPE_FAILED_ STORAGE_DOMAIN_STATUS_ ILLEGAL2,$status PreparingForMaintenance<br id="yiv2961678542m_1856169086348407128yiv5069007669m_-8653308167440808248yiv0115319588m_-4605301492274769828yui_3_16_0_ym19_1_1519085807369_10686" clear="none">2018-02-20 08:01:51,866+08 INFO [org.ovirt.engine.core.bll. eventqueue.EventQueueMonitor] (org.ovirt.thread.pool-7- thread-49) [2c11a866] Finished reconstruct for pool '5a865884-0366-0330-02b8- 0000000002d4'. Clearing event queue<br id="yiv2961678542m_1856169086348407128yiv5069007669m_-8653308167440808248yiv0115319588m_-4605301492274769828yui_3_16_0_ym19_1_1519085807369_10687" clear="none"><br clear="none"></font></font></div></div></div></div><br clear="none">______________________________ _________________<br clear="none">
Users mailing list<br clear="none">
<a rel="nofollow" shape="rect" ymailto="mailto:Users@ovirt.org" target="_blank" href="mailto:Users@ovirt.org">Users@ovirt.org</a><br clear="none">
<a rel="nofollow" shape="rect" target="_blank" href="http://lists.ovirt.org/mailman/listinfo/users">http://lists.ovirt.org/ mailman/listinfo/users</a><br clear="none">
<br clear="none"></blockquote></div><span class="yiv2961678542m_1856169086348407128yiv5069007669HOEnZb"><font color="#888888"><br clear="none"><br clear="all"></font></span><span class="yiv2961678542HOEnZb"><font color="#888888"></font></span><div><br clear="none"></div>-- <br clear="none"><div class="yiv2961678542m_1856169086348407128yiv5069007669m_-8653308167440808248yiv0115319588gmail_signature"><div dir="ltr">Regards,<div>Eyal Shenitzky</div></div></div>
</div></div></div><span class="yiv2961678542HOEnZb"><font color="#888888"><br clear="none"><br clear="none"></font></span></div><span class="yiv2961678542HOEnZb"><font color="#888888"> </font></span></div></div><span class="yiv2961678542HOEnZb"><font color="#888888"></font></span><div class="yiv2961678542m_1856169086348407128yiv5069007669m_-8653308167440808248yiv0115319588yqt2980525560" id="yiv2961678542m_1856169086348407128yiv5069007669m_-8653308167440808248yiv0115319588yqtfd94549"> </div></div><span class="yiv2961678542HOEnZb"><font color="#888888"></font></span><div class="yiv2961678542m_1856169086348407128yiv5069007669m_-8653308167440808248yiv0115319588yqt2980525560" id="yiv2961678542m_1856169086348407128yiv5069007669m_-8653308167440808248yiv0115319588yqtfd67072"> </div></div></div></div></div><span class="yiv2961678542HOEnZb"><font color="#888888"><br clear="none"><br clear="none"></font></span></div><span class="yiv2961678542HOEnZb"><font color="#888888"> </font></span></div><span class="yiv2961678542HOEnZb"><font color="#888888"> </font></span></div><span class="yiv2961678542HOEnZb"><font color="#888888"> </font></span></div></div></div></blockquote></div></div><span class="yiv2961678542HOEnZb"><font color="#888888"><br clear="none"><br clear="all"></font></span><div><br clear="none"></div>-- <br clear="none"><div class="yiv2961678542m_1856169086348407128yiv5069007669gmail_signature"><div dir="ltr">Regards,<div>Eyal Shenitzky</div></div></div>
</div></div></div><br clear="none"><br clear="none"></div> </div> </div> </div></div></div></blockquote></div></div><br clear="none"><br clear="all"><div><br clear="none"></div>-- <br clear="none"><div class="yiv2961678542gmail_signature"><div dir="ltr">Regards,<div>Eyal Shenitzky</div></div></div>
</div></div></div><br clear="none"><br clear="none"></div> </div> </div> </div></div></div></div></div><br><br></div> </div> </div> </div></div></body></html>