<div dir="ltr">Hi,<br>i have the same problem with gluster.<br>this is a bug that i opened <a href="https://bugzilla.redhat.com/show_bug.cgi?id=1461029">https://bugzilla.redhat.com/show_bug.cgi?id=1461029</a> .<br>In the bug i used single disk vm but i start to notice the problem with multiple disk vm.<br><div><div><br></div></div></div><div class="gmail_extra"><br><div class="gmail_quote">2017-07-13 0:07 GMT+02:00 Devin Acosta <span dir="ltr"><<a href="mailto:devin@pabstatencio.com" target="_blank">devin@pabstatencio.com</a>></span>:<br><blockquote class="gmail_quote" style="margin:0 0 0 .8ex;border-left:1px #ccc solid;padding-left:1ex"><div style="word-wrap:break-word"><div id="m_-4245500177440494772bloop_customfont" style="margin:0px"><font face="Input Mono">We are running a fresh install of oVIRT 4.1.3, using ISCSI, the VM in question has multiple Disks (4 to be exact). It snapshotted OK while on iSCSI however when I went to delete the single snapshot that existed it went into Locked state and never came back. The deletion has been going for well over an hour, and I am not convinced since the snapshot is less than 12 hours old that it’s really doing anything.</font></div><div id="m_-4245500177440494772bloop_customfont" style="margin:0px"><font face="Input Mono"><br></font></div><div id="m_-4245500177440494772bloop_customfont" style="margin:0px"><font face="Input Mono">I have seen that doing some Googling indicates there might be some known issues with iSCSI/Block Storage/Multiple Disk Snapshot issues. </font></div><div id="m_-4245500177440494772bloop_customfont" style="margin:0px"><font face="Input Mono"><br></font></div><div id="m_-4245500177440494772bloop_customfont" style="margin:0px"><font face="Input Mono">In the logs on the engine it shows:</font></div><div id="m_-4245500177440494772bloop_customfont" style="color:rgb(0,0,0);margin:0px"><font face="Input Mono"><br></font></div><div id="m_-4245500177440494772bloop_customfont" style="margin:0px"><div id="m_-4245500177440494772bloop_customfont" style="color:rgb(0,0,0);margin:0px"><font face="Input Mono">2017-07-12 21:59:42,473Z INFO [org.ovirt.engine.core.bll.<wbr>SerialChildCommandsExecutionCa<wbr>llback] (DefaultQuartzScheduler2) [a5f6eaf2-7996-4d51-ba62-<wbr>050272d1f097] Command 'ColdMergeSnapshotSingleDisk' (id: '75c535fd-4558-459a-9992-<wbr>875c48578a97') waiting on child command id: 'd92e9a22-5f0f-4b61-aac6-<wbr>5601f8ac2cda' type:'PrepareMerge' to complete</font></div><div id="m_-4245500177440494772bloop_customfont" style="color:rgb(0,0,0);margin:0px"><font face="Input Mono">2017-07-12 21:59:52,480Z INFO [org.ovirt.engine.core.bll.<wbr>ConcurrentChildCommandsExecuti<wbr>onCallback] (DefaultQuartzScheduler2) [a5f6eaf2-7996-4d51-ba62-<wbr>050272d1f097] Command 'RemoveSnapshot' (id: '40482d09-8a7c-4dbd-8324-<wbr>3e789296887a') waiting on child command id: '75c535fd-4558-459a-9992-<wbr>875c48578a97' type:'<wbr>ColdMergeSnapshotSingleDisk' to complete</font></div><div id="m_-4245500177440494772bloop_customfont" style="color:rgb(0,0,0);margin:0px"><font face="Input Mono">2017-07-12 21:59:52,483Z INFO [org.ovirt.engine.core.bll.<wbr>SerialChildCommandsExecutionCa<wbr>llback] (DefaultQuartzScheduler2) [a5f6eaf2-7996-4d51-ba62-<wbr>050272d1f097] Command 'ColdMergeSnapshotSingleDisk' (id: '75c535fd-4558-459a-9992-<wbr>875c48578a97') waiting on child command id: 'd92e9a22-5f0f-4b61-aac6-<wbr>5601f8ac2cda' type:'PrepareMerge' to complete</font></div><div id="m_-4245500177440494772bloop_customfont" style="color:rgb(0,0,0);margin:0px"><font face="Input Mono">2017-07-12 22:00:02,490Z INFO [org.ovirt.engine.core.bll.<wbr>ConcurrentChildCommandsExecuti<wbr>onCallback] (DefaultQuartzScheduler6) [a5f6eaf2-7996-4d51-ba62-<wbr>050272d1f097] Command 'RemoveSnapshot' (id: '40482d09-8a7c-4dbd-8324-<wbr>3e789296887a') waiting on child command id: '75c535fd-4558-459a-9992-<wbr>875c48578a97' type:'<wbr>ColdMergeSnapshotSingleDisk' to complete</font></div><div id="m_-4245500177440494772bloop_customfont" style="color:rgb(0,0,0);margin:0px"><font face="Input Mono">2017-07-12 22:00:02,493Z INFO [org.ovirt.engine.core.bll.<wbr>SerialChildCommandsExecutionCa<wbr>llback] (DefaultQuartzScheduler6) [a5f6eaf2-7996-4d51-ba62-<wbr>050272d1f097] Command 'ColdMergeSnapshotSingleDisk' (id: '75c535fd-4558-459a-9992-<wbr>875c48578a97') waiting on child command id: 'd92e9a22-5f0f-4b61-aac6-<wbr>5601f8ac2cda' type:'PrepareMerge' to complete</font></div><div id="m_-4245500177440494772bloop_customfont" style="color:rgb(0,0,0);margin:0px"><font face="Input Mono">2017-07-12 22:00:12,498Z INFO [org.ovirt.engine.core.bll.<wbr>ConcurrentChildCommandsExecuti<wbr>onCallback] (DefaultQuartzScheduler3) [a5f6eaf2-7996-4d51-ba62-<wbr>050272d1f097] Command 'RemoveSnapshot' (id: '40482d09-8a7c-4dbd-8324-<wbr>3e789296887a') waiting on child command id: '75c535fd-4558-459a-9992-<wbr>875c48578a97' type:'<wbr>ColdMergeSnapshotSingleDisk' to complete</font></div><div id="m_-4245500177440494772bloop_customfont" style="color:rgb(0,0,0);margin:0px"><font face="Input Mono">2017-07-12 22:00:12,501Z INFO [org.ovirt.engine.core.bll.<wbr>SerialChildCommandsExecutionCa<wbr>llback] (DefaultQuartzScheduler3) [a5f6eaf2-7996-4d51-ba62-<wbr>050272d1f097] Command 'ColdMergeSnapshotSingleDisk' (id: '75c535fd-4558-459a-9992-<wbr>875c48578a97') waiting on child command id: 'd92e9a22-5f0f-4b61-aac6-<wbr>5601f8ac2cda' type:'PrepareMerge' to complete</font></div><div id="m_-4245500177440494772bloop_customfont" style="color:rgb(0,0,0);margin:0px"><font face="Input Mono">2017-07-12 22:00:22,508Z INFO [org.ovirt.engine.core.bll.<wbr>ConcurrentChildCommandsExecuti<wbr>onCallback] (DefaultQuartzScheduler5) [a5f6eaf2-7996-4d51-ba62-<wbr>050272d1f097] Command 'RemoveSnapshot' (id: '40482d09-8a7c-4dbd-8324-<wbr>3e789296887a') waiting on child command id: '75c535fd-4558-459a-9992-<wbr>875c48578a97' type:'<wbr>ColdMergeSnapshotSingleDisk' to complete</font></div><div id="m_-4245500177440494772bloop_customfont" style="color:rgb(0,0,0);margin:0px"><font face="Input Mono">2017-07-12 22:00:22,511Z INFO [org.ovirt.engine.core.bll.<wbr>SerialChildCommandsExecutionCa<wbr>llback] (DefaultQuartzScheduler5) [a5f6eaf2-7996-4d51-ba62-<wbr>050272d1f097] Command 'ColdMergeSnapshotSingleDisk' (id: '75c535fd-4558-459a-9992-<wbr>875c48578a97') waiting on child command id: 'd92e9a22-5f0f-4b61-aac6-<wbr>5601f8ac2cda' type:'PrepareMerge' to complete</font></div><div id="m_-4245500177440494772bloop_customfont" style="color:rgb(0,0,0);margin:0px"><font face="Input Mono"><br></font></div><div id="m_-4245500177440494772bloop_customfont" style="margin:0px"><font face="Input Mono">This is what I seen on the SPM when I grep’d the Snapshot ID.</font></div><div id="m_-4245500177440494772bloop_customfont" style="color:rgb(0,0,0);margin:0px"><font face="Input Mono"><br></font></div><div id="m_-4245500177440494772bloop_customfont" style="color:rgb(0,0,0);margin:0px"><div id="m_-4245500177440494772bloop_customfont" style="margin:0px"><font face="Input Mono">2017-07-12 14:22:18,773-0700 INFO (jsonrpc/6) [vdsm.api] START createVolume(sdUUID=u'<wbr>0c02a758-4295-4199-97de-<wbr>b041744b3b15', spUUID=u'00000001-0001-0001-<wbr>0001-000000000311', imgUUID=u'6a887015-67cd-4f7b-<wbr>b709-eef97142258d', size=u'107374182400', volFormat=4, preallocate=2, diskType=2, volUUID=u'5921ba71-0f00-46cd-<wbr>b0be-3c2ac1396845', desc=u'', srcImgUUID=u'6a887015-67cd-<wbr>4f7b-b709-eef97142258d', srcVolUUID=u'0c3de1a8-ac18-<wbr>4d7b-b348-3b097bf0a0ae', initialSize=None) from=::ffff:10.4.64.7,60016, flow_id=e94eebf8-75dc-407a-<wbr>8916-f4ff632f843e (api:46)</font></div><div id="m_-4245500177440494772bloop_customfont" style="margin:0px"><font face="Input Mono">2017-07-12 14:22:19,095-0700 WARN (tasks/6) [root] File: /rhev/data-center/00000001-<wbr>0001-0001-0001-000000000311/<wbr>0c02a758-4295-4199-97de-<wbr>b041744b3b15/images/6a887015-<wbr>67cd-4f7b-b709-eef97142258d/<wbr>5921ba71-0f00-46cd-b0be-<wbr>3c2ac1396845 already removed (utils:120)</font></div><div id="m_-4245500177440494772bloop_customfont" style="margin:0px"><font face="Input Mono">2017-07-12 14:22:19,096-0700 INFO (tasks/6) [storage.Volume] Request to create snapshot 6a887015-67cd-4f7b-b709-<wbr>eef97142258d/5921ba71-0f00-<wbr>46cd-b0be-3c2ac1396845 of volume 6a887015-67cd-4f7b-b709-<wbr>eef97142258d/0c3de1a8-ac18-<wbr>4d7b-b348-3b097bf0a0ae (blockVolume:545)</font></div><div id="m_-4245500177440494772bloop_customfont" style="margin:0px"><font face="Input Mono">2017-07-12 14:22:19,676-0700 INFO (tasks/6) [storage.LVM] Change LV tags (vg=0c02a758-4295-4199-97de-<wbr>b041744b3b15, lv=5921ba71-0f00-46cd-b0be-<wbr>3c2ac1396845, delTags=['OVIRT_VOL_<wbr>INITIALIZING'], addTags=['MD_10', u'PU_0c3de1a8-ac18-4d7b-b348-<wbr>3b097bf0a0ae', u'IU_6a887015-67cd-4f7b-b709-<wbr>eef97142258d']) (lvm:1344)</font></div><div id="m_-4245500177440494772bloop_customfont" style="margin:0px"><font face="Input Mono">2017-07-12 14:22:36,010-0700 INFO (jsonrpc/5) [vdsm.api] START getVolumeInfo(sdUUID=u'<wbr>0c02a758-4295-4199-97de-<wbr>b041744b3b15', spUUID=u'00000001-0001-0001-<wbr>0001-000000000311', imgUUID=u'6a887015-67cd-4f7b-<wbr>b709-eef97142258d', volUUID=u'5921ba71-0f00-46cd-<wbr>b0be-3c2ac1396845', options=None) from=::ffff:10.4.64.7,59664, flow_id=e94eebf8-75dc-407a-<wbr>8916-f4ff632f843e (api:46)</font></div><div id="m_-4245500177440494772bloop_customfont" style="margin:0px"><font face="Input Mono">2017-07-12 14:22:36,077-0700 INFO (jsonrpc/5) [storage.VolumeManifest] Info request: sdUUID=0c02a758-4295-4199-<wbr>97de-b041744b3b15 imgUUID=6a887015-67cd-4f7b-<wbr>b709-eef97142258d volUUID = 5921ba71-0f00-46cd-b0be-<wbr>3c2ac1396845 (volume:238)</font></div><div id="m_-4245500177440494772bloop_customfont" style="margin:0px"><font face="Input Mono">2017-07-12 14:22:36,185-0700 INFO (jsonrpc/5) [storage.VolumeManifest] 0c02a758-4295-4199-97de-<wbr>b041744b3b15/6a887015-67cd-<wbr>4f7b-b709-eef97142258d/<wbr>5921ba71-0f00-46cd-b0be-<wbr>3c2ac1396845 info is {'status': 'OK', 'domain': '0c02a758-4295-4199-97de-<wbr>b041744b3b15', 'voltype': 'LEAF', 'description': '', 'parent': '0c3de1a8-ac18-4d7b-b348-<wbr>3b097bf0a0ae', 'format': 'COW', 'generation': 0, 'image': '6a887015-67cd-4f7b-b709-<wbr>eef97142258d', 'ctime': '1499894539', 'disktype': '2', 'legality': 'LEGAL', 'mtime': '0', 'apparentsize': '1073741824', 'children': [], 'pool': '', 'capacity': '107374182400', 'uuid': u'5921ba71-0f00-46cd-b0be-<wbr>3c2ac1396845', 'truesize': '1073741824', 'type': 'SPARSE', 'lease': {'owners': [], 'version': None}} (volume:272)</font></div><div id="m_-4245500177440494772bloop_customfont" style="margin:0px"><font face="Input Mono">2017-07-12 14:22:36,186-0700 INFO (jsonrpc/5) [vdsm.api] FINISH getVolumeInfo return={'info': {'status': 'OK', 'domain': '0c02a758-4295-4199-97de-<wbr>b041744b3b15', 'voltype': 'LEAF', 'description': '', 'parent': '0c3de1a8-ac18-4d7b-b348-<wbr>3b097bf0a0ae', 'format': 'COW', 'generation': 0, 'image': '6a887015-67cd-4f7b-b709-<wbr>eef97142258d', 'ctime': '1499894539', 'disktype': '2', 'legality': 'LEGAL', 'mtime': '0', 'apparentsize': '1073741824', 'children': [], 'pool': '', 'capacity': '107374182400', 'uuid': u'5921ba71-0f00-46cd-b0be-<wbr>3c2ac1396845', 'truesize': '1073741824', 'type': 'SPARSE', 'lease': {'owners': [], 'version': None}}} from=::ffff:10.4.64.7,59664, flow_id=e94eebf8-75dc-407a-<wbr>8916-f4ff632f843e (api:52)</font></div><div id="m_-4245500177440494772bloop_customfont" style="margin:0px"><font face="Input Mono">2017-07-12 14:24:24,854-0700 INFO (jsonrpc/1) [vdsm.api] START deleteVolume(sdUUID=u'<wbr>0c02a758-4295-4199-97de-<wbr>b041744b3b15', spUUID=u'00000001-0001-0001-<wbr>0001-000000000311', imgUUID=u'6a887015-67cd-4f7b-<wbr>b709-eef97142258d', volumes=[u'5921ba71-0f00-46cd-<wbr>b0be-3c2ac1396845'], postZero=u'false', force=u'false', discard=False) from=::ffff:10.4.64.7,60016, flow_id=c5e4bda4-9cd3-461d-<wbr>8164-51d5614b995e (api:46)</font></div><div id="m_-4245500177440494772bloop_customfont" style="margin:0px"><font face="Input Mono">2017-07-12 14:24:25,010-0700 INFO (tasks/7) [storage.Volume] Request to delete LV 5921ba71-0f00-46cd-b0be-<wbr>3c2ac1396845 of image 6a887015-67cd-4f7b-b709-<wbr>eef97142258d in VG 0c02a758-4295-4199-97de-<wbr>b041744b3b15 (blockVolume:579)</font></div><div id="m_-4245500177440494772bloop_customfont" style="margin:0px"><font face="Input Mono">2017-07-12 14:24:25,130-0700 INFO (tasks/7) [storage.VolumeManifest] sdUUID=0c02a758-4295-4199-<wbr>97de-b041744b3b15 imgUUID=6a887015-67cd-4f7b-<wbr>b709-eef97142258d volUUID = 5921ba71-0f00-46cd-b0be-<wbr>3c2ac1396845 legality = ILLEGAL (volume:398)</font></div><div id="m_-4245500177440494772bloop_customfont" style="margin:0px"><font face="Input Mono">2017-07-12 14:24:38,881-0700 INFO (jsonrpc/2) [vdsm.api] START getVolumeInfo(sdUUID=u'<wbr>0c02a758-4295-4199-97de-<wbr>b041744b3b15', spUUID=u'00000001-0001-0001-<wbr>0001-000000000311', imgUUID=u'6a887015-67cd-4f7b-<wbr>b709-eef97142258d', volUUID=u'5921ba71-0f00-46cd-<wbr>b0be-3c2ac1396845', options=None) from=::ffff:10.4.64.7,59664, flow_id=c5e4bda4-9cd3-461d-<wbr>8164-51d5614b995e (api:46)</font></div><div id="m_-4245500177440494772bloop_customfont" style="margin:0px"><font face="Input Mono">2017-07-12 14:24:49,911-0700 INFO (jsonrpc/1) [vdsm.api] START getVolumeInfo(sdUUID=u'<wbr>0c02a758-4295-4199-97de-<wbr>b041744b3b15', spUUID=u'00000001-0001-0001-<wbr>0001-000000000311', imgUUID=u'6a887015-67cd-4f7b-<wbr>b709-eef97142258d', volUUID=u'0c3de1a8-ac18-4d7b-<wbr>b348-3b097bf0a0ae', options=None) from=::ffff:10.4.64.7,59664, flow_id=c5e4bda4-9cd3-461d-<wbr>8164-51d5614b995e (api:46)</font></div><div id="m_-4245500177440494772bloop_customfont" style="margin:0px"><font face="Input Mono">2017-07-12 14:24:49,912-0700 INFO (jsonrpc/1) [storage.VolumeManifest] Info request: sdUUID=0c02a758-4295-4199-<wbr>97de-b041744b3b15 imgUUID=6a887015-67cd-4f7b-<wbr>b709-eef97142258d volUUID = 0c3de1a8-ac18-4d7b-b348-<wbr>3b097bf0a0ae (volume:238)</font></div><div id="m_-4245500177440494772bloop_customfont" style="margin:0px"><font face="Input Mono">2017-07-12 14:24:50,036-0700 INFO (jsonrpc/1) [storage.VolumeManifest] 0c02a758-4295-4199-97de-<wbr>b041744b3b15/6a887015-67cd-<wbr>4f7b-b709-eef97142258d/<wbr>0c3de1a8-ac18-4d7b-b348-<wbr>3b097bf0a0ae info is {'status': 'OK', 'domain': '0c02a758-4295-4199-97de-<wbr>b041744b3b15', 'voltype': 'LEAF', 'description': '', 'parent': '00000000-0000-0000-0000-<wbr>000000000000', 'format': 'COW', 'generation': 0, 'image': '6a887015-67cd-4f7b-b709-<wbr>eef97142258d', 'ctime': '1499885619', 'disktype': '2', 'legality': 'LEGAL', 'mtime': '0', 'apparentsize': '110729625600', 'children': [], 'pool': '', 'capacity': '107374182400', 'uuid': u'0c3de1a8-ac18-4d7b-b348-<wbr>3b097bf0a0ae', 'truesize': '110729625600', 'type': 'SPARSE', 'lease': {'owners': [], 'version': None}} (volume:272)</font></div><div id="m_-4245500177440494772bloop_customfont" style="margin:0px"><font face="Input Mono">2017-07-12 14:24:50,037-0700 INFO (jsonrpc/1) [vdsm.api] FINISH getVolumeInfo return={'info': {'status': 'OK', 'domain': '0c02a758-4295-4199-97de-<wbr>b041744b3b15', 'voltype': 'LEAF', 'description': '', 'parent': '00000000-0000-0000-0000-<wbr>000000000000', 'format': 'COW', 'generation': 0, 'image': '6a887015-67cd-4f7b-b709-<wbr>eef97142258d', 'ctime': '1499885619', 'disktype': '2', 'legality': 'LEGAL', 'mtime': '0', 'apparentsize': '110729625600', 'children': [], 'pool': '', 'capacity': '107374182400', 'uuid': u'0c3de1a8-ac18-4d7b-b348-<wbr>3b097bf0a0ae', 'truesize': '110729625600', 'type': 'SPARSE', 'lease': {'owners': [], 'version': None}}} from=::ffff:10.4.64.7,59664, flow_id=c5e4bda4-9cd3-461d-<wbr>8164-51d5614b995e (api:52)</font></div></div><div id="m_-4245500177440494772bloop_customfont" style="color:rgb(0,0,0);margin:0px"><br></div><div id="m_-4245500177440494772bloop_customfont" style="margin:0px"><font face="Input Mono">HELP, Right now I am starting to think Block Storage and oVIRT = BAD!</font></div><div id="m_-4245500177440494772bloop_customfont" style="margin:0px"><font face="Input Mono"><br></font></div><div id="m_-4245500177440494772bloop_customfont" style="color:rgb(0,0,0);margin:0px"><br></div><div id="m_-4245500177440494772bloop_customfont" style="color:rgb(0,0,0);margin:0px"><br></div></div><br><div class="m_-4245500177440494772bloop_sign" id="m_-4245500177440494772bloop_sign_1499896436188286976"><span style="font-family:'helvetica Neue',helvetica;font-size:14px">--</span><br style="font-family:'helvetica Neue',helvetica;font-size:14px"><div class="m_-4245500177440494772gmail_signature" style="font-family:'helvetica Neue',helvetica;font-size:14px"><div dir="ltr"><div><br></div><div>Devin Acosta</div><div>Red Hat Certified Architect, LinuxStack </div><div><br></div><div><br></div></div></div></div></div>
<br>______________________________<wbr>_________________<br>
Users mailing list<br>
<a href="mailto:Users@ovirt.org">Users@ovirt.org</a><br>
<a href="http://lists.ovirt.org/mailman/listinfo/users" rel="noreferrer" target="_blank">http://lists.ovirt.org/<wbr>mailman/listinfo/users</a><br>
<br></blockquote></div><br></div>