<div dir="ltr">Seems very similar to this bug - <a href="https://bugzilla.redhat.com/show_bug.cgi?id=1213157">https://bugzilla.redhat.com/show_bug.cgi?id=1213157</a> </div><div class="gmail_extra"><br><div class="gmail_quote">On Thu, Apr 30, 2015 at 3:54 PM, Patrick Russell <span dir="ltr"><<a href="mailto:patrick_russell@volusion.com" target="_blank">patrick_russell@volusion.com</a>></span> wrote:<br><blockquote class="gmail_quote" style="margin:0 0 0 .8ex;border-left:1px #ccc solid;padding-left:1ex">
<div style="word-wrap:break-word">
Our packages in my earlier email were incorrect. Here are the correct packages and some more logs:
<div><br>
</div>
<div><span class="">
<div># rpm -qa |grep ovirt</div>
<div>ovirt-engine-cli-3.5.0.5-1.el6.noarch</div>
</span><div>ovirt-engine-setup-plugin-ovirt-engine-3.5.2-1.el6.noarch</div>
<div>ovirt-engine-setup-plugin-allinone-3.5.2-1.el6.noarch</div>
<div>ovirt-engine-setup-3.5.2-1.el6.noarch</div>
<div>ovirt-engine-dbscripts-3.5.2-1.el6.noarch</div>
<div>ovirt-engine-backend-3.5.2-1.el6.noarch</div>
<div>ovirt-guest-tools-3.5.0-0.5.master.noarch</div>
<div>ovirt-host-deploy-1.3.1-1.el6.noarch</div>
<div>ovirt-engine-sdk-python-3.5.2.1-1.el6.noarch</div>
<div>ovirt-engine-setup-plugin-ovirt-engine-common-3.5.2-1.el6.noarch</div>
<div>ovirt-engine-restapi-3.5.2-1.el6.noarch</div>
<div>ovirt-engine-userportal-3.5.2-1.el6.noarch</div>
<div>ovirt-engine-3.5.2-1.el6.noarch</div><span class="">
<div>ovirt-host-deploy-offline-1.3.1-1.el6.x86_64</div>
<div>ovirt-engine-setup-plugin-websocket-proxy-3.5.2-1.el6.noarch</div>
<div>ovirt-engine-websocket-proxy-3.5.2-1.el6.noarch</div>
<div>ovirt-iso-uploader-3.5.2-1.el6.noarch</div>
<div>ovirt-engine-extensions-api-impl-3.5.2-1.el6.noarch</div>
<div>ovirt-engine-jboss-as-7.1.1-1.el6.x86_64</div>
</span><span class=""><div>ovirt-guest-tools-iso-3.5-7.noarch</div>
<div>ovirt-image-uploader-3.5.1-1.el6.noarch</div>
<div>ovirt-engine-lib-3.5.2-1.el6.noarch</div>
<div>ovirt-engine-setup-base-3.5.2-1.el6.noarch</div>
<div>ovirt-release35-003-1.noarch</div>
</span><div>ovirt-engine-tools-3.5.2-1.el6.noarch</div>
<div>ovirt-engine-webadmin-portal-3.5.2-1.el6.noarch</div>
<div>ovirt-host-deploy-java-1.3.1-1.el6.noarch</div>
<div><br>
</div>
<div>Logs:</div>
<div><br>
</div>
<div>
<div>2015-04-30 19:50:24,578 INFO [org.ovirt.engine.core.bll.RemoveSnapshotCommand] (ajp--127.0.0.1-8702-6) [4cbab1c6] Lock Acquired to object EngineLock [exclusiveLocks= key: f2afb8c7-6b1d-4822-a43a-0889178de719 value: VM</div>
<div>, sharedLocks= ]</div>
<div>2015-04-30 19:50:24,689 INFO [org.ovirt.engine.core.bll.RemoveSnapshotCommand] (ajp--127.0.0.1-8702-6) [4cbab1c6] Running command: RemoveSnapshotCommand internal: false. Entities affected : ID: f2afb8c7-6b1d-4822-a43a-0889178de719 Type: VMAction
group MANIPULATE_VM_SNAPSHOTS with role type USER</div>
<div>2015-04-30 19:50:24,723 INFO [org.ovirt.engine.core.bll.RemoveSnapshotCommand] (ajp--127.0.0.1-8702-6) [4cbab1c6] Lock freed to object EngineLock [exclusiveLocks= key: f2afb8c7-6b1d-4822-a43a-0889178de719 value: VM</div>
<div>, sharedLocks= ]</div>
<div>2015-04-30 19:50:24,762 INFO [org.ovirt.engine.core.bll.RemoveSnapshotSingleDiskLiveCommand] (pool-7-thread-7) [598dbb1e] Running command: RemoveSnapshotSingleDiskLiveCommand internal: true. Entities affected : ID: 00000000-0000-0000-0000-000000000000
Type: Storage</div>
<div>2015-04-30 19:50:24,771 INFO [org.ovirt.engine.core.bll.RemoveSnapshotSingleDiskLiveCommand] (pool-7-thread-8) [5c8285eb] Running command: RemoveSnapshotSingleDiskLiveCommand internal: true. Entities affected : ID: 00000000-0000-0000-0000-000000000000
Type: Storage</div>
<div>2015-04-30 19:50:24,805 INFO [org.ovirt.engine.core.dal.dbbroker.auditloghandling.AuditLogDirector] (ajp--127.0.0.1-8702-6) Correlation ID: 4cbab1c6, Job ID: b0057939-d15a-4da7-b782-c835127a1413, Call Stack: null, Custom Event ID: -1, Message:
Snapshot 'test_snap1' deletion for VM '3.5.2.wintest1' was initiated by <a href="mailto:patrick_russell@corp.volusion.com" target="_blank">
patrick_russell@corp.volusion.com</a>.</div>
<div>2015-04-30 19:50:29,642 INFO [org.ovirt.engine.core.bll.RemoveSnapshotSingleDiskLiveCommand] (DefaultQuartzScheduler_Worker-13) [5c8285eb] Executing Live Merge command step MERGE</div>
<div>2015-04-30 19:50:29,665 INFO [org.ovirt.engine.core.bll.RemoveSnapshotCommandCallback] (DefaultQuartzScheduler_Worker-13) Waiting on Live Merge child commands to complete</div>
<div>2015-04-30 19:50:29,671 INFO [org.ovirt.engine.core.bll.MergeCommand] (pool-7-thread-9) [e10f6a3] Running command: MergeCommand internal: true. Entities affected : ID: 58ef50e1-2efb-4081-a3ca-493c2ee7e7b6 Type: Storage</div>
<div>2015-04-30 19:50:29,679 INFO [org.ovirt.engine.core.bll.RemoveSnapshotSingleDiskLiveCommand] (DefaultQuartzScheduler_Worker-13) [598dbb1e] Executing Live Merge command step MERGE</div>
<div>2015-04-30 19:50:29,697 INFO [org.ovirt.engine.core.bll.MergeCommand] (pool-7-thread-10) [4898b008] Running command: MergeCommand internal: true. Entities affected : ID: 58ef50e1-2efb-4081-a3ca-493c2ee7e7b6 Type: Storage</div>
<div>2015-04-30 19:50:29,703 INFO [org.ovirt.engine.core.vdsbroker.vdsbroker.MergeVDSCommand] (pool-7-thread-9) [e10f6a3] START, MergeVDSCommand(HostName =
<a href="http://aus02gdkvm01.dev.volusion.com" target="_blank">aus02gdkvm01.dev.volusion.com</a>, MergeVDSCommandParameters{HostId = cfc227ef-c9ab-4c35-aa22-138c169f2ba6, vmId=f2afb8c7-6b1d-4822-a43a-0889178de719, storagePoolId=edca3dc6-d9ca-410f-af84-9e24d49df3e0,
storageDomainId=58ef50e1-2efb-4081-a3ca-493c2ee7e7b6, imageGroupId=bb8359c6-1dba-45e2-97cb-929d1c83a7c8, imageId=87f26ef2-6a84-4f7c-9771-be76650c703c, baseImageId=3f90ced5-58ea-43cc-955a-559e8d89fbb2, topImageId=87f26ef2-6a84-4f7c-9771-be76650c703c, bandwidth=0}),
log id: 5859dc0</div>
<div>2015-04-30 19:50:29,728 INFO [org.ovirt.engine.core.vdsbroker.vdsbroker.MergeVDSCommand] (pool-7-thread-10) [4898b008] START, MergeVDSCommand(HostName =
<a href="http://aus02gdkvm01.dev.volusion.com" target="_blank">aus02gdkvm01.dev.volusion.com</a>, MergeVDSCommandParameters{HostId = cfc227ef-c9ab-4c35-aa22-138c169f2ba6, vmId=f2afb8c7-6b1d-4822-a43a-0889178de719, storagePoolId=edca3dc6-d9ca-410f-af84-9e24d49df3e0,
storageDomainId=58ef50e1-2efb-4081-a3ca-493c2ee7e7b6, imageGroupId=99e44df8-fa01-4255-bc1a-a4c240eb01e2, imageId=2ca5b861-52a8-4c0f-92a2-a8f0a3ed4129, baseImageId=7ad1cf7d-f2e6-41d4-a3c4-57458e80c43d, topImageId=2ca5b861-52a8-4c0f-92a2-a8f0a3ed4129, bandwidth=0}),
log id: 160b45fb</div>
<div>2015-04-30 19:50:29,819 ERROR [org.ovirt.engine.core.vdsbroker.vdsbroker.MergeVDSCommand] (pool-7-thread-9) [e10f6a3] Failed in MergeVDS method</div>
<div>2015-04-30 19:50:29,820 INFO [org.ovirt.engine.core.vdsbroker.vdsbroker.MergeVDSCommand] (pool-7-thread-9) [e10f6a3] Command org.ovirt.engine.core.vdsbroker.vdsbroker.MergeVDSCommand return value </div><span class="">
<div> StatusOnlyReturnForXmlRpc [mStatus=StatusForXmlRpc [mCode=52, mMessage=Merge failed]]</div>
</span><div>2015-04-30 19:50:29,821 INFO [org.ovirt.engine.core.vdsbroker.vdsbroker.MergeVDSCommand] (pool-7-thread-9) [e10f6a3] HostName =
<a href="http://aus02gdkvm01.dev.volusion.com" target="_blank">aus02gdkvm01.dev.volusion.com</a></div>
<div>2015-04-30 19:50:29,822 ERROR [org.ovirt.engine.core.vdsbroker.vdsbroker.MergeVDSCommand] (pool-7-thread-9) [e10f6a3] Command MergeVDSCommand(HostName =
<a href="http://aus02gdkvm01.dev.volusion.com" target="_blank">aus02gdkvm01.dev.volusion.com</a>, MergeVDSCommandParameters{HostId = cfc227ef-c9ab-4c35-aa22-138c169f2ba6, vmId=f2afb8c7-6b1d-4822-a43a-0889178de719, storagePoolId=edca3dc6-d9ca-410f-af84-9e24d49df3e0,
storageDomainId=58ef50e1-2efb-4081-a3ca-493c2ee7e7b6, imageGroupId=bb8359c6-1dba-45e2-97cb-929d1c83a7c8, imageId=87f26ef2-6a84-4f7c-9771-be76650c703c, baseImageId=3f90ced5-58ea-43cc-955a-559e8d89fbb2, topImageId=87f26ef2-6a84-4f7c-9771-be76650c703c, bandwidth=0})
execution failed. Exception: VDSErrorException: VDSGenericException: VDSErrorException: Failed to MergeVDS, error = Merge failed, code = 52</div>
<div>2015-04-30 19:50:29,827 INFO [org.ovirt.engine.core.vdsbroker.vdsbroker.MergeVDSCommand] (pool-7-thread-9) [e10f6a3] FINISH, MergeVDSCommand, log id: 5859dc0</div>
<div>2015-04-30 19:50:29,828 ERROR [org.ovirt.engine.core.bll.MergeCommand] (pool-7-thread-9) [e10f6a3] Command org.ovirt.engine.core.bll.MergeCommand throw Vdc Bll exception. With error message VdcBLLException: org.ovirt.engine.core.vdsbroker.vdsbroker.VDSErrorException:
VDSGenericException: VDSErrorException: Failed to MergeVDS, error = Merge failed, code = 52 (Failed with error mergeErr and code 52)</div>
<div>2015-04-30 19:50:29,834 ERROR [org.ovirt.engine.core.bll.MergeCommand] (pool-7-thread-9) [e10f6a3] Transaction rolled-back for command: org.ovirt.engine.core.bll.MergeCommand.</div>
<div>2015-04-30 19:50:29,858 ERROR [org.ovirt.engine.core.vdsbroker.vdsbroker.MergeVDSCommand] (pool-7-thread-10) [4898b008] Failed in MergeVDS method</div>
<div>2015-04-30 19:50:29,859 INFO [org.ovirt.engine.core.vdsbroker.vdsbroker.MergeVDSCommand] (pool-7-thread-10) [4898b008] Command org.ovirt.engine.core.vdsbroker.vdsbroker.MergeVDSCommand return value </div><span class="">
<div> StatusOnlyReturnForXmlRpc [mStatus=StatusForXmlRpc [mCode=52, mMessage=Merge failed]]</div>
</span><div>2015-04-30 19:50:29,860 INFO [org.ovirt.engine.core.vdsbroker.vdsbroker.MergeVDSCommand] (pool-7-thread-10) [4898b008] HostName =
<a href="http://aus02gdkvm01.dev.volusion.com" target="_blank">aus02gdkvm01.dev.volusion.com</a></div>
<div>2015-04-30 19:50:29,861 ERROR [org.ovirt.engine.core.vdsbroker.vdsbroker.MergeVDSCommand] (pool-7-thread-10) [4898b008] Command MergeVDSCommand(HostName =
<a href="http://aus02gdkvm01.dev.volusion.com" target="_blank">aus02gdkvm01.dev.volusion.com</a>, MergeVDSCommandParameters{HostId = cfc227ef-c9ab-4c35-aa22-138c169f2ba6, vmId=f2afb8c7-6b1d-4822-a43a-0889178de719, storagePoolId=edca3dc6-d9ca-410f-af84-9e24d49df3e0,
storageDomainId=58ef50e1-2efb-4081-a3ca-493c2ee7e7b6, imageGroupId=99e44df8-fa01-4255-bc1a-a4c240eb01e2, imageId=2ca5b861-52a8-4c0f-92a2-a8f0a3ed4129, baseImageId=7ad1cf7d-f2e6-41d4-a3c4-57458e80c43d, topImageId=2ca5b861-52a8-4c0f-92a2-a8f0a3ed4129, bandwidth=0})
execution failed. Exception: VDSErrorException: VDSGenericException: VDSErrorException: Failed to MergeVDS, error = Merge failed, code = 52</div>
<div>2015-04-30 19:50:29,864 INFO [org.ovirt.engine.core.vdsbroker.vdsbroker.MergeVDSCommand] (pool-7-thread-10) [4898b008] FINISH, MergeVDSCommand, log id: 160b45fb</div>
<div>2015-04-30 19:50:29,865 ERROR [org.ovirt.engine.core.bll.MergeCommand] (pool-7-thread-10) [4898b008] Command org.ovirt.engine.core.bll.MergeCommand throw Vdc Bll exception. With error message VdcBLLException: org.ovirt.engine.core.vdsbroker.vdsbroker.VDSErrorException:
VDSGenericException: VDSErrorException: Failed to MergeVDS, error = Merge failed, code = 52 (Failed with error mergeErr and code 52)</div>
<div>2015-04-30 19:50:29,871 ERROR [org.ovirt.engine.core.bll.MergeCommand] (pool-7-thread-10) [4898b008] Transaction rolled-back for command: org.ovirt.engine.core.bll.MergeCommand.</div>
<div>2015-04-30 19:50:39,698 ERROR [org.ovirt.engine.core.bll.RemoveSnapshotSingleDiskLiveCommand] (DefaultQuartzScheduler_Worker-73) [5c8285eb] Failed child command status for step MERGE</div>
<div>2015-04-30 19:50:39,701 INFO [org.ovirt.engine.core.bll.RemoveSnapshotCommandCallback] (DefaultQuartzScheduler_Worker-73) [5c8285eb] Waiting on Live Merge child commands to complete</div>
<div>2015-04-30 19:50:39,710 ERROR [org.ovirt.engine.core.bll.RemoveSnapshotSingleDiskLiveCommand] (DefaultQuartzScheduler_Worker-73) [598dbb1e] Failed child command status for step MERGE</div>
<div>2015-04-30 19:50:49,720 ERROR [org.ovirt.engine.core.bll.RemoveSnapshotSingleDiskLiveCommand] (DefaultQuartzScheduler_Worker-36) [5c8285eb] Merging of snapshot 470b525e-fb30-4956-a702-f4c6389bcb60 images 3f90ced5-58ea-43cc-955a-559e8d89fbb2..87f26ef2-6a84-4f7c-9771-be76650c703c
failed. Images have been marked illegal and can no longer be previewed or reverted to. Please retry Live Merge on the snapshot to complete the operation.</div>
<div>2015-04-30 19:50:49,725 INFO [org.ovirt.engine.core.bll.RemoveSnapshotCommandCallback] (DefaultQuartzScheduler_Worker-36) [4cbab1c6] All Live Merge child commands have completed, status FAILED</div>
<div>2015-04-30 19:50:49,734 ERROR [org.ovirt.engine.core.bll.RemoveSnapshotSingleDiskLiveCommand] (DefaultQuartzScheduler_Worker-36) [598dbb1e] Merging of snapshot 470b525e-fb30-4956-a702-f4c6389bcb60 images 7ad1cf7d-f2e6-41d4-a3c4-57458e80c43d..2ca5b861-52a8-4c0f-92a2-a8f0a3ed4129
failed. Images have been marked illegal and can no longer be previewed or reverted to. Please retry Live Merge on the snapshot to complete the operation.</div>
<div>2015-04-30 19:50:59,787 ERROR [org.ovirt.engine.core.bll.RemoveSnapshotCommand] (DefaultQuartzScheduler_Worker-16) [4cbab1c6] Ending command with failure: org.ovirt.engine.core.bll.RemoveSnapshotCommand</div>
<div>2015-04-30 19:50:59,837 ERROR [org.ovirt.engine.core.dal.dbbroker.auditloghandling.AuditLogDirector] (DefaultQuartzScheduler_Worker-16) [4cbab1c6] Correlation ID: 4cbab1c6, Call Stack: null, Custom Event ID: -1, Message: Failed to delete snapshot
'test_snap1' for VM '3.5.2.wintest1’.</div>
</div>
<div><br>
</div>
<div><br>
</div>
<div>
<blockquote type="cite"><div><div class="h5">
<div>On Apr 30, 2015, at 9:54 AM, Patrick Russell <<a href="mailto:patrick_russell@volusion.com" target="_blank">patrick_russell@volusion.com</a>> wrote:</div>
<br>
</div></div><div><div><div class="h5">
<div style="word-wrap:break-word">
<div>Hi everyone,</div>
<div><br>
</div>
We’re not seeing live merge working as of the 3.5.2 update. We’ve tested using fibre channel and NFS attached storage. Both throwing the same error code. Are other people seeing success with live-merge after the update?
<div><br>
</div>
<div>Here’s the environment:</div>
<div><br>
</div>
<div>Engine Running on CentOS 6x64 updated to 3.5.2 via yum update (standalone physical box, dual socket hex core + hyperthreading, 16GB memory)</div>
<div><br>
</div>
<div>
<div># rpm -qa |grep ovirt</div>
<div>ovirt-engine-cli-3.5.0.5-1.el6.noarch</div>
<div>ovirt-engine-3.5.1.1-1.el6.noarch</div>
<div>ovirt-engine-setup-plugin-ovirt-engine-3.5.2-1.el6.noarch</div>
<div>ovirt-engine-setup-plugin-allinone-3.5.2-1.el6.noarch</div>
<div>ovirt-engine-setup-3.5.2-1.el6.noarch</div>
<div>ovirt-guest-tools-3.5.0-0.5.master.noarch</div>
<div>ovirt-host-deploy-1.3.1-1.el6.noarch</div>
<div>ovirt-engine-sdk-python-3.5.2.1-1.el6.noarch</div>
<div>ovirt-engine-setup-plugin-ovirt-engine-common-3.5.2-1.el6.noarch</div>
<div>ovirt-engine-backend-3.5.1.1-1.el6.noarch</div>
<div>ovirt-engine-userportal-3.5.1.1-1.el6.noarch</div>
<div>ovirt-engine-dbscripts-3.5.1.1-1.el6.noarch</div>
<div>ovirt-engine-tools-3.5.1.1-1.el6.noarch</div>
<div>ovirt-host-deploy-offline-1.3.1-1.el6.x86_64</div>
<div>ovirt-engine-setup-plugin-websocket-proxy-3.5.2-1.el6.noarch</div>
<div>ovirt-engine-websocket-proxy-3.5.2-1.el6.noarch</div>
<div>ovirt-iso-uploader-3.5.2-1.el6.noarch</div>
<div>ovirt-engine-extensions-api-impl-3.5.2-1.el6.noarch</div>
<div>ovirt-engine-jboss-as-7.1.1-1.el6.x86_64</div>
<div>ovirt-engine-webadmin-portal-3.5.1.1-1.el6.noarch</div>
<div>ovirt-engine-restapi-3.5.1.1-1.el6.noarch</div>
<div>ovirt-guest-tools-iso-3.5-7.noarch</div>
<div>ovirt-image-uploader-3.5.1-1.el6.noarch</div>
<div>ovirt-engine-lib-3.5.2-1.el6.noarch</div>
<div>ovirt-engine-setup-base-3.5.2-1.el6.noarch</div>
<div>ovirt-release35-003-1.noarch</div>
<div>ovirt-host-deploy-java-1.3.1-1.el6.noarch</div>
</div>
<div><br>
</div>
<div>Hypervisors are running ovirt-node, upgraded from ISO : <a href="http://resources.ovirt.org/pub/ovirt-3.5/iso/ovirt-node/el7-3.5.2/ovirt-node-iso-3.5-0.999.201504280931.el7.centos.iso" target="_blank">http://resources.ovirt.org/pub/ovirt-3.5/iso/ovirt-node/el7-3.5.2/ovirt-node-iso-3.5-0.999.201504280931.el7.centos.iso</a></div>
<div><br>
</div>
<div><br>
</div>
<div>Here’s a snippet from the logs:
<div><br>
</div>
<div>
<div>2015-04-29 18:47:16,947 INFO [org.ovirt.engine.core.vdsbroker.vdsbroker.MergeVDSCommand] (pool-7-thread-2) [48eb0b1d] FINISH, MergeVDSCommand, log id: 5121ecc9</div>
<div>2015-04-29 18:47:16,947 ERROR [org.ovirt.engine.core.bll.MergeCommand] (pool-7-thread-2) [48eb0b1d] Command org.ovirt.engine.core.bll.MergeCommand throw Vdc Bll exception. With error message VdcBLLException: org.ovirt.engine.core.vdsbroker.vdsbroker.VDSErrorException:
VDSGenericException: VDSErrorException: Failed to MergeVDS, error = Merge failed, code = 52 (Failed with error mergeErr and code 52)</div>
<div>2015-04-29 18:47:16,954 ERROR [org.ovirt.engine.core.bll.MergeCommand] (pool-7-thread-2) [48eb0b1d] Transaction rolled-back for command: org.ovirt.engine.core.bll.MergeCommand.</div>
<div>2015-04-29 18:47:16,981 ERROR [org.ovirt.engine.core.vdsbroker.vdsbroker.MergeVDSCommand] (pool-7-thread-3) [5495bde7] Failed in MergeVDS method</div>
<div>2015-04-29 18:47:16,982 INFO [org.ovirt.engine.core.vdsbroker.vdsbroker.MergeVDSCommand] (pool-7-thread-3) [5495bde7] Command org.ovirt.engine.core.vdsbroker.vdsbroker.MergeVDSCommand return value </div>
<div> StatusOnlyReturnForXmlRpc [mStatus=StatusForXmlRpc [mCode=52, mMessage=Merge failed]]</div>
</div>
</div>
<div><br>
</div>
<div><br>
</div>
</div></div></div>
_______________________________________________<br>
Users mailing list<br>
<a href="mailto:Users@ovirt.org" target="_blank">Users@ovirt.org</a><br>
<a href="http://lists.ovirt.org/mailman/listinfo/users" target="_blank">http://lists.ovirt.org/mailman/listinfo/users</a><br>
</div>
</blockquote>
</div>
<br>
</div>
</div>
<br>_______________________________________________<br>
Users mailing list<br>
<a href="mailto:Users@ovirt.org">Users@ovirt.org</a><br>
<a href="http://lists.ovirt.org/mailman/listinfo/users" target="_blank">http://lists.ovirt.org/mailman/listinfo/users</a><br>
<br></blockquote></div><br></div>