Re: [ovirt-users] Ghost Snapshot Disk

Hi All, I've a similar question , I can't remove a snapshot. oVirt version is the last stable 4.2.1.7 with engine running in non hosted mode. Before remove snapshot I've shutdown vm. These are logs from engine: 2018-03-08 16:57:47,153+01 INFO [org.ovirt.engine.core.bll.ProcessDownVmCommand] (EE-ManagedThreadFactory-engine-Thread-40253) [711eba7d] Running command: ProcessDownVmCommand internal: true. 2018-03-08 16:57:55,589+01 INFO [org.ovirt.engine.core. vdsbroker.monitoring.VmsStatisticsFetcher] (EE-ManagedThreadFactory-engineScheduled-Thread-71) [] Fetched 8 VMs from VDS '85bbf811-1069-4e67-ba86-e50dec9f5da9' 2018-03-08 16:59:00,561+01 INFO [org.ovirt.engine.core.bll.snapshots.RemoveDiskSnapshotsCommand] (default task-31) [ceb64d67-ce9e-41a8-93dd-ffa512bfd18e] Lock Acquired to object 'EngineLock:{exclusiveLocks='[281a869f-8541-49cf-894b-f583bd26083d=DISK]', sharedLocks=''}' 2018-03-08 16:59:00,599+01 INFO [org.ovirt.engine.core.bll.snapshots.RemoveDiskSnapshotsCommand] (EE-ManagedThreadFactory-engine-Thread-40284) [ceb64d67-ce9e-41a8-93dd-ffa512bfd18e] Running command: RemoveDiskSnapshotsCommand internal: false. Entities affected : ID: 01f9b5f2-9e48-4c24-80e5-dca7f1d4d128 Type: VMAction group MANIPULATE_VM_SNAPSHOTS with role type USER 2018-03-08 16:59:00,613+01 INFO [org.ovirt.engine.core.dal. dbbroker.auditloghandling.AuditLogDirector] (EE-ManagedThreadFactory-engine-Thread-40284) [ceb64d67-ce9e-41a8-93dd-ffa512bfd18e] EVENT_ID: USER_REMOVE_DISK_SNAPSHOT(373), Disk 'SOL_Disk1' from Snapshot(s) 'PHP 5.6.29' of VM 'SOL-DEV' deletion was initiated by admin@internal-authz. 2018-03-08 16:59:00,615+01 INFO [org.ovirt.engine.core.bll.snapshots.RemoveDiskSnapshotsCommand] (EE-ManagedThreadFactory-engine-Thread-40284) [ceb64d67-ce9e-41a8-93dd-ffa512bfd18e] Lock freed to object 'EngineLock:{exclusiveLocks='[ 281a869f-8541-49cf-894b-f583bd26083d=DISK]', sharedLocks=''}' 2018-03-08 16:59:00,993+01 INFO [org.ovirt.engine.core.bll.snapshots.RemoveDiskSnapshotsCommand] (EE-ManagedThreadFactory-engineScheduled-Thread-25) [ceb64d67-ce9e-41a8-93dd-ffa512bfd18e] Starting child command 1 of 1, image '64e8f28d-6c00-41d8-9f60-26a87d51cb8c' 2018-03-08 16:59:01,026+01 INFO [org.ovirt.engine.core.bll.snapshots. ColdMergeSnapshotSingleDiskCommand] (EE-ManagedThreadFactory-commandCoordinator-Thread-7) [ceb64d67-ce9e-41a8-93dd-ffa512bfd18e] Running command: ColdMergeSnapshotSingleDiskCommand internal: true. Entities affected : ID: 00000000-0000-0000-0000-000000000000 Type: Storage 2018-03-08 16:59:02,026+01 INFO [org.ovirt.engine.core.bll.snapshots. ColdMergeSnapshotSingleDiskCommand] (EE-ManagedThreadFactory-engineScheduled-Thread-98) [ceb64d67-ce9e-41a8-93dd-ffa512bfd18e] Command 'ColdMergeSnapshotSingleDisk' id '2bbd81c3-9fa1-4e48-ab69-5588e1367539' executing step 'PREPARE_MERGE' 2018-03-08 16:59:02,048+01 INFO [org.ovirt.engine.core.bll. storage.disk.image.PrepareMergeCommand] (EE-ManagedThreadFactory-engineScheduled-Thread-98) [ceb64d67-ce9e-41a8-93dd-ffa512bfd18e] Running command: PrepareMergeCommand internal: true. Entities affected : ID: 00000000-0000-0000-0000-000000000000 Type: Storage 2018-03-08 16:59:02,049+01 INFO [org.ovirt.engine.core. vdsbroker.irsbroker.PrepareMergeVDSCommand] (EE-ManagedThreadFactory-engineScheduled-Thread-98) [ceb64d67-ce9e-41a8-93dd-ffa512bfd18e] START, PrepareMergeVDSCommand( SPMColdMergeVDSCommandParameters:{storagePoolId='18d57688-6ed4-43b8-bd7c-0665b55950b7', ignoreFailoverLimit='false'}), log id: 447f2b58 2018-03-08 16:59:02,178+01 INFO [org.ovirt.engine.core. vdsbroker.irsbroker.PrepareMergeVDSCommand] (EE-ManagedThreadFactory-engineScheduled-Thread-98) [ceb64d67-ce9e-41a8-93dd-ffa512bfd18e] FINISH, PrepareMergeVDSCommand, log id: 447f2b58 2018-03-08 16:59:02,221+01 INFO [org.ovirt.engine.core.bll.tasks.CommandAsyncTask] (EE-ManagedThreadFactory-engineScheduled-Thread-98) [ceb64d67-ce9e-41a8-93dd-ffa512bfd18e] CommandAsyncTask::Adding CommandMultiAsyncTasks object for command 'b5983000-d637-47da-8aa2- fb8fec50b480' 2018-03-08 16:59:02,221+01 INFO [org.ovirt.engine.core.bll.CommandMultiAsyncTasks] (EE-ManagedThreadFactory-engineScheduled-Thread-98) [ceb64d67-ce9e-41a8-93dd-ffa512bfd18e] CommandMultiAsyncTasks::attachTask: Attaching task '0f762196-21ed-4b70-995d-729f3ed72425' to command 'b5983000-d637-47da-8aa2-fb8fec50b480'. 2018-03-08 16:59:02,235+01 INFO [org.ovirt.engine.core.bll.tasks.AsyncTaskManager] (EE-ManagedThreadFactory-engineScheduled-Thread-98) [ceb64d67-ce9e-41a8-93dd-ffa512bfd18e] Adding task '0f762196-21ed-4b70-995d-729f3ed72425' (Parent Command 'PrepareMerge', Parameters Type 'org.ovirt.engine.core.common.asynctasks.AsyncTaskParameters'), polling hasn't started yet.. 2018-03-08 16:59:02,241+01 INFO [org.ovirt.engine.core.bll.tasks.SPMAsyncTask] (EE-ManagedThreadFactory-engineScheduled-Thread-98) [ceb64d67-ce9e-41a8-93dd-ffa512bfd18e] BaseAsyncTask::startPollingTask: Starting to poll task '0f762196-21ed-4b70-995d-729f3ed72425'. 2018-03-08 16:59:03,262+01 INFO [org.ovirt.engine.core.bll. SerialChildCommandsExecutionCallback] (EE-ManagedThreadFactory-engineScheduled-Thread-79) [ceb64d67-ce9e-41a8-93dd-ffa512bfd18e] Command 'RemoveDiskSnapshots' (id: '8f600a64-5ec5-4f81-98de-2ced76193aa4') waiting on child command id: '2bbd81c3-9fa1-4e48-ab69-5588e1367539' type:'ColdMergeSnapshotSingleDisk' to complete 2018-03-08 16:59:04,271+01 INFO [org.ovirt.engine.core.bll. SerialChildCommandsExecutionCallback] (EE-ManagedThreadFactory-engineScheduled-Thread-67) [ceb64d67-ce9e-41a8-93dd-ffa512bfd18e] Command 'ColdMergeSnapshotSingleDisk' (id: '2bbd81c3-9fa1-4e48-ab69-5588e1367539') waiting on child command id: 'b5983000-d637-47da-8aa2-fb8fec50b480' type:'PrepareMerge' to complete 2018-03-08 16:59:07,322+01 INFO [org.ovirt.engine.core.bll. SerialChildCommandsExecutionCallback] (EE-ManagedThreadFactory-engineScheduled-Thread-86) [ceb64d67-ce9e-41a8-93dd-ffa512bfd18e] Command 'RemoveDiskSnapshots' (id: '8f600a64-5ec5-4f81-98de-2ced76193aa4') waiting on child command id: '2bbd81c3-9fa1-4e48-ab69-5588e1367539' type:'ColdMergeSnapshotSingleDisk' to complete 2018-03-08 16:59:08,331+01 INFO [org.ovirt.engine.core.bll. SerialChildCommandsExecutionCallback] (EE-ManagedThreadFactory-engineScheduled-Thread-100) [ceb64d67-ce9e-41a8-93dd-ffa512bfd18e] Command 'ColdMergeSnapshotSingleDisk' (id: '2bbd81c3-9fa1-4e48-ab69-5588e1367539') waiting on child command id: 'b5983000-d637-47da-8aa2-fb8fec50b480' type:'PrepareMerge' to complete 2018-03-08 16:59:09,986+01 INFO [org.ovirt.engine.core.bll.tasks.AsyncTaskManager] (EE-ManagedThreadFactory-engineScheduled-Thread-38) [] Polling and updating Async Tasks: 1 tasks, 1 tasks to poll now 2018-03-08 16:59:09,998+01 ERROR [org.ovirt.engine.core. vdsbroker.vdsbroker.HSMGetAllTasksStatusesVDSCommand] (EE-ManagedThreadFactory-engineScheduled-Thread-38) [] Failed in 'HSMGetAllTasksStatusesVDS' method 2018-03-08 16:59:10,000+01 ERROR [org.ovirt.engine.core.dal. dbbroker.auditloghandling.AuditLogDirector] (EE-ManagedThreadFactory-engineScheduled-Thread-38) [] EVENT_ID: VDS_BROKER_COMMAND_FAILURE(10,802), VDSM infn-vm05.management command HSMGetAllTasksStatusesVDS failed: Volume does not exist: ('64e8f28d-6c00-41d8-9f60-26a87d51cb8c',) 2018-03-08 16:59:10,000+01 INFO [org.ovirt.engine.core.bll.tasks.SPMAsyncTask] (EE-ManagedThreadFactory-engineScheduled-Thread-38) [] SPMAsyncTask::PollTask: Polling task '0f762196-21ed-4b70-995d-729f3ed72425' (Parent Command 'PrepareMerge', Parameters Type 'org.ovirt.engine.core.common.asynctasks.AsyncTaskParameters') returned status 'finished', result 'cleanSuccess'. 2018-03-08 16:59:10,000+01 ERROR [org.ovirt.engine.core.bll.tasks.SPMAsyncTask] (EE-ManagedThreadFactory-engineScheduled-Thread-38) [] BaseAsyncTask::logEndTaskFailure: Task '0f762196-21ed-4b70-995d-729f3ed72425' (Parent Command 'PrepareMerge', Parameters Type 'org.ovirt.engine.core.common.asynctasks.AsyncTaskParameters') ended with failure: -- Result: 'cleanSuccess' -- Message: 'VDSGenericException: VDSErrorException: Failed to HSMGetAllTasksStatusesVDS, error = Volume does not exist: ('64e8f28d-6c00-41d8-9f60-26a87d51cb8c',), code = 201', -- Exception: 'VDSGenericException: VDSErrorException: Failed to HSMGetAllTasksStatusesVDS, error = Volume does not exist: ('64e8f28d-6c00-41d8-9f60-26a87d51cb8c',), code = 201' 2018-03-08 16:59:10,001+01 INFO [org.ovirt.engine.core.bll.tasks.CommandAsyncTask] (EE-ManagedThreadFactory-engineScheduled-Thread-38) [] CommandAsyncTask::endActionIfNecessary: All tasks of command 'b5983000-d637-47da-8aa2-fb8fec50b480' has ended -> executing 'endAction' 2018-03-08 16:59:10,001+01 INFO [org.ovirt.engine.core.bll.tasks.CommandAsyncTask] (EE-ManagedThreadFactory-engineScheduled-Thread-38) [] CommandAsyncTask::endAction: Ending action for '1' tasks (command ID: 'b5983000-d637-47da-8aa2-fb8fec50b480'): calling endAction '. 2018-03-08 16:59:10,001+01 INFO [org.ovirt.engine.core.bll.tasks.CommandAsyncTask] (EE-ManagedThreadFactory-engine-Thread-40287) [] CommandAsyncTask::endCommandAction [within thread] context: Attempting to endAction 'PrepareMerge', 2018-03-08 16:59:10,005+01 ERROR [org.ovirt.engine.core.bll. storage.disk.image.PrepareMergeCommand] (EE-ManagedThreadFactory-engine-Thread-40287) [ceb64d67-ce9e-41a8-93dd-ffa512bfd18e] Ending command 'org.ovirt.engine.core.bll.storage.disk.image.PrepareMergeCommand' with failure. 2018-03-08 16:59:10,013+01 INFO [org.ovirt.engine.core.bll.tasks.CommandAsyncTask] (EE-ManagedThreadFactory-engine-Thread-40287) [ceb64d67-ce9e-41a8-93dd-ffa512bfd18e] CommandAsyncTask::HandleEndActionResult [within thread]: endAction for action type 'PrepareMerge' completed, handling the result. 2018-03-08 16:59:10,013+01 INFO [org.ovirt.engine.core.bll.tasks.CommandAsyncTask] (EE-ManagedThreadFactory-engine-Thread-40287) [ceb64d67-ce9e-41a8-93dd-ffa512bfd18e] CommandAsyncTask::HandleEndActionResult [within thread]: endAction for action type 'PrepareMerge' succeeded, clearing tasks. 2018-03-08 16:59:10,014+01 INFO [org.ovirt.engine.core.bll.tasks.SPMAsyncTask] (EE-ManagedThreadFactory-engine-Thread-40287) [ceb64d67-ce9e-41a8-93dd-ffa512bfd18e] SPMAsyncTask::ClearAsyncTask: Attempting to clear task '0f762196-21ed-4b70-995d-729f3ed72425' 2018-03-08 16:59:10,016+01 INFO [org.ovirt.engine.core. vdsbroker.irsbroker.SPMClearTaskVDSCommand] (EE-ManagedThreadFactory-engine-Thread-40287) [ceb64d67-ce9e-41a8-93dd-ffa512bfd18e] START, SPMClearTaskVDSCommand( SPMTaskGuidBaseVDSCommandParameters:{storagePoolId=' 18d57688-6ed4-43b8-bd7c-0665b55950b7', ignoreFailoverLimit='false', taskId='0f762196-21ed-4b70-995d-729f3ed72425'}), log id: 5a0c3cc1 2018-03-08 16:59:10,017+01 INFO [org.ovirt.engine.core. vdsbroker.vdsbroker.HSMClearTaskVDSCommand] (EE-ManagedThreadFactory-engine-Thread-40287) [ceb64d67-ce9e-41a8-93dd-ffa512bfd18e] START, HSMClearTaskVDSCommand(HostName = infn-vm05.management, HSMTaskGuidBaseVDSCommandParam eters:{hostId='751f3e99-b95e-4c31-bc38-77f5661a0bdc', taskId='0f762196-21ed-4b70-995d-729f3ed72425'}), log id: 608b0bf 2018-03-08 16:59:10,033+01 INFO [org.ovirt.engine.core. vdsbroker.vdsbroker.HSMClearTaskVDSCommand] (EE-ManagedThreadFactory-engine-Thread-40287) [ceb64d67-ce9e-41a8-93dd-ffa512bfd18e] FINISH, HSMClearTaskVDSCommand, log id: 608b0bf 2018-03-08 16:59:10,033+01 INFO [org.ovirt.engine.core. vdsbroker.irsbroker.SPMClearTaskVDSCommand] (EE-ManagedThreadFactory-engine-Thread-40287) [ceb64d67-ce9e-41a8-93dd-ffa512bfd18e] FINISH, SPMClearTaskVDSCommand, log id: 5a0c3cc1 2018-03-08 16:59:10,035+01 INFO [org.ovirt.engine.core.bll.tasks.SPMAsyncTask] (EE-ManagedThreadFactory-engine-Thread-40287) [ceb64d67-ce9e-41a8-93dd-ffa512bfd18e] BaseAsyncTask::removeTaskFromDB: Removed task '0f762196-21ed-4b70-995d-729f3ed72425' from DataBase 2018-03-08 16:59:10,035+01 INFO [org.ovirt.engine.core.bll.tasks.CommandAsyncTask] (EE-ManagedThreadFactory-engine-Thread-40287) [ceb64d67-ce9e-41a8-93dd-ffa512bfd18e] CommandAsyncTask::HandleEndActionResult [within thread]: Removing CommandMultiAsyncTasks object for entity 'b5983000-d637-47da-8aa2- fb8fec50b480' 2018-03-08 16:59:15,375+01 INFO [org.ovirt.engine.core.bll. SerialChildCommandsExecutionCallback] (EE-ManagedThreadFactory-engineScheduled-Thread-57) [ceb64d67-ce9e-41a8-93dd-ffa512bfd18e] Command 'RemoveDiskSnapshots' (id: '8f600a64-5ec5-4f81-98de-2ced76193aa4') waiting on child command id: '2bbd81c3-9fa1-4e48-ab69-5588e1367539' type:'ColdMergeSnapshotSingleDisk' to complete 2018-03-08 16:59:16,384+01 ERROR [org.ovirt.engine.core.bll.snapshots. ColdMergeSnapshotSingleDiskCommand] (EE-ManagedThreadFactory-engineScheduled-Thread-27) [ceb64d67-ce9e-41a8-93dd-ffa512bfd18e] Command 'ColdMergeSnapshotSingleDisk' id '2bbd81c3-9fa1-4e48-ab69-5588e1367539' failed executing step 'PREPARE_MERGE' 2018-03-08 16:59:16,384+01 INFO [org.ovirt.engine.core.bll. SerialChildCommandsExecutionCallback] (EE-ManagedThreadFactory-engineScheduled-Thread-27) [ceb64d67-ce9e-41a8-93dd-ffa512bfd18e] Command 'ColdMergeSnapshotSingleDisk' id: '2bbd81c3-9fa1-4e48-ab69-5588e1367539' child commands '[b5983000-d637-47da-8aa2-fb8fec50b480]' executions were completed, status 'FAILED' 2018-03-08 16:59:17,398+01 ERROR [org.ovirt.engine.core.bll.snapshots. ColdMergeSnapshotSingleDiskCommand] (EE-ManagedThreadFactory-engineScheduled-Thread-77) [ceb64d67-ce9e-41a8-93dd-ffa512bfd18e] Ending command 'org.ovirt.engine.core.bll.snapshots.ColdMergeSnapshotSingleDiskCommand' with failure. 2018-03-08 16:59:17,409+01 INFO [org.ovirt.engine.core.bll. SerialChildCommandsExecutionCallback] (EE-ManagedThreadFactory-engineScheduled-Thread-77) [ceb64d67-ce9e-41a8-93dd-ffa512bfd18e] Command 'RemoveDiskSnapshots' id: '8f600a64-5ec5-4f81-98de-2ced76193aa4' child commands '[2bbd81c3-9fa1-4e48-ab69-5588e1367539]' executions were completed, status 'FAILED' 2018-03-08 16:59:18,424+01 ERROR [org.ovirt.engine.core.bll.snapshots.RemoveDiskSnapshotsCommand] (EE-ManagedThreadFactory-engineScheduled-Thread-36) [ceb64d67-ce9e-41a8-93dd-ffa512bfd18e] Ending command 'org.ovirt.engine.core.bll.snapshots.RemoveDiskSnapshotsCommand' with failure. 2018-03-08 16:59:18,460+01 ERROR [org.ovirt.engine.core.dal. dbbroker.auditloghandling.AuditLogDirector] (EE-ManagedThreadFactory-engineScheduled-Thread-36) [ceb64d67-ce9e-41a8-93dd-ffa512bfd18e] EVENT_ID: USER_REMOVE_DISK_SNAPSHOT_FINISHED_FAILURE(376), Failed to complete deletion of Disk 'SOL_Disk1' from snapshot(s) 'PHP 5.6.29' of VM 'SOL-DEV' (User: admin@internal-authz). 2018-03-08 17:00:18,145+01 INFO [org.ovirt.engine.core.bll.tasks.AsyncTaskManager] (EE-ManagedThreadFactory-engineScheduled-Thread-46) [] Setting new tasks map. The map contains now 0 tasks 2018-03-08 17:00:18,145+01 INFO [org.ovirt.engine.core.bll.tasks.AsyncTaskManager] (EE-ManagedThreadFactory-engineScheduled-Thread-46) [] Cleared all tasks of pool '18d57688-6ed4-43b8-bd7c-0665b55950b7'. Thanks a lot Best Regards Enrico Il 08/03/18 16:30, Shani Leviim ha scritto: Hi Lionel, Can you please share once again your engine log (or at least the relevant part where that error message occurred)? *Regards, * *Shani Leviim * On Thu, Mar 8, 2018 at 1:56 PM, Lionel Caignec <caignec(a)cines.fr&gt; wrote: > Hi, > > i finished to move my data, but now when i want to remove my old disk i > get stuck to this error : > "Cannot detach Virtual Machine Disk. The disk is already configured in a > snapshot. In order to detach it, remove the disk's snapshots". > But like i said before there is no snapshot anymore. > So what can i do? Delete manually inside database? So where? > Delete manually lvm volume, so how can i find the good one? > > Please help ;). > > Lionel > > ----- Mail original ----- > De: "Lionel Caignec" <caignec(a)cines.fr&gt; > À: "Shani Leviim" <sleviim(a)redhat.com&gt; > Cc: "users" <users(a)ovirt.org&gt; > Envoyé: Mardi 6 Mars 2018 08:22:30 > Objet: Re: [ovirt-users] Ghost Snapshot Disk > > Hi, > > ok thank you for information (sorry for late response). > > I will do that. > > ----- Mail original ----- > De: "Shani Leviim" <sleviim(a)redhat.com&gt; > À: "Lionel Caignec" <caignec(a)cines.fr&gt; > Cc: "users" <users(a)ovirt.org&gt; > Envoyé: Mardi 27 Février 2018 14:19:45 > Objet: Re: [ovirt-users] Ghost Snapshot Disk > > Hi Lionel, > > Sorry for the delay in replying you. > > If it's possible from your side, syncing the data and destroying old disk > sounds about right. > > In addition, it seems like you're having this bug: > https://bugzilla.redhat.com/show_bug.cgi?id=1509629 > And it was fixed for version 4.1.9. and above. > > > > *Regards,* > > *Shani Leviim* > > On Mon, Feb 26, 2018 at 4:18 PM, Lionel Caignec <caignec(a)cines.fr&gt; wrote: > > > Ok so i reply myself, > > > > Version is 4.1.7.6-1 > > > > I just delete manually a snapshot previously created. But this is an io > > intensive vm, whit big disk (2,5To, and 5To). > > > > For the log, i cannot paste all my log on public list security reason, i > > will send you full in private. > > Here is an extract relevant to my error > > engine.log-20180210:2018-02-09 23:00:03,200+01 INFO > > [org.ovirt.engine.core.dal.dbbroker.auditloghandling.AuditLogDirector] > > (default task-312) [44402a8c-3196-43f0-ba33-307ea78e6f49] EVENT_ID: > > USER_CREATE_SNAPSHOT(45), Correlation ID: 44402a8c-3196-43f0-ba33-307ea7 > 8e6f49, > > Job ID: 030cd310-fec9-4a89-8c3f-7888504fe973, Call Stack: null, Custom > > ID: null, Custom Event ID: -1, Message: Snapshot > 'AUTO_7D_zz_nil_20180209_220002' > > creation for VM 'zz_nil' was initiated by snap_user@internal. > > engine.log-20180210:2018-02-09 23:01:06,578+01 INFO > > [org.ovirt.engine.core.dal.dbbroker.auditloghandling.AuditLogDirector] > > (DefaultQuartzScheduler5) [] EVENT_ID: USER_CREATE_SNAPSHOT_FINISHED_ > SUCCESS(68), > > Correlation ID: 44402a8c-3196-43f0-ba33-307ea78e6f49, Job ID: > > 030cd310-fec9-4a89-8c3f-7888504fe973, Call Stack: null, Custom ID: > null, > > Custom Event ID: -1, Message: Snapshot 'AUTO_7D_zz_nil_20180209_220002' > > creation for VM 'zz_nil' has been completed. > > engine.log-20180220:2018-02-19 17:01:23,800+01 INFO > > [org.ovirt.engine.core.dal.dbbroker.auditloghandling.AuditLogDirector] > > (default task-113) [] EVENT_ID: USER_REMOVE_SNAPSHOT(342), Correlation > ID: > > 06a9efa4-1b80-4021-bf3e-41ecebe58a88, Job ID: > c9a918a7-b00c-43cf-b6de-3659ac0765da, > > Call Stack: null, Custom ID: null, Custom Event ID: -1, Message: > Snapshot > > 'AUTO_7D_zz_nil_20180209_220002' deletion for VM 'zz_nil' was initiated > > by acaignec@ldap-cines-authz. > > engine.log-20180221:2018-02-20 22:24:45,174+01 ERROR > > [org.ovirt.engine.core.dal.dbbroker.auditloghandling.AuditLogDirector] > > (DefaultQuartzScheduler6) [06a9efa4-1b80-4021-bf3e-41ecebe58a88] > > EVENT_ID: USER_REMOVE_SNAPSHOT_FINISHED_FAILURE(357), Correlation ID: > > 06a9efa4-1b80-4021-bf3e-41ecebe58a88, Job ID: > c9a918a7-b00c-43cf-b6de-3659ac0765da, > > Call Stack: null, Custom ID: null, Custom Event ID: -1, Message: Failed > to > > delete snapshot 'AUTO_7D_zz_nil_20180209_220002' for VM 'zz_nil'. > > 2018-02-20 22:24:46,266+01 INFO [org.ovirt.engine.core.bll.tas > ks.SPMAsyncTask] > > (DefaultQuartzScheduler3) [516079c3] SPMAsyncTask::PollTask: Polling > task > > '34137342-4f30-476d-b16c-1cb7e0ea0ac0' (Parent Command 'DestroyImage', > > Parameters Type 'org.ovirt.engine.core.common. > asynctasks.AsyncTaskParameters') > > returned status 'finished', result 'success'. > > 2018-02-20 22:24:46,267+01 INFO [org.ovirt.engine.core.bll.tas > ks.SPMAsyncTask] > > (DefaultQuartzScheduler3) [516079c3] BaseAsyncTask::onTaskEndSuccess: > > Task '34137342-4f30-476d-b16c-1cb7e0ea0ac0' (Parent Command > > 'DestroyImage', Parameters Type 'org.ovirt.engine.core.common. > asynctasks.AsyncTaskParameters') > > ended successfully. > > 2018-02-20 22:24:46,268+01 INFO [org.ovirt.engine.core.bll.tas > ks.CommandAsyncTask] > > (DefaultQuartzScheduler3) [516079c3] CommandAsyncTask::endActionIfN > ecessary: > > All tasks of command 'fe8c91f2-386b-4b3f-bbf3-aeda8e9244c6' has ended > -> > > executing 'endAction' > > 2018-02-20 22:24:46,268+01 INFO [org.ovirt.engine.core.bll.tas > ks.CommandAsyncTask] > > (DefaultQuartzScheduler3) [516079c3] CommandAsyncTask::endAction: Ending > > action for '1' tasks (command ID: 'fe8c91f2-386b-4b3f-bbf3-aeda8 > e9244c6'): > > calling endAction '. > > 2018-02-20 22:24:46,268+01 INFO [org.ovirt.engine.core.bll.tas > ks.CommandAsyncTask] > > (org.ovirt.thread.pool-6-thread-20) [516079c3] > CommandAsyncTask::endCommandAction > > [within thread] context: Attempting to endAction 'DestroyImage', > > 2018-02-20 22:24:46,269+01 ERROR [org.ovirt.engine.core.bll.tas > ks.CommandAsyncTask] > > (org.ovirt.thread.pool-6-thread-20) [516079c3] [within thread]: > endAction > > for action type DestroyImage threw an exception.: > > java.lang.NullPointerException > > at org.ovirt.engine.core.bll.tasks.CoCoAsyncTaskHelper. > > endAction(CoCoAsyncTaskHelper.java:335) [bll.jar:] > > at org.ovirt.engine.core.bll.tasks.CommandCoordinatorImpl. > > endAction(CommandCoordinatorImpl.java:340) [bll.jar:] > > at org.ovirt.engine.core.bll.tasks.CommandAsyncTask. > > endCommandAction(CommandAsyncTask.java:154) [bll.jar:] > > at org.ovirt.engine.core.bll.tasks.CommandAsyncTask.lambda$ > > endActionIfNecessary$0(CommandAsyncTask.java:106) [bll.jar:] > > at org.ovirt.engine.core.utils.threadpool.ThreadPoolUtil$ > > InternalWrapperRunnable.run(ThreadPoolUtil.java:84) [utils.jar:] > > at java.util.concurrent.Executors$RunnableAdapter.call( > Executors.java:511) > > [rt.jar:1.8.0_161] > > at java.util.concurrent.FutureTask.run(FutureTask.java:266) > > [rt.jar:1.8.0_161] > > at java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPool > Executor.java:1149) > > [rt.jar:1.8.0_161] > > at java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoo > lExecutor.java:624) > > [rt.jar:1.8.0_161] > > at java.lang.Thread.run(Thread.java:748) [rt.jar:1.8.0_161] > > > > ----- Mail original ----- > > De: "Shani Leviim" <sleviim(a)redhat.com&gt; > > À: "Lionel Caignec" <caignec(a)cines.fr&gt; > > Envoyé: Lundi 26 Février 2018 14:42:38 > > Objet: Re: [ovirt-users] Ghost Snapshot Disk > > > > Yes, please. > > Can you detail a bit more regarding the actions you've done? > > > > I'm assuming that since the snapshot had no description, trying to > operate > > it caused the nullPointerException you've got. > > But I want to examine what was the cause for that. > > > > Also, can you please answer back to the list? > > > > > > > > *Regards,* > > > > *Shani Leviim* > > > > On Mon, Feb 26, 2018 at 3:37 PM, Lionel Caignec <caignec(a)cines.fr&gt; > wrote: > > > > > Version is 4.1.7.6-1 > > > > > > Do you want the log from the day i delete snapshot? > > > > > > ----- Mail original ----- > > > De: "Shani Leviim" <sleviim(a)redhat.com&gt; > > > À: "Lionel Caignec" <caignec(a)cines.fr&gt; > > > Cc: "users" <users(a)ovirt.org&gt; > > > Envoyé: Lundi 26 Février 2018 14:29:16 > > > Objet: Re: [ovirt-users] Ghost Snapshot Disk > > > > > > Hi, > > > > > > What is your engine version, please? > > > I'm trying to reproduce your steps, for understanding better was is > the > > > cause for that error. Therefore, a full engine log is needed. > > > Can you please attach it? > > > > > > Thanks, > > > > > > > > > *Shani Leviim* > > > > > > On Mon, Feb 26, 2018 at 2:48 PM, Lionel Caignec <caignec(a)cines.fr&gt; > > wrote: > > > > > > > Hi > > > > > > > > 1) this is error message from ui.log > > > > > > > > 2018-02-26 13:44:10,001+01 ERROR [org.ovirt.engine.ui.frontend. > > > > server.gwt.OvirtRemoteLoggingService] (default task-3) [] > Permutation > > > > name: 8C01181C3B121D0AAE1312275CC96415 > > > > 2018-02-26 13:44:10,001+01 ERROR [org.ovirt.engine.ui.frontend. > > > server.gwt.OvirtRemoteLoggingService] > > > > (default task-3) [] Uncaught exception: com.google.gwt.core.client. > > > JavaScriptException: > > > > (TypeError) > > > > __gwt$exception: <skipped>: Cannot read property 'F' of null > > > > at org.ovirt.engine.ui.uicommonweb.models.storage. > > > > DisksAllocationModel$3.$onSuccess(DisksAllocationModel.java:120) > > > > at org.ovirt.engine.ui.uicommonweb.models.storage. > > > > DisksAllocationModel$3.onSuccess(DisksAllocationModel.java:120) > > > > at org.ovirt.engine.ui.frontend.Frontend$2.$onSuccess( > > > Frontend.java:233) > > > > [frontend.jar:] > > > > at org.ovirt.engine.ui.frontend.F > rontend$2.onSuccess(Frontend. > > > java:233) > > > > [frontend.jar:] > > > > at org.ovirt.engine.ui.frontend.communication. > > > > OperationProcessor$2.$onSuccess(OperationProcessor.java:139) > > > > [frontend.jar:] > > > > at org.ovirt.engine.ui.frontend.communication. > > > > OperationProcessor$2.onSuccess(OperationProcessor.java:139) > > > > [frontend.jar:] > > > > at org.ovirt.engine.ui.frontend.communication. > > > > GWTRPCCommunicationProvider$5$1.$onSuccess( > > GWTRPCCommunicationProvider. > > > java:269) > > > > [frontend.jar:] > > > > at org.ovirt.engine.ui.frontend.communication. > > > > GWTRPCCommunicationProvider$5$1.onSuccess(GWTRPCCommunicatio > nProvider. > > > java:269) > > > > [frontend.jar:] > > > > at com.google.gwt.user.client.rpc > .impl.RequestCallbackAdapter. > > > > onResponseReceived(RequestCallbackAdapter.java:198) > [gwt-servlet.jar:] > > > > at com.google.gwt.http.client.Req > uest.$fireOnResponseReceived( > > > Request.java:237) > > > > [gwt-servlet.jar:] > > > > at com.google.gwt.http.client.RequestBuilder$1. > > > onReadyStateChange(RequestBuilder.java:409) > > > > [gwt-servlet.jar:] > > > > at Unknown.eval(webadmin-0.js@65) > > > > at com.google.gwt.core.client.imp > l.Impl.apply(Impl.java:296) > > > > [gwt-servlet.jar:] > > > > at com.google.gwt.core.client.imp > l.Impl.entry0(Impl.java:335) > > > > [gwt-servlet.jar:] > > > > at Unknown.eval(webadmin-0.js@54) > > > > > > > > > > > > 2) This line seems to be about the bad disk : > > > > > > > > f242cc9a-56c1-4ae4-aef0-f75eb01f74b1 | 2018-01-17 22:02:00+01 | > > > > 2748779069440 | 00000000-0000-0000-0000-000000000000 | > > > > 00000000-0000-0000-0000-000000000000 | 4 | 2018-01-18 > > > > 22:01:20.5+01 | 0dd2090c-3491-4fa1-98c3-54ae88be793c > > > > > > > > > > > > 3) Snapshot table is empty for the concerned vm_id. > > > > > > > > ----- Mail original ----- > > > > De: "Shani Leviim" <sleviim(a)redhat.com&gt; > > > > À: "Lionel Caignec" <caignec(a)cines.fr&gt; > > > > Cc: "users" <users(a)ovirt.org&gt; > > > > Envoyé: Lundi 26 Février 2018 13:31:23 > > > > Objet: Re: [ovirt-users] Ghost Snapshot Disk > > > > > > > > Hi Lionel, > > > > > > > > The error message you've mentioned sounds like a UI error. > > > > Can you please attach your ui log? > > > > > > > > Also, on the data from 'images' table you've uploaded, can you > describe > > > > which line is the relevant disk? > > > > > > > > Finally (for now), in case the snapshot was deleted, can you please > > > > validate it by viewing the output of: > > > > $ select * from snapshots; > > > > > > > > > > > > > > > > *Regards,* > > > > > > > > *Shani Leviim* > > > > > > > > On Mon, Feb 26, 2018 at 9:20 AM, Lionel Caignec <caignec(a)cines.fr&gt; > > > wrote: > > > > > > > > > Hi Shani, > > > > > thank you for helping me with your reply, > > > > > i juste make a little mistake on explanation. In fact it's the > > snapshot > > > > > does not exist anymore. This is the disk(s) relative to her wich > > still > > > > > exist, and perhaps LVM volume. > > > > > So can i delete manually this disk in database? what about the lvm > > > > volume? > > > > > Is it better to recreate disk sync data and destroy old one? > > > > > > > > > > > > > > > > > > > > ----- Mail original ----- > > > > > De: "Shani Leviim" <sleviim(a)redhat.com&gt; > > > > > À: "Lionel Caignec" <caignec(a)cines.fr&gt; > > > > > Cc: "users" <users(a)ovirt.org&gt; > > > > > Envoyé: Dimanche 25 Février 2018 14:26:41 > > > > > Objet: Re: [ovirt-users] Ghost Snapshot Disk > > > > > > > > > > Hi Lionel, > > > > > > > > > > You can try to delete that snapshot directly from the database. > > > > > > > > > > In case of using psql [1], once you've logged in to your > database, > > you > > > > can > > > > > run this query: > > > > > $ select * from snapshots where vm_id = '<your_vm_id>'; > > > > > This one would list the snapshots associated with a VM by its id. > > > > > > > > > > In case you don't have you vm_id, you can locate it by querying: > > > > > $ select * from vms where vm_name = 'nil'; > > > > > This one would show you some details about a VM by its name > > (including > > > > the > > > > > vm's id). > > > > > > > > > > Once you've found the relevant snapshot, you can delete it by > > running: > > > > > $ delete from snapshots where snapshot_id = '<your_snapshot_id>'; > > > > > This one would delete the desired snapshot from the database. > > > > > > > > > > Since it's a delete operation, I would suggest confirming the ids > > > before > > > > > executing it. > > > > > > > > > > Hope you've found it useful! > > > > > > > > > > [1] > > > > > https://www.ovirt.org/documentation/install-guide/ > > > > appe-Preparing_a_Remote_ > > > > > PostgreSQL_Database_for_Use_with_the_oVirt_Engine/ > > > > > > > > > > > > > > > *Regards,* > > > > > > > > > > *Shani Leviim* > > > > > > > > > > On Fri, Feb 23, 2018 at 9:25 AM, Lionel Caignec <caignec(a)cines.fr > > > > > > wrote: > > > > > > > > > > > Hi, > > > > > > > > > > > > i've a problem with snapshot. On one VM i've a "snapshot" ghost > > > without > > > > > > name or uuid, only information is size (see attachment). In the > > > > snapshot > > > > > > tab there is no trace about this disk. > > > > > > > > > > > > In database (table images) i found this : > > > > > > f242cc9a-56c1-4ae4-aef0-f75eb01f74b1 | 2018-01-17 > 22:02:00+01 | > > > > > > 2748779069440 | 00000000-0000-0000-0000-000000000000 | > > > > > > 00000000-0000-0000-0000-000000000000 | 4 | 2018-01-18 > > > > > > 22:01:20.5+01 | 0dd2090c-3491-4fa1-98c3-54ae88be793c > > > > > > | 2 | 4 | 17e26476-cecb-441d-a5f7- > > > 46ab3ef387ee > > > > | > > > > > > 2018-01-17 22:01:29.663334+01 | 2018-01-19 08:40:14.345229+01 | > f > > > > | > > > > > > 1 | 2 > > > > > > 1c7650fa-542b-4ec2-83a1-d2c1c31be5fd | 2018-01-17 22:02:03+01 > | > > > > > > 5368709120000 | 00000000-0000-0000-0000-000000000000 | > > > > > > 00000000-0000-0000-0000-000000000000 | 4 | 2018-01-18 > > > > > > 22:01:20.84+01 | 0dd2090c-3491-4fa1-98c3-54ae88be793c > > > > > > | 2 | 4 | bf834a91-c69f-4d2c-b639- > > > 116ed58296d8 > > > > | > > > > > > 2018-01-17 22:01:29.836133+01 | 2018-01-19 08:40:19.083508+01 | > f > > > > | > > > > > > 1 | 2 > > > > > > 8614b21f-c0de-40f2-b4fb-e5cf193b0743 | 2018-02-09 23:00:44+01 > | > > > > > > 5368709120000 | 00000000-0000-0000-0000-000000000000 | > > > > > > 00000000-0000-0000-0000-000000000000 | 4 | 2018-02-16 > > > > > > 23:00:02.855+01 | 390175dc-baf4-4831-936a-5ea68fa4c969 > > > > > > > > > > > > > > > > > > But i does not know which line is my disk. Is it possible to > > delete > > > > > > directly into database? > > > > > > Or is it better to dump my disk to another new and delete the > > > > "corrupted > > > > > > one"? > > > > > > > > > > > > Another thing, when i try to move the disk to another storage > > > domain i > > > > > > always get "uncaght exeption occured ..." and no error in > > engine.log. > > > > > > > > > > > > > > > > > > Thank you for helping. > > > > > > > > > > > > -- > > > > > > Lionel Caignec > > > > > > > > > > > > _______________________________________________ > > > > > > Users mailing list > > > > > > Users(a)ovirt.org > > > > > > http://lists.ovirt.org/mailman/listinfo/users > > > > > > > > > > > > > > > > > > > > > > > > > > > _______________________________________________ > Users mailing list > Users(a)ovirt.org > http://lists.ovirt.org/mailman/listinfo/users > _______________________________________________ Users mailing listUsers@ovirt.orghttp://lists.ovirt.org/mailman/listinfo/users _______________________________________________ Users mailing list Users(a)ovirt.org http://lists.ovirt.org/mailman/listinfo/users