11:47 p.m.
On 02/16/2016 07:10 AM, Marcelo Leandro wrote:
Hello,
I have the same problem, i tried delete snapshot but it did not
success, the status snapshot as illegal , look the engine.log follow
you can see the messages error:
Hi Marcelo,
The problem in your log,
error = Drive image file could not be found, code = 13
is a little different but may have been triggered by the same bug in a
previous merge attempt.
In this case, would you run the "VM disk info gathering tool" from [1]?
See Adam's comment in the bug about its execution.
If this shows that the VM is no longer dependent on that missing volume,
then please shut down engine, back up the db, and run the "post-merge
failure repair script", also attached to [1].
Arguments in your case would be the VM name, snapshot name, and the UUID
of the image that is missing from your storage. (You may need to manually
mark the image as illegal first, [2]).
HTH,
Greg
[1] https://bugzilla.redhat.com/show_bug.cgi?id=1306741
[2] UPDATE images SET imagestatus = 4 WHERE image_guid = '<imgId>'::UUID;
2016-02-16 08:46:20,059 INFO
[org.ovirt.engine.core.bll.RemoveSnapshotCommandCallback]
(DefaultQuartzScheduler_Worker-57) [46dd2ef7] Waiting on Live Merge
child commands to complete
2016-02-16 08:46:21,069 INFO
[org.ovirt.engine.core.bll.RemoveSnapshotSingleDiskLiveCommand]
(DefaultQuartzScheduler_Worker-40) [15b703ee] Waiting on Live Merge
command step 'MERGE' to complete
2016-02-16 08:46:22,072 INFO
[org.ovirt.engine.core.bll.MergeCommandCallback]
(DefaultQuartzScheduler_Worker-65) [30cdf6ed] Waiting on merge command
to complete
2016-02-16 08:46:23,670 INFO
[org.ovirt.engine.core.bll.RemoveSnapshotCommand] (default task-48)
[5e0c088f] Lock Acquired to object
'EngineLock:{exclusiveLocks='[94d788f4-eba4-49ee-8091-80028cc46627=<VM,
ACTION_TYPE_FAILED_OBJECT_LOCKED>]', sharedLocks='null'}'
2016-02-16 08:46:23,795 INFO
[org.ovirt.engine.core.bll.RemoveSnapshotCommand] (default task-48)
[5e0c088f] Running command: RemoveSnapshotCommand internal: false.
Entities affected : ID: 94d788f4-eba4-49ee-8091-80028cc46627 Type:
VMAction group MANIPULATE_VM_SNAPSHOTS with role type USER
2016-02-16 08:46:23,824 INFO
[org.ovirt.engine.core.bll.RemoveSnapshotCommand] (default task-48)
[5e0c088f] Lock freed to object
'EngineLock:{exclusiveLocks='[94d788f4-eba4-49ee-8091-80028cc46627=<VM,
ACTION_TYPE_FAILED_OBJECT_LOCKED>]', sharedLocks='null'}'
2016-02-16 08:46:23,876 INFO
[org.ovirt.engine.core.bll.RemoveSnapshotSingleDiskLiveCommand]
(pool-7-thread-5) [1be123ac] Running command:
RemoveSnapshotSingleDiskLiveCommand internal: true. Entities affected
: ID: 00000000-0000-0000-0000-000000000000 Type: Storage
2016-02-16 08:46:23,921 INFO
[org.ovirt.engine.core.dal.dbbroker.auditloghandling.AuditLogDirector]
(default task-48) [] Correlation ID: 5e0c088f, Job ID:
aa811e83-24fb-4658-b849-d36439f58d95, Call Stack: null, Custom Event
ID: -1, Message: Snapshot 'BKP the VM' deletion for VM
'Servidor-Cliente' was initiated by admin@internal.
2016-02-16 08:46:24,093 INFO
[org.ovirt.engine.core.bll.RemoveSnapshotSingleDiskLiveCommand]
(DefaultQuartzScheduler_Worker-14) [1be123ac] Executing Live Merge
command step 'EXTEND'
2016-02-16 08:46:24,122 INFO
[org.ovirt.engine.core.bll.RemoveSnapshotCommandCallback]
(DefaultQuartzScheduler_Worker-14) [] Waiting on Live Merge child
commands to complete
2016-02-16 08:46:24,133 INFO
[org.ovirt.engine.core.bll.MergeExtendCommand] (pool-7-thread-6)
[766ffc9f] Running command: MergeExtendCommand internal: true.
Entities affected : ID: c2dc0101-748e-4a7b-9913-47993eaa52bd Type:
Storage
2016-02-16 08:46:24,134 INFO
[org.ovirt.engine.core.bll.MergeExtendCommand] (pool-7-thread-6)
[766ffc9f] Base and top image sizes are the same; no image size update
required
2016-02-16 08:46:25,133 INFO
[org.ovirt.engine.core.bll.RemoveSnapshotSingleDiskLiveCommand]
(DefaultQuartzScheduler_Worker-16) [1be123ac] Executing Live Merge
command step 'MERGE'
2016-02-16 08:46:25,168 INFO [org.ovirt.engine.core.bll.MergeCommand]
(pool-7-thread-7) [1b7bc421] Running command: MergeCommand internal:
true. Entities affected : ID: c2dc0101-748e-4a7b-9913-47993eaa52bd
Type: Storage
2016-02-16 08:46:25,169 INFO
[org.ovirt.engine.core.vdsbroker.vdsbroker.MergeVDSCommand]
(pool-7-thread-7) [1b7bc421] START, MergeVDSCommand(HostName = Host01,
MergeVDSCommandParameters:{runAsync='true',
hostId='d4f29978-1540-44d9-ab22-1e6ff750059f',
vmId='94d788f4-eba4-49ee-8091-80028cc46627',
storagePoolId='77e24b20-9d21-4952-a089-3c5c592b4e6d',
storageDomainId='c2dc0101-748e-4a7b-9913-47993eaa52bd',
imageGroupId='b7a27d0c-57cc-490e-a3f8-b4981310a9b0',
imageId='7f8bb099-9a18-4e89-bf48-57e56e5770d2',
baseImageId='2e59f7f2-9e30-460e-836a-5e0d3d625059',
topImageId='7f8bb099-9a18-4e89-bf48-57e56e5770d2', bandwidth='0'}),
log id: 2a7ab7b7
2016-02-16 08:46:25,176 ERROR
[org.ovirt.engine.core.vdsbroker.vdsbroker.MergeVDSCommand]
(pool-7-thread-7) [1b7bc421] Failed in 'MergeVDS' method
2016-02-16 08:46:25,179 ERROR
[org.ovirt.engine.core.dal.dbbroker.auditloghandling.AuditLogDirector]
(pool-7-thread-7) [1b7bc421] Correlation ID: null, Call Stack: null,
Custom Event ID: -1, Message: VDSM Host01 command failed: Drive image
file could not be found
2016-02-16 08:46:25,179 INFO
[org.ovirt.engine.core.vdsbroker.vdsbroker.MergeVDSCommand]
(pool-7-thread-7) [1b7bc421] Command
'org.ovirt.engine.core.vdsbroker.vdsbroker.MergeVDSCommand' return
value 'StatusOnlyReturnForXmlRpc [status=StatusForXmlRpc [code=13,
message=Drive image file could not be found]]'
2016-02-16 08:46:25,179 INFO
[org.ovirt.engine.core.vdsbroker.vdsbroker.MergeVDSCommand]
(pool-7-thread-7) [1b7bc421] HostName = Host01
2016-02-16 08:46:25,179 ERROR
[org.ovirt.engine.core.vdsbroker.vdsbroker.MergeVDSCommand]
(pool-7-thread-7) [1b7bc421] Command 'MergeVDSCommand(HostName =
Host01, MergeVDSCommandParameters:{runAsync='true',
hostId='d4f29978-1540-44d9-ab22-1e6ff750059f',
vmId='94d788f4-eba4-49ee-8091-80028cc46627',
storagePoolId='77e24b20-9d21-4952-a089-3c5c592b4e6d',
storageDomainId='c2dc0101-748e-4a7b-9913-47993eaa52bd',
imageGroupId='b7a27d0c-57cc-490e-a3f8-b4981310a9b0',
imageId='7f8bb099-9a18-4e89-bf48-57e56e5770d2',
baseImageId='2e59f7f2-9e30-460e-836a-5e0d3d625059',
topImageId='7f8bb099-9a18-4e89-bf48-57e56e5770d2', bandwidth='0'})'
execution failed: VDSGenericException: VDSErrorException: Failed to
MergeVDS, error = Drive image file could not be found, code = 13
2016-02-16 08:46:25,179 INFO
[org.ovirt.engine.core.vdsbroker.vdsbroker.MergeVDSCommand]
(pool-7-thread-7) [1b7bc421] FINISH, MergeVDSCommand, log id: 2a7ab7b7
2016-02-16 08:46:25,180 ERROR [org.ovirt.engine.core.bll.MergeCommand]
(pool-7-thread-7) [1b7bc421] Command
'org.ovirt.engine.core.bll.MergeCommand' failed: EngineException:
org.ovirt.engine.core.vdsbroker.vdsbroker.VDSErrorException:
VDSGenericException: VDSErrorException: Failed to MergeVDS, error =
Drive image file could not be found, code = 13 (Failed with error
imageErr and code 13)
2016-02-16 08:46:25,186 ERROR [org.ovirt.engine.core.bll.MergeCommand]
(pool-7-thread-7) [1b7bc421] Transaction rolled-back for command
'org.ovirt.engine.core.bll.MergeCommand'.
2016-02-16 08:46:26,159 INFO
[org.ovirt.engine.core.bll.RemoveSnapshotCommandCallback]
(DefaultQuartzScheduler_Worker-25) [15b703ee] Waiting on Live Merge
child commands to complete
2016-02-16 08:46:27,164 ERROR
[org.ovirt.engine.core.bll.RemoveSnapshotSingleDiskLiveCommand]
(DefaultQuartzScheduler_Worker-32) [1be123ac] Failed child command
status for step 'MERGE'
2016-02-16 08:46:27,497 INFO
[org.ovirt.engine.core.vdsbroker.VmAnalyzer]
(DefaultQuartzScheduler_Worker-37) [30cdf6ed] VM job
'77669e28-4aa2-4038-b7b6-1a949a1d039e': In progress, updating
2016-02-16 08:46:28,192 ERROR
[org.ovirt.engine.core.bll.RemoveSnapshotSingleDiskLiveCommand]
(DefaultQuartzScheduler_Worker-87) [1be123ac] Merging of snapshot
'119f668e-af60-49ea-aa08-735be8af0a7d' images
'2e59f7f2-9e30-460e-836a-5e0d3d625059'..'7f8bb099-9a18-4e89-bf48-57e56e5770d2'
failed. Images have been marked illegal and can no longer be previewed
or reverted to. Please retry Live Merge on the snapshot to complete
the operation.
2016-02-16 08:46:28,204 INFO
[org.ovirt.engine.core.bll.RemoveSnapshotCommandCallback]
(DefaultQuartzScheduler_Worker-87) [5e0c088f] All Live Merge child
commands have completed, status 'FAILED'
2016-02-16 08:46:29,216 ERROR
[org.ovirt.engine.core.bll.RemoveSnapshotCommand]
(DefaultQuartzScheduler_Worker-89) [5e0c088f] Ending command
'org.ovirt.engine.core.bll.RemoveSnapshotCommand' with failure.
2016-02-16 08:46:29,263 ERROR
[org.ovirt.engine.core.dal.dbbroker.auditloghandling.AuditLogDirector]
(DefaultQuartzScheduler_Worker-89) [5e0c088f] Correlation ID:
5e0c088f, Job ID: aa811e83-24fb-4658-b849-d36439f58d95, Call Stack:
null, Custom Event ID: -1, Message: Failed to delete snapshot 'BKP the
VM' for VM 'Servidor-Cliente'.
2016-02-16 08:46:30,287 INFO
[org.ovirt.engine.core.bll.RemoveSnapshotCommandCallback]
(DefaultQuartzScheduler_Worker-33) [] Waiting on Live Merge child
commands to complete
2016-02-16 08:46:31,298 INFO
[org.ovirt.engine.core.bll.RemoveSnapshotSingleDiskLiveCommand]
(DefaultQuartzScheduler_Worker-21) [15b703ee] Waiting on Live Merge
command step 'MERGE' to complete
2016-02-16 08:46:32,301 INFO
[org.ovirt.engine.core.bll.MergeCommandCallback]
(DefaultQuartzScheduler_Worker-68) [30cdf6ed] Waiting on merge command
to complete
2016-02-16 08:46:40,304 INFO
[org.ovirt.engine.core.bll.RemoveSnapshotCommandCallback]
(DefaultQuartzScheduler_Worker-55) [280a8a32] Waiting on Live Merge
child commands to complete
2016-02-16 08:46:41,308 INFO
[org.ovirt.engine.core.bll.RemoveSnapshotSingleDiskLiveCommand]
(DefaultQuartzScheduler_Worker-54) [15b703ee] Waiting on Live Merge
command step 'MERGE' to complete
2016-02-16 08:46:42,312 INFO
[org.ovirt.engine.core.bll.MergeCommandCallback]
(DefaultQuartzScheduler_Worker-57) [30cdf6ed] Waiting on merge command
to complete
2016-02-16 08:46:42,850 INFO
[org.ovirt.engine.core.vdsbroker.VmAnalyzer]
(DefaultQuartzScheduler_Worker-84) [] VM job
'77669e28-4aa2-4038-b7b6-1a949a1d039e': In progress, updating
2016-02-16 08:46:42,854 INFO
[org.ovirt.engine.core.vdsbroker.vdsbroker.FullListVDSCommand]
(DefaultQuartzScheduler_Worker-84) [] START,
FullListVDSCommand(HostName = ,
FullListVDSCommandParameters:{runAsync='true',
hostId='aebc403a-ec4e-4346-9029-6353d5d76f01',
vds='Host[,aebc403a-ec4e-4346-9029-6353d5d76f01]',
vmIds='[6af1f9c3-7210-45c3-90dc-bd7793346c0c]'}), log id: 74961dad
I cannot see the snapshot disk at the storage domain:
[root@ ~]# cd
/rhev/data-center/77e24b20-9d21-4952-a089-3c5c592b4e6d/c1938052-7524-404c-bac9-f238227269ea/images/b7a27d0c-57cc-490e-a3f8-b4981310a9b0/
[root@ b7a27d0c-57cc-490e-a3f8-b4981310a9b0]# ls
2e59f7f2-9e30-460e-836a-5e0d3d625059 2e59f7f2-9e30-460e-836a-5e0d3d625059.meta
Thanks.
2016-02-09 21:30 GMT-03:00 Greg Padgett <gpadgett(a)redhat.com>:
> On 02/09/2016 06:08 AM, Michal Skrivanek wrote:
>>
>>
>>> On 03 Feb 2016, at 10:37, Rik Theys <Rik.Theys(a)esat.kuleuven.be>
wrote:
>>>
>>> Hi,
>>>
>>> In the mean time I've noticed the following entries in our periodic
>>> logcheck output:
>>>
>>> Feb 3 09:05:53 orinoco journal: block copy still active: disk 'vda'
not
>>> ready for pivot yet
>>> Feb 3 09:05:53 orinoco journal: vdsm root ERROR Unhandled
>>> exception#012Traceback (most recent call last):#012 File
>>> "/usr/lib/python2.7/site-packages/vdsm/utils.py", line 734, in
>>> wrapper#012 return f(*a, **kw)#012 File
>>> "/usr/share/vdsm/virt/vm.py", line 5168, in run#012
>>> self.tryPivot()#012 File "/usr/share/vdsm/virt/vm.py", line 5137,
in
>>> tryPivot#012 ret = self.vm._dom.blockJobAbort(self.drive.name,
>>> flags)#012 File "/usr/share/vdsm/virt/virdomain.py", line 68, in
f#012
>>> ret = attr(*args, **kwargs)#012 File
>>> "/usr/lib/python2.7/site-packages/vdsm/libvirtconnection.py", line
124,
>>> in wrapper#012 ret = f(*args, **kwargs)#012 File
>>> "/usr/lib64/python2.7/site-packages/libvirt.py", line 733, in
>>> blockJobAbort#012 if ret == -1: raise libvirtError
>>> ('virDomainBlockJobAbort() failed', dom=self)#012libvirtError: block
>>> copy still active: disk 'vda' not ready for pivot yet
>>>
>>> This is from the host running the VM.
>>>
>>> Note that this host is not the SPM of the cluster. I always thought all
>>> operations on disk volumes happened on the SPM host?
>>>
>>> My question still remains:
>>>
>>>> I can see the snapshot in the "Disk snapshot" tab of the
storage. It has
>>>> a status of "illegal". Is it OK to (try to) remove this
snapshot? Will
>>>> this impact the running VM and/or disk image?
>>
>>
>> No, it’s not ok to remove it while live merge(apparently) is still ongoing
>> I guess that’s a live merge bug?
>
>
> Indeed, this is bug 1302215.
>
> I wrote a sql script to help with cleanup in this scenario, which you can
> find attached to the bug along with a description of how to use it[1].
>
> However, Rik, before trying that, would you be able to run the attached
> script [2] (or just the db query within) and forward the output to me? I'd
> like to make sure everything looks as it should before modifying the db
> directly.
>
> Thanks,
> Greg
>
> [1] https://bugzilla.redhat.com/show_bug.cgi?id=1302215#c13
> (Also note that the engine should be stopped before running this.)
>
> [2] Arguments are the ovirt db name, db user, and the name of the vm you
> were performing live merge on.
>
>
>> Thanks,
>> michal
>>
>>>
>>>
>>> Regards,
>>>
>>> Rik
>>>
>>> On 02/03/2016 10:26 AM, Rik Theys wrote:
>>>>
>>>> Hi,
>>>>
>>>> I created a snapshot of a running VM prior to an OS upgrade. The OS
>>>> upgrade has now been succesful and I would like to remove the snapshot.
>>>> I've selected the snapshot in the UI and clicked Delete to start the
>>>> task.
>>>>
>>>> After a few minutes, the task has failed. When I click delete again on
>>>> the same snapshot, the failed message is returned after a few seconds.
>>>>
>>>>> From browsing through the engine log (attached) it seems the
snapshot
>>>>
>>>> was correctly merged in the first try but something went wrong in the
>>>> finalizing fase. On retries, the log indicates the snapshot/disk image
>>>> no longer exists and the removal of the snapshot fails for this reason.
>>>>
>>>> Is there any way to clean up this snapshot?
>>>>
>>>> I can see the snapshot in the "Disk snapshot" tab of the
storage. It has
>>>> a status of "illegal". Is it OK to (try to) remove this
snapshot? Will
>>>> this impact the running VM and/or disk image?
>>>>
>>>> Regards,
>>>>
>>>> Rik
>>>>
>>>>
>>>>
>>>> _______________________________________________
>>>> Users mailing list
>>>> Users(a)ovirt.org
>>>> http://lists.ovirt.org/mailman/listinfo/users
>>>>
>>>
>>>
>>> --
>>> Rik Theys
>>> System Engineer
>>> KU Leuven - Dept. Elektrotechniek (ESAT)
>>> Kasteelpark Arenberg 10 bus 2440 - B-3001 Leuven-Heverlee
>>> +32(0)16/32.11.07
>>> ----------------------------------------------------------------
>>> <<Any errors in spelling, tact or fact are transmission errors>>
>>> _______________________________________________
>>> Users mailing list
>>> Users(a)ovirt.org
>>> http://lists.ovirt.org/mailman/listinfo/users
>>
>>
>> _______________________________________________
>> Users mailing list
>> Users(a)ovirt.org
>> http://lists.ovirt.org/mailman/listinfo/users
>>
>
>
> _______________________________________________
> Users mailing list
> Users(a)ovirt.org
> http://lists.ovirt.org/mailman/listinfo/users
>