[ovirt-users] Deleting large snapshots blocks the whole cluster

[Stefan Wendler]

I see. Then I will keep an eye on this tonight where we have a huge "all
ovirt downtime" just to delete those snapshots ^^


Thanks and Cheers,
Stefan


On 10/27/14 13:07, Markus Stockhausen wrote:
>> Von: Stefan Wendler [stefan.wendler at tngtech.com]
>> Gesendet: Montag, 27. Oktober 2014 11:39
>> An: Markus Stockhausen
>> Cc: users at ovirt.org
>> Betreff: Re: [ovirt-users] Deleting large snapshots blocks the whole cluster
>>
>> Hi,
>>
>> do you mean during snapshot deletion or in general?
>>
>> In general we do not have any swap usage at all. On none of the 4 hosts.
>>
>> We had some swap I/O on the host that is SPM most of the time. But not
>> during that period where we tried to delete the snapshot.
>>
> 
> I thought of the bug described in RH bugzilla 1138690 "[SCALE] snapshot deletion -> 
> heavy swapping on SPM". In case you have no access to it (as it is RHEV tagged): 
> 
> qemu-img reads through pagecache during snapshot deletion. A new flag has been 
> intrduced in August this year that allows to avoid page cache. This flag must be
> provided by OVirt/RHEV. Target release is 3.6. 
> 
> Until then the only bugfix is to drop pagecache manually on regular intervals on the 
> hypversior and especially the SPM node.
> 
> Markus
> 
>>
>> Cheers,
>> Stefan
>>
>> On 10/27/14 11:08, Markus Stockhausen wrote:
>>> Do you see swapping on the SPM? If yes a regular echo 3 > drop_caches could help.
>>>
>>> Markus
>>>
>>> Am 27.10.2014 10:57 schrieb Stefan Wendler <stefan.wendler at tngtech.com>:
>>> Hi,
>>>
>>> we have some really large snapshots left from a migration. Since our
>>> store is almost full now we have to delete them now.
>>>
>>> Some snapshots are around 1TB already.
>>>
>>> The last time we tried to delete a ~500GB snapshot the delete task
>> blocked the whole diskstore's IO and the whole cluster and all hosts
>> became unavailable.
>>
>> Is there a way to delete large snapshots in a "humane" way that will not
>> block everything?
>>
>> Cheers,
>> Stefan
>>
>>
> 

-------------- next part --------------
A non-text attachment was scrubbed...
Name: signature.asc
Type: application/pgp-signature
Size: 181 bytes
Desc: OpenPGP digital signature
URL: <http://lists.ovirt.org/pipermail/users/attachments/20141027/ce0e3eb4/attachment.sig>