Re: [ovirt-users] Deleting large snapshots blocks the whole cluster
This is an OpenPGP/MIME signed message (RFC 4880 and 3156)
--pRrw2Nflj1DiHOHuNiU2AGGrNfrnE9WXR
Content-Type: text/plain; charset=windows-1252
Content-Transfer-Encoding: quoted-printable
I see. Then I will keep an eye on this tonight where we have a huge "all
ovirt downtime" just to delete those snapshots ^^
Thanks and Cheers,
Stefan
On 10/27/14 13:07, Markus Stockhausen wrote:
> Von: Stefan Wendler [stefan.wendler(a)tngtech.com]
> Gesendet: Montag, 27. Oktober 2014 11:39
> An: Markus Stockhausen
> Cc: users(a)ovirt.org
> Betreff: Re: [ovirt-users] Deleting large snapshots blocks the whole c=
luster
>
> Hi,
>
> do you mean during snapshot deletion or in general?
>
> In general we do not have any swap usage at all. On none of the 4 host=
s.
>
> We had some swap I/O on the host that is SPM most of the time. But not=
> during that period where we tried to delete the snapshot.
>
=20
I thought of the bug described in RH bugzilla 1138690 "[SCALE] snapshot=
deletion ->=20
heavy swapping on SPM". In case you have no access to it (as it
is RHEV=
tagged):=20
=20
qemu-img reads through pagecache during snapshot deletion. A new flag h=
as
been=20
intrduced in August this year that allows to avoid page cache. This
fla=
g must be
provided by OVirt/RHEV. Target release is 3.6.=20
=20
Until then the only bugfix is to drop pagecache manually on regular int=
ervals on
the=20
hypversior and especially the SPM node.
=20
Markus
=20
>
> Cheers,
> Stefan
>
> On 10/27/14 11:08, Markus Stockhausen wrote:
>> Do you see swapping on the SPM? If yes a regular echo 3 > drop_caches=
could help.
>>
>> Markus
>>
>> Am 27.10.2014 10:57 schrieb Stefan Wendler <stefan.wendler(a)tngtech.co=
m>:
>> Hi,
>>
>> we have some really large snapshots left from a migration. Since our
>> store is almost full now we have to delete them now.
>>
>> Some snapshots are around 1TB already.
>>
>> The last time we tried to delete a ~500GB snapshot the delete task
> blocked the whole diskstore's IO and the whole cluster and all hosts
> became unavailable.
>
> Is there a way to delete large snapshots in a "humane" way that will n=
ot
> block everything?
>
> Cheers,
> Stefan
>
>
=20
--pRrw2Nflj1DiHOHuNiU2AGGrNfrnE9WXR
Content-Type: application/pgp-signature; name="signature.asc"
Content-Description: OpenPGP digital signature
Content-Disposition: attachment; filename="signature.asc"
-----BEGIN PGP SIGNATURE-----
Version: GnuPG v2
iEYEARECAAYFAlROPm0ACgkQ01vvrYDXSqsNugCgmCvjPjgrTzUIWb5OEi2QAaTJ
/NEAn1ahQxu3bb7OGpUcZgjBldYHn6st
=etB8
-----END PGP SIGNATURE-----
--pRrw2Nflj1DiHOHuNiU2AGGrNfrnE9WXR--
Attachments:
- attachment.bin (multipart/signed — 2.6 KB)