Scheduling daily Snapshot
Hello, What is the best way to set up a daily live snapshot for all VM, and have the possibility to recover, for example, a specific VM to a specific day? I use a Hyperconverged Infrastructure with 3 nodes, gluster storage. Thank you,
On Wed, Dec 6, 2017 at 6:01 PM, Jason Lelievre <jlelievre@folksvfx.com> wrote:
Hello,
What is the best way to set up a daily live snapshot for all VM, and have the possibility to recover, for example, a specific VM to a specific day?
I use a Hyperconverged Infrastructure with 3 nodes, gluster storage.
Thank you,
_______________________________________________ Users mailing list Users@ovirt.org http://lists.ovirt.org/mailman/listinfo/users
One idea is to use crontab to run a daily script which will use the engine-sdk to grep all VMs and for each one, create a snapshot.
On Wed, Dec 6, 2017 at 6:02 PM Jason Lelievre <jlelievre@folksvfx.com> wrote:
Hello,
What is the best way to set up a daily live snapshot for all VM, and have the possibility to recover, for example, a specific VM to a specific day?
Each snapshot you create make reads and writes slower, as qemu has to lookup data through the entire chain. When we take a snapshot, we create a new file (or block device) and make the new file the active layer of the chain. For example, assuming you have a base image in raw format: image-1.raw (top) After taking a snapshot, you have: image-1.raw <- image-2.qcow2 (top) Now when qemu need to read data from the image, it will try to get the data from the top layer (image-2), if it does not exists it will try the backing file (image-1). Same when writing data, if qemu need to write small amount of data, it may need to get entire sector from a another layer in the chain and copy it to the top layer. So after a month of backups, will have 30 layer in the chain. I don't know how much slower it will be, I never measured this. For backups, I think you like to have something like a daily backup for the last week, a weekly backup for the last month, and so on. For daily backup of the last week you can take a snapshot every day, and delete the 7th snapshot every day. Then backup the snapshot bellow the active layer. You can download the snapshot using the SDK, see the download_snapshot example (requires 4.2). All this can be done while the vm is running. On your backup, you will have a snapshot per day. You can keep all snapshot or merge old snapshot using qemu-img commit. It is also should be possible to reconstruct an image on storage from your backup by uploading the the snapshots back to storage - but it is tricky to get right. I think the best option for users is to get some 3rd party solution based on the oVirt SDK. Adding qemu-block mailing list, they can probably add more info on how much you pay for large chain of images, and have more ideas about backing up qcow2 chains. I use a Hyperconverged Infrastructure with 3 nodes, gluster storage.
Maybe a solution based on gluster or the layers below (e.g. lvm snapshots) will be easier to use and more efficient. You can ask in the gluster mailing list. Adding Sahina to help with the gluster side. Nir
Thank you, _______________________________________________ Users mailing list Users@ovirt.org http://lists.ovirt.org/mailman/listinfo/users
Am 07.12.2017 um 23:19 hat Nir Soffer geschrieben:
On Wed, Dec 6, 2017 at 6:02 PM Jason Lelievre <jlelievre@folksvfx.com> wrote:
Hello,
What is the best way to set up a daily live snapshot for all VM, and have the possibility to recover, for example, a specific VM to a specific day?
Each snapshot you create make reads and writes slower, as qemu has to lookup data through the entire chain.
This is true in principle. However, as long as the lookup is purely in memory and doesn't involve I/O, you won't even notice this in average use cases. Whether additional I/O is necessary depends on whether the metadata caches already cover the part of the image that you're accessing. By choosing the right cache size values for the use case, it can normally be achieved that everything is already in memory.
When we take a snapshot, we create a new file (or block device) and make the new file the active layer of the chain.
For example, assuming you have a base image in raw format:
image-1.raw (top)
After taking a snapshot, you have:
image-1.raw <- image-2.qcow2 (top)
Now when qemu need to read data from the image, it will try to get the data from the top layer (image-2), if it does not exists it will try the backing file (image-1). Same when writing data, if qemu need to write small amount of data, it may need to get entire sector from a another layer in the chain and copy it to the top layer.
Yes, though for this operation it doesn't matter whether it has to copy it from the second image in the chain or the thirtieth. As soon as you do a partial write to a cluster that hasn't been written yet since the last snapshot was taken, you get to copy data, no matter the length of the chain. Kevin
On Fri, Dec 8, 2017 at 4:08 PM Kevin Wolf <kwolf@redhat.com> wrote:
Am 07.12.2017 um 23:19 hat Nir Soffer geschrieben:
On Wed, Dec 6, 2017 at 6:02 PM Jason Lelievre <jlelievre@folksvfx.com> wrote:
Hello,
What is the best way to set up a daily live snapshot for all VM, and have the possibility to recover, for example, a specific VM to a specific day?
Each snapshot you create make reads and writes slower, as qemu has to lookup data through the entire chain.
This is true in principle. However, as long as the lookup is purely in memory and doesn't involve I/O, you won't even notice this in average use cases. Whether additional I/O is necessary depends on whether the metadata caches already cover the part of the image that you're accessing.
By choosing the right cache size values for the use case, it can normally be achieved that everything is already in memory.
Can you give more details about selecting the cache size?
When we take a snapshot, we create a new file (or block device) and make the new file the active layer of the chain.
For example, assuming you have a base image in raw format:
image-1.raw (top)
After taking a snapshot, you have:
image-1.raw <- image-2.qcow2 (top)
Now when qemu need to read data from the image, it will try to get the data from the top layer (image-2), if it does not exists it will try the backing file (image-1). Same when writing data, if qemu need to write small amount of data, it may need to get entire sector from a another layer in the chain and copy it to the top layer.
Yes, though for this operation it doesn't matter whether it has to copy it from the second image in the chain or the thirtieth. As soon as you do a partial write to a cluster that hasn't been written yet since the last snapshot was taken, you get to copy data, no matter the length of the chain.
So do you think keeping 30 snapshots for backup/restore purpose is a practical solution with negligible effect on performance?
Kevin
This is an OpenPGP/MIME signed message (RFC 4880 and 3156) --hR72digdxOGpffpOAtEa44reDgnaxGDrI Content-Type: multipart/mixed; boundary="WQeH5Mg5P2FQw10eUHPbddub9lX0rt2P4"; protected-headers="v1" From: Max Reitz <mreitz@redhat.com> To: Nir Soffer <nsoffer@redhat.com>, Jason Lelievre <jlelievre@folksvfx.com>, "qemu-block@nongnu.org" <qemu-block@nongnu.org>, Sahina Bose <sabose@redhat.com> Cc: users@ovirt.org Message-ID: <6ff20c92-c9a5-0ae8-3571-f03bd7fc57f1@redhat.com> Subject: Re: [Qemu-block] [ovirt-users] Scheduling daily Snapshot References: <CAHy3Vd3d4-L_UtVE6uD5F0jiyLoUcxdS+3wnCN2Dtt_pSYBUtw@mail.gmail.com> <CAMRbyyvTh0PDeJZmpCK6+qoOprxd=XFEuYn5r8_99C0SFCe=VA@mail.gmail.com> In-Reply-To: <CAMRbyyvTh0PDeJZmpCK6+qoOprxd=XFEuYn5r8_99C0SFCe=VA@mail.gmail.com> --WQeH5Mg5P2FQw10eUHPbddub9lX0rt2P4 Content-Type: text/plain; charset=utf-8 Content-Language: en-US Content-Transfer-Encoding: quoted-printable On 2017-12-07 23:19, Nir Soffer wrote:
On Wed, Dec 6, 2017 at 6:02 PM Jason Lelievre <jlelievre@folksvfx.com <mailto:jlelievre@folksvfx.com>> wrote: =20 Hello, =20 What is the best way to set up a daily live snapshot for all VM, an= d have the possibility to recover, for example, a specific VM to a specific day? =20 =20 Each snapshot you create make reads and writes slower, as qemu has to lookup data through the entire chain. =20 When we take a snapshot, we create a new file (or block device) and mak= e the new file the active layer of the chain.
I'm not sure how much this is going to slow you down exactly, but I can tell you that there are also incremental backups to look into. (e.g. https://wiki.qemu.org/Features/IncrementalBackup) Max --WQeH5Mg5P2FQw10eUHPbddub9lX0rt2P4-- --hR72digdxOGpffpOAtEa44reDgnaxGDrI Content-Type: application/pgp-signature; name="signature.asc" Content-Description: OpenPGP digital signature Content-Disposition: attachment; filename="signature.asc" -----BEGIN PGP SIGNATURE----- iQFGBAEBCAAwFiEEkb62CjDbPohX0Rgp9AfbAGHVz0AFAloqruoSHG1yZWl0ekBy ZWRoYXQuY29tAAoJEPQH2wBh1c9ACSYIAIX/txzs4mP81hlJookZfAGv958zDdiw umTq2jzMvpDQVYf6HgjrzuVELxMg7DA6T4oI08dGCVyfDq4qyw0xqodvE84wWZkH oWxLQAmR3c8vVBkxkcx5lRM8qs0NydaipZGUSpN65MEGOL0B9j5+06fmmLTHjUc9 nqsmJUjza3tuuWwX9K9yWxdiBqj/58z0/1zp/HaG0/TKJgpwG20OF8xITAf70HwU qLYg6YEUsLu7z6qL5H15g4lAdiMfZ1dSgezdTY4Sm07MBn5vt236XygS3Wag14Y+ 7Sx9e5TLyoNUB6Z+DJ/vG5Y0H98NzyB8flgLqFG0kDVuyjiaF+MV1LU= =B4Uz -----END PGP SIGNATURE----- --hR72digdxOGpffpOAtEa44reDgnaxGDrI--
On Fri, Dec 8, 2017 at 5:25 PM Max Reitz <mreitz@redhat.com> wrote:
On 2017-12-07 23:19, Nir Soffer wrote:
On Wed, Dec 6, 2017 at 6:02 PM Jason Lelievre <jlelievre@folksvfx.com <mailto:jlelievre@folksvfx.com>> wrote:
Hello,
What is the best way to set up a daily live snapshot for all VM, and have the possibility to recover, for example, a specific VM to a specific day?
Each snapshot you create make reads and writes slower, as qemu has to lookup data through the entire chain.
When we take a snapshot, we create a new file (or block device) and make the new file the active layer of the chain.
I'm not sure how much this is going to slow you down exactly, but I can tell you that there are also incremental backups to look into.
checking, thanks!
Max
participants (6)
-
Jason Lelievre -
Kevin Wolf -
Maor Lipchuk -
Max Reitz -
Nir Soffer -
Nir Soffer