7:01 p.m.
Hello,
What is the best way to set up a daily live snapshot for all VM, and have
the possibility to recover, for example, a specific VM to a specific day?
I use a Hyperconverged Infrastructure with 3 nodes, gluster storage.
Thank you,
12:39 a.m.
On Wed, Dec 6, 2017 at 6:01 PM, Jason Lelievre <jlelievre(a)folksvfx.com>
wrote:
Hello,
What is the best way to set up a daily live snapshot for all VM, and have
the possibility to recover, for example, a specific VM to a specific day?
I use a Hyperconverged Infrastructure with 3 nodes, gluster storage.
Thank you,
_______________________________________________
Users mailing list
Users(a)ovirt.org
http://lists.ovirt.org/mailman/listinfo/users
One idea is to use crontab to run a daily script which will use the
engine-sdk to grep all VMs and for each one, create a snapshot.
1:19 a.m.
On Wed, Dec 6, 2017 at 6:02 PM Jason Lelievre <jlelievre(a)folksvfx.com wrote:
Maybe a solution based on gluster or the layers below (e.g. lvm snapshots)
will
be easier to use and more efficient. You can ask in the gluster mailing
list.
Adding Sahina to help with the gluster side.
Nir
> Thank you,
> _______________________________________________
> Users mailing list
> Users(a)ovirt.org
> http://lists.ovirt.org/mailman/listinfo/users
Hello,
What is the best way to set up a daily live snapshot for all VM, and have
the possibility to recover, for example, a specific VM to a specific day?
Each snapshot you create make reads and writes slower, as qemu has to
lookup data through the entire chain.
When we take a snapshot, we create a new file (or block device) and make
the new file the active layer of the chain.
For example, assuming you have a base image in raw format:
image-1.raw (top)
After taking a snapshot, you have:
image-1.raw <- image-2.qcow2 (top)
Now when qemu need to read data from the image, it will try to get the data
from
the top layer (image-2), if it does not exists it will try the backing file
(image-1).
Same when writing data, if qemu need to write small amount of data, it may
need
to get entire sector from a another layer in the chain and copy it to the
top layer.
So after a month of backups, will have 30 layer in the chain. I don't know
how
much slower it will be, I never measured this.
For backups, I think you like to have something like a daily backup for the
last week,
a weekly backup for the last month, and so on.
For daily backup of the last week you can take a snapshot every day, and
delete
the 7th snapshot every day. Then backup the snapshot bellow the active
layer.
You can download the snapshot using the SDK, see the download_snapshot
example (requires 4.2). All this can be done while the vm is running.
On your backup, you will have a snapshot per day. You can keep all snapshot
or merge old snapshot using qemu-img commit.
It is also should be possible to reconstruct an image on storage from your
backup
by uploading the the snapshots back to storage - but it is tricky to get
right.
I think the best option for users is to get some 3rd party solution based
on
the oVirt SDK.
Adding qemu-block mailing list, they can probably add more info on how much
you pay for large chain of images, and have more ideas about backing up
qcow2 chains.
I use a Hyperconverged Infrastructure with 3 nodes, gluster storage.
5:08 p.m.
New subject: [Qemu-block] Scheduling daily Snapshot
Am 07.12.2017 um 23:19 hat Nir Soffer geschrieben:
On Wed, Dec 6, 2017 at 6:02 PM Jason Lelievre
<jlelievre(a)folksvfx.com>
wrote:
> Hello,
>
> What is the best way to set up a daily live snapshot for all VM, and have
> the possibility to recover, for example, a specific VM to a specific day?
Each snapshot you create make reads and writes slower, as qemu has to
lookup data through the entire chain.
This is true in principle. However, as long as the lookup is purely in
memory and doesn't involve I/O, you won't even notice this in average
use cases. Whether additional I/O is necessary depends on whether the
metadata caches already cover the part of the image that you're
accessing.
By choosing the right cache size values for the use case, it can
normally be achieved that everything is already in memory.
When we take a snapshot, we create a new file (or block device) and
make
the new file the active layer of the chain.
For example, assuming you have a base image in raw format:
image-1.raw (top)
After taking a snapshot, you have:
image-1.raw <- image-2.qcow2 (top)
Now when qemu need to read data from the image, it will try to get the
data from the top layer (image-2), if it does not exists it will try
the backing file (image-1). Same when writing data, if qemu need to
write small amount of data, it may need to get entire sector from a
another layer in the chain and copy it to the top layer.
Yes, though for this operation it doesn't matter whether it has to copy
it from the second image in the chain or the thirtieth. As soon as you
do a partial write to a cluster that hasn't been written yet since the
last snapshot was taken, you get to copy data, no matter the length of
the chain.
Kevin
5:46 p.m.
New subject: [Qemu-block] Scheduling daily Snapshot
On Fri, Dec 8, 2017 at 4:08 PM Kevin Wolf <kwolf(a)redhat.com> wrote:
Am 07.12.2017 um 23:19 hat Nir Soffer geschrieben:
> On Wed, Dec 6, 2017 at 6:02 PM Jason Lelievre <jlelievre(a)folksvfx.com>
> wrote:
>
> > Hello,
> >
> > What is the best way to set up a daily live snapshot for all VM, and
have
> > the possibility to recover, for example, a specific VM to a specific
day?
>
> Each snapshot you create make reads and writes slower, as qemu has to
> lookup data through the entire chain.
This is true in principle. However, as long as the lookup is purely in
memory and doesn't involve I/O, you won't even notice this in average
use cases. Whether additional I/O is necessary depends on whether the
metadata caches already cover the part of the image that you're
accessing.
By choosing the right cache size values for the use case, it can
normally be achieved that everything is already in memory.
Can you give more details about selecting the cache size?
> When we take a snapshot, we create a new file (or block device) and make
> the new file the active layer of the chain.
>
> For example, assuming you have a base image in raw format:
>
> image-1.raw (top)
>
> After taking a snapshot, you have:
>
> image-1.raw <- image-2.qcow2 (top)
>
> Now when qemu need to read data from the image, it will try to get the
> data from the top layer (image-2), if it does not exists it will try
> the backing file (image-1). Same when writing data, if qemu need to
> write small amount of data, it may need to get entire sector from a
> another layer in the chain and copy it to the top layer.
Yes, though for this operation it doesn't matter whether it has to copy
it from the second image in the chain or the thirtieth. As soon as you
do a partial write to a cluster that hasn't been written yet since the
last snapshot was taken, you get to copy data, no matter the length of
the chain.
So do you think keeping 30 snapshots for backup/restore purpose is
a practical solution with negligible effect on performance?
Kevin
6:25 p.m.
New subject: [Qemu-block] Scheduling daily Snapshot
This is an OpenPGP/MIME signed message (RFC 4880 and 3156)
--hR72digdxOGpffpOAtEa44reDgnaxGDrI
Content-Type: multipart/mixed; boundary="WQeH5Mg5P2FQw10eUHPbddub9lX0rt2P4";
protected-headers="v1"
From: Max Reitz <mreitz(a)redhat.com>
To: Nir Soffer <nsoffer(a)redhat.com>, Jason Lelievre <jlelievre(a)folksvfx.com>,
"qemu-block(a)nongnu.org" <qemu-block(a)nongnu.org>,
Sahina Bose <sabose(a)redhat.com>
Cc: users(a)ovirt.org
Message-ID: <6ff20c92-c9a5-0ae8-3571-f03bd7fc57f1(a)redhat.com>
Subject: Re: [Qemu-block] [ovirt-users] Scheduling daily Snapshot
References: <CAHy3Vd3d4-L_UtVE6uD5F0jiyLoUcxdS+3wnCN2Dtt_pSYBUtw(a)mail.gmail.com>
<CAMRbyyvTh0PDeJZmpCK6+qoOprxd=XFEuYn5r8_99C0SFCe=VA(a)mail.gmail.com>
In-Reply-To: <CAMRbyyvTh0PDeJZmpCK6+qoOprxd=XFEuYn5r8_99C0SFCe=VA(a)mail.gmail.com>
--WQeH5Mg5P2FQw10eUHPbddub9lX0rt2P4
Content-Type: text/plain; charset=utf-8
Content-Language: en-US
Content-Transfer-Encoding: quoted-printable
On 2017-12-07 23:19, Nir Soffer wrote:
On Wed, Dec 6, 2017 at 6:02 PM Jason Lelievre
<jlelievre(a)folksvfx.com
<mailto:jlelievre@folksvfx.com>> wrote:
=20
Hello,
=20
What is the best way to set up a daily live snapshot for all VM, an=
d
have the possibility to recover, for example, a specific VM to a
specific day?
=20
=20
Each snapshot you create make reads and writes slower, as qemu has to
lookup data through the entire chain.
=20
When we take a snapshot, we create a new file (or block device) and mak=
e
the new file the active layer of the chain.
I'm not sure how much this is going to slow you down exactly, but I can
tell you that there are also incremental backups to look into.
(e.g. https://wiki.qemu.org/Features/IncrementalBackup)
Max
--WQeH5Mg5P2FQw10eUHPbddub9lX0rt2P4--
--hR72digdxOGpffpOAtEa44reDgnaxGDrI
Content-Type: application/pgp-signature; name="signature.asc"
Content-Description: OpenPGP digital signature
Content-Disposition: attachment; filename="signature.asc"
-----BEGIN PGP SIGNATURE-----
iQFGBAEBCAAwFiEEkb62CjDbPohX0Rgp9AfbAGHVz0AFAloqruoSHG1yZWl0ekBy
ZWRoYXQuY29tAAoJEPQH2wBh1c9ACSYIAIX/txzs4mP81hlJookZfAGv958zDdiw
umTq2jzMvpDQVYf6HgjrzuVELxMg7DA6T4oI08dGCVyfDq4qyw0xqodvE84wWZkH
oWxLQAmR3c8vVBkxkcx5lRM8qs0NydaipZGUSpN65MEGOL0B9j5+06fmmLTHjUc9
nqsmJUjza3tuuWwX9K9yWxdiBqj/58z0/1zp/HaG0/TKJgpwG20OF8xITAf70HwU
qLYg6YEUsLu7z6qL5H15g4lAdiMfZ1dSgezdTY4Sm07MBn5vt236XygS3Wag14Y+
7Sx9e5TLyoNUB6Z+DJ/vG5Y0H98NzyB8flgLqFG0kDVuyjiaF+MV1LU=
=B4Uz
-----END PGP SIGNATURE-----
--hR72digdxOGpffpOAtEa44reDgnaxGDrI--
12:03 a.m.
New subject: [Qemu-block] Scheduling daily Snapshot
On Fri, Dec 8, 2017 at 5:25 PM Max Reitz <mreitz(a)redhat.com> wrote:
On 2017-12-07 23:19, Nir Soffer wrote:
> On Wed, Dec 6, 2017 at 6:02 PM Jason Lelievre <jlelievre(a)folksvfx.com
> <mailto:jlelievre@folksvfx.com>> wrote:
>
> Hello,
>
> What is the best way to set up a daily live snapshot for all VM, and
> have the possibility to recover, for example, a specific VM to a
> specific day?
>
>
> Each snapshot you create make reads and writes slower, as qemu has to
> lookup data through the entire chain.
>
> When we take a snapshot, we create a new file (or block device) and make
> the new file the active layer of the chain.
I'm not sure how much this is going to slow you down exactly, but I can
tell you that there are also incremental backups to look into.
(e.g. https://wiki.qemu.org/Features/IncrementalBackup)
checking, thanks!
Max
2541
days inactive
2543
days old
6 comments
6 participants
participants (6)
-
Jason Lelievre -
Kevin Wolf -
Maor Lipchuk -
Max Reitz -
Nir Soffer -
Nir Soffer