----- Original Message -----
From: "Daniel Erez" <derez@redhat.com> To: "Nir Soffer" <nsoffer@redhat.com> Cc: "devel" <devel@ovirt.org>, "Eric Blake" <eblake@redhat.com>, "Francesco Romani" <fromani@redhat.com>, "Adam Litke" <alitke@redhat.com>, "Federico Simoncelli" <fsimonce@redhat.com>, "Yaniv Dary" <ydary@redhat.com> Sent: Sunday, June 21, 2015 10:05:41 AM Subject: Re: [VDSM] Live snapshot with ceph disks
----- Original Message -----
From: "Nir Soffer" <nsoffer@redhat.com> To: "devel" <devel@ovirt.org> Cc: "Eric Blake" <eblake@redhat.com>, "Daniel Erez" <derez@redhat.com>, "Francesco Romani" <fromani@redhat.com>, "Adam Litke" <alitke@redhat.com>, "Federico Simoncelli" <fsimonce@redhat.com>, "Yaniv Dary" <ydary@redhat.com> Sent: Friday, June 19, 2015 11:40:23 PM Subject: [VDSM] Live snapshot with ceph disks
Hi all,
For 3.6, we will not support live vm snapshot, but this is a must for the next release.
It is trivial to create a disk snapshot in ceph (using cinder apis). The snapshot is transparent to libvirt, qmeu and the guest os.
However, we want to create a consistent snapshot, so you can revert to the disk snapshot and get a consistent file system state.
We also want to create a complete vm snapshot, including all disks and vm memory. Libvirt and qemu provides that when given a new disk for the active layer, but when using ceph disk, we don't change the active layer - we continue to use the same disk.
Since 1.2.5, libvirt provides virDomainFSFreeze and virDomainFSThaw: https://libvirt.org/hvsupport.html
So here is possible flows (ignoring engine side stuff like locking vms and disks)
Disk snapshot -------------
1. Engine invoke VM.freezeFileSystems 2. Vdsm invokes libvirt.virDomainFSFreeze 3. Engine creates snapshot via cinder 4. Engine invokes VM.thawFileSystems 5. Vdsm invokes livbirt.virDomainFSThaw
Vm snapshot -----------
1. Engine invoke VM.freezeFileSystems 2. Vdsm invokes libvirt.virDomainFSFreeze 3. Engine creates snapshot via cinder 4. Engine invokes VM.snapshot 5. Vdsm creates snapshot, skipping ceph disks 6. Engine invokes VM.thawFileSystems 7. Vdsm invokes livbirt.virDomainFSThaw
API changes -----------
New verbs: - VM.freezeFileSystems - basically invokes virDomainFSFreeze - VM.thawFileSystems - basically invokes virDomainFSThaw
What do you think?
OpenStack uses two different approaches for live snapshot: 1. When taking a snapshot of an instance, a new image (of the entire instance) is created on Glance in qcow2 format - orchestrated by Nova and libvirt (Snapshot xml).
This does not sound like compatible solution like snaphost using vdsm images.
2. When taking a snapshot of a single volume while the VM is running (i.e. the volume is attached to an instance), the snapshot is taken using Cinder with the relevant driver. The following message is displayed in Horizon: "This volume is currently attached to an instance. In some cases, creating a snapshot from an attached volume can result in a corrupted snapshot." (see attached screenshot)
Since the current integration is directly with Cinder and VM snapshot is handled by oVirt engine, we should go with a variant of option 2...
We support consistent snapshots with vdsm images, and should support consistent snapshots with ceph as well.
If it's too late to integrate the new verbs into 3.6, maybe we could just settle with a similar warning when creating a live snapshot? Or block the feature for now to avoid possible data inconsistency?
I think we plan live snapshot for next release, not 3.6, so we can add any api we need. I think our goal should be similar live snapshot as we have with other storage types. Do we agree on this goal? Nir