[ovirt-devel] [VDSM] Live snapshot with ceph disks

[Nir Soffer]

----- Original Message -----
> From: "Daniel Erez" <derez at redhat.com>
> To: "Nir Soffer" <nsoffer at redhat.com>
> Cc: "devel" <devel at ovirt.org>, "Eric Blake" <eblake at redhat.com>, "Francesco Romani" <fromani at redhat.com>, "Adam
> Litke" <alitke at redhat.com>, "Federico Simoncelli" <fsimonce at redhat.com>, "Yaniv Dary" <ydary at redhat.com>
> Sent: Sunday, June 21, 2015 10:05:41 AM
> Subject: Re: [VDSM] Live snapshot with ceph disks
> 
> 
> 
> ----- Original Message -----
> > From: "Nir Soffer" <nsoffer at redhat.com>
> > To: "devel" <devel at ovirt.org>
> > Cc: "Eric Blake" <eblake at redhat.com>, "Daniel Erez" <derez at redhat.com>,
> > "Francesco Romani" <fromani at redhat.com>,
> > "Adam Litke" <alitke at redhat.com>, "Federico Simoncelli"
> > <fsimonce at redhat.com>, "Yaniv Dary" <ydary at redhat.com>
> > Sent: Friday, June 19, 2015 11:40:23 PM
> > Subject: [VDSM] Live snapshot with ceph disks
> > 
> > Hi all,
> > 
> > For 3.6, we will not support live vm snapshot, but this is a must for the
> > next
> > release.
> > 
> > It is trivial to create a disk snapshot in ceph (using cinder apis). The
> > snapshot
> > is transparent to libvirt, qmeu and the guest os.
> > 
> > However, we want to create a consistent snapshot, so you can revert to the
> > disk
> > snapshot and get a consistent file system state.
> > 
> > We also want to create a complete vm snapshot, including all disks and vm
> > memory.
> > Libvirt and qemu provides that when given a new disk for the active layer,
> > but
> > when using ceph disk, we don't change the active layer - we continue to use
> > the
> > same disk.
> > 
> > Since 1.2.5, libvirt provides virDomainFSFreeze and virDomainFSThaw:
> > https://libvirt.org/hvsupport.html
> > 
> > So here is possible flows (ignoring engine side stuff like locking vms and
> > disks)
> > 
> > Disk snapshot
> > -------------
> > 
> > 1. Engine invoke VM.freezeFileSystems
> > 2. Vdsm invokes libvirt.virDomainFSFreeze
> > 3. Engine creates snapshot via cinder
> > 4. Engine invokes VM.thawFileSystems
> > 5. Vdsm invokes livbirt.virDomainFSThaw
> > 
> > Vm snapshot
> > -----------
> > 
> > 1. Engine invoke VM.freezeFileSystems
> > 2. Vdsm invokes libvirt.virDomainFSFreeze
> > 3. Engine creates snapshot via cinder
> > 4. Engine invokes VM.snapshot
> > 5. Vdsm creates snapshot, skipping ceph disks
> > 6. Engine invokes VM.thawFileSystems
> > 7. Vdsm invokes livbirt.virDomainFSThaw
> > 
> > API changes
> > -----------
> > 
> > New verbs:
> > - VM.freezeFileSystems - basically invokes virDomainFSFreeze
> > - VM.thawFileSystems - basically invokes virDomainFSThaw
> > 
> > 
> > What do you think?
> 
> OpenStack uses two different approaches for live snapshot:
> 1. When taking a snapshot of an instance, a new image (of the entire
>    instance) is created on Glance in qcow2 format - orchestrated by
>    Nova and libvirt (Snapshot xml).

This does not sound like compatible solution like snaphost using vdsm images.

> 2. When taking a snapshot of a single volume while the VM is running
>    (i.e. the volume is attached to an instance), the snapshot is taken
>    using Cinder with the relevant driver. The following message is displayed
>    in Horizon: "This volume is currently attached to an instance.
>    In some cases, creating a snapshot from an attached volume can result
>    in a corrupted snapshot." (see attached screenshot)
> 
> Since the current integration is directly with Cinder and VM snapshot
> is handled by oVirt engine, we should go with a variant of option 2...

We support consistent snapshots with vdsm images, and should support
consistent snapshots with ceph as well.

> If it's too late to integrate the new verbs into 3.6, maybe we could just
> settle with a similar warning when creating a live snapshot? Or block
> the feature for now to avoid possible data inconsistency?

I think we plan live snapshot for next release, not 3.6, so we can add any
api we need.

I think our goal should be similar live snapshot as we have with other
storage types. Do we agree on this goal?

Nir