[ovirt-devel] [VDSM] Live snapshot with ceph disks

[Yaniv Dary]

On 06/21/2015 11:07 AM, Nir Soffer wrote:
>
> ----- Original Message -----
>> From: "Daniel Erez" <derez at redhat.com>
>> To: "Nir Soffer" <nsoffer at redhat.com>
>> Cc: "devel" <devel at ovirt.org>, "Eric Blake" <eblake at redhat.com>, "Francesco Romani" <fromani at redhat.com>, "Adam
>> Litke" <alitke at redhat.com>, "Federico Simoncelli" <fsimonce at redhat.com>, "Yaniv Dary" <ydary at redhat.com>
>> Sent: Sunday, June 21, 2015 10:05:41 AM
>> Subject: Re: [VDSM] Live snapshot with ceph disks
>>
>>
>>
>> ----- Original Message -----
>>> From: "Nir Soffer" <nsoffer at redhat.com>
>>> To: "devel" <devel at ovirt.org>
>>> Cc: "Eric Blake" <eblake at redhat.com>, "Daniel Erez" <derez at redhat.com>,
>>> "Francesco Romani" <fromani at redhat.com>,
>>> "Adam Litke" <alitke at redhat.com>, "Federico Simoncelli"
>>> <fsimonce at redhat.com>, "Yaniv Dary" <ydary at redhat.com>
>>> Sent: Friday, June 19, 2015 11:40:23 PM
>>> Subject: [VDSM] Live snapshot with ceph disks
>>>
>>> Hi all,
>>>
>>> For 3.6, we will not support live vm snapshot, but this is a must for the
>>> next
>>> release.
>>>
>>> It is trivial to create a disk snapshot in ceph (using cinder apis). The
>>> snapshot
>>> is transparent to libvirt, qmeu and the guest os.
>>>
>>> However, we want to create a consistent snapshot, so you can revert to the
>>> disk
>>> snapshot and get a consistent file system state.
>>>
>>> We also want to create a complete vm snapshot, including all disks and vm
>>> memory.
>>> Libvirt and qemu provides that when given a new disk for the active layer,
>>> but
>>> when using ceph disk, we don't change the active layer - we continue to use
>>> the
>>> same disk.
>>>
>>> Since 1.2.5, libvirt provides virDomainFSFreeze and virDomainFSThaw:
>>> https://libvirt.org/hvsupport.html
>>>
>>> So here is possible flows (ignoring engine side stuff like locking vms and
>>> disks)
>>>
>>> Disk snapshot
>>> -------------
>>>
>>> 1. Engine invoke VM.freezeFileSystems
>>> 2. Vdsm invokes libvirt.virDomainFSFreeze
>>> 3. Engine creates snapshot via cinder
>>> 4. Engine invokes VM.thawFileSystems
>>> 5. Vdsm invokes livbirt.virDomainFSThaw
>>>
>>> Vm snapshot
>>> -----------
>>>
>>> 1. Engine invoke VM.freezeFileSystems
>>> 2. Vdsm invokes libvirt.virDomainFSFreeze
>>> 3. Engine creates snapshot via cinder
>>> 4. Engine invokes VM.snapshot
>>> 5. Vdsm creates snapshot, skipping ceph disks
>>> 6. Engine invokes VM.thawFileSystems
>>> 7. Vdsm invokes livbirt.virDomainFSThaw
>>>
>>> API changes
>>> -----------
>>>
>>> New verbs:
>>> - VM.freezeFileSystems - basically invokes virDomainFSFreeze
>>> - VM.thawFileSystems - basically invokes virDomainFSThaw
>>>
>>>
>>> What do you think?
>> OpenStack uses two different approaches for live snapshot:
>> 1. When taking a snapshot of an instance, a new image (of the entire
>>     instance) is created on Glance in qcow2 format - orchestrated by
>>     Nova and libvirt (Snapshot xml).
> This does not sound like compatible solution like snaphost using vdsm images.
>
>> 2. When taking a snapshot of a single volume while the VM is running
>>     (i.e. the volume is attached to an instance), the snapshot is taken
>>     using Cinder with the relevant driver. The following message is displayed
>>     in Horizon: "This volume is currently attached to an instance.
>>     In some cases, creating a snapshot from an attached volume can result
>>     in a corrupted snapshot." (see attached screenshot)
>>
>> Since the current integration is directly with Cinder and VM snapshot
>> is handled by oVirt engine, we should go with a variant of option 2...
> We support consistent snapshots with vdsm images, and should support
> consistent snapshots with ceph as well.
>
>> If it's too late to integrate the new verbs into 3.6, maybe we could just
>> settle with a similar warning when creating a live snapshot? Or block
>> the feature for now to avoid possible data inconsistency?
> I think we plan live snapshot for next release, not 3.6, so we can add any
> api we need.
>
> I think our goal should be similar live snapshot as we have with other
> storage types. Do we agree on this goal?

Ack on that, we should remember we can have a VM with both Cinder and 
engine managed disks.

>
> Nir

-- 
Yaniv Dary
Technical Product Manager
Red Hat Israel Ltd.
34 Jerusalem Road
Building A, 4th floor
Ra'anana, Israel 4350109

Tel : +972 (9) 7692306
       8272306
Email: ydary at redhat.com
IRC : ydary