[ovirt-devel] Libvirt secrets management - take 2

[Nir Soffer]

----- Original Message -----
> From: "Michal Skrivanek" <mskrivan at redhat.com>
> To: "Nir Soffer" <nsoffer at redhat.com>
> Cc: "Francesco Romani" <fromani at redhat.com>, "devel" <devel at ovirt.org>, "Adam Litke" <alitke at redhat.com>, "Federico
> Simoncelli" <fsimonce at redhat.com>, "Dan Kenigsberg" <danken at redhat.com>, "Allon Mureinik" <amureini at redhat.com>,
> "Daniel Erez" <derez at redhat.com>, "Eric Blake" <eblake at redhat.com>
> Sent: Monday, June 15, 2015 6:20:34 PM
> Subject: Re: Libvirt secrets management - take 2
> 
> 
> On 15 Jun 2015, at 17:06, Nir Soffer wrote:
> 
> > 
> > 
> > ----- Original Message -----
> >> From: "Francesco Romani" <fromani at redhat.com>
> >> To: "devel" <devel at ovirt.org>
> >> Cc: "Adam Litke" <alitke at redhat.com>, "Federico Simoncelli"
> >> <fsimonce at redhat.com>, "Dan Kenigsberg"
> >> <danken at redhat.com>, "Allon Mureinik" <amureini at redhat.com>, "Daniel Erez"
> >> <derez at redhat.com>, "Michal Skrivanek"
> >> <mskrivan at redhat.com>, "Eric Blake" <eblake at redhat.com>, "Nir Soffer"
> >> <nsoffer at redhat.com>
> >> Sent: Monday, June 15, 2015 4:08:09 PM
> >> Subject: Re: Libvirt secrets management - take 2
> >> 
> >> ----- Original Message -----
> >>> From: "Nir Soffer" <nsoffer at redhat.com>
> >>> To: "Adam Litke" <alitke at redhat.com>
> >>> Cc: "devel" <devel at ovirt.org>, "Francesco Romani" <fromani at redhat.com>,
> >>> "Federico Simoncelli" <fsimonce at redhat.com>,
> >>> "Dan Kenigsberg" <danken at redhat.com>, "Allon Mureinik"
> >>> <amureini at redhat.com>, "Daniel Erez" <derez at redhat.com>,
> >>> "Michal Skrivanek" <mskrivan at redhat.com>, "Eric Blake"
> >>> <eblake at redhat.com>
> >>> Sent: Saturday, June 13, 2015 11:14:03 PM
> >>> Subject: Re: Libvirt secrets management - take 2
> >> 
> >> [...]
> >>>>> Flows
> >>>>> =====
> >>>>> 
> >>>>> Start vm
> >>>>> --------
> >>>>> - Engine add required secrets to vm description
> >>>>> - Vdsm register temporary secrets with libvirt
> >>>>> - When vm is up or if operation failed, Vdsm remove the temporary
> >>>>> secret
> >>>>> 
> >>>>> Migrate vm
> >>>>> ----------
> >>>>> - Engine add required secrets to vm description
> >>>>> - Vdsm add secrets to the vm description sent to the destination
> >>>>> - On the destination, Vdsm register temporary secrets with libvirt
> >>>>> - On the destination, when vm is up or if operation failed, Vdsm remove
> >>>>> the temporary secret
> >>> 
> >>> On issue with migration - do we have to keep the same auth information as
> >>> in
> >>> the original vm xml, or we can create the vm on the destination using
> >>> different
> >>> auth xml?
> >> 
> >> I don't have an answer for this. Need to check and play with this a bit.
> >> 
> >>>>> Hot-plug disk
> >>>>> -------------
> >>>>> - Engine add secret to disk description
> >>>>> - Vdsm register temporary secret with libvirt
> >>>>> - When disk is successfully plugged, or if operation failed, Vdsm
> >>>>> remove
> >>>>> the temporary secret
> >> 
> >> OK, what if this VM is migrate afterwards? destination VDSM needs to
> >> register
> >> this secret
> >> to libvirt.
> >> 
> >> The fact here is that is the source VDSM that orchestrates the migration,
> >> and
> >> if it has
> >> forgorgotten the secret, then we need get it back again from Engine and
> >> pass
> >> it through the destination VM.
> >> 
> >> On migration, it is the source VM which orchestrates the flow. Engine
> >> polls
> >> source VM for updates,
> >> and consider the VM migrated only after source exited correctly.
> >> I don't recall the fine details engine side (need to check), but on VDSM
> >> side:
> >> 
> >> - source VDSM receives VM.migrate(destination_uri)
> >> - source VDSM sends migrationCreate to destination VDSM
> >> - once destination VDSM created special-purpose VM, source VDSM instructs
> >> libvirt to start the migration
> >> - source VDSM monitors the migration, report progress, updates the
> >> downtime
> >> - migration fails: source VDSM automatically send VM.destroy to
> >> destination
> >> VDSM
> >> - migration succeeds: source VDSM destroy source VM, execution now
> >> continues
> >> on destination VM
> 
> the source side is "destroyed" by qemu automatically. Engine just sends an
> explicit destroy after the migration is deemed successful in the engine too
> 
> > 
> > Ok, when we create the special purpose vm, do we include all the devices?
> 
> it's an exact copy of the source vm. yes
> 
> > 
> > If this special vm (waiting for migration) is connected to all the disks,
> > then
> 
> it is (it goes though preparePaths as any other VM creation)
> 
> > we only need to register the secrets sent from the source in
> > migrationCreate,
> > start the special vm, and one it is running, delete the secrets.
> 
> probably. Does the source know those secrets?

At the time of migration, the source does not know much about the secrerts.

- qemu got the secret key and connected to the ceph disks.
- libvirt knew about the secret when starting qemu/ hot-plugging a disk, but
  now it does not know anything about the secret, except the secret uuid
  in the <disk> <auth> element.
- vdsm knows about the auth dicts in the disk devices, but does not know the
  key (the key cannot be persisted)

So engine will have to send the secrets again, so vdsm can send them to the destination.

> 
> > 
> >> 
> >> so far, Engine talks with destination VM only if migration completed
> >> succesfully (unless I'm missing something :))
> >> 
> >> so we need to pass back again all the needed secrets as part of VM.migrate
> >> verb
> >> 
> >> This is only the very first thought of mine, we can probably think
> >> something
> >> cleaner.
> >> 
> >> Bests,
> >> 
> >> --
> >> Francesco Romani
> >> RedHat Engineering Virtualization R & D
> >> Phone: 8261328
> >> IRC: fromani
> >> 
> 
>