Re: [ovirt-devel] Libvirt secrets management - take 2

----- Original Message ----- > From: "Michal Skrivanek" <mskrivan(a)redhat.com&gt; > To: "Nir Soffer" <nsoffer(a)redhat.com&gt; > Cc: "Francesco Romani" <fromani(a)redhat.com&gt;, "devel" <devel(a)ovirt.org&gt;, "Adam Litke" <alitke(a)redhat.com&gt;, "Federico > Simoncelli" <fsimonce(a)redhat.com&gt;, "Dan Kenigsberg" <danken(a)redhat.com&gt;, "Allon Mureinik" <amureini(a)redhat.com&gt;, > "Daniel Erez" <derez(a)redhat.com&gt;, "Eric Blake" <eblake(a)redhat.com&gt; > Sent: Monday, June 15, 2015 6:20:34 PM > Subject: Re: Libvirt secrets management - take 2 > > > On 15 Jun 2015, at 17:06, Nir Soffer wrote: > >> >> >> ----- Original Message ----- >>> From: "Francesco Romani" <fromani(a)redhat.com&gt; >>> To: "devel" <devel(a)ovirt.org&gt; >>> Cc: "Adam Litke" <alitke(a)redhat.com&gt;, "Federico Simoncelli" >>> <fsimonce(a)redhat.com&gt;, "Dan Kenigsberg" >>> <danken(a)redhat.com&gt;, "Allon Mureinik" <amureini(a)redhat.com&gt;, "Daniel Erez" >>> <derez(a)redhat.com&gt;, "Michal Skrivanek" >>> <mskrivan(a)redhat.com&gt;, "Eric Blake" <eblake(a)redhat.com&gt;, "Nir Soffer" >>> <nsoffer(a)redhat.com&gt; >>> Sent: Monday, June 15, 2015 4:08:09 PM >>> Subject: Re: Libvirt secrets management - take 2 >>> >>> ----- Original Message ----- >>>> From: "Nir Soffer" <nsoffer(a)redhat.com&gt; >>>> To: "Adam Litke" <alitke(a)redhat.com&gt; >>>> Cc: "devel" <devel(a)ovirt.org&gt;, "Francesco Romani" <fromani(a)redhat.com&gt;, >>>> "Federico Simoncelli" <fsimonce(a)redhat.com&gt;, >>>> "Dan Kenigsberg" <danken(a)redhat.com&gt;, "Allon Mureinik" >>>> <amureini(a)redhat.com&gt;, "Daniel Erez" <derez(a)redhat.com&gt;, >>>> "Michal Skrivanek" <mskrivan(a)redhat.com&gt;, "Eric Blake" >>>> <eblake(a)redhat.com&gt; >>>> Sent: Saturday, June 13, 2015 11:14:03 PM >>>> Subject: Re: Libvirt secrets management - take 2 >>> >>> [...] >>>>>> Flows >>>>>> ===== >>>>>> >>>>>> Start vm >>>>>> -------- >>>>>> - Engine add required secrets to vm description >>>>>> - Vdsm register temporary secrets with libvirt >>>>>> - When vm is up or if operation failed, Vdsm remove the temporary >>>>>> secret >>>>>> >>>>>> Migrate vm >>>>>> ---------- >>>>>> - Engine add required secrets to vm description >>>>>> - Vdsm add secrets to the vm description sent to the destination >>>>>> - On the destination, Vdsm register temporary secrets with libvirt >>>>>> - On the destination, when vm is up or if operation failed, Vdsm remove >>>>>> the temporary secret >>>> >>>> On issue with migration - do we have to keep the same auth information as >>>> in >>>> the original vm xml, or we can create the vm on the destination using >>>> different >>>> auth xml? >>> >>> I don't have an answer for this. Need to check and play with this a bit. >>> >>>>>> Hot-plug disk >>>>>> ------------- >>>>>> - Engine add secret to disk description >>>>>> - Vdsm register temporary secret with libvirt >>>>>> - When disk is successfully plugged, or if operation failed, Vdsm >>>>>> remove >>>>>> the temporary secret >>> >>> OK, what if this VM is migrate afterwards? destination VDSM needs to >>> register >>> this secret >>> to libvirt. >>> >>> The fact here is that is the source VDSM that orchestrates the migration, >>> and >>> if it has >>> forgorgotten the secret, then we need get it back again from Engine and >>> pass >>> it through the destination VM. >>> >>> On migration, it is the source VM which orchestrates the flow. Engine >>> polls >>> source VM for updates, >>> and consider the VM migrated only after source exited correctly. >>> I don't recall the fine details engine side (need to check), but on VDSM >>> side: >>> >>> - source VDSM receives VM.migrate(destination_uri) >>> - source VDSM sends migrationCreate to destination VDSM >>> - once destination VDSM created special-purpose VM, source VDSM instructs >>> libvirt to start the migration >>> - source VDSM monitors the migration, report progress, updates the >>> downtime >>> - migration fails: source VDSM automatically send VM.destroy to >>> destination >>> VDSM >>> - migration succeeds: source VDSM destroy source VM, execution now >>> continues >>> on destination VM > > the source side is "destroyed" by qemu automatically. Engine just sends an > explicit destroy after the migration is deemed successful in the engine too > >> >> Ok, when we create the special purpose vm, do we include all the devices? > > it's an exact copy of the source vm. yes > >> >> If this special vm (waiting for migration) is connected to all the disks, >> then > > it is (it goes though preparePaths as any other VM creation) > >> we only need to register the secrets sent from the source in >> migrationCreate, >> start the special vm, and one it is running, delete the secrets. > > probably. Does the source know those secrets? At the time of migration, the source does not know much about the secrerts. - qemu got the secret key and connected to the ceph disks. - libvirt knew about the secret when starting qemu/ hot-plugging a disk, but now it does not know anything about the secret, except the secret uuid in the <disk> <auth> element.

- vdsm knows about the auth dicts in the disk devices, but does not know the key (the key cannot be persisted)

So engine will have to send the secrets again, so vdsm can send them to the destination.

> >> >>> >>> so far, Engine talks with destination VM only if migration completed >>> succesfully (unless I'm missing something :)) >>> >>> so we need to pass back again all the needed secrets as part of VM.migrate >>> verb >>> >>> This is only the very first thought of mine, we can probably think >>> something >>> cleaner. >>> >>> Bests, >>> >>> -- >>> Francesco Romani >>> RedHat Engineering Virtualization R & D >>> Phone: 8261328 >>> IRC: fromani > >