Re: [ovirt-devel] Libvirt secrets management - take 2

----- Original Message ----- > From: "Oved Ourfali" <oourfali(a)redhat.com&gt; > To: "Nir Soffer" <nsoffer(a)redhat.com&gt; > Cc: "Eric Blake" <eblake(a)redhat.com&gt;, "devel" <devel(a)ovirt.org&gt;, "Michal Skrivanek" <mskrivan(a)redhat.com&gt; > Sent: Friday, June 12, 2015 2:35:30 PM > Subject: Re: [ovirt-devel] Libvirt secrets management - take 2 > > > On Jun 12, 2015 14:21, Nir Soffer <nsoffer(a)redhat.com&gt; wrote: >> Hi all, >> >> Recently support for Ceph network disk landed in master. It its possible >> now to start a vm using Ceph network disk or hot-plug/unplug such disk >> using Cephx authentication. >> >> However, to make it work, you must add the relevant Ceph secret to >> libvirt manually, in the same way it is done in OpenStack deployment. >> Our goal is to manage secrets automatically and use ephemeral (safer) >> secrets. >> >> The next patches in the Ceph topic [1], implement secret management in >> the same way we manage storage domains or server connections: >> >> The concept is - all hosts can use all secrets, so you can migrate a vm >> using Ceph disk to any host in the cluster. >> >> 1. When host becomes up, we register the secrets associated with all the >> current active domains with libvirt >> >> 2. When activating a domain, we register the secrets associated with the >> new domain with libvirt >> >> 3. When deactivating a domain, we unregister the secrets associated with >> the domain from libvirt >> >> 4. When moving host to maintenance, we clear all secrets >> >> 5. When vdsm shutdown or starts, clear all secrets to ensure that we don't >> keep >> stale or unneeded secrets on a host >> >> This system seems to work, but Federico pointed few issues and suggested >> a new (simpler?) approach. >> >> In future libvirt version, libvirt will support the concept of transient >> secrets so you can start a transient vm using secret without registering >> the secret with libvirt before starting the vm. The secret will be >> specified in the vm XML (for starting a vm) or disk XML (for hot-plug). >> This will make our secret management system and APIs useless.

> When is this planned to be added to libvirt? I have no idea > iiuc, once added, you will no > longer need to register the secrets with libvirt, right? Right >> Managing state on multiple hosts is hard; we will probably have to deal >> with nasty edge cases (e.g. lost messages, network errors), which may >> lead to host with missing secret, which cannot run some vms. We probably >> do this right for storage domains (after 8 years?), and we should not >> assume that we are smarter and secret management will work in the first >> try. >> >> The new approach is to *not* manage state or multiple hosts. Instead, >> send the required secrets only to the host that starting a vm or >> hot-plugging a disk that need a libvirt secret: >> >> 1. When starting a vm, add the required secrets to the vm description. >> On the host, vdsm will register these secrets with libvirt before >> starting the vm. >> >> 2. When migrating a vm, add the required secrets to the vm description. >> On the host, vdsm will send these secrets to the destination host, >> and on the destination host, vdsm will register the secrets with libvirt >> before starting the vm. >> > Will these secrets be part of the domain xml? > If so, no need for special treatment in case of VM migration. > >> 3. When hot-plugging a disk, send the secret if needed in the disk >> description. On the host, vdsm will register the secrets with libvirt. >> >> 4. When vdsm shutdown or starts, clear all secrets to ensure that we don't >> keep >> stale or unneeded secrets on a host >> >> 5. We never unregister secrets, since they are ephemeral anyway. >> >> 6. Alternatively, we can implement secrets reference counting so when a vm >> stops or disk is hot-unplugged we decrease the reference count on the >> secrets associated with this vm/disk, and if no other vms need the >> secret, we can unregister the secret from libvirt. > Doesn't seem necessary to me. > >> The new approach is simpler, if we avoid the fancy secret reference >> counting. I believe we can get it merged in couple of weeks with help >> from the virt team. >> >> Please share your thoughts on these alternative solutions. >> > Keep in mind also hosted engine. Sounds to me like the new approach will be a > better fit for it, as he won't need to deal with storage domain secrets, but > just pass it in the VM description... Although I'm not a hosted engine > expert, so not sure it makes a difference, but it sounds simpler in that > aspect as well. Good point, the first option require hosted engine installer to register the required secrets when installing the engine vm, and maybe also when adding a new host. Nir _______________________________________________ Devel mailing list Devel(a)ovirt.org http://lists.ovirt.org/mailman/listinfo/devel