[ovirt-devel] Libvirt secrets management - take 2

[Yaniv Dary]

On 06/12/2015 02:40 PM, Nir Soffer wrote:
> ----- Original Message -----
>> From: "Oved Ourfali" <oourfali at redhat.com>
>> To: "Nir Soffer" <nsoffer at redhat.com>
>> Cc: "Eric Blake" <eblake at redhat.com>, "devel" <devel at ovirt.org>, "Michal Skrivanek" <mskrivan at redhat.com>
>> Sent: Friday, June 12, 2015 2:35:30 PM
>> Subject: Re: [ovirt-devel] Libvirt secrets management - take 2
>>
>>
>> On Jun 12, 2015 14:21, Nir Soffer <nsoffer at redhat.com> wrote:
>>> Hi all,
>>>
>>> Recently support for Ceph network disk landed in master. It its possible
>>> now to start a vm using Ceph network disk or hot-plug/unplug such disk
>>> using Cephx authentication.
>>>
>>> However, to make it work, you must add the relevant Ceph secret to
>>> libvirt manually, in the same way it is done in OpenStack deployment.
>>> Our goal is to manage secrets automatically and use ephemeral (safer)
>>> secrets.
>>>
>>> The next patches in the Ceph topic [1], implement secret management in
>>> the same way we manage storage domains or server connections:
>>>
>>> The concept is - all hosts can use all secrets, so you can migrate a vm
>>> using Ceph disk to any host in the cluster.
>>>
>>> 1. When host becomes up, we register the secrets associated with all the
>>>     current active domains with libvirt
>>>
>>> 2. When activating a domain, we register the secrets associated with the
>>>     new domain with libvirt
>>>
>>> 3. When deactivating a domain, we unregister the secrets associated with
>>>     the domain from libvirt
>>>
>>> 4. When moving host to maintenance, we clear all secrets
>>>
>>> 5. When vdsm shutdown or starts, clear all secrets to ensure that we don't
>>> keep
>>>     stale or unneeded secrets on a host
>>>
>>> This system seems to work, but Federico pointed few issues and suggested
>>> a new (simpler?) approach.
>>>
>>> In future libvirt version, libvirt will support the concept of transient
>>> secrets so you can start a transient vm using secret without registering
>>> the secret with libvirt before starting the vm. The secret will be
>>> specified in the vm XML (for starting a vm) or disk XML (for hot-plug).
>>> This will make our secret management system and APIs useless.

Can you open a RFE tracking the libvirt RFE on secrets?

>> When is this planned to be added to libvirt?
> I have no idea
>
>> iiuc, once added, you will no
>> longer need to register the secrets with libvirt, right?
> Right
>
>>> Managing state on multiple hosts is hard; we will probably have to deal
>>> with nasty edge cases (e.g. lost messages, network errors), which may
>>> lead to host with missing secret, which cannot run some vms. We probably
>>> do this right for storage domains (after 8 years?), and we should not
>>> assume that we are smarter and secret management will work in the first
>>> try.
>>>
>>> The new approach is to *not* manage state or multiple hosts. Instead,
>>> send the required secrets only to the host that starting a vm or
>>> hot-plugging a disk that need a libvirt secret:
>>>
>>> 1. When starting a vm, add the required secrets to the vm description.
>>>     On the host, vdsm will register these secrets with libvirt before
>>>     starting the vm.
>>>
>>> 2. When migrating a vm, add the required secrets to the vm description.
>>>     On the host, vdsm will send these secrets to the destination host,
>>>     and on the destination host, vdsm will register the secrets with libvirt
>>>     before starting the vm.
>>>
>> Will these secrets be part of the domain xml?
>> If so, no need for special treatment in case of VM migration.
>>
>>> 3. When hot-plugging a disk, send the secret if needed in the disk
>>>     description.  On the host, vdsm will register the secrets with libvirt.
>>>
>>> 4. When vdsm shutdown or starts, clear all secrets to ensure that we don't
>>> keep
>>>     stale or unneeded secrets on a host
>>>
>>> 5. We never unregister secrets, since they are ephemeral anyway.
>>>
>>> 6. Alternatively, we can implement secrets reference counting so when a vm
>>>     stops or disk is hot-unplugged we decrease the reference count on the
>>>     secrets associated with this vm/disk, and if no other vms need the
>>>     secret, we can unregister the secret from libvirt.
>> Doesn't seem necessary to me.
>>
>>> The new approach is simpler, if we avoid the fancy secret reference
>>> counting. I believe we can get it merged in couple of weeks with help
>>> from the virt team.
>>>
>>> Please share your thoughts on these alternative solutions.
>>>
>> Keep in mind also hosted engine. Sounds to me like the new approach will be a
>> better fit for it, as he won't need to deal with storage domain secrets, but
>> just pass it in the VM description... Although I'm not a hosted engine
>> expert, so not sure it makes a difference, but it sounds simpler in that
>> aspect as well.
> Good point, the first option require hosted engine installer to register
> the required secrets when installing the engine vm, and maybe also when
> adding a new host.
>
> Nir
> _______________________________________________
> Devel mailing list
> Devel at ovirt.org
> http://lists.ovirt.org/mailman/listinfo/devel

-- 
Yaniv Dary
Technical Product Manager
Red Hat Israel Ltd.
34 Jerusalem Road
Building A, 4th floor
Ra'anana, Israel 4350109

Tel : +972 (9) 7692306
       8272306
Email: ydary at redhat.com
IRC : ydary