[Engine-devel] [vdsm] unmanaged devices thrown into 'custom' feature

[Ayal Baron]

On 10/24/2012 05:54 AM, Itamar Heim wrote:
> On 10/23/2012 10:52 PM, Ayal Baron wrote:
>>
>>
>> ----- Original Message -----
>>> On Tue, Oct 23, 2012 at 05:53:20AM -0400, Doron Fediuck wrote:
>>>>
>>>>
>>>> ----- Original Message -----
>>>>> From: "Livnat Peer" <lpeer at redhat.com>
>>>>> To: "Dan Kenigsberg" <danken at redhat.com>
>>>>> Cc: engine-devel at ovirt.org, "Genadi Chereshnya"
>>>>> <gcheresh at redhat.com>, vdsm-devel at fedorahosted.org
>>>>> Sent: Monday, October 22, 2012 8:29:20 AM
>>>>> Subject: Re: [Engine-devel] [vdsm] unmanaged devices thrown into
>>>>> 'custom' feature
>>>>>
>>>>> On 21/10/12 23:49, Dan Kenigsberg wrote:
>>>>>> On Sun, Oct 21, 2012 at 11:57:10AM -0400, Eli Mesika wrote:
>>>>>>>
>>>>>>>
>>>>>>> ----- Original Message -----
>>>>>>>> From: "Yair Zaslavsky" <yzaslavs at redhat.com>
>>>>>>>> To: "Livnat Peer" <lpeer at redhat.com>
>>>>>>>> Cc: "Genadi Chereshnya" <gcheresh at redhat.com>,
>>>>>>>> engine-devel at ovirt.org, vdsm-devel at fedorahosted.org
>>>>>>>> Sent: Sunday, October 21, 2012 5:38:54 PM
>>>>>>>> Subject: Re: [Engine-devel] unmanaged devices thrown into
>>>>>>>> 'custom' feature
>>>>>>>>
>>>>>>>>
>>>>>>>>
>>>>>>>> ----- Original Message -----
>>>>>>>>> From: "Livnat Peer" <lpeer at redhat.com>
>>>>>>>>> To: "Dan Kenigsberg" <danken at redhat.com>
>>>>>>>>> Cc: "Genadi Chereshnya" <gcheresh at redhat.com>,
>>>>>>>>> engine-devel at ovirt.org, vdsm-devel at fedorahosted.org
>>>>>>>>> Sent: Sunday, October 21, 2012 5:18:31 PM
>>>>>>>>> Subject: Re: [Engine-devel] unmanaged devices thrown into
>>>>>>>>> 'custom'
>>>>>>>>> feature
>>>>>>>>>
>>>>>>>>> On 21/10/12 16:42, Dan Kenigsberg wrote:
>>>>>>>>>> I have just noticed that when a VM is started for the
>>>>>>>>>> second
>>>>>>>>>> time,
>>>>>>>>>> Engine
>>>>>>>>>> issues the "create" vdsm verb with some information
>>>>>>>>>> regarding
>>>>>>>>>> "unmanaged" devices (an example is shown below[1]) in the
>>>>>>>>>> 'custom'
>>>>>>>>>> propery bag.
>>>>>>>>>>
>>>>>>>>>> I'm surprised about this, as I was not aware of this usage
>>>>>>>>>> of
>>>>>>>>>> the
>>>>>>>>>> 'custom' dictionary, and Vdsm is not doing anything with
>>>>>>>>>> the
>>>>>>>>>> data.
>>>>>>>>>>
>>>>>>>>>
>>>>>>>>> If I recall correctly the idea of passing the unmanaged
>>>>>>>>> devices
>>>>>>>>> data
>>>>>>>>> in
>>>>>>>>> the custom property was to enable managing stable device
>>>>>>>>> addresses
>>>>>>>>> in
>>>>>>>>> the hooks (to devices that were added to the VM via hooks
>>>>>>>>> from
>>>>>>>>> the
>>>>>>>>> first
>>>>>>>>> place), so this info is there not for VDSM use.
>>>>>>>>> For example if you add a device in a hook it will be kept in
>>>>>>>>> the
>>>>>>>>> engine
>>>>>>>>> as a non managed device. later when starting the VM again
>>>>>>>>> you
>>>>>>>>> would
>>>>>>>>> like
>>>>>>>>> to assign the same device address to your device, and you
>>>>>>>>> can do
>>>>>>>>> so
>>>>>>>>> because you have access to the original address in the
>>>>>>>>> custom
>>>>>>>>> properties
>>>>>>>>> of the VM.
>>>>>>>>
>>>>>>>> This is exactly what Eli has explained Gendai and Dan today.
>>>>>>
>>>>>> (I was asking here because I did not understand the verbal
>>>>>> explanation.)
>>>>>>
>>>>>>>
>>>>>>> This is taken from the Stable Device Address design in
>>>>>>> http://wiki.ovirt.org/wiki/Features/Design/StableDeviceAddresses
>>>>>>>
>>>>>>> Unmanaged Device
>>>>>>> -----------------
>>>>>>> Unmanaged Device will be supported in the new format and will
>>>>>>> include all unhandled devices as sound/controller/etc and
>>>>>>> future
>>>>>>> devices. Those devices will be persistent and will have Type ,
>>>>>>> SubType (device specific) and an Address. For 3.1 an unmanaged
>>>>>>> Device is not exposed to any GUI/REST API. Unmanaged devices
>>>>>>> are
>>>>>>> passed to vdsm inside a Custom property. VDSM in it turn is
>>>>>>> passing this as is for possible hook processing.
>>>>>>
>>>>>> Thanks for the elaboration. Too bad that I've missed this issue
>>>>>> before.
>>>>>>
>>>>>> Are you aware of any hook making use of this?  I hope that hook
>>>>>> writers
>>>>>> are not using APIs that are not documented in vdsmd(8).
>>>>>>
>>>>>> It seems as a classic case where a generic bag interface is
>>>>>> coerced
>>>>>> into
>>>>>> an awkward partially-documented interface.
>>>>>>
>>>>>> I think that a better approach would have been to pass all
>>>>>> devices
>>>>>> (managed and unmanaged alike) in the 'devices' property, and
>>>>>> let
>>>>>> vdsm
>>>>>> expose whatever is needed to the before_vm_start hook.
>>>>>>
>>>>>> Maybe we can still do this.
>>>>>
>>>>> That was the original idea but Ayal objected and I think Igor did
>>>>> not
>>>>> like it as well...
>>>>>
>>>>
>>>> +2.
>>>> The original design had an 'unmanaged' (or generic) device type,
>>>> and all
>>>> devices should have been normalized. But as explained, this was
>>>> strongly
>>>> rejected in the VDSM side, causing Eli write some special handling
>>>> for this anomaly.
>>>
>>> Can someone (Ayal?) explain the rejection on Vdsm side?
>>> Hiding part of the API in the custom propery bag requires strong
>>> reasoning indeed.
>>
>> It's not hiding anything.
>> Today vdsm passes the libvirt xml to the hooks + the custom properties.
>> If you pass 'non-managed devices' through the devices API then 
>> basically you're saying to vdsm 'here is a bunch of things I want you 
>> to ignore but be a sport and pass it on to the hooks'.
>> Last I checked, that mechanism is custom properties.  I don't see any 
>> reason to add another one.
>
> 1. this means hooks don't actually get the unmanaged devices today?

Yes they do, in the custom properties.

> 2. custom property are to pass parameters to the hook user can 
> reasonably control. device management should not be part of that if 
> not strictly related to the hook (from user's side).

No problem, then we should not pass these devices at all.

> 3. I'm guessing (didn't get this first time i read your answer), you 
> are hinting the unmanaged devices should be passed as a custom 
> property to vdsm, auto generated by engine?
I'm not saying this is what *should* happen, I'm saying this is what 
*is* happening.
vdsm passes to hooks 2 things:
1. libvirt xml (which obviously would not contain 'unmanaged devices')
2. whatever is passed in 'custom properties' by engine (afaik engine 
passes the entire bag of properties to all hooks, relevant or not).
For some reason someone insisted on being able to pass all the devices 
engine gets back from the vm back to the hooks even if these were not 
passed by engine when starting up the VM (iirc the idea was that if a 
hook adds a device in this way it is able to know what address the 
device got on previous run and maintain stable device addresses).
What I opposed originally was the ability when running a VM to pass in 
the devices section a list of devices that should not be attached to the 
vm (i.e. 'here is a list of devices that I don't want you to use devices 
in the vm').  I think it's self evident why I was against this and why I 
still oppose the idea.
If you want a bag of rubbish to pass to hooks then that is 'custom 
properties'.  No need to make the API any uglier.
Personally, I think that any hook that adds devices should be able to 
manage addresses (e.g. choose an empty address, or add a post hook for 
running VMs and persist the info).
In fact, this is just another case where hooks are used in the entirely 
wrong place.  The place to do this is in the engine with an engine side 
pre-run vm hook and then any devices you want to add to the VM become 
'managed' as far as vdsm is concerned.

Note that from what I could understand, Dan was really trying to 
understand why we don't 'manage' these devices (actually passing them to 
libvirt on subsequent runs of the VM).