[Engine-devel] API design and plan

[Itamar Heim]

On 12/12/2011 08:18 PM, Jon Choate wrote:
> On 12/08/2011 04:34 PM, Ayal Baron wrote:
>> I just noticed, why are we cross-posting to engine-devel?
>>
>> Anyway, comments inline
>>
>> ----- Original Message -----
>>> On Thu, Dec 08, 2011 at 04:56:17AM -0500, Ayal Baron wrote:
>>>>
>>>> ----- Original Message -----
>>>>> On Tue, Dec 06, 2011 at 08:46:57AM -0600, Adam Litke wrote:
>>>>>> On Tue, Dec 06, 2011 at 02:58:59PM +0200, Dan Kenigsberg wrote:
>>>>>>> On Mon, Dec 05, 2011 at 11:34:18AM -0600, Adam Litke wrote:
>>>>>>>> Hi everyone. On today's VDSM call we discussed the
>>>>>>>> requirements, design, and
>>>>>>>> plan for updating the API to include support for QMF and
>>>>>>>> single-host REST API.
>>>>>>>> All members present arrived at a general consensus on the
>>>>>>>> best
>>>>>>>> way to design the
>>>>>>>> next-generation API. I have tried to capture this
>>>>>>>> discussion
>>>>>>>> in the oVirt wiki:
>>>>>>>>
>>>>>>>> http://ovirt.org/wiki/Vdsm_API
>>>>>>>>
>>>>>>>> Please take a look at this page and let's discuss any
>>>>>>>> changes
>>>>>>>> that may be needed
>>>>>>>> in order to adopt it as a working plan that we can begin to
>>>>>>>> execute. Thanks!
>>>>>>>>
>>>>>>> Very nice, I've fixed two bullets about the future of the
>>>>>>> xml-rpc.
>>>>>> Thanks... Updates look good to me.
>>>>>>
>>>>>>> I think that we are missing something here: how do we model
>>>>>>> Vdsm-to-Vdsm
>>>>>>> communication, in a binding-blind way? I'm less worried about
>>>>>>> the
>>>>>>> storage-based mailbox used for lvextend requests: my problem
>>>>>>> is
>>>>>>> with
>>>>>>> migration command.
>>>>>> Ok, interesting... Besides migration, are there other features
>>>>>> (current or
>>>>>> planned) that would involve P2P communication? I want to
>>>>>> ensure we
>>>>>> consider the
>>>>>> full problem space.
>>>>> Well, I can imagine we would like a host in distress to migrate
>>>>> VMs
>>>>> to
>>>>> whomever can take them, without central management driving this
>>>>> process.
>>>>> (CAVE split brain)
>>>>>
>>>>> At the momemt I cannot think of something that cannot be
>>>>> implemented
>>>>> by
>>>>> QMF events. Ayal?
>>>>>
>>>>>>> Currently, the implementation of the "migrate" verb includes
>>>>>>> contacting
>>>>>>> the remote Vdsm over xml-rpc before issuing the libvirt
>>>>>>> migrateToURI2
>>>>>>> command ('migrationCreate' verb).
>>>>>>>
>>>>>>> A Vdsm user who choose to use the REST binding, is likely to
>>>>>>> want
>>>>>>> this to
>>>>>>> be implemented this using a REST request to the destination.
>>>>>>> This
>>>>>>> means
>>>>>>> that the implementation of Vdsm depends on the chosen
>>>>>>> binding.
>>>>>>>
>>>>>>> The issue can be mitigating by requiring the binding level to
>>>>>>> provide a
>>>>>>> "callback" for migrationCreate (and any other future
>>>>>>> Vdsm->world
>>>>>>> requests).
>>>>>>> This would complicate the beautiful png at
>>>>>>> http://ovirt.org/wiki/Vdsm_API#Design ... Does anyone have
>>>>>>> another
>>>>>>> suggestion?
>>>>>> Actually, I think you are blending the external API with vdsm
>>>>>> internals. As a
>>>>>> management server or ovirt-engine, I don't care about the
>>>>>> protocol
>>>>>> that vdsm
>>>>>> uses to contact the migration recipient. As far as I am
>>>>>> concerned
>>>>>> this is a
>>>>>> special case internal function call. For that purpose, I think
>>>>>> xmlrpc is
>>>>>> perfectly well-suited to the task and should be used
>>>>>> unconditionally, regardless
>>>>>> of the bindings used to initiate the migration.
>>>>>>
>>>>>> So I would propose that we modify the design such that we keep
>>>>>> an
>>>>>> extremely thin
>>>>>> xmlrpc server active whose sole purpose is to service internal
>>>>>> P2P
>>>>>> requests.
>>>>> Interesting. We could avoid even that, if we could register a
>>>>> callback
>>>>> with libvirt, so that destination libvirtd called destination
>>>>> Vdsm to
>>>>> verify that all storage and networking resources are ready,
>>>>> before
>>>>> executing qemu. DanPB, can something like that be done? (I guess
>>>>> it
>>>>> is
>>>>> not realistic since we may need to pass vdsm-specific data from
>>>>> source
>>>>> to dest, and libvirt is not supposed to be a general purpose
>>>>> transport.)
>>>>>
>>>>> Dan.
>>>> I don't understand the issue. The whole point of the REST API is
>>>> to be an
>>>> easily consumable *single* node management API. Once you start
>>>> coordinating
>>>> among different nodes then you need clustering and management
>>>> (either
>>>> distributed or centralized), in both cases it is fine to require
>>>> having a bus
>>>> in which case you have your method of communications between hosts
>>>> to replace
>>>> current xml-rpc.
>>> Implicit in this statement is an assertion that live migration
>>> between two vdsm
>>> instances will not be supported without orchestration from an
>>> ovirt-engine
>>> instance. I don't agree with placing such a limitation on vdsm since
>>> p2p
>>> migration is already well-supported by the underlying components
>>> (libvirt and
>>> qemu).
>> proper migration requires a lot of coordination.
>> e.g. making sure that both source and target have the same networks
>> (required by guest) available, access to shared storage, access to
>> same vm configuration, handling split brain scenarios, etc.
>> This has to be managed somehow...
>> I don't think that ovirt-engine should be a requirement for this, but
>> something should take care of this. This entity could just initiate
>> migrateCreate on destination if not using qmf.
>>
>>>> Requiring an additional xml-rpc server sounds wrong to me.
>>> The other option is to support a migrateCreate binding in REST and
>>> QMF.
>> I see no problem with this, I don't view preparing to accept a
>> migration as an internal flow, it can be directed by either another
>> vdsm or a management server or anything else. Seeing as it is by any
>> event run by an external entity, it should be a public API.
>>
>>
>>> --
>>> Adam Litke<agl at us.ibm.com>
>>> IBM Linux Technology Center
>>>
>>>
>> _______________________________________________
>> Engine-devel mailing list
>> Engine-devel at ovirt.org
>> http://lists.ovirt.org/mailman/listinfo/engine-devel
> Is there a bias towards using the mailing list and against the
> discussion tab on the wiki page when discussing wiki pages? Just want to
> know. I'm used to commenting on wikis on the discussion page.

mailing list - shouldn't be.
wiki - may be lost on the larger audience - I think discussing on the 
mailing list makes more sense since many don't subscribe to the wiki 
pages after their first review from when it was published.