[Engine-devel] API design and plan

[Ayal Baron]

----- Original Message -----
> On 12/12/2011 08:18 PM, Jon Choate wrote:
> > On 12/08/2011 04:34 PM, Ayal Baron wrote:
> >> I just noticed, why are we cross-posting to engine-devel?
> >>
> >> Anyway, comments inline
> >>
> >> ----- Original Message -----
> >>> On Thu, Dec 08, 2011 at 04:56:17AM -0500, Ayal Baron wrote:
> >>>>
> >>>> ----- Original Message -----
> >>>>> On Tue, Dec 06, 2011 at 08:46:57AM -0600, Adam Litke wrote:
> >>>>>> On Tue, Dec 06, 2011 at 02:58:59PM +0200, Dan Kenigsberg
> >>>>>> wrote:
> >>>>>>> On Mon, Dec 05, 2011 at 11:34:18AM -0600, Adam Litke wrote:
> >>>>>>>> Hi everyone. On today's VDSM call we discussed the
> >>>>>>>> requirements, design, and
> >>>>>>>> plan for updating the API to include support for QMF and
> >>>>>>>> single-host REST API.
> >>>>>>>> All members present arrived at a general consensus on the
> >>>>>>>> best
> >>>>>>>> way to design the
> >>>>>>>> next-generation API. I have tried to capture this
> >>>>>>>> discussion
> >>>>>>>> in the oVirt wiki:
> >>>>>>>>
> >>>>>>>> http://ovirt.org/wiki/Vdsm_API
> >>>>>>>>
> >>>>>>>> Please take a look at this page and let's discuss any
> >>>>>>>> changes
> >>>>>>>> that may be needed
> >>>>>>>> in order to adopt it as a working plan that we can begin to
> >>>>>>>> execute. Thanks!
> >>>>>>>>
> >>>>>>> Very nice, I've fixed two bullets about the future of the
> >>>>>>> xml-rpc.
> >>>>>> Thanks... Updates look good to me.
> >>>>>>
> >>>>>>> I think that we are missing something here: how do we model
> >>>>>>> Vdsm-to-Vdsm
> >>>>>>> communication, in a binding-blind way? I'm less worried about
> >>>>>>> the
> >>>>>>> storage-based mailbox used for lvextend requests: my problem
> >>>>>>> is
> >>>>>>> with
> >>>>>>> migration command.
> >>>>>> Ok, interesting... Besides migration, are there other features
> >>>>>> (current or
> >>>>>> planned) that would involve P2P communication? I want to
> >>>>>> ensure we
> >>>>>> consider the
> >>>>>> full problem space.
> >>>>> Well, I can imagine we would like a host in distress to migrate
> >>>>> VMs
> >>>>> to
> >>>>> whomever can take them, without central management driving this
> >>>>> process.
> >>>>> (CAVE split brain)
> >>>>>
> >>>>> At the momemt I cannot think of something that cannot be
> >>>>> implemented
> >>>>> by
> >>>>> QMF events. Ayal?
> >>>>>
> >>>>>>> Currently, the implementation of the "migrate" verb includes
> >>>>>>> contacting
> >>>>>>> the remote Vdsm over xml-rpc before issuing the libvirt
> >>>>>>> migrateToURI2
> >>>>>>> command ('migrationCreate' verb).
> >>>>>>>
> >>>>>>> A Vdsm user who choose to use the REST binding, is likely to
> >>>>>>> want
> >>>>>>> this to
> >>>>>>> be implemented this using a REST request to the destination.
> >>>>>>> This
> >>>>>>> means
> >>>>>>> that the implementation of Vdsm depends on the chosen
> >>>>>>> binding.
> >>>>>>>
> >>>>>>> The issue can be mitigating by requiring the binding level to
> >>>>>>> provide a
> >>>>>>> "callback" for migrationCreate (and any other future
> >>>>>>> Vdsm->world
> >>>>>>> requests).
> >>>>>>> This would complicate the beautiful png at
> >>>>>>> http://ovirt.org/wiki/Vdsm_API#Design ... Does anyone have
> >>>>>>> another
> >>>>>>> suggestion?
> >>>>>> Actually, I think you are blending the external API with vdsm
> >>>>>> internals. As a
> >>>>>> management server or ovirt-engine, I don't care about the
> >>>>>> protocol
> >>>>>> that vdsm
> >>>>>> uses to contact the migration recipient. As far as I am
> >>>>>> concerned
> >>>>>> this is a
> >>>>>> special case internal function call. For that purpose, I think
> >>>>>> xmlrpc is
> >>>>>> perfectly well-suited to the task and should be used
> >>>>>> unconditionally, regardless
> >>>>>> of the bindings used to initiate the migration.
> >>>>>>
> >>>>>> So I would propose that we modify the design such that we keep
> >>>>>> an
> >>>>>> extremely thin
> >>>>>> xmlrpc server active whose sole purpose is to service internal
> >>>>>> P2P
> >>>>>> requests.
> >>>>> Interesting. We could avoid even that, if we could register a
> >>>>> callback
> >>>>> with libvirt, so that destination libvirtd called destination
> >>>>> Vdsm to
> >>>>> verify that all storage and networking resources are ready,
> >>>>> before
> >>>>> executing qemu. DanPB, can something like that be done? (I
> >>>>> guess
> >>>>> it
> >>>>> is
> >>>>> not realistic since we may need to pass vdsm-specific data from
> >>>>> source
> >>>>> to dest, and libvirt is not supposed to be a general purpose
> >>>>> transport.)
> >>>>>
> >>>>> Dan.
> >>>> I don't understand the issue. The whole point of the REST API is
> >>>> to be an
> >>>> easily consumable *single* node management API. Once you start
> >>>> coordinating
> >>>> among different nodes then you need clustering and management
> >>>> (either
> >>>> distributed or centralized), in both cases it is fine to require
> >>>> having a bus
> >>>> in which case you have your method of communications between
> >>>> hosts
> >>>> to replace
> >>>> current xml-rpc.
> >>> Implicit in this statement is an assertion that live migration
> >>> between two vdsm
> >>> instances will not be supported without orchestration from an
> >>> ovirt-engine
> >>> instance. I don't agree with placing such a limitation on vdsm
> >>> since
> >>> p2p
> >>> migration is already well-supported by the underlying components
> >>> (libvirt and
> >>> qemu).
> >> proper migration requires a lot of coordination.
> >> e.g. making sure that both source and target have the same
> >> networks
> >> (required by guest) available, access to shared storage, access to
> >> same vm configuration, handling split brain scenarios, etc.
> >> This has to be managed somehow...
> >> I don't think that ovirt-engine should be a requirement for this,
> >> but
> >> something should take care of this. This entity could just
> >> initiate
> >> migrateCreate on destination if not using qmf.
> >>
> >>>> Requiring an additional xml-rpc server sounds wrong to me.
> >>> The other option is to support a migrateCreate binding in REST
> >>> and
> >>> QMF.
> >> I see no problem with this, I don't view preparing to accept a
> >> migration as an internal flow, it can be directed by either
> >> another
> >> vdsm or a management server or anything else. Seeing as it is by
> >> any
> >> event run by an external entity, it should be a public API.
> >>
> >>
> >>> --
> >>> Adam Litke<agl at us.ibm.com>
> >>> IBM Linux Technology Center
> >>>
> >>>
> >> _______________________________________________
> >> Engine-devel mailing list
> >> Engine-devel at ovirt.org
> >> http://lists.ovirt.org/mailman/listinfo/engine-devel
> > Is there a bias towards using the mailing list and against the
> > discussion tab on the wiki page when discussing wiki pages? Just
> > want to
> > know. I'm used to commenting on wikis on the discussion page.
> 
> mailing list - shouldn't be.
> wiki - may be lost on the larger audience - I think discussing on the
> mailing list makes more sense since many don't subscribe to the wiki
> pages after their first review from when it was published.

No offense, but the only thing which is more broken than email for discussing such things is a wiki discussion (mailing list at least pushes changes by default. has clients to manage replies by threads, responses can be sent concurrently etc).


> _______________________________________________
> Engine-devel mailing list
> Engine-devel at ovirt.org
> http://lists.ovirt.org/mailman/listinfo/engine-devel
>