[Engine-devel] API design and plan

[Ayal Baron]

I just noticed, why are we cross-posting to engine-devel?

Anyway, comments inline

----- Original Message -----
> On Thu, Dec 08, 2011 at 04:56:17AM -0500, Ayal Baron wrote:
> > 
> > 
> > ----- Original Message -----
> > > On Tue, Dec 06, 2011 at 08:46:57AM -0600, Adam Litke wrote:
> > > > On Tue, Dec 06, 2011 at 02:58:59PM +0200, Dan Kenigsberg wrote:
> > > > > On Mon, Dec 05, 2011 at 11:34:18AM -0600, Adam Litke wrote:
> > > > > > Hi everyone.  On today's VDSM call we discussed the
> > > > > > requirements, design, and
> > > > > > plan for updating the API to include support for QMF and
> > > > > > single-host REST API.
> > > > > > All members present arrived at a general consensus on the
> > > > > > best
> > > > > > way to design the
> > > > > > next-generation API.  I have tried to capture this
> > > > > > discussion
> > > > > > in the oVirt wiki:
> > > > > > 
> > > > > >      http://ovirt.org/wiki/Vdsm_API
> > > > > > 
> > > > > > Please take a look at this page and let's discuss any
> > > > > > changes
> > > > > > that may be needed
> > > > > > in order to adopt it as a working plan that we can begin to
> > > > > > execute.  Thanks!
> > > > > >
> > > > > 
> > > > > Very nice, I've fixed two bullets about the future of the
> > > > > xml-rpc.
> > > > 
> > > > Thanks... Updates look good to me.
> > > > 
> > > > > I think that we are missing something here: how do we model
> > > > > Vdsm-to-Vdsm
> > > > > communication, in a binding-blind way? I'm less worried about
> > > > > the
> > > > > storage-based mailbox used for lvextend requests: my problem
> > > > > is
> > > > > with
> > > > > migration command.
> > > > 
> > > > Ok, interesting...  Besides migration, are there other features
> > > > (current or
> > > > planned) that would involve P2P communication?  I want to
> > > > ensure we
> > > > consider the
> > > > full problem space.
> > > 
> > > Well, I can imagine we would like a host in distress to migrate
> > > VMs
> > > to
> > > whomever can take them, without central management driving this
> > > process.
> > > (CAVE split brain)
> > > 
> > > At the momemt I cannot think of something that cannot be
> > > implemented
> > > by
> > > QMF events. Ayal?
> > > 
> > > > 
> > > > > Currently, the implementation of the "migrate" verb includes
> > > > > contacting
> > > > > the remote Vdsm over xml-rpc before issuing the libvirt
> > > > > migrateToURI2
> > > > > command ('migrationCreate' verb).
> > > > > 
> > > > > A Vdsm user who choose to use the REST binding, is likely to
> > > > > want
> > > > > this to
> > > > > be implemented this using a REST request to the destination.
> > > > > This
> > > > > means
> > > > > that the implementation of Vdsm depends on the chosen
> > > > > binding.
> > > > > 
> > > > > The issue can be mitigating by requiring the binding level to
> > > > > provide a
> > > > > "callback" for migrationCreate (and any other future
> > > > > Vdsm->world
> > > > > requests).
> > > > > This would complicate the beautiful png at
> > > > > http://ovirt.org/wiki/Vdsm_API#Design ... Does anyone have
> > > > > another
> > > > > suggestion?
> > > > 
> > > > Actually, I think you are blending the external API with vdsm
> > > > internals.  As a
> > > > management server or ovirt-engine, I don't care about the
> > > > protocol
> > > > that vdsm
> > > > uses to contact the migration recipient.  As far as I am
> > > > concerned
> > > > this is a
> > > > special case internal function call.  For that purpose, I think
> > > > xmlrpc is
> > > > perfectly well-suited to the task and should be used
> > > > unconditionally, regardless
> > > > of the bindings used to initiate the migration.
> > > > 
> > > > So I would propose that we modify the design such that we keep
> > > > an
> > > > extremely thin
> > > > xmlrpc server active whose sole purpose is to service internal
> > > > P2P
> > > > requests.
> > > 
> > > Interesting. We could avoid even that, if we could register a
> > > callback
> > > with libvirt, so that destination libvirtd called destination
> > > Vdsm to
> > > verify that all storage and networking resources are ready,
> > > before
> > > executing qemu. DanPB, can something like that be done? (I guess
> > > it
> > > is
> > > not realistic since we may need to pass vdsm-specific data from
> > > source
> > > to dest, and libvirt is not supposed to be a general purpose
> > > transport.)
> > > 
> > > Dan.
> > 
> > I don't understand the issue.  The whole point of the REST API is
> > to be an
> > easily consumable *single* node management API.  Once you start
> > coordinating
> > among different nodes then you need clustering and management
> > (either
> > distributed or centralized), in both cases it is fine to require
> > having a bus
> > in which case you have your method of communications between hosts
> > to replace
> > current xml-rpc.
> 
> Implicit in this statement is an assertion that live migration
> between two vdsm
> instances will not be supported without orchestration from an
> ovirt-engine
> instance.  I don't agree with placing such a limitation on vdsm since
> p2p
> migration is already well-supported by the underlying components
> (libvirt and
> qemu).

proper migration requires a lot of coordination.
e.g. making sure that both source and target have the same networks (required by guest) available, access to shared storage, access to same vm configuration, handling split brain scenarios, etc.
This has to be managed somehow...
I don't think that ovirt-engine should be a requirement for this, but something should take care of this.  This entity could just initiate migrateCreate on destination if not using qmf.

> 
> > Requiring an additional xml-rpc server sounds wrong to me.
> 
> The other option is to support a migrateCreate binding in REST and
> QMF.

I see no problem with this, I don't view preparing to accept a migration as an internal flow, it can be directed by either another vdsm or a management server or anything else.  Seeing as it is by any event run by an external entity, it should be a public API.


> 
> --
> Adam Litke <agl at us.ibm.com>
> IBM Linux Technology Center
> 
>