[Engine-devel] Announcing a proof of concept REST API for VDSM

[Itamar Heim]

On 11/30/2011 05:15 PM, Adam Litke wrote:
> On Wed, Nov 30, 2011 at 03:21:07PM +0200, Itamar Heim wrote:
>> On 11/30/2011 11:29 AM, Daniel P. Berrange wrote:
>>> On Wed, Nov 30, 2011 at 09:14:16AM +0200, Itamar Heim wrote:
>>>> On 11/29/2011 11:36 PM, Adam Litke wrote:
>>>>> On Tue, Nov 29, 2011 at 12:54:44PM -0800, Chris Wright wrote:
>>>>>> * Adam Litke (agl at us.ibm.com) wrote:
>>>>>>> On Tue, Nov 29, 2011 at 11:39:08AM -0800, Chris Wright wrote:
>>>>>>>> * Adam Litke (agl at us.ibm.com) wrote:
>>>>>>>>>
>>>>>>>>> https://github.com/aglitke/vdsm-rest/
>>>>>>>>>
>>>>>>>>> Today I am releasing a proof of concept implementation of a REST API for vdsm.
>>>>>>>>> You can find the code on github.  My goal is to eventually replace the current
>>>>>>>>> xmlrpc interface with a REST API.  Once completed, ovirt-engine could switch to
>>>>>>>>> this new API.  The major advantages to making this change are: 1) VDSM will gain
>>>>>>>>> a structured API that conceptually, structurally, and functionally aligns with
>>>>>>>>> the ovirt-engine REST API, 2) this new API can be made public, thus providing an
>>>>>>>>> entry point for direct virtualization management at the node level.
>>>>>>>>
>>>>>>>> Adam, this looks like a nice PoC.  I didn't see how API versioning is
>>>>>>>> handled.  Any VDSM developers willing to review this work?
>>>>>>>
>>>>>>> Thanks for taking a look.  I am not handling versioning yet.  I think we can add
>>>>>>> a version field to the root API object.  As for compatibility, we'll just have
>>>>>>> to decide on an API backwards-compat support policy.  Would this be enough to
>>>>>>> handle versioning issues?  We shouldn't need anything like capabilities because
>>>>>>> the API is discoverable.
>>>>>>
>>>>>> Right, that seems sensible.
>>>>>>
>>>>>> Did you find cases where RPC to REST resource mapping was difficult?
>>>>>
>>>>> I haven't yet fully implemented the current vdsm API but I suspect that certain
>>>>> calls (like the ones you mention below) will require some extensions to what I
>>>>> have available currently.  The main missing piece is probably events and a nice
>>>>> polling API.  Another big piece of work will be to rebase onto the newly
>>>>> redesigned storage model.
>>>>>
>>>>>> I could see something like migrate() plus migrateStatus() and
>>>>>> migrateCancel() needing to add some kind of operational state that to the
>>>>>> resource.  And something like monitorCommand() which has both a possible
>>>>>> side-effect and some freefrom response...
>>>>>
>>>>> Hopefully monitorCommand will not be too bad, since vdsm should be asking
>>>>> libvirt for the VM details when they are needed.  Of course we'll need to be
>>>>> testing to make sure we aren't keeping state around.  Also, I would expect
>>>>> monitorCommand to 'taint' the VM in the eyes of the vdsm API (as it does for
>>>>> libvirt).
>>>>>
>>>>>>>> <snip>
>>>>>>>>> ovirt-engine wants to subscribe to asynchronous events.  REST APIs do not
>>>>>>>>> typically support async events and instead rely on polling of resources.  I am
>>>>>>>>> investigating what options are available for supporting async events via REST.
>>>>>>>>
>>>>>>>> I think typical is either polling or long polling.  If it's a single
>>>>>>>> resource, then perhaps long polling would be fine (won't be a large
>>>>>>>> number of connections tied up if it's only a single resource).
>>>>>>>
>>>>>>> Not sure if this is what you are referring to, but I was thinking we could do a
>>>>>>> batch-polling mechanism where an API user passes in a list of task UUIDs and/or
>>>>>>> event URIs.  The server would respond with the status of these resources in one
>>>>>>> response.  I have some ideas on how we could wire up asynchronous events on the
>>>>>>> server side to reduce the amount of actual work that such a batch-polling
>>>>>>> operation would require.
>>>>>>
>>>>>> Oh, I just meant this:
>>>>>>
>>>>>> Polling (GET /event + 404 loop)
>>>>>> Long polling (GET + block ... can chew up a thread connection)
>>>>>
>>>>> Yep.  And we can talk later about building an API for efficient, repeated
>>>>> polling.  I wonder if the ovirt-engine guys could weigh in as to whether a REST
>>>>
>>>> cc-ing engine-devel...
>>>>
>>>>> interface with event polling would be acceptable to them.  It is critical that
>>>>> we settle on an API that can become _the_ first-class vehicle for interacting
>>>>> with vdsm.
>>>>
>>>> i have two points for consideration around this:
>>>> 1. as the api between ovirt-engine and vdsm, I had a preference for the
>>>> bus like nature of QMF, as it would allow multiple ovirt-engine to load
>>>> balance handling of messages from the queue, and multiple consumers for
>>>> some messages (say, history service picking up the stats in parallel to
>>>> engine picking them, rather than copying them from engine).
>>>
>>> I tend to agree, using a bus like QMF between ovirt-engine and vdsm is
>>> an inherantly more scalable network architecture, since it avoids the
>>> need to have a direct point-to-point connection between the engine and
>>> every single node, instead you can build a resilient grid by stategically
>>> deploying QPid brokers.
>>>
>>> As compared to REST, it is also much better at coping with async events,
>>> since you can have a push, rather than pull, model which VDSM just puts
>>> events onto the bus as they're generated, to be lazily consumed by any
>>> remote nodes when desired.
>>>
>>> NB, I don't mean to imply that there should not *also* be a REST API for
>>> the node level. A REST API has the very compelling property that it is
>>> trivially accessible from anything with HTTP client support, which is
>>> basically everything in existance today.
>>>
>>>> 2. as node level api, i think a lightweight ovirt-engine managing a
>>>> single node and exposing the exact same REST API and behavior of the
>>>> multi-node ovirt engine would be easier to cosnume for someone that
>>>> wants to interact with a single node same way they would interact with
>>>> ovirt-engine.
>>>
>>> If the REST API is well specified, you wouldn't need to necessarily have
>>> a lightweight ovirt-engine deployed at the node level, you could just
>>> have VDSM implement the same REST specification natively, as is impl
>>> in the engine. Or perhaps provide a some shered code that can be plugged
>>> into both VDSM&   the enegine to facilitate provision of the same REST
>>> interface.
>>
>> Indeed, but an API is not just the transport, it is also behavior
>> and error code numbering, etc.
>> trying to develop two different projects trying to do the same thing
>> sounds like quite the overhead to me.
>
> Agreed.
>
>> I prefer investing the time in making a lighter version of the
>> engine for a single node.
>> then you would also get the web admin and power user portal if you
>> want them.
>
> Could you flesh out some details on the effort required to implement a
> light-weight engine?  I am concerned that this is more complex than it seems.
> Also, how would we avoid dual maintenance between the full engine and the light
> one when a new feature/api is added?
>

I would use the same engine, moduling out parts not relevant for single 
node.
first we need to move to JBoss AS 7 though to get it to be lighter, then 
we can look at other angles (make parts of the engine optional, allow it 
to run with/without jboss, etc.)