[Engine-devel] Using REST API in web UI - review call summary

Tue Dec 3 11:45:35 UTC 2013

On 11/28/2013 09:37 PM, Vojtech Szocs wrote:
> 
> 
> ----- Original Message -----
>> From: "Michael Pasternak" <mpastern at redhat.com>
>> To: "Itamar Heim" <iheim at redhat.com>
>> Cc: "Vojtech Szocs" <vszocs at redhat.com>, "engine-devel" <engine-devel at ovirt.org>, "Einav Cohen" <ecohen at redhat.com>
>> Sent: Sunday, November 24, 2013 9:37:22 AM
>> Subject: Re: Using REST API in web UI - review call summary
>>
>> On 11/22/2013 12:00 AM, Itamar Heim wrote:
>>> On 11/21/2013 11:56 PM, Vojtech Szocs wrote:
>>>>
>>>>
>>>> ----- Original Message -----
>>>>> From: "Itamar Heim" <iheim at redhat.com>
>>>>> To: "Vojtech Szocs" <vszocs at redhat.com>, "engine-devel"
>>>>> <engine-devel at ovirt.org>
>>>>> Cc: "Einav Cohen" <ecohen at redhat.com>
>>>>> Sent: Thursday, November 21, 2013 10:25:04 PM
>>>>> Subject: Re: Using REST API in web UI - review call summary
>>>>>
>>>>> On 11/21/2013 11:18 PM, Vojtech Szocs wrote:
>>>>>> Hi guys,
>>>>>>
>>>>>> this is a summary of yesterday's review call, I'll try to highlight
>>>>>> important Q/A and things we agreed on. Feel free to add anything in case
>>>>>> I've missed something.
>>>>>>
>>>>>> --
>>>>>>
>>>>>> Q: Why don't we simply try to use existing Java SDK and adapt it for GWT
>>>>>> apps? (asked by Michael & Gilad)
>>>>>>
>>>>>> A: This might be a viable option to consider if we wanted to skip
>>>>>> JavaScript-based SDK altogether and target Java/GWT code directly; we
>>>>>> could simply take Java SDK and customize its abstractions where
>>>>>> necessary,
>>>>>> i.e. using HTTP transport layer implementation that works with GWT. In
>>>>>> any
>>>>>> case, this would mean coupling ourselves to Java SDK (which has its own
>>>>>> release cycle) and I think this would complicate things for us.
>>>>>>
>>>>>> As proposed on the meeting, I think it's best to aim for JavaScript SDK
>>>>>> as
>>>>>> the lowest common denominator for *any* web application that wants to
>>>>>> work
>>>>>> with REST API. oVirt GWT-based UI can simply bind to JavaScript SDK,
>>>>>> i.e.
>>>>>> Java/GWT code that just overlays objects and functions provided by
>>>>>> JavaScript SDK. Another reason is ease of maintenance - I'd rather see
>>>>>> JavaScript SDK's code generation process to be independent of any other
>>>>>> SDK (people responsible for maintaining JavaScript SDK should have full
>>>>>> control over generated code).
>>>>>>
>>>>>> --
>>>>>>
>>>>>> Q: What about functionality currently used by oVirt UI but not supported
>>>>>> by
>>>>>> REST API? (asked by Einav)
>>>>>>      [For example, fetching VM entity over GWT RPC also returns related
>>>>>>      data
>>>>>>      such as Cluster name.]
>>>>>>
>>>>>> A: Based on discussion I've had with other colleagues after yesterday's
>>>>>> review call, I don't think that separate support-like backend layer is a
>>>>>> good idea. Instead, this is the kind of functionality that could be
>>>>>> placed
>>>>>> in oVirt.js library. Logical operations like "get VMs and related data"
>>>>>> would be exposed through oVirt.js (callback-based) API and ultimately
>>>>>> realized as multiple physical requests to REST API via JavaScript
>>>>>> Binding.
>>>>>>
>>>>>> oVirt.js client would be completely oblivious to the fact that multiple
>>>>>> physical requests are dispatched. In fact, since HTTP communication is
>>>>>> asynchronous in nature, oVirt.js client wouldn't even notice any
>>>>>> difference in terms of API consumption. This assumes JavaScript SDK
>>>>>> would
>>>>>> use callback-based (non-blocking) API instead of blocking one - after
>>>>>> all,
>>>>>> blocking API on top of non-blocking implementation sounds pretty much
>>>>>> like
>>>>>> leaky abstraction [1].
>>>>>>
>>>>>> For example:
>>>>>>
>>>>>>       sdk.getVmsWithExtraData(
>>>>>>           callbackToGetExtraDataForGivenVm, // might cause extra
>>>>>>           physical
>>>>>>           requests to REST API
>>>>>>           callbackFiredWhenAllDataIsReady   // update client only when
>>>>>>           all
>>>>>>           data is ready
>>>>>>       )
>>>>>
>>>>> would this also resolve RunMultipleActions?
>>>>
>>>> Yes, I think so. There could be API to pass multiple "actions" and get
>>>> notified when they complete.
>>>>
>>>>> sounds like no reason to have RunMultipleQueries, although i'm still
>>>>> sure a single call to engine for multiple keys would be much more
>>>>> efficient than multiple async calls?
>>>>> (I understand we may not be able to model this).
>>>>
>>>> Efficiency-wise, yes, single call to get all data seems optimal. API-wise,
>>>> I don't think it really matters from oVirt.js client perspective.
>>>>
>>>> We can proceed with simple (possibly inefficient) solution and improve as
>>>> needed. We're making baby steps now..
>>>>
>>>>>
>>>>>>
>>>>>> [1] http://en.wikipedia.org/wiki/Leaky_abstraction
>>>>>>
>>>>>> --
>>>>>>
>>>>>> Last but not least, where to maintain JavaScript SDK projects: low-level
>>>>>> JavaScript Binding + high-level oVirt.js library.
>>>>>>
>>>>>> I agree that conceptually both above mentioned projects should go into
>>>>>> dedicated "ovirt-engine-sdk-js" git repository and have their own
>>>>>> build/release process. However, for now, we're just making baby steps so
>>>>>> let's keep things simple and prototype these projects as part of
>>>>>> "ovirt-engine" git repository.
>>>>>>
>>>>>> ... we can complicate things anytime, but we should know that any
>>>>>> complex
>>>>>> system that works has inevitably evolved from simple system that works
>>>>>> ...
>>>>>> (quote from http://en.wikipedia.org/wiki/Gall%27s_law)
>>>>>
>>>>> I think the entities should just be generated from the xsd.
>>>>
>>>> +1
>>>>
>>>> The JavaScript Binding (aka low-level SDK) module should follow same
>>>> concepts as existing Java SDK - generated entities decorated with
>>>> operations to form fluent API.
>>>>
>>>> Everything Java SDK currently offers should be available in JavaScript
>>>> Binding. oVirt.js is our opportunity to build on top of that.
>>>>
>>>>> for the rsdl, makes sense to start with clean code to see what works
>>>>> best, then see about generating it (but you should adhere the rsdl as
>>>>> guidlines i guess).
>>>>
>>>> +1
>>>>
>>>> The initial prototype should be written by hand, things will get generated
>>>> as soon as we have better idea how the end result should look like.
>>>
>>> i can understand that for the methods and maybe for populating the entities
>>> for the first few.
>>> the entities themselves, no point in hand coding - just generated them from
>>> the api.xsd.
>>> and once you decide how you want to fill them, not worth hand coding this -
>>> either json gives this out of the box, or should be generated as well.
>>>
>>>>
>>>>>
>>>>> lets try to plan for lightweight entities while at it - the API has a
>>>>> mechanism for different level of details - maybe we need a custom level
>>>>> where the client specifies which fields they want back or something like
>>>>> that.
>>>>
>>>> Good idea! We should definitely think about the granularity of entities.
>>>>
>>>> I didn't know REST API supports different level of detail per entity, is
>>>> there some documentation for this feature?
>>
>> currently it's about adding adding extra information (by supplying header
>> [1]),
>> this is driven by the properties that require extra query against the engine
>> to
>> fill them,
>>
>> obviously the default is [2]
>>
>> [1] All-Content: True
>> [2] All-Content: False
> 
> OK so if I understand correctly, "All-Content: False" (default) just returns data of given entity plus IDs of related entities, whereas "All-Content: True" returns data of given entity plus (embedded) data of related entities, is that correct?

not exactly, "All-Content: True" will return other fields of this entity
(not the entity's your resource has relation with, as it was non-RESTful),
the differences is in cost of feeling these properties, if it's expensive
like running extra query to the engine (like Payload/Certificate/Ballooning in vm)
it's not shown by default.

> 
>>
>>>>
>>>> Since JavaScript is dynamic, one possible solution would be to let client
>>>> define the entity structure (i.e. what data client needs) on the fly,
>>>> during runtime :)
>>
>> we talked about this solution couple of years ago, but it was obvious
>> that there is no real "demand" for it till UI moves to REST,
> 
> I assume you mean to let client selectively specify properties of given entity when making REST call, correct? If yes, are you planning to support such feature?

obviously this is one of the steps to take down the road to enhance the restapi.

> 
>>
>> btw it won't solve the problem where to fill UI entity you need combination
>> of N REST entities, you still will have to perform N + 1 queries to API
> 
> Of course. But we can make improvements with regard to:
> 
> - optimizing queries so that client receives/parses only data it needs (see my question above)
> - if necessary, combine such optimized queries into bigger "logical" queries (utilizing callback mechanism)

not sure about "bigger" (you can't change api interface), but combining few requests for the same entity
(in sake of less verbose communication) in to single GET call, is a well known pattern in distributed
systems.

> 
>> (maybe asking for entity with fewer details), - all it address is a capacity
>> of the data being send.
> 
> Not only that, capacity of data sent goes proportionally with time needed to parse & process such data on client. (Imagine big XML marshalling on client. This is why everyone prefers JSON nowadays, JSON naturally maps to native JavaScript structures with minimal computational overhead.)
> 
>>
>>>
>>> michael?
>>>
>>>>
>>>>>
>>>>> Good luck,
>>>>>      Itamar
>>>>>
>>>
>>
>>
>> --
>>
>> Michael Pasternak
>> RedHat, ENG-Virtualization R&D
>>

-- 

Michael Pasternak
RedHat, ENG-Virtualization R&D