[Engine-devel] Using REST API in web UI - review call summary

Thu Nov 28 19:37:36 UTC 2013

----- Original Message -----
> From: "Michael Pasternak" <mpastern at redhat.com>
> To: "Itamar Heim" <iheim at redhat.com>
> Cc: "Vojtech Szocs" <vszocs at redhat.com>, "engine-devel" <engine-devel at ovirt.org>, "Einav Cohen" <ecohen at redhat.com>
> Sent: Sunday, November 24, 2013 9:37:22 AM
> Subject: Re: Using REST API in web UI - review call summary
> 
> On 11/22/2013 12:00 AM, Itamar Heim wrote:
> > On 11/21/2013 11:56 PM, Vojtech Szocs wrote:
> >>
> >>
> >> ----- Original Message -----
> >>> From: "Itamar Heim" <iheim at redhat.com>
> >>> To: "Vojtech Szocs" <vszocs at redhat.com>, "engine-devel"
> >>> <engine-devel at ovirt.org>
> >>> Cc: "Einav Cohen" <ecohen at redhat.com>
> >>> Sent: Thursday, November 21, 2013 10:25:04 PM
> >>> Subject: Re: Using REST API in web UI - review call summary
> >>>
> >>> On 11/21/2013 11:18 PM, Vojtech Szocs wrote:
> >>>> Hi guys,
> >>>>
> >>>> this is a summary of yesterday's review call, I'll try to highlight
> >>>> important Q/A and things we agreed on. Feel free to add anything in case
> >>>> I've missed something.
> >>>>
> >>>> --
> >>>>
> >>>> Q: Why don't we simply try to use existing Java SDK and adapt it for GWT
> >>>> apps? (asked by Michael & Gilad)
> >>>>
> >>>> A: This might be a viable option to consider if we wanted to skip
> >>>> JavaScript-based SDK altogether and target Java/GWT code directly; we
> >>>> could simply take Java SDK and customize its abstractions where
> >>>> necessary,
> >>>> i.e. using HTTP transport layer implementation that works with GWT. In
> >>>> any
> >>>> case, this would mean coupling ourselves to Java SDK (which has its own
> >>>> release cycle) and I think this would complicate things for us.
> >>>>
> >>>> As proposed on the meeting, I think it's best to aim for JavaScript SDK
> >>>> as
> >>>> the lowest common denominator for *any* web application that wants to
> >>>> work
> >>>> with REST API. oVirt GWT-based UI can simply bind to JavaScript SDK,
> >>>> i.e.
> >>>> Java/GWT code that just overlays objects and functions provided by
> >>>> JavaScript SDK. Another reason is ease of maintenance - I'd rather see
> >>>> JavaScript SDK's code generation process to be independent of any other
> >>>> SDK (people responsible for maintaining JavaScript SDK should have full
> >>>> control over generated code).
> >>>>
> >>>> --
> >>>>
> >>>> Q: What about functionality currently used by oVirt UI but not supported
> >>>> by
> >>>> REST API? (asked by Einav)
> >>>>      [For example, fetching VM entity over GWT RPC also returns related
> >>>>      data
> >>>>      such as Cluster name.]
> >>>>
> >>>> A: Based on discussion I've had with other colleagues after yesterday's
> >>>> review call, I don't think that separate support-like backend layer is a
> >>>> good idea. Instead, this is the kind of functionality that could be
> >>>> placed
> >>>> in oVirt.js library. Logical operations like "get VMs and related data"
> >>>> would be exposed through oVirt.js (callback-based) API and ultimately
> >>>> realized as multiple physical requests to REST API via JavaScript
> >>>> Binding.
> >>>>
> >>>> oVirt.js client would be completely oblivious to the fact that multiple
> >>>> physical requests are dispatched. In fact, since HTTP communication is
> >>>> asynchronous in nature, oVirt.js client wouldn't even notice any
> >>>> difference in terms of API consumption. This assumes JavaScript SDK
> >>>> would
> >>>> use callback-based (non-blocking) API instead of blocking one - after
> >>>> all,
> >>>> blocking API on top of non-blocking implementation sounds pretty much
> >>>> like
> >>>> leaky abstraction [1].
> >>>>
> >>>> For example:
> >>>>
> >>>>       sdk.getVmsWithExtraData(
> >>>>           callbackToGetExtraDataForGivenVm, // might cause extra
> >>>>           physical
> >>>>           requests to REST API
> >>>>           callbackFiredWhenAllDataIsReady   // update client only when
> >>>>           all
> >>>>           data is ready
> >>>>       )
> >>>
> >>> would this also resolve RunMultipleActions?
> >>
> >> Yes, I think so. There could be API to pass multiple "actions" and get
> >> notified when they complete.
> >>
> >>> sounds like no reason to have RunMultipleQueries, although i'm still
> >>> sure a single call to engine for multiple keys would be much more
> >>> efficient than multiple async calls?
> >>> (I understand we may not be able to model this).
> >>
> >> Efficiency-wise, yes, single call to get all data seems optimal. API-wise,
> >> I don't think it really matters from oVirt.js client perspective.
> >>
> >> We can proceed with simple (possibly inefficient) solution and improve as
> >> needed. We're making baby steps now..
> >>
> >>>
> >>>>
> >>>> [1] http://en.wikipedia.org/wiki/Leaky_abstraction
> >>>>
> >>>> --
> >>>>
> >>>> Last but not least, where to maintain JavaScript SDK projects: low-level
> >>>> JavaScript Binding + high-level oVirt.js library.
> >>>>
> >>>> I agree that conceptually both above mentioned projects should go into
> >>>> dedicated "ovirt-engine-sdk-js" git repository and have their own
> >>>> build/release process. However, for now, we're just making baby steps so
> >>>> let's keep things simple and prototype these projects as part of
> >>>> "ovirt-engine" git repository.
> >>>>
> >>>> ... we can complicate things anytime, but we should know that any
> >>>> complex
> >>>> system that works has inevitably evolved from simple system that works
> >>>> ...
> >>>> (quote from http://en.wikipedia.org/wiki/Gall%27s_law)
> >>>
> >>> I think the entities should just be generated from the xsd.
> >>
> >> +1
> >>
> >> The JavaScript Binding (aka low-level SDK) module should follow same
> >> concepts as existing Java SDK - generated entities decorated with
> >> operations to form fluent API.
> >>
> >> Everything Java SDK currently offers should be available in JavaScript
> >> Binding. oVirt.js is our opportunity to build on top of that.
> >>
> >>> for the rsdl, makes sense to start with clean code to see what works
> >>> best, then see about generating it (but you should adhere the rsdl as
> >>> guidlines i guess).
> >>
> >> +1
> >>
> >> The initial prototype should be written by hand, things will get generated
> >> as soon as we have better idea how the end result should look like.
> > 
> > i can understand that for the methods and maybe for populating the entities
> > for the first few.
> > the entities themselves, no point in hand coding - just generated them from
> > the api.xsd.
> > and once you decide how you want to fill them, not worth hand coding this -
> > either json gives this out of the box, or should be generated as well.
> > 
> >>
> >>>
> >>> lets try to plan for lightweight entities while at it - the API has a
> >>> mechanism for different level of details - maybe we need a custom level
> >>> where the client specifies which fields they want back or something like
> >>> that.
> >>
> >> Good idea! We should definitely think about the granularity of entities.
> >>
> >> I didn't know REST API supports different level of detail per entity, is
> >> there some documentation for this feature?
> 
> currently it's about adding adding extra information (by supplying header
> [1]),
> this is driven by the properties that require extra query against the engine
> to
> fill them,
> 
> obviously the default is [2]
> 
> [1] All-Content: True
> [2] All-Content: False

OK so if I understand correctly, "All-Content: False" (default) just returns data of given entity plus IDs of related entities, whereas "All-Content: True" returns data of given entity plus (embedded) data of related entities, is that correct?

> 
> >>
> >> Since JavaScript is dynamic, one possible solution would be to let client
> >> define the entity structure (i.e. what data client needs) on the fly,
> >> during runtime :)
> 
> we talked about this solution couple of years ago, but it was obvious
> that there is no real "demand" for it till UI moves to REST,

I assume you mean to let client selectively specify properties of given entity when making REST call, correct? If yes, are you planning to support such feature?

> 
> btw it won't solve the problem where to fill UI entity you need combination
> of N REST entities, you still will have to perform N + 1 queries to API

Of course. But we can make improvements with regard to:

- optimizing queries so that client receives/parses only data it needs (see my question above)
- if necessary, combine such optimized queries into bigger "logical" queries (utilizing callback mechanism)

> (maybe asking for entity with fewer details), - all it address is a capacity
> of the data being send.

Not only that, capacity of data sent goes proportionally with time needed to parse & process such data on client. (Imagine big XML marshalling on client. This is why everyone prefers JSON nowadays, JSON naturally maps to native JavaScript structures with minimal computational overhead.)

> 
> > 
> > michael?
> > 
> >>
> >>>
> >>> Good luck,
> >>>      Itamar
> >>>
> > 
> 
> 
> --
> 
> Michael Pasternak
> RedHat, ENG-Virtualization R&D
> 