[Engine-devel] Using REST API in web UI - review call summary
Hi guys, this is a summary of yesterday's review call, I'll try to highlight important Q/A and things we agreed on. Feel free to add anything in case I've missed something. -- Q: Why don't we simply try to use existing Java SDK and adapt it for GWT apps? (asked by Michael & Gilad) A: This might be a viable option to consider if we wanted to skip JavaScript-based SDK altogether and target Java/GWT code directly; we could simply take Java SDK and customize its abstractions where necessary, i.e. using HTTP transport layer implementation that works with GWT. In any case, this would mean coupling ourselves to Java SDK (which has its own release cycle) and I think this would complicate things for us. As proposed on the meeting, I think it's best to aim for JavaScript SDK as the lowest common denominator for *any* web application that wants to work with REST API. oVirt GWT-based UI can simply bind to JavaScript SDK, i.e. Java/GWT code that just overlays objects and functions provided by JavaScript SDK. Another reason is ease of maintenance - I'd rather see JavaScript SDK's code generation process to be independent of any other SDK (people responsible for maintaining JavaScript SDK should have full control over generated code). -- Q: What about functionality currently used by oVirt UI but not supported by REST API? (asked by Einav) [For example, fetching VM entity over GWT RPC also returns related data such as Cluster name.] A: Based on discussion I've had with other colleagues after yesterday's review call, I don't think that separate support-like backend layer is a good idea. Instead, this is the kind of functionality that could be placed in oVirt.js library. Logical operations like "get VMs and related data" would be exposed through oVirt.js (callback-based) API and ultimately realized as multiple physical requests to REST API via JavaScript Binding. oVirt.js client would be completely oblivious to the fact that multiple physical requests are dispatched. In fact, since HTTP communication is asynchronous in nature, oVirt.js client wouldn't even notice any difference in terms of API consumption. This assumes JavaScript SDK would use callback-based (non-blocking) API instead of blocking one - after all, blocking API on top of non-blocking implementation sounds pretty much like leaky abstraction [1]. For example: sdk.getVmsWithExtraData( callbackToGetExtraDataForGivenVm, // might cause extra physical requests to REST API callbackFiredWhenAllDataIsReady // update client only when all data is ready ) [1] http://en.wikipedia.org/wiki/Leaky_abstraction -- Last but not least, where to maintain JavaScript SDK projects: low-level JavaScript Binding + high-level oVirt.js library. I agree that conceptually both above mentioned projects should go into dedicated "ovirt-engine-sdk-js" git repository and have their own build/release process. However, for now, we're just making baby steps so let's keep things simple and prototype these projects as part of "ovirt-engine" git repository. ... we can complicate things anytime, but we should know that any complex system that works has inevitably evolved from simple system that works ... (quote from http://en.wikipedia.org/wiki/Gall%27s_law) Regards, Vojtech
On 11/21/2013 11:18 PM, Vojtech Szocs wrote:
Hi guys,
this is a summary of yesterday's review call, I'll try to highlight important Q/A and things we agreed on. Feel free to add anything in case I've missed something.
--
Q: Why don't we simply try to use existing Java SDK and adapt it for GWT apps? (asked by Michael & Gilad)
A: This might be a viable option to consider if we wanted to skip JavaScript-based SDK altogether and target Java/GWT code directly; we could simply take Java SDK and customize its abstractions where necessary, i.e. using HTTP transport layer implementation that works with GWT. In any case, this would mean coupling ourselves to Java SDK (which has its own release cycle) and I think this would complicate things for us.
As proposed on the meeting, I think it's best to aim for JavaScript SDK as the lowest common denominator for *any* web application that wants to work with REST API. oVirt GWT-based UI can simply bind to JavaScript SDK, i.e. Java/GWT code that just overlays objects and functions provided by JavaScript SDK. Another reason is ease of maintenance - I'd rather see JavaScript SDK's code generation process to be independent of any other SDK (people responsible for maintaining JavaScript SDK should have full control over generated code).
--
Q: What about functionality currently used by oVirt UI but not supported by REST API? (asked by Einav) [For example, fetching VM entity over GWT RPC also returns related data such as Cluster name.]
A: Based on discussion I've had with other colleagues after yesterday's review call, I don't think that separate support-like backend layer is a good idea. Instead, this is the kind of functionality that could be placed in oVirt.js library. Logical operations like "get VMs and related data" would be exposed through oVirt.js (callback-based) API and ultimately realized as multiple physical requests to REST API via JavaScript Binding.
oVirt.js client would be completely oblivious to the fact that multiple physical requests are dispatched. In fact, since HTTP communication is asynchronous in nature, oVirt.js client wouldn't even notice any difference in terms of API consumption. This assumes JavaScript SDK would use callback-based (non-blocking) API instead of blocking one - after all, blocking API on top of non-blocking implementation sounds pretty much like leaky abstraction [1].
For example:
sdk.getVmsWithExtraData( callbackToGetExtraDataForGivenVm, // might cause extra physical requests to REST API callbackFiredWhenAllDataIsReady // update client only when all data is ready )
would this also resolve RunMultipleActions? sounds like no reason to have RunMultipleQueries, although i'm still sure a single call to engine for multiple keys would be much more efficient than multiple async calls? (I understand we may not be able to model this).
[1] http://en.wikipedia.org/wiki/Leaky_abstraction
--
Last but not least, where to maintain JavaScript SDK projects: low-level JavaScript Binding + high-level oVirt.js library.
I agree that conceptually both above mentioned projects should go into dedicated "ovirt-engine-sdk-js" git repository and have their own build/release process. However, for now, we're just making baby steps so let's keep things simple and prototype these projects as part of "ovirt-engine" git repository.
... we can complicate things anytime, but we should know that any complex system that works has inevitably evolved from simple system that works ... (quote from http://en.wikipedia.org/wiki/Gall%27s_law)
I think the entities should just be generated from the xsd. for the rsdl, makes sense to start with clean code to see what works best, then see about generating it (but you should adhere the rsdl as guidlines i guess). lets try to plan for lightweight entities while at it - the API has a mechanism for different level of details - maybe we need a custom level where the client specifies which fields they want back or something like that. Good luck, Itamar
----- Original Message -----
From: "Itamar Heim" <iheim@redhat.com> To: "Vojtech Szocs" <vszocs@redhat.com>, "engine-devel" <engine-devel@ovirt.org> Cc: "Einav Cohen" <ecohen@redhat.com> Sent: Thursday, November 21, 2013 10:25:04 PM Subject: Re: Using REST API in web UI - review call summary
On 11/21/2013 11:18 PM, Vojtech Szocs wrote:
Hi guys,
this is a summary of yesterday's review call, I'll try to highlight important Q/A and things we agreed on. Feel free to add anything in case I've missed something.
--
Q: Why don't we simply try to use existing Java SDK and adapt it for GWT apps? (asked by Michael & Gilad)
A: This might be a viable option to consider if we wanted to skip JavaScript-based SDK altogether and target Java/GWT code directly; we could simply take Java SDK and customize its abstractions where necessary, i.e. using HTTP transport layer implementation that works with GWT. In any case, this would mean coupling ourselves to Java SDK (which has its own release cycle) and I think this would complicate things for us.
As proposed on the meeting, I think it's best to aim for JavaScript SDK as the lowest common denominator for *any* web application that wants to work with REST API. oVirt GWT-based UI can simply bind to JavaScript SDK, i.e. Java/GWT code that just overlays objects and functions provided by JavaScript SDK. Another reason is ease of maintenance - I'd rather see JavaScript SDK's code generation process to be independent of any other SDK (people responsible for maintaining JavaScript SDK should have full control over generated code).
--
Q: What about functionality currently used by oVirt UI but not supported by REST API? (asked by Einav) [For example, fetching VM entity over GWT RPC also returns related data such as Cluster name.]
A: Based on discussion I've had with other colleagues after yesterday's review call, I don't think that separate support-like backend layer is a good idea. Instead, this is the kind of functionality that could be placed in oVirt.js library. Logical operations like "get VMs and related data" would be exposed through oVirt.js (callback-based) API and ultimately realized as multiple physical requests to REST API via JavaScript Binding.
oVirt.js client would be completely oblivious to the fact that multiple physical requests are dispatched. In fact, since HTTP communication is asynchronous in nature, oVirt.js client wouldn't even notice any difference in terms of API consumption. This assumes JavaScript SDK would use callback-based (non-blocking) API instead of blocking one - after all, blocking API on top of non-blocking implementation sounds pretty much like leaky abstraction [1].
For example:
sdk.getVmsWithExtraData( callbackToGetExtraDataForGivenVm, // might cause extra physical requests to REST API callbackFiredWhenAllDataIsReady // update client only when all data is ready )
would this also resolve RunMultipleActions?
Yes, I think so. There could be API to pass multiple "actions" and get notified when they complete.
sounds like no reason to have RunMultipleQueries, although i'm still sure a single call to engine for multiple keys would be much more efficient than multiple async calls? (I understand we may not be able to model this).
Efficiency-wise, yes, single call to get all data seems optimal. API-wise, I don't think it really matters from oVirt.js client perspective. We can proceed with simple (possibly inefficient) solution and improve as needed. We're making baby steps now..
[1] http://en.wikipedia.org/wiki/Leaky_abstraction
--
Last but not least, where to maintain JavaScript SDK projects: low-level JavaScript Binding + high-level oVirt.js library.
I agree that conceptually both above mentioned projects should go into dedicated "ovirt-engine-sdk-js" git repository and have their own build/release process. However, for now, we're just making baby steps so let's keep things simple and prototype these projects as part of "ovirt-engine" git repository.
... we can complicate things anytime, but we should know that any complex system that works has inevitably evolved from simple system that works ... (quote from http://en.wikipedia.org/wiki/Gall%27s_law)
I think the entities should just be generated from the xsd.
+1 The JavaScript Binding (aka low-level SDK) module should follow same concepts as existing Java SDK - generated entities decorated with operations to form fluent API. Everything Java SDK currently offers should be available in JavaScript Binding. oVirt.js is our opportunity to build on top of that.
for the rsdl, makes sense to start with clean code to see what works best, then see about generating it (but you should adhere the rsdl as guidlines i guess).
+1 The initial prototype should be written by hand, things will get generated as soon as we have better idea how the end result should look like.
lets try to plan for lightweight entities while at it - the API has a mechanism for different level of details - maybe we need a custom level where the client specifies which fields they want back or something like that.
Good idea! We should definitely think about the granularity of entities. I didn't know REST API supports different level of detail per entity, is there some documentation for this feature? Since JavaScript is dynamic, one possible solution would be to let client define the entity structure (i.e. what data client needs) on the fly, during runtime :)
Good luck, Itamar
On 11/21/2013 11:56 PM, Vojtech Szocs wrote:
----- Original Message -----
From: "Itamar Heim" <iheim@redhat.com> To: "Vojtech Szocs" <vszocs@redhat.com>, "engine-devel" <engine-devel@ovirt.org> Cc: "Einav Cohen" <ecohen@redhat.com> Sent: Thursday, November 21, 2013 10:25:04 PM Subject: Re: Using REST API in web UI - review call summary
On 11/21/2013 11:18 PM, Vojtech Szocs wrote:
Hi guys,
this is a summary of yesterday's review call, I'll try to highlight important Q/A and things we agreed on. Feel free to add anything in case I've missed something.
--
Q: Why don't we simply try to use existing Java SDK and adapt it for GWT apps? (asked by Michael & Gilad)
A: This might be a viable option to consider if we wanted to skip JavaScript-based SDK altogether and target Java/GWT code directly; we could simply take Java SDK and customize its abstractions where necessary, i.e. using HTTP transport layer implementation that works with GWT. In any case, this would mean coupling ourselves to Java SDK (which has its own release cycle) and I think this would complicate things for us.
As proposed on the meeting, I think it's best to aim for JavaScript SDK as the lowest common denominator for *any* web application that wants to work with REST API. oVirt GWT-based UI can simply bind to JavaScript SDK, i.e. Java/GWT code that just overlays objects and functions provided by JavaScript SDK. Another reason is ease of maintenance - I'd rather see JavaScript SDK's code generation process to be independent of any other SDK (people responsible for maintaining JavaScript SDK should have full control over generated code).
--
Q: What about functionality currently used by oVirt UI but not supported by REST API? (asked by Einav) [For example, fetching VM entity over GWT RPC also returns related data such as Cluster name.]
A: Based on discussion I've had with other colleagues after yesterday's review call, I don't think that separate support-like backend layer is a good idea. Instead, this is the kind of functionality that could be placed in oVirt.js library. Logical operations like "get VMs and related data" would be exposed through oVirt.js (callback-based) API and ultimately realized as multiple physical requests to REST API via JavaScript Binding.
oVirt.js client would be completely oblivious to the fact that multiple physical requests are dispatched. In fact, since HTTP communication is asynchronous in nature, oVirt.js client wouldn't even notice any difference in terms of API consumption. This assumes JavaScript SDK would use callback-based (non-blocking) API instead of blocking one - after all, blocking API on top of non-blocking implementation sounds pretty much like leaky abstraction [1].
For example:
sdk.getVmsWithExtraData( callbackToGetExtraDataForGivenVm, // might cause extra physical requests to REST API callbackFiredWhenAllDataIsReady // update client only when all data is ready )
would this also resolve RunMultipleActions?
Yes, I think so. There could be API to pass multiple "actions" and get notified when they complete.
sounds like no reason to have RunMultipleQueries, although i'm still sure a single call to engine for multiple keys would be much more efficient than multiple async calls? (I understand we may not be able to model this).
Efficiency-wise, yes, single call to get all data seems optimal. API-wise, I don't think it really matters from oVirt.js client perspective.
We can proceed with simple (possibly inefficient) solution and improve as needed. We're making baby steps now..
[1] http://en.wikipedia.org/wiki/Leaky_abstraction
--
Last but not least, where to maintain JavaScript SDK projects: low-level JavaScript Binding + high-level oVirt.js library.
I agree that conceptually both above mentioned projects should go into dedicated "ovirt-engine-sdk-js" git repository and have their own build/release process. However, for now, we're just making baby steps so let's keep things simple and prototype these projects as part of "ovirt-engine" git repository.
... we can complicate things anytime, but we should know that any complex system that works has inevitably evolved from simple system that works ... (quote from http://en.wikipedia.org/wiki/Gall%27s_law)
I think the entities should just be generated from the xsd.
+1
The JavaScript Binding (aka low-level SDK) module should follow same concepts as existing Java SDK - generated entities decorated with operations to form fluent API.
Everything Java SDK currently offers should be available in JavaScript Binding. oVirt.js is our opportunity to build on top of that.
for the rsdl, makes sense to start with clean code to see what works best, then see about generating it (but you should adhere the rsdl as guidlines i guess).
+1
The initial prototype should be written by hand, things will get generated as soon as we have better idea how the end result should look like.
i can understand that for the methods and maybe for populating the entities for the first few. the entities themselves, no point in hand coding - just generated them from the api.xsd. and once you decide how you want to fill them, not worth hand coding this - either json gives this out of the box, or should be generated as well.
lets try to plan for lightweight entities while at it - the API has a mechanism for different level of details - maybe we need a custom level where the client specifies which fields they want back or something like that.
Good idea! We should definitely think about the granularity of entities.
I didn't know REST API supports different level of detail per entity, is there some documentation for this feature?
Since JavaScript is dynamic, one possible solution would be to let client define the entity structure (i.e. what data client needs) on the fly, during runtime :)
michael?
Good luck, Itamar
----- Original Message -----
From: "Itamar Heim" <iheim@redhat.com> To: "Vojtech Szocs" <vszocs@redhat.com> Cc: "engine-devel" <engine-devel@ovirt.org>, "Einav Cohen" <ecohen@redhat.com>, "Michael Pasternak" <mpastern@redhat.com> Sent: Thursday, November 21, 2013 11:00:25 PM Subject: Re: Using REST API in web UI - review call summary
On 11/21/2013 11:56 PM, Vojtech Szocs wrote:
----- Original Message -----
From: "Itamar Heim" <iheim@redhat.com> To: "Vojtech Szocs" <vszocs@redhat.com>, "engine-devel" <engine-devel@ovirt.org> Cc: "Einav Cohen" <ecohen@redhat.com> Sent: Thursday, November 21, 2013 10:25:04 PM Subject: Re: Using REST API in web UI - review call summary
On 11/21/2013 11:18 PM, Vojtech Szocs wrote:
Hi guys,
this is a summary of yesterday's review call, I'll try to highlight important Q/A and things we agreed on. Feel free to add anything in case I've missed something.
--
Q: Why don't we simply try to use existing Java SDK and adapt it for GWT apps? (asked by Michael & Gilad)
A: This might be a viable option to consider if we wanted to skip JavaScript-based SDK altogether and target Java/GWT code directly; we could simply take Java SDK and customize its abstractions where necessary, i.e. using HTTP transport layer implementation that works with GWT. In any case, this would mean coupling ourselves to Java SDK (which has its own release cycle) and I think this would complicate things for us.
As proposed on the meeting, I think it's best to aim for JavaScript SDK as the lowest common denominator for *any* web application that wants to work with REST API. oVirt GWT-based UI can simply bind to JavaScript SDK, i.e. Java/GWT code that just overlays objects and functions provided by JavaScript SDK. Another reason is ease of maintenance - I'd rather see JavaScript SDK's code generation process to be independent of any other SDK (people responsible for maintaining JavaScript SDK should have full control over generated code).
--
Q: What about functionality currently used by oVirt UI but not supported by REST API? (asked by Einav) [For example, fetching VM entity over GWT RPC also returns related data such as Cluster name.]
A: Based on discussion I've had with other colleagues after yesterday's review call, I don't think that separate support-like backend layer is a good idea. Instead, this is the kind of functionality that could be placed in oVirt.js library. Logical operations like "get VMs and related data" would be exposed through oVirt.js (callback-based) API and ultimately realized as multiple physical requests to REST API via JavaScript Binding.
oVirt.js client would be completely oblivious to the fact that multiple physical requests are dispatched. In fact, since HTTP communication is asynchronous in nature, oVirt.js client wouldn't even notice any difference in terms of API consumption. This assumes JavaScript SDK would use callback-based (non-blocking) API instead of blocking one - after all, blocking API on top of non-blocking implementation sounds pretty much like leaky abstraction [1].
For example:
sdk.getVmsWithExtraData( callbackToGetExtraDataForGivenVm, // might cause extra physical requests to REST API callbackFiredWhenAllDataIsReady // update client only when all data is ready )
would this also resolve RunMultipleActions?
Yes, I think so. There could be API to pass multiple "actions" and get notified when they complete.
sounds like no reason to have RunMultipleQueries, although i'm still sure a single call to engine for multiple keys would be much more efficient than multiple async calls? (I understand we may not be able to model this).
Efficiency-wise, yes, single call to get all data seems optimal. API-wise, I don't think it really matters from oVirt.js client perspective.
We can proceed with simple (possibly inefficient) solution and improve as needed. We're making baby steps now..
[1] http://en.wikipedia.org/wiki/Leaky_abstraction
--
Last but not least, where to maintain JavaScript SDK projects: low-level JavaScript Binding + high-level oVirt.js library.
I agree that conceptually both above mentioned projects should go into dedicated "ovirt-engine-sdk-js" git repository and have their own build/release process. However, for now, we're just making baby steps so let's keep things simple and prototype these projects as part of "ovirt-engine" git repository.
... we can complicate things anytime, but we should know that any complex system that works has inevitably evolved from simple system that works ... (quote from http://en.wikipedia.org/wiki/Gall%27s_law)
I think the entities should just be generated from the xsd.
+1
The JavaScript Binding (aka low-level SDK) module should follow same concepts as existing Java SDK - generated entities decorated with operations to form fluent API.
Everything Java SDK currently offers should be available in JavaScript Binding. oVirt.js is our opportunity to build on top of that.
for the rsdl, makes sense to start with clean code to see what works best, then see about generating it (but you should adhere the rsdl as guidlines i guess).
+1
The initial prototype should be written by hand, things will get generated as soon as we have better idea how the end result should look like.
i can understand that for the methods and maybe for populating the entities for the first few. the entities themselves, no point in hand coding - just generated them from the api.xsd. and once you decide how you want to fill them, not worth hand coding this - either json gives this out of the box, or should be generated as well.
OK, now I get you :) sure, entities aren't too interesting by themselves, but populating (decorating) them is something that requires more thought.
lets try to plan for lightweight entities while at it - the API has a mechanism for different level of details - maybe we need a custom level where the client specifies which fields they want back or something like that.
Good idea! We should definitely think about the granularity of entities.
I didn't know REST API supports different level of detail per entity, is there some documentation for this feature?
Since JavaScript is dynamic, one possible solution would be to let client define the entity structure (i.e. what data client needs) on the fly, during runtime :)
michael?
Good luck, Itamar
On 11/22/2013 12:08 AM, Vojtech Szocs wrote:
----- Original Message -----
From: "Itamar Heim" <iheim@redhat.com> To: "Vojtech Szocs" <vszocs@redhat.com> Cc: "engine-devel" <engine-devel@ovirt.org>, "Einav Cohen" <ecohen@redhat.com>, "Michael Pasternak" <mpastern@redhat.com> Sent: Thursday, November 21, 2013 11:00:25 PM Subject: Re: Using REST API in web UI - review call summary
On 11/21/2013 11:56 PM, Vojtech Szocs wrote:
----- Original Message -----
From: "Itamar Heim" <iheim@redhat.com> To: "Vojtech Szocs" <vszocs@redhat.com>, "engine-devel" <engine-devel@ovirt.org> Cc: "Einav Cohen" <ecohen@redhat.com> Sent: Thursday, November 21, 2013 10:25:04 PM Subject: Re: Using REST API in web UI - review call summary
On 11/21/2013 11:18 PM, Vojtech Szocs wrote:
Hi guys,
this is a summary of yesterday's review call, I'll try to highlight important Q/A and things we agreed on. Feel free to add anything in case I've missed something.
--
Q: Why don't we simply try to use existing Java SDK and adapt it for GWT apps? (asked by Michael & Gilad)
A: This might be a viable option to consider if we wanted to skip JavaScript-based SDK altogether and target Java/GWT code directly; we could simply take Java SDK and customize its abstractions where necessary, i.e. using HTTP transport layer implementation that works with GWT. In any case, this would mean coupling ourselves to Java SDK (which has its own release cycle) and I think this would complicate things for us.
As proposed on the meeting, I think it's best to aim for JavaScript SDK as the lowest common denominator for *any* web application that wants to work with REST API. oVirt GWT-based UI can simply bind to JavaScript SDK, i.e. Java/GWT code that just overlays objects and functions provided by JavaScript SDK. Another reason is ease of maintenance - I'd rather see JavaScript SDK's code generation process to be independent of any other SDK (people responsible for maintaining JavaScript SDK should have full control over generated code).
--
Q: What about functionality currently used by oVirt UI but not supported by REST API? (asked by Einav) [For example, fetching VM entity over GWT RPC also returns related data such as Cluster name.]
A: Based on discussion I've had with other colleagues after yesterday's review call, I don't think that separate support-like backend layer is a good idea. Instead, this is the kind of functionality that could be placed in oVirt.js library. Logical operations like "get VMs and related data" would be exposed through oVirt.js (callback-based) API and ultimately realized as multiple physical requests to REST API via JavaScript Binding.
oVirt.js client would be completely oblivious to the fact that multiple physical requests are dispatched. In fact, since HTTP communication is asynchronous in nature, oVirt.js client wouldn't even notice any difference in terms of API consumption. This assumes JavaScript SDK would use callback-based (non-blocking) API instead of blocking one - after all, blocking API on top of non-blocking implementation sounds pretty much like leaky abstraction [1].
For example:
sdk.getVmsWithExtraData( callbackToGetExtraDataForGivenVm, // might cause extra physical requests to REST API callbackFiredWhenAllDataIsReady // update client only when all data is ready )
would this also resolve RunMultipleActions?
Yes, I think so. There could be API to pass multiple "actions" and get notified when they complete.
sounds like no reason to have RunMultipleQueries, although i'm still sure a single call to engine for multiple keys would be much more efficient than multiple async calls? (I understand we may not be able to model this).
Efficiency-wise, yes, single call to get all data seems optimal. API-wise, I don't think it really matters from oVirt.js client perspective.
We can proceed with simple (possibly inefficient) solution and improve as needed. We're making baby steps now..
[1] http://en.wikipedia.org/wiki/Leaky_abstraction
--
Last but not least, where to maintain JavaScript SDK projects: low-level JavaScript Binding + high-level oVirt.js library.
I agree that conceptually both above mentioned projects should go into dedicated "ovirt-engine-sdk-js" git repository and have their own build/release process. However, for now, we're just making baby steps so let's keep things simple and prototype these projects as part of "ovirt-engine" git repository.
... we can complicate things anytime, but we should know that any complex system that works has inevitably evolved from simple system that works ... (quote from http://en.wikipedia.org/wiki/Gall%27s_law)
I think the entities should just be generated from the xsd.
+1
The JavaScript Binding (aka low-level SDK) module should follow same concepts as existing Java SDK - generated entities decorated with operations to form fluent API.
Everything Java SDK currently offers should be available in JavaScript Binding. oVirt.js is our opportunity to build on top of that.
for the rsdl, makes sense to start with clean code to see what works best, then see about generating it (but you should adhere the rsdl as guidlines i guess).
+1
The initial prototype should be written by hand, things will get generated as soon as we have better idea how the end result should look like.
i can understand that for the methods and maybe for populating the entities for the first few. the entities themselves, no point in hand coding - just generated them from the api.xsd. and once you decide how you want to fill them, not worth hand coding this - either json gives this out of the box, or should be generated as well.
OK, now I get you :) sure, entities aren't too interesting by themselves, but populating (decorating) them is something that requires more thought.
you have Java-SDK that does this already, so it should be relativity easy to take it from there.
lets try to plan for lightweight entities while at it - the API has a mechanism for different level of details - maybe we need a custom level where the client specifies which fields they want back or something like that.
Good idea! We should definitely think about the granularity of entities.
I didn't know REST API supports different level of detail per entity, is there some documentation for this feature?
Since JavaScript is dynamic, one possible solution would be to let client define the entity structure (i.e. what data client needs) on the fly, during runtime :)
michael?
Good luck, Itamar
-- Michael Pasternak RedHat, ENG-Virtualization R&D
----- Original Message -----
From: "Michael Pasternak" <mpastern@redhat.com> To: "Vojtech Szocs" <vszocs@redhat.com> Cc: "Itamar Heim" <iheim@redhat.com>, "engine-devel" <engine-devel@ovirt.org>, "Einav Cohen" <ecohen@redhat.com> Sent: Sunday, November 24, 2013 9:38:45 AM Subject: Re: Using REST API in web UI - review call summary
On 11/22/2013 12:08 AM, Vojtech Szocs wrote:
----- Original Message -----
From: "Itamar Heim" <iheim@redhat.com> To: "Vojtech Szocs" <vszocs@redhat.com> Cc: "engine-devel" <engine-devel@ovirt.org>, "Einav Cohen" <ecohen@redhat.com>, "Michael Pasternak" <mpastern@redhat.com> Sent: Thursday, November 21, 2013 11:00:25 PM Subject: Re: Using REST API in web UI - review call summary
On 11/21/2013 11:56 PM, Vojtech Szocs wrote:
----- Original Message -----
From: "Itamar Heim" <iheim@redhat.com> To: "Vojtech Szocs" <vszocs@redhat.com>, "engine-devel" <engine-devel@ovirt.org> Cc: "Einav Cohen" <ecohen@redhat.com> Sent: Thursday, November 21, 2013 10:25:04 PM Subject: Re: Using REST API in web UI - review call summary
On 11/21/2013 11:18 PM, Vojtech Szocs wrote:
Hi guys,
this is a summary of yesterday's review call, I'll try to highlight important Q/A and things we agreed on. Feel free to add anything in case I've missed something.
--
Q: Why don't we simply try to use existing Java SDK and adapt it for GWT apps? (asked by Michael & Gilad)
A: This might be a viable option to consider if we wanted to skip JavaScript-based SDK altogether and target Java/GWT code directly; we could simply take Java SDK and customize its abstractions where necessary, i.e. using HTTP transport layer implementation that works with GWT. In any case, this would mean coupling ourselves to Java SDK (which has its own release cycle) and I think this would complicate things for us.
As proposed on the meeting, I think it's best to aim for JavaScript SDK as the lowest common denominator for *any* web application that wants to work with REST API. oVirt GWT-based UI can simply bind to JavaScript SDK, i.e. Java/GWT code that just overlays objects and functions provided by JavaScript SDK. Another reason is ease of maintenance - I'd rather see JavaScript SDK's code generation process to be independent of any other SDK (people responsible for maintaining JavaScript SDK should have full control over generated code).
--
Q: What about functionality currently used by oVirt UI but not supported by REST API? (asked by Einav) [For example, fetching VM entity over GWT RPC also returns related data such as Cluster name.]
A: Based on discussion I've had with other colleagues after yesterday's review call, I don't think that separate support-like backend layer is a good idea. Instead, this is the kind of functionality that could be placed in oVirt.js library. Logical operations like "get VMs and related data" would be exposed through oVirt.js (callback-based) API and ultimately realized as multiple physical requests to REST API via JavaScript Binding.
oVirt.js client would be completely oblivious to the fact that multiple physical requests are dispatched. In fact, since HTTP communication is asynchronous in nature, oVirt.js client wouldn't even notice any difference in terms of API consumption. This assumes JavaScript SDK would use callback-based (non-blocking) API instead of blocking one - after all, blocking API on top of non-blocking implementation sounds pretty much like leaky abstraction [1].
For example:
sdk.getVmsWithExtraData( callbackToGetExtraDataForGivenVm, // might cause extra physical requests to REST API callbackFiredWhenAllDataIsReady // update client only when all data is ready )
would this also resolve RunMultipleActions?
Yes, I think so. There could be API to pass multiple "actions" and get notified when they complete.
sounds like no reason to have RunMultipleQueries, although i'm still sure a single call to engine for multiple keys would be much more efficient than multiple async calls? (I understand we may not be able to model this).
Efficiency-wise, yes, single call to get all data seems optimal. API-wise, I don't think it really matters from oVirt.js client perspective.
We can proceed with simple (possibly inefficient) solution and improve as needed. We're making baby steps now..
[1] http://en.wikipedia.org/wiki/Leaky_abstraction
--
Last but not least, where to maintain JavaScript SDK projects: low-level JavaScript Binding + high-level oVirt.js library.
I agree that conceptually both above mentioned projects should go into dedicated "ovirt-engine-sdk-js" git repository and have their own build/release process. However, for now, we're just making baby steps so let's keep things simple and prototype these projects as part of "ovirt-engine" git repository.
... we can complicate things anytime, but we should know that any complex system that works has inevitably evolved from simple system that works ... (quote from http://en.wikipedia.org/wiki/Gall%27s_law)
I think the entities should just be generated from the xsd.
+1
The JavaScript Binding (aka low-level SDK) module should follow same concepts as existing Java SDK - generated entities decorated with operations to form fluent API.
Everything Java SDK currently offers should be available in JavaScript Binding. oVirt.js is our opportunity to build on top of that.
for the rsdl, makes sense to start with clean code to see what works best, then see about generating it (but you should adhere the rsdl as guidlines i guess).
+1
The initial prototype should be written by hand, things will get generated as soon as we have better idea how the end result should look like.
i can understand that for the methods and maybe for populating the entities for the first few. the entities themselves, no point in hand coding - just generated them from the api.xsd. and once you decide how you want to fill them, not worth hand coding this - either json gives this out of the box, or should be generated as well.
OK, now I get you :) sure, entities aren't too interesting by themselves, but populating (decorating) them is something that requires more thought.
you have Java-SDK that does this already, so it should be relativity easy to take it from there.
Yes, Java SDK will be our reference when developing JavaScript binding for REST API [1]: "It would contain everything currently offered by existing Java SDK". [1] http://www.ovirt.org/Features/Design/Using_REST_API_In_Web_UI#REST_API_JavaS...
lets try to plan for lightweight entities while at it - the API has a mechanism for different level of details - maybe we need a custom level where the client specifies which fields they want back or something like that.
Good idea! We should definitely think about the granularity of entities.
I didn't know REST API supports different level of detail per entity, is there some documentation for this feature?
Since JavaScript is dynamic, one possible solution would be to let client define the entity structure (i.e. what data client needs) on the fly, during runtime :)
michael?
Good luck, Itamar
--
Michael Pasternak RedHat, ENG-Virtualization R&D
On 11/22/2013 12:00 AM, Itamar Heim wrote:
On 11/21/2013 11:56 PM, Vojtech Szocs wrote:
----- Original Message -----
From: "Itamar Heim" <iheim@redhat.com> To: "Vojtech Szocs" <vszocs@redhat.com>, "engine-devel" <engine-devel@ovirt.org> Cc: "Einav Cohen" <ecohen@redhat.com> Sent: Thursday, November 21, 2013 10:25:04 PM Subject: Re: Using REST API in web UI - review call summary
On 11/21/2013 11:18 PM, Vojtech Szocs wrote:
Hi guys,
this is a summary of yesterday's review call, I'll try to highlight important Q/A and things we agreed on. Feel free to add anything in case I've missed something.
--
Q: Why don't we simply try to use existing Java SDK and adapt it for GWT apps? (asked by Michael & Gilad)
A: This might be a viable option to consider if we wanted to skip JavaScript-based SDK altogether and target Java/GWT code directly; we could simply take Java SDK and customize its abstractions where necessary, i.e. using HTTP transport layer implementation that works with GWT. In any case, this would mean coupling ourselves to Java SDK (which has its own release cycle) and I think this would complicate things for us.
As proposed on the meeting, I think it's best to aim for JavaScript SDK as the lowest common denominator for *any* web application that wants to work with REST API. oVirt GWT-based UI can simply bind to JavaScript SDK, i.e. Java/GWT code that just overlays objects and functions provided by JavaScript SDK. Another reason is ease of maintenance - I'd rather see JavaScript SDK's code generation process to be independent of any other SDK (people responsible for maintaining JavaScript SDK should have full control over generated code).
--
Q: What about functionality currently used by oVirt UI but not supported by REST API? (asked by Einav) [For example, fetching VM entity over GWT RPC also returns related data such as Cluster name.]
A: Based on discussion I've had with other colleagues after yesterday's review call, I don't think that separate support-like backend layer is a good idea. Instead, this is the kind of functionality that could be placed in oVirt.js library. Logical operations like "get VMs and related data" would be exposed through oVirt.js (callback-based) API and ultimately realized as multiple physical requests to REST API via JavaScript Binding.
oVirt.js client would be completely oblivious to the fact that multiple physical requests are dispatched. In fact, since HTTP communication is asynchronous in nature, oVirt.js client wouldn't even notice any difference in terms of API consumption. This assumes JavaScript SDK would use callback-based (non-blocking) API instead of blocking one - after all, blocking API on top of non-blocking implementation sounds pretty much like leaky abstraction [1].
For example:
sdk.getVmsWithExtraData( callbackToGetExtraDataForGivenVm, // might cause extra physical requests to REST API callbackFiredWhenAllDataIsReady // update client only when all data is ready )
would this also resolve RunMultipleActions?
Yes, I think so. There could be API to pass multiple "actions" and get notified when they complete.
sounds like no reason to have RunMultipleQueries, although i'm still sure a single call to engine for multiple keys would be much more efficient than multiple async calls? (I understand we may not be able to model this).
Efficiency-wise, yes, single call to get all data seems optimal. API-wise, I don't think it really matters from oVirt.js client perspective.
We can proceed with simple (possibly inefficient) solution and improve as needed. We're making baby steps now..
[1] http://en.wikipedia.org/wiki/Leaky_abstraction
--
Last but not least, where to maintain JavaScript SDK projects: low-level JavaScript Binding + high-level oVirt.js library.
I agree that conceptually both above mentioned projects should go into dedicated "ovirt-engine-sdk-js" git repository and have their own build/release process. However, for now, we're just making baby steps so let's keep things simple and prototype these projects as part of "ovirt-engine" git repository.
... we can complicate things anytime, but we should know that any complex system that works has inevitably evolved from simple system that works ... (quote from http://en.wikipedia.org/wiki/Gall%27s_law)
I think the entities should just be generated from the xsd.
+1
The JavaScript Binding (aka low-level SDK) module should follow same concepts as existing Java SDK - generated entities decorated with operations to form fluent API.
Everything Java SDK currently offers should be available in JavaScript Binding. oVirt.js is our opportunity to build on top of that.
for the rsdl, makes sense to start with clean code to see what works best, then see about generating it (but you should adhere the rsdl as guidlines i guess).
+1
The initial prototype should be written by hand, things will get generated as soon as we have better idea how the end result should look like.
i can understand that for the methods and maybe for populating the entities for the first few. the entities themselves, no point in hand coding - just generated them from the api.xsd. and once you decide how you want to fill them, not worth hand coding this - either json gives this out of the box, or should be generated as well.
lets try to plan for lightweight entities while at it - the API has a mechanism for different level of details - maybe we need a custom level where the client specifies which fields they want back or something like that.
Good idea! We should definitely think about the granularity of entities.
I didn't know REST API supports different level of detail per entity, is there some documentation for this feature?
currently it's about adding adding extra information (by supplying header [1]), this is driven by the properties that require extra query against the engine to fill them, obviously the default is [2] [1] All-Content: True [2] All-Content: False
Since JavaScript is dynamic, one possible solution would be to let client define the entity structure (i.e. what data client needs) on the fly, during runtime :)
we talked about this solution couple of years ago, but it was obvious that there is no real "demand" for it till UI moves to REST, btw it won't solve the problem where to fill UI entity you need combination of N REST entities, you still will have to perform N + 1 queries to API (maybe asking for entity with fewer details), - all it address is a capacity of the data being send.
michael?
Good luck, Itamar
-- Michael Pasternak RedHat, ENG-Virtualization R&D
----- Original Message -----
From: "Michael Pasternak" <mpastern@redhat.com> To: "Itamar Heim" <iheim@redhat.com> Cc: "engine-devel" <engine-devel@ovirt.org> Sent: Sunday, November 24, 2013 9:37:22 AM Subject: Re: [Engine-devel] Using REST API in web UI - review call summary
On 11/22/2013 12:00 AM, Itamar Heim wrote:
On 11/21/2013 11:56 PM, Vojtech Szocs wrote:
----- Original Message -----
From: "Itamar Heim" <iheim@redhat.com> To: "Vojtech Szocs" <vszocs@redhat.com>, "engine-devel" <engine-devel@ovirt.org> Cc: "Einav Cohen" <ecohen@redhat.com> Sent: Thursday, November 21, 2013 10:25:04 PM Subject: Re: Using REST API in web UI - review call summary
On 11/21/2013 11:18 PM, Vojtech Szocs wrote:
Hi guys,
this is a summary of yesterday's review call, I'll try to highlight important Q/A and things we agreed on. Feel free to add anything in case I've missed something.
--
Q: Why don't we simply try to use existing Java SDK and adapt it for GWT apps? (asked by Michael & Gilad)
A: This might be a viable option to consider if we wanted to skip JavaScript-based SDK altogether and target Java/GWT code directly; we could simply take Java SDK and customize its abstractions where necessary, i.e. using HTTP transport layer implementation that works with GWT. In any case, this would mean coupling ourselves to Java SDK (which has its own release cycle) and I think this would complicate things for us.
As proposed on the meeting, I think it's best to aim for JavaScript SDK as the lowest common denominator for *any* web application that wants to work with REST API. oVirt GWT-based UI can simply bind to JavaScript SDK, i.e. Java/GWT code that just overlays objects and functions provided by JavaScript SDK. Another reason is ease of maintenance - I'd rather see JavaScript SDK's code generation process to be independent of any other SDK (people responsible for maintaining JavaScript SDK should have full control over generated code).
--
Q: What about functionality currently used by oVirt UI but not supported by REST API? (asked by Einav) [For example, fetching VM entity over GWT RPC also returns related data such as Cluster name.]
A: Based on discussion I've had with other colleagues after yesterday's review call, I don't think that separate support-like backend layer is a good idea. Instead, this is the kind of functionality that could be placed in oVirt.js library. Logical operations like "get VMs and related data" would be exposed through oVirt.js (callback-based) API and ultimately realized as multiple physical requests to REST API via JavaScript Binding.
oVirt.js client would be completely oblivious to the fact that multiple physical requests are dispatched. In fact, since HTTP communication is asynchronous in nature, oVirt.js client wouldn't even notice any difference in terms of API consumption. This assumes JavaScript SDK would use callback-based (non-blocking) API instead of blocking one - after all, blocking API on top of non-blocking implementation sounds pretty much like leaky abstraction [1].
For example:
sdk.getVmsWithExtraData( callbackToGetExtraDataForGivenVm, // might cause extra physical requests to REST API callbackFiredWhenAllDataIsReady // update client only when all data is ready )
would this also resolve RunMultipleActions?
Yes, I think so. There could be API to pass multiple "actions" and get notified when they complete.
sounds like no reason to have RunMultipleQueries, although i'm still sure a single call to engine for multiple keys would be much more efficient than multiple async calls? (I understand we may not be able to model this).
Efficiency-wise, yes, single call to get all data seems optimal. API-wise, I don't think it really matters from oVirt.js client perspective.
We can proceed with simple (possibly inefficient) solution and improve as needed. We're making baby steps now..
[1] http://en.wikipedia.org/wiki/Leaky_abstraction
--
Last but not least, where to maintain JavaScript SDK projects: low-level JavaScript Binding + high-level oVirt.js library.
I agree that conceptually both above mentioned projects should go into dedicated "ovirt-engine-sdk-js" git repository and have their own build/release process. However, for now, we're just making baby steps so let's keep things simple and prototype these projects as part of "ovirt-engine" git repository.
... we can complicate things anytime, but we should know that any complex system that works has inevitably evolved from simple system that works ... (quote from http://en.wikipedia.org/wiki/Gall%27s_law)
I think the entities should just be generated from the xsd.
+1
The JavaScript Binding (aka low-level SDK) module should follow same concepts as existing Java SDK - generated entities decorated with operations to form fluent API.
Everything Java SDK currently offers should be available in JavaScript Binding. oVirt.js is our opportunity to build on top of that.
for the rsdl, makes sense to start with clean code to see what works best, then see about generating it (but you should adhere the rsdl as guidlines i guess).
+1
The initial prototype should be written by hand, things will get generated as soon as we have better idea how the end result should look like.
i can understand that for the methods and maybe for populating the entities for the first few. the entities themselves, no point in hand coding - just generated them from the api.xsd. and once you decide how you want to fill them, not worth hand coding this - either json gives this out of the box, or should be generated as well.
lets try to plan for lightweight entities while at it - the API has a mechanism for different level of details - maybe we need a custom level where the client specifies which fields they want back or something like that.
Good idea! We should definitely think about the granularity of entities.
I didn't know REST API supports different level of detail per entity, is there some documentation for this feature?
currently it's about adding adding extra information (by supplying header [1]), this is driven by the properties that require extra query against the engine to fill them,
obviously the default is [2]
[1] All-Content: True [2] All-Content: False
Since JavaScript is dynamic, one possible solution would be to let client define the entity structure (i.e. what data client needs) on the fly, during runtime :)
we talked about this solution couple of years ago, but it was obvious that there is no real "demand" for it till UI moves to REST,
btw it won't solve the problem where to fill UI entity you need combination of N REST entities, you still will have to perform N + 1 queries to API (maybe asking for entity with fewer details), - all it address is a capacity of the data being send.
For this problem of content filtering and retrieve associated data in a "restful" way I saw Yoga http://yoga.skyscreamer.org/ some example usage: http://www.infoq.com/articles/json-yoga
michael?
Good luck, Itamar
--
Michael Pasternak RedHat, ENG-Virtualization R&D _______________________________________________ Engine-devel mailing list Engine-devel@ovirt.org http://lists.ovirt.org/mailman/listinfo/engine-devel
----- Original Message -----
From: "Antoni Segura Puimedon" <asegurap@redhat.com> To: "Michael Pasternak" <mpastern@redhat.com> Cc: "engine-devel" <engine-devel@ovirt.org> Sent: Sunday, November 24, 2013 10:45:15 AM Subject: Re: [Engine-devel] Using REST API in web UI - review call summary
----- Original Message -----
From: "Michael Pasternak" <mpastern@redhat.com> To: "Itamar Heim" <iheim@redhat.com> Cc: "engine-devel" <engine-devel@ovirt.org> Sent: Sunday, November 24, 2013 9:37:22 AM Subject: Re: [Engine-devel] Using REST API in web UI - review call summary
On 11/22/2013 12:00 AM, Itamar Heim wrote:
On 11/21/2013 11:56 PM, Vojtech Szocs wrote:
----- Original Message -----
From: "Itamar Heim" <iheim@redhat.com> To: "Vojtech Szocs" <vszocs@redhat.com>, "engine-devel" <engine-devel@ovirt.org> Cc: "Einav Cohen" <ecohen@redhat.com> Sent: Thursday, November 21, 2013 10:25:04 PM Subject: Re: Using REST API in web UI - review call summary
On 11/21/2013 11:18 PM, Vojtech Szocs wrote:
Hi guys,
this is a summary of yesterday's review call, I'll try to highlight important Q/A and things we agreed on. Feel free to add anything in case I've missed something.
--
Q: Why don't we simply try to use existing Java SDK and adapt it for GWT apps? (asked by Michael & Gilad)
A: This might be a viable option to consider if we wanted to skip JavaScript-based SDK altogether and target Java/GWT code directly; we could simply take Java SDK and customize its abstractions where necessary, i.e. using HTTP transport layer implementation that works with GWT. In any case, this would mean coupling ourselves to Java SDK (which has its own release cycle) and I think this would complicate things for us.
As proposed on the meeting, I think it's best to aim for JavaScript SDK as the lowest common denominator for *any* web application that wants to work with REST API. oVirt GWT-based UI can simply bind to JavaScript SDK, i.e. Java/GWT code that just overlays objects and functions provided by JavaScript SDK. Another reason is ease of maintenance - I'd rather see JavaScript SDK's code generation process to be independent of any other SDK (people responsible for maintaining JavaScript SDK should have full control over generated code).
--
Q: What about functionality currently used by oVirt UI but not supported by REST API? (asked by Einav) [For example, fetching VM entity over GWT RPC also returns related data such as Cluster name.]
A: Based on discussion I've had with other colleagues after yesterday's review call, I don't think that separate support-like backend layer is a good idea. Instead, this is the kind of functionality that could be placed in oVirt.js library. Logical operations like "get VMs and related data" would be exposed through oVirt.js (callback-based) API and ultimately realized as multiple physical requests to REST API via JavaScript Binding.
oVirt.js client would be completely oblivious to the fact that multiple physical requests are dispatched. In fact, since HTTP communication is asynchronous in nature, oVirt.js client wouldn't even notice any difference in terms of API consumption. This assumes JavaScript SDK would use callback-based (non-blocking) API instead of blocking one - after all, blocking API on top of non-blocking implementation sounds pretty much like leaky abstraction [1].
For example:
sdk.getVmsWithExtraData( callbackToGetExtraDataForGivenVm, // might cause extra physical requests to REST API callbackFiredWhenAllDataIsReady // update client only when all data is ready )
would this also resolve RunMultipleActions?
Yes, I think so. There could be API to pass multiple "actions" and get notified when they complete.
sounds like no reason to have RunMultipleQueries, although i'm still sure a single call to engine for multiple keys would be much more efficient than multiple async calls? (I understand we may not be able to model this).
Efficiency-wise, yes, single call to get all data seems optimal. API-wise, I don't think it really matters from oVirt.js client perspective.
We can proceed with simple (possibly inefficient) solution and improve as needed. We're making baby steps now..
[1] http://en.wikipedia.org/wiki/Leaky_abstraction
--
Last but not least, where to maintain JavaScript SDK projects: low-level JavaScript Binding + high-level oVirt.js library.
I agree that conceptually both above mentioned projects should go into dedicated "ovirt-engine-sdk-js" git repository and have their own build/release process. However, for now, we're just making baby steps so let's keep things simple and prototype these projects as part of "ovirt-engine" git repository.
... we can complicate things anytime, but we should know that any complex system that works has inevitably evolved from simple system that works ... (quote from http://en.wikipedia.org/wiki/Gall%27s_law)
I think the entities should just be generated from the xsd.
+1
The JavaScript Binding (aka low-level SDK) module should follow same concepts as existing Java SDK - generated entities decorated with operations to form fluent API.
Everything Java SDK currently offers should be available in JavaScript Binding. oVirt.js is our opportunity to build on top of that.
for the rsdl, makes sense to start with clean code to see what works best, then see about generating it (but you should adhere the rsdl as guidlines i guess).
+1
The initial prototype should be written by hand, things will get generated as soon as we have better idea how the end result should look like.
i can understand that for the methods and maybe for populating the entities for the first few. the entities themselves, no point in hand coding - just generated them from the api.xsd. and once you decide how you want to fill them, not worth hand coding this - either json gives this out of the box, or should be generated as well.
lets try to plan for lightweight entities while at it - the API has a mechanism for different level of details - maybe we need a custom level where the client specifies which fields they want back or something like that.
Good idea! We should definitely think about the granularity of entities.
I didn't know REST API supports different level of detail per entity, is there some documentation for this feature?
currently it's about adding adding extra information (by supplying header [1]), this is driven by the properties that require extra query against the engine to fill them,
obviously the default is [2]
[1] All-Content: True [2] All-Content: False
Since JavaScript is dynamic, one possible solution would be to let client define the entity structure (i.e. what data client needs) on the fly, during runtime :)
we talked about this solution couple of years ago, but it was obvious that there is no real "demand" for it till UI moves to REST,
btw it won't solve the problem where to fill UI entity you need combination of N REST entities, you still will have to perform N + 1 queries to API (maybe asking for entity with fewer details), - all it address is a capacity of the data being send.
For this problem of content filtering and retrieve associated data in a "restful" way I saw Yoga http://yoga.skyscreamer.org/
some example usage: http://www.infoq.com/articles/json-yoga
This looks exactly what we'd like to do :) filtering entity data plus retrieving related data all in one request. Michael, what is your opinion on this? Could we integrate Yoga with our RESTEasy-based infra?
michael?
Good luck, Itamar
--
Michael Pasternak RedHat, ENG-Virtualization R&D _______________________________________________ Engine-devel mailing list Engine-devel@ovirt.org http://lists.ovirt.org/mailman/listinfo/engine-devel
_______________________________________________ Engine-devel mailing list Engine-devel@ovirt.org http://lists.ovirt.org/mailman/listinfo/engine-devel
----- Original Message -----
From: "Michael Pasternak" <mpastern@redhat.com> To: "Itamar Heim" <iheim@redhat.com> Cc: "Vojtech Szocs" <vszocs@redhat.com>, "engine-devel" <engine-devel@ovirt.org>, "Einav Cohen" <ecohen@redhat.com> Sent: Sunday, November 24, 2013 9:37:22 AM Subject: Re: Using REST API in web UI - review call summary
On 11/22/2013 12:00 AM, Itamar Heim wrote:
On 11/21/2013 11:56 PM, Vojtech Szocs wrote:
----- Original Message -----
From: "Itamar Heim" <iheim@redhat.com> To: "Vojtech Szocs" <vszocs@redhat.com>, "engine-devel" <engine-devel@ovirt.org> Cc: "Einav Cohen" <ecohen@redhat.com> Sent: Thursday, November 21, 2013 10:25:04 PM Subject: Re: Using REST API in web UI - review call summary
On 11/21/2013 11:18 PM, Vojtech Szocs wrote:
Hi guys,
this is a summary of yesterday's review call, I'll try to highlight important Q/A and things we agreed on. Feel free to add anything in case I've missed something.
--
Q: Why don't we simply try to use existing Java SDK and adapt it for GWT apps? (asked by Michael & Gilad)
A: This might be a viable option to consider if we wanted to skip JavaScript-based SDK altogether and target Java/GWT code directly; we could simply take Java SDK and customize its abstractions where necessary, i.e. using HTTP transport layer implementation that works with GWT. In any case, this would mean coupling ourselves to Java SDK (which has its own release cycle) and I think this would complicate things for us.
As proposed on the meeting, I think it's best to aim for JavaScript SDK as the lowest common denominator for *any* web application that wants to work with REST API. oVirt GWT-based UI can simply bind to JavaScript SDK, i.e. Java/GWT code that just overlays objects and functions provided by JavaScript SDK. Another reason is ease of maintenance - I'd rather see JavaScript SDK's code generation process to be independent of any other SDK (people responsible for maintaining JavaScript SDK should have full control over generated code).
--
Q: What about functionality currently used by oVirt UI but not supported by REST API? (asked by Einav) [For example, fetching VM entity over GWT RPC also returns related data such as Cluster name.]
A: Based on discussion I've had with other colleagues after yesterday's review call, I don't think that separate support-like backend layer is a good idea. Instead, this is the kind of functionality that could be placed in oVirt.js library. Logical operations like "get VMs and related data" would be exposed through oVirt.js (callback-based) API and ultimately realized as multiple physical requests to REST API via JavaScript Binding.
oVirt.js client would be completely oblivious to the fact that multiple physical requests are dispatched. In fact, since HTTP communication is asynchronous in nature, oVirt.js client wouldn't even notice any difference in terms of API consumption. This assumes JavaScript SDK would use callback-based (non-blocking) API instead of blocking one - after all, blocking API on top of non-blocking implementation sounds pretty much like leaky abstraction [1].
For example:
sdk.getVmsWithExtraData( callbackToGetExtraDataForGivenVm, // might cause extra physical requests to REST API callbackFiredWhenAllDataIsReady // update client only when all data is ready )
would this also resolve RunMultipleActions?
Yes, I think so. There could be API to pass multiple "actions" and get notified when they complete.
sounds like no reason to have RunMultipleQueries, although i'm still sure a single call to engine for multiple keys would be much more efficient than multiple async calls? (I understand we may not be able to model this).
Efficiency-wise, yes, single call to get all data seems optimal. API-wise, I don't think it really matters from oVirt.js client perspective.
We can proceed with simple (possibly inefficient) solution and improve as needed. We're making baby steps now..
[1] http://en.wikipedia.org/wiki/Leaky_abstraction
--
Last but not least, where to maintain JavaScript SDK projects: low-level JavaScript Binding + high-level oVirt.js library.
I agree that conceptually both above mentioned projects should go into dedicated "ovirt-engine-sdk-js" git repository and have their own build/release process. However, for now, we're just making baby steps so let's keep things simple and prototype these projects as part of "ovirt-engine" git repository.
... we can complicate things anytime, but we should know that any complex system that works has inevitably evolved from simple system that works ... (quote from http://en.wikipedia.org/wiki/Gall%27s_law)
I think the entities should just be generated from the xsd.
+1
The JavaScript Binding (aka low-level SDK) module should follow same concepts as existing Java SDK - generated entities decorated with operations to form fluent API.
Everything Java SDK currently offers should be available in JavaScript Binding. oVirt.js is our opportunity to build on top of that.
for the rsdl, makes sense to start with clean code to see what works best, then see about generating it (but you should adhere the rsdl as guidlines i guess).
+1
The initial prototype should be written by hand, things will get generated as soon as we have better idea how the end result should look like.
i can understand that for the methods and maybe for populating the entities for the first few. the entities themselves, no point in hand coding - just generated them from the api.xsd. and once you decide how you want to fill them, not worth hand coding this - either json gives this out of the box, or should be generated as well.
lets try to plan for lightweight entities while at it - the API has a mechanism for different level of details - maybe we need a custom level where the client specifies which fields they want back or something like that.
Good idea! We should definitely think about the granularity of entities.
I didn't know REST API supports different level of detail per entity, is there some documentation for this feature?
currently it's about adding adding extra information (by supplying header [1]), this is driven by the properties that require extra query against the engine to fill them,
obviously the default is [2]
[1] All-Content: True [2] All-Content: False
OK so if I understand correctly, "All-Content: False" (default) just returns data of given entity plus IDs of related entities, whereas "All-Content: True" returns data of given entity plus (embedded) data of related entities, is that correct?
Since JavaScript is dynamic, one possible solution would be to let client define the entity structure (i.e. what data client needs) on the fly, during runtime :)
we talked about this solution couple of years ago, but it was obvious that there is no real "demand" for it till UI moves to REST,
I assume you mean to let client selectively specify properties of given entity when making REST call, correct? If yes, are you planning to support such feature?
btw it won't solve the problem where to fill UI entity you need combination of N REST entities, you still will have to perform N + 1 queries to API
Of course. But we can make improvements with regard to: - optimizing queries so that client receives/parses only data it needs (see my question above) - if necessary, combine such optimized queries into bigger "logical" queries (utilizing callback mechanism)
(maybe asking for entity with fewer details), - all it address is a capacity of the data being send.
Not only that, capacity of data sent goes proportionally with time needed to parse & process such data on client. (Imagine big XML marshalling on client. This is why everyone prefers JSON nowadays, JSON naturally maps to native JavaScript structures with minimal computational overhead.)
michael?
Good luck, Itamar
--
Michael Pasternak RedHat, ENG-Virtualization R&D
On 11/28/2013 09:37 PM, Vojtech Szocs wrote:
----- Original Message -----
From: "Michael Pasternak" <mpastern@redhat.com> To: "Itamar Heim" <iheim@redhat.com> Cc: "Vojtech Szocs" <vszocs@redhat.com>, "engine-devel" <engine-devel@ovirt.org>, "Einav Cohen" <ecohen@redhat.com> Sent: Sunday, November 24, 2013 9:37:22 AM Subject: Re: Using REST API in web UI - review call summary
On 11/22/2013 12:00 AM, Itamar Heim wrote:
On 11/21/2013 11:56 PM, Vojtech Szocs wrote:
----- Original Message -----
From: "Itamar Heim" <iheim@redhat.com> To: "Vojtech Szocs" <vszocs@redhat.com>, "engine-devel" <engine-devel@ovirt.org> Cc: "Einav Cohen" <ecohen@redhat.com> Sent: Thursday, November 21, 2013 10:25:04 PM Subject: Re: Using REST API in web UI - review call summary
On 11/21/2013 11:18 PM, Vojtech Szocs wrote:
Hi guys,
this is a summary of yesterday's review call, I'll try to highlight important Q/A and things we agreed on. Feel free to add anything in case I've missed something.
--
Q: Why don't we simply try to use existing Java SDK and adapt it for GWT apps? (asked by Michael & Gilad)
A: This might be a viable option to consider if we wanted to skip JavaScript-based SDK altogether and target Java/GWT code directly; we could simply take Java SDK and customize its abstractions where necessary, i.e. using HTTP transport layer implementation that works with GWT. In any case, this would mean coupling ourselves to Java SDK (which has its own release cycle) and I think this would complicate things for us.
As proposed on the meeting, I think it's best to aim for JavaScript SDK as the lowest common denominator for *any* web application that wants to work with REST API. oVirt GWT-based UI can simply bind to JavaScript SDK, i.e. Java/GWT code that just overlays objects and functions provided by JavaScript SDK. Another reason is ease of maintenance - I'd rather see JavaScript SDK's code generation process to be independent of any other SDK (people responsible for maintaining JavaScript SDK should have full control over generated code).
--
Q: What about functionality currently used by oVirt UI but not supported by REST API? (asked by Einav) [For example, fetching VM entity over GWT RPC also returns related data such as Cluster name.]
A: Based on discussion I've had with other colleagues after yesterday's review call, I don't think that separate support-like backend layer is a good idea. Instead, this is the kind of functionality that could be placed in oVirt.js library. Logical operations like "get VMs and related data" would be exposed through oVirt.js (callback-based) API and ultimately realized as multiple physical requests to REST API via JavaScript Binding.
oVirt.js client would be completely oblivious to the fact that multiple physical requests are dispatched. In fact, since HTTP communication is asynchronous in nature, oVirt.js client wouldn't even notice any difference in terms of API consumption. This assumes JavaScript SDK would use callback-based (non-blocking) API instead of blocking one - after all, blocking API on top of non-blocking implementation sounds pretty much like leaky abstraction [1].
For example:
sdk.getVmsWithExtraData( callbackToGetExtraDataForGivenVm, // might cause extra physical requests to REST API callbackFiredWhenAllDataIsReady // update client only when all data is ready )
would this also resolve RunMultipleActions?
Yes, I think so. There could be API to pass multiple "actions" and get notified when they complete.
sounds like no reason to have RunMultipleQueries, although i'm still sure a single call to engine for multiple keys would be much more efficient than multiple async calls? (I understand we may not be able to model this).
Efficiency-wise, yes, single call to get all data seems optimal. API-wise, I don't think it really matters from oVirt.js client perspective.
We can proceed with simple (possibly inefficient) solution and improve as needed. We're making baby steps now..
[1] http://en.wikipedia.org/wiki/Leaky_abstraction
--
Last but not least, where to maintain JavaScript SDK projects: low-level JavaScript Binding + high-level oVirt.js library.
I agree that conceptually both above mentioned projects should go into dedicated "ovirt-engine-sdk-js" git repository and have their own build/release process. However, for now, we're just making baby steps so let's keep things simple and prototype these projects as part of "ovirt-engine" git repository.
... we can complicate things anytime, but we should know that any complex system that works has inevitably evolved from simple system that works ... (quote from http://en.wikipedia.org/wiki/Gall%27s_law)
I think the entities should just be generated from the xsd.
+1
The JavaScript Binding (aka low-level SDK) module should follow same concepts as existing Java SDK - generated entities decorated with operations to form fluent API.
Everything Java SDK currently offers should be available in JavaScript Binding. oVirt.js is our opportunity to build on top of that.
for the rsdl, makes sense to start with clean code to see what works best, then see about generating it (but you should adhere the rsdl as guidlines i guess).
+1
The initial prototype should be written by hand, things will get generated as soon as we have better idea how the end result should look like.
i can understand that for the methods and maybe for populating the entities for the first few. the entities themselves, no point in hand coding - just generated them from the api.xsd. and once you decide how you want to fill them, not worth hand coding this - either json gives this out of the box, or should be generated as well.
lets try to plan for lightweight entities while at it - the API has a mechanism for different level of details - maybe we need a custom level where the client specifies which fields they want back or something like that.
Good idea! We should definitely think about the granularity of entities.
I didn't know REST API supports different level of detail per entity, is there some documentation for this feature?
currently it's about adding adding extra information (by supplying header [1]), this is driven by the properties that require extra query against the engine to fill them,
obviously the default is [2]
[1] All-Content: True [2] All-Content: False
OK so if I understand correctly, "All-Content: False" (default) just returns data of given entity plus IDs of related entities, whereas "All-Content: True" returns data of given entity plus (embedded) data of related entities, is that correct?
not exactly, "All-Content: True" will return other fields of this entity (not the entity's your resource has relation with, as it was non-RESTful), the differences is in cost of feeling these properties, if it's expensive like running extra query to the engine (like Payload/Certificate/Ballooning in vm) it's not shown by default.
Since JavaScript is dynamic, one possible solution would be to let client define the entity structure (i.e. what data client needs) on the fly, during runtime :)
we talked about this solution couple of years ago, but it was obvious that there is no real "demand" for it till UI moves to REST,
I assume you mean to let client selectively specify properties of given entity when making REST call, correct? If yes, are you planning to support such feature?
obviously this is one of the steps to take down the road to enhance the restapi.
btw it won't solve the problem where to fill UI entity you need combination of N REST entities, you still will have to perform N + 1 queries to API
Of course. But we can make improvements with regard to:
- optimizing queries so that client receives/parses only data it needs (see my question above) - if necessary, combine such optimized queries into bigger "logical" queries (utilizing callback mechanism)
not sure about "bigger" (you can't change api interface), but combining few requests for the same entity (in sake of less verbose communication) in to single GET call, is a well known pattern in distributed systems.
(maybe asking for entity with fewer details), - all it address is a capacity of the data being send.
Not only that, capacity of data sent goes proportionally with time needed to parse & process such data on client. (Imagine big XML marshalling on client. This is why everyone prefers JSON nowadays, JSON naturally maps to native JavaScript structures with minimal computational overhead.)
michael?
Good luck, Itamar
--
Michael Pasternak RedHat, ENG-Virtualization R&D
-- Michael Pasternak RedHat, ENG-Virtualization R&D
Hi Vojtech, First of all it was a good "presentation" of requirements + suggested solutions - well done!, few comments/questions inline. On 11/21/2013 11:18 PM, Vojtech Szocs wrote:
Hi guys,
this is a summary of yesterday's review call, I'll try to highlight important Q/A and things we agreed on. Feel free to add anything in case I've missed something.
--
Q: Why don't we simply try to use existing Java SDK and adapt it for GWT apps? (asked by Michael & Gilad)
A: This might be a viable option to consider if we wanted to skip JavaScript-based SDK altogether and target Java/GWT code directly; we could simply take Java SDK and customize its abstractions where necessary, i.e. using HTTP transport layer implementation that works with GWT. In any case, this would mean coupling ourselves to Java SDK (which has its own release cycle) and I think this would complicate things for us.
not sure i buy this one :), this is the purpose of any sdk, including the one you about to write, people that will use it, will be "coupling" to it ...
As proposed on the meeting, I think it's best to aim for JavaScript SDK as the lowest common denominator for *any* web application that wants to work with REST API. oVirt GWT-based UI can simply bind to JavaScript SDK, i.e. Java/GWT code that just overlays objects and functions provided by JavaScript SDK. Another reason is ease of maintenance - I'd rather see JavaScript SDK's code generation process to be independent of any other SDK (people responsible for maintaining JavaScript SDK should have full control over generated code).
what do you mean by "people should have full control over generated code"? the purpose of code generation is to ease maintenance, i.e you/maintainer should not write the feature once it available in api, just run CodeGen and you'll get it for free, but this is zero control over code.
--
Q: What about functionality currently used by oVirt UI but not supported by REST API? (asked by Einav) [For example, fetching VM entity over GWT RPC also returns related data such as Cluster name.]
A: Based on discussion I've had with other colleagues after yesterday's review call, I don't think that separate support-like backend layer is a good idea. Instead, this is the kind of functionality that could be placed in oVirt.js library. Logical operations like "get VMs and related data" would be exposed through oVirt.js (callback-based) API and ultimately realized as multiple physical requests to REST API via JavaScript Binding.
oVirt.js client would be completely oblivious to the fact that multiple physical requests are dispatched. In fact, since HTTP communication is asynchronous in nature, oVirt.js client wouldn't even notice any difference in terms of API consumption. This assumes JavaScript SDK would use callback-based (non-blocking) API instead of blocking one - after all, blocking API on top of non-blocking implementation sounds pretty much like leaky abstraction [1].
For example:
sdk.getVmsWithExtraData( callbackToGetExtraDataForGivenVm, // might cause extra physical requests to REST API callbackFiredWhenAllDataIsReady // update client only when all data is ready )
actually this the main bottleneck in moving UI to work on top of REST, and most interesting/complex part of this project, you should think of very wise polling mechanism cause callbacks is a nice thing on paper, but behind the scene it all about polling: - the entity/s till action got accomplished - add to this updating different grids - running multiple actions - showing events - and obviously much more and don't forget that every polled entity should be marshalled from xml to the javascript entity so at the end, "callbacks" mechanism will be extremely CPU consuming.
[1] http://en.wikipedia.org/wiki/Leaky_abstraction
--
Last but not least, where to maintain JavaScript SDK projects: low-level JavaScript Binding + high-level oVirt.js library.
I agree that conceptually both above mentioned projects should go into dedicated "ovirt-engine-sdk-js" git repository and have their own build/release process. However, for now, we're just making baby steps so let's keep things simple and prototype these projects as part of "ovirt-engine" git repository.
... we can complicate things anytime, but we should know that any complex system that works has inevitably evolved from simple system that works ... (quote from http://en.wikipedia.org/wiki/Gall%27s_law)
Regards, Vojtech _______________________________________________ Engine-devel mailing list Engine-devel@ovirt.org http://lists.ovirt.org/mailman/listinfo/engine-devel
-- Michael Pasternak RedHat, ENG-Virtualization R&D
----- Original Message -----
From: "Michael Pasternak" <mpastern@redhat.com> To: "Vojtech Szocs" <vszocs@redhat.com> Cc: "engine-devel" <engine-devel@ovirt.org> Sent: Sunday, November 24, 2013 9:07:01 AM Subject: Re: [Engine-devel] Using REST API in web UI - review call summary
Hi Vojtech,
First of all it was a good "presentation" of requirements + suggested solutions - well done!,
Thank you :)
few comments/questions inline.
On 11/21/2013 11:18 PM, Vojtech Szocs wrote:
Hi guys,
this is a summary of yesterday's review call, I'll try to highlight important Q/A and things we agreed on. Feel free to add anything in case I've missed something.
--
Q: Why don't we simply try to use existing Java SDK and adapt it for GWT apps? (asked by Michael & Gilad)
A: This might be a viable option to consider if we wanted to skip JavaScript-based SDK altogether and target Java/GWT code directly; we could simply take Java SDK and customize its abstractions where necessary, i.e. using HTTP transport layer implementation that works with GWT. In any case, this would mean coupling ourselves to Java SDK (which has its own release cycle) and I think this would complicate things for us.
not sure i buy this one :), this is the purpose of any sdk, including the one you about to write, people that will use it, will be "coupling" to it ...
Of course, but by saying "coupling ourselves to Java SDK" I meant SDK perspective, not client perspective: - someone else (you) maintains Java SDK and therefore controls generated sources (JAR or RPM isn't relevant here) - another guy (me) maintains (fictional) Java/GWT SDK that relies on Java SDK + some (supported) customizations - the only way I can impose changes in my SDK is through supported customizations as you control original (Java SDK) sources, i.e. the whole code generation process is driven by your SDK, so my SDK is coupled to your SDK's build/release cycle For the sake of simplicity, I guess it's best to start with SDK that has no dependencies whatsoever. After all, there's no common dependency (aside from running Engine to provide XSD & RSDL) between Java & Python SDK too, if I understand correctly. In other words, building on top of something existing (just because we can do that) isn't always appropriate/flexible/efficient, it always depends on given context and requirements.
As proposed on the meeting, I think it's best to aim for JavaScript SDK as the lowest common denominator for *any* web application that wants to work with REST API. oVirt GWT-based UI can simply bind to JavaScript SDK, i.e. Java/GWT code that just overlays objects and functions provided by JavaScript SDK. Another reason is ease of maintenance - I'd rather see JavaScript SDK's code generation process to be independent of any other SDK (people responsible for maintaining JavaScript SDK should have full control over generated code).
what do you mean by "people should have full control over generated code"?
It's related to "coupling from SDK perspective" I mentioned above: "the only way I can impose changes in my SDK is through supported customizations as you control original (Java SDK) sources" (by "people" I meant "JavaScript SDK developers") Full control means ability to change generated sources in whatever way desired, but assuming the idea of reusing/customizing existing SDK code, aspect of full control is lost in favor of reusing existing code. And of course, this assumes that existing code (Java SDK) provides everything we need, which might or might not be the case. So I just vote for simplicity, generate JavaScript SDK the way like other SDKs (Java/Python) - not trying to reuse anything, just grab XSD & RSDL and generate sources.
the purpose of code generation is to ease maintenance, i.e you/maintainer should not write the feature once it available in api, just run CodeGen and you'll get it for free, but this is zero control over code.
+1 I agree with you on this.
--
Q: What about functionality currently used by oVirt UI but not supported by REST API? (asked by Einav) [For example, fetching VM entity over GWT RPC also returns related data such as Cluster name.]
A: Based on discussion I've had with other colleagues after yesterday's review call, I don't think that separate support-like backend layer is a good idea. Instead, this is the kind of functionality that could be placed in oVirt.js library. Logical operations like "get VMs and related data" would be exposed through oVirt.js (callback-based) API and ultimately realized as multiple physical requests to REST API via JavaScript Binding.
oVirt.js client would be completely oblivious to the fact that multiple physical requests are dispatched. In fact, since HTTP communication is asynchronous in nature, oVirt.js client wouldn't even notice any difference in terms of API consumption. This assumes JavaScript SDK would use callback-based (non-blocking) API instead of blocking one - after all, blocking API on top of non-blocking implementation sounds pretty much like leaky abstraction [1].
For example:
sdk.getVmsWithExtraData( callbackToGetExtraDataForGivenVm, // might cause extra physical requests to REST API callbackFiredWhenAllDataIsReady // update client only when all data is ready )
actually this the main bottleneck in moving UI to work on top of REST, and most interesting/complex part of this project,
Agreed, it's because UI "got used to" using internal backend interface concepts (actions, queries etc.) in the first place.. So we'll have to emulate what we used to use to prevent regressions, maybe improve/refactor in future.
you should think of very wise polling mechanism cause callbacks is a nice thing on paper, but behind the scene it all about polling:
- the entity/s till action got accomplished - add to this updating different grids - running multiple actions - showing events - and obviously much more
IMHO polling is just a workaround and indicates lack of proper notification solution. Apparently, oVirt web UI isn't some CLI program for which HTTP request/response style is sufficient. oVirt web UI is dynamic, interactive web application that displays/updates data in real time. This is, in my opinion, quite a big difference. I don't think callbacks are just a nice thing on paper. Callbacks are needed because the underlying communcation is async in nature: - caller invokes API function and provides callback to execute when operation completes -> API is non-blocking - polling attempts to detect change (i.e. operation completed) and notify the caller, so it's also some sort of callback -> this is more complicated compared to simple callback
and don't forget that every polled entity should be marshalled from xml to the javascript entity so at the end, "callbacks" mechanism will be extremely CPU consuming.
First of all, I don't understand how callback mechanism can be CPU consuming, can you please provide some explanation or use case? Does Java SDK provide ability to poll Engine in order to get recent updates, and if yes, why? Finally, polling makes things stateful, whereas SDK code should be stateless instead. If client wants to get recent updates, it should just use (stateless) SDK code to achieve this goal.
[1] http://en.wikipedia.org/wiki/Leaky_abstraction
--
Last but not least, where to maintain JavaScript SDK projects: low-level JavaScript Binding + high-level oVirt.js library.
I agree that conceptually both above mentioned projects should go into dedicated "ovirt-engine-sdk-js" git repository and have their own build/release process. However, for now, we're just making baby steps so let's keep things simple and prototype these projects as part of "ovirt-engine" git repository.
... we can complicate things anytime, but we should know that any complex system that works has inevitably evolved from simple system that works ... (quote from http://en.wikipedia.org/wiki/Gall%27s_law)
Regards, Vojtech _______________________________________________ Engine-devel mailing list Engine-devel@ovirt.org http://lists.ovirt.org/mailman/listinfo/engine-devel
--
Michael Pasternak RedHat, ENG-Virtualization R&D
On 11/28/2013 09:22 PM, Vojtech Szocs wrote:
----- Original Message -----
From: "Michael Pasternak" <mpastern@redhat.com> To: "Vojtech Szocs" <vszocs@redhat.com> Cc: "engine-devel" <engine-devel@ovirt.org> Sent: Sunday, November 24, 2013 9:07:01 AM Subject: Re: [Engine-devel] Using REST API in web UI - review call summary
Hi Vojtech,
First of all it was a good "presentation" of requirements + suggested solutions - well done!,
Thank you :)
few comments/questions inline.
On 11/21/2013 11:18 PM, Vojtech Szocs wrote:
Hi guys,
this is a summary of yesterday's review call, I'll try to highlight important Q/A and things we agreed on. Feel free to add anything in case I've missed something.
--
Q: Why don't we simply try to use existing Java SDK and adapt it for GWT apps? (asked by Michael & Gilad)
A: This might be a viable option to consider if we wanted to skip JavaScript-based SDK altogether and target Java/GWT code directly; we could simply take Java SDK and customize its abstractions where necessary, i.e. using HTTP transport layer implementation that works with GWT. In any case, this would mean coupling ourselves to Java SDK (which has its own release cycle) and I think this would complicate things for us.
not sure i buy this one :), this is the purpose of any sdk, including the one you about to write, people that will use it, will be "coupling" to it ...
Of course, but by saying "coupling ourselves to Java SDK" I meant SDK perspective, not client perspective:
of course, but you told something different, that you want js-sdk to be aware of the client, and this is actually why you taking this path.
- someone else (you) maintains Java SDK and therefore controls generated sources (JAR or RPM isn't relevant here) - another guy (me) maintains (fictional) Java/GWT SDK that relies on Java SDK + some (supported) customizations - the only way I can impose changes in my SDK is through supported customizations as you control original (Java SDK) sources, i.e. the whole code generation process is driven by your SDK, so my SDK is coupled to your SDK's build/release cycle
that's how things working in software, you always depending on the certain version of the component you're working against, as it expose set of features you need, i don't think that having control over framework features, justifying rewriting the framework ... (please note that i'm not against the js-sdk, go ahead, this is a nice initiative indeed, i just can't see the business case for not reusing existent infrastructure cause it works for all your needs and eventually both worlds would benefiting from it UI and java-sdk users cause you where extending it with additional capabilities they may also need)
For the sake of simplicity, I guess it's best to start with SDK that has no dependencies whatsoever.
so why won't you rewrite the engine in Java-script? your js-sdk eventually will be depending on it, this way you'll have control over it (and it's features) as well ;-)
After all, there's no common dependency (aside from running Engine to provide XSD & RSDL) between Java & Python SDK too, if I understand correctly.
In other words, building on top of something existing (just because we can do that) isn't always appropriate/flexible/efficient, it always depends on given context and requirements.
it would be true, if your requirements would make existing infrastructure inappropriate.
As proposed on the meeting, I think it's best to aim for JavaScript SDK as the lowest common denominator for *any* web application that wants to work with REST API. oVirt GWT-based UI can simply bind to JavaScript SDK, i.e. Java/GWT code that just overlays objects and functions provided by JavaScript SDK. Another reason is ease of maintenance - I'd rather see JavaScript SDK's code generation process to be independent of any other SDK (people responsible for maintaining JavaScript SDK should have full control over generated code).
what do you mean by "people should have full control over generated code"?
It's related to "coupling from SDK perspective" I mentioned above: "the only way I can impose changes in my SDK is through supported customizations as you control original (Java SDK) sources"
if you need additional functionality in java-sdk, you could do the following: 1. submit a patch to java-sdk 2. build new java-sdk locally and use it along with new feature you've added 3. make UI depending on next version of java-sdk (which includes your new feature) we (and all other SW projects) doing that day by day in engine,api,etc. (as i mentioned this would also benefit java-sdk users with additional features they might find useful as well)
(by "people" I meant "JavaScript SDK developers")
Full control means ability to change generated sources in whatever way desired, but assuming the idea of reusing/customizing existing SDK code, aspect of full control is lost in favor of reusing existing code.
i disagree on this one, you have all control you need over java-sdk at any time as it one of indoor projects.
And of course, this assumes that existing code (Java SDK) provides everything we need, which might or might not be the case.
So I just vote for simplicity, generate JavaScript SDK the way like other SDKs (Java/Python) - not trying to reuse anything, just grab XSD & RSDL and generate sources.
the purpose of code generation is to ease maintenance, i.e you/maintainer should not write the feature once it available in api, just run CodeGen and you'll get it for free, but this is zero control over code.
+1
I agree with you on this.
--
Q: What about functionality currently used by oVirt UI but not supported by REST API? (asked by Einav) [For example, fetching VM entity over GWT RPC also returns related data such as Cluster name.]
A: Based on discussion I've had with other colleagues after yesterday's review call, I don't think that separate support-like backend layer is a good idea. Instead, this is the kind of functionality that could be placed in oVirt.js library. Logical operations like "get VMs and related data" would be exposed through oVirt.js (callback-based) API and ultimately realized as multiple physical requests to REST API via JavaScript Binding.
oVirt.js client would be completely oblivious to the fact that multiple physical requests are dispatched. In fact, since HTTP communication is asynchronous in nature, oVirt.js client wouldn't even notice any difference in terms of API consumption. This assumes JavaScript SDK would use callback-based (non-blocking) API instead of blocking one - after all, blocking API on top of non-blocking implementation sounds pretty much like leaky abstraction [1].
For example:
sdk.getVmsWithExtraData( callbackToGetExtraDataForGivenVm, // might cause extra physical requests to REST API callbackFiredWhenAllDataIsReady // update client only when all data is ready )
actually this the main bottleneck in moving UI to work on top of REST, and most interesting/complex part of this project,
Agreed, it's because UI "got used to" using internal backend interface concepts (actions, queries etc.) in the first place.. So we'll have to emulate what we used to use to prevent regressions, maybe improve/refactor in future.
you should think of very wise polling mechanism cause callbacks is a nice thing on paper, but behind the scene it all about polling:
- the entity/s till action got accomplished - add to this updating different grids - running multiple actions - showing events - and obviously much more
IMHO polling is just a workaround and indicates lack of proper notification solution.
Apparently, oVirt web UI isn't some CLI program for which HTTP request/response style is sufficient. oVirt web UI is dynamic, interactive web application that displays/updates data in real time. This is, in my opinion, quite a big difference.
I don't think callbacks are just a nice thing on paper. Callbacks are needed because the underlying communcation is async in nature:
Vojtech, you've got all wrong, i told that you *do need* callbacks, but implementing them only sounds easy, while actually it will be a quite complicated task.
- caller invokes API function and provides callback to execute when operation completes -> API is non-blocking - polling attempts to detect change (i.e. operation completed) and notify the caller, so it's also some sort of callback -> this is more complicated compared to simple callback
and don't forget that every polled entity should be marshalled from xml to the javascript entity so at the end, "callbacks" mechanism will be extremely CPU consuming.
First of all, I don't understand how callback mechanism can be CPU consuming, can you please provide some explanation or use case?
of course, you'll have to do per call-back call: 1. request/response to the server 2. decompression of data from gzip 3. object mapping (in 99% of cases) note you'll have a lot of callback consumers that monitoring resource state, waiting for new events.
Does Java SDK provide ability to poll Engine in order to get recent updates, and if yes, why?
i was kinda hoping that you'll add it, but you've chosen to write your own sdk ;-)
Finally, polling makes things stateful, whereas SDK code should be stateless instead. If client wants to get recent updates, it should just use (stateless) SDK code to achieve this goal.
sdk cannot be stateful by definition simply because server is stateless, (also pooling != keeping state, being stateful means that you save data for request on server side)
[1] http://en.wikipedia.org/wiki/Leaky_abstraction
--
Last but not least, where to maintain JavaScript SDK projects: low-level JavaScript Binding + high-level oVirt.js library.
I agree that conceptually both above mentioned projects should go into dedicated "ovirt-engine-sdk-js" git repository and have their own build/release process. However, for now, we're just making baby steps so let's keep things simple and prototype these projects as part of "ovirt-engine" git repository.
... we can complicate things anytime, but we should know that any complex system that works has inevitably evolved from simple system that works ... (quote from http://en.wikipedia.org/wiki/Gall%27s_law)
Regards, Vojtech _______________________________________________ Engine-devel mailing list Engine-devel@ovirt.org http://lists.ovirt.org/mailman/listinfo/engine-devel
--
Michael Pasternak RedHat, ENG-Virtualization R&D
-- Michael Pasternak RedHat, ENG-Virtualization R&D
On 11/29/2013 11:45 AM, Michael Pasternak wrote:
On 11/28/2013 09:22 PM, Vojtech Szocs wrote:
----- Original Message -----
From: "Michael Pasternak" <mpastern@redhat.com> To: "Vojtech Szocs" <vszocs@redhat.com> Cc: "engine-devel" <engine-devel@ovirt.org> Sent: Sunday, November 24, 2013 9:07:01 AM Subject: Re: [Engine-devel] Using REST API in web UI - review call summary
Hi Vojtech,
First of all it was a good "presentation" of requirements + suggested solutions - well done!,
Thank you :)
few comments/questions inline.
On 11/21/2013 11:18 PM, Vojtech Szocs wrote:
Hi guys,
this is a summary of yesterday's review call, I'll try to highlight important Q/A and things we agreed on. Feel free to add anything in case I've missed something.
--
Q: Why don't we simply try to use existing Java SDK and adapt it for GWT apps? (asked by Michael & Gilad)
A: This might be a viable option to consider if we wanted to skip JavaScript-based SDK altogether and target Java/GWT code directly; we could simply take Java SDK and customize its abstractions where necessary, i.e. using HTTP transport layer implementation that works with GWT. In any case, this would mean coupling ourselves to Java SDK (which has its own release cycle) and I think this would complicate things for us.
not sure i buy this one :), this is the purpose of any sdk, including the one you about to write, people that will use it, will be "coupling" to it ...
Of course, but by saying "coupling ourselves to Java SDK" I meant SDK perspective, not client perspective:
of course, but you told something different, that you want js-sdk to be aware of the client, and this is actually why you taking this path.
- someone else (you) maintains Java SDK and therefore controls generated sources (JAR or RPM isn't relevant here) - another guy (me) maintains (fictional) Java/GWT SDK that relies on Java SDK + some (supported) customizations - the only way I can impose changes in my SDK is through supported customizations as you control original (Java SDK) sources, i.e. the whole code generation process is driven by your SDK, so my SDK is coupled to your SDK's build/release cycle
that's how things working in software, you always depending on the certain version of the component you're working against, as it expose set of features you need, i don't think that having control over framework features, justifying rewriting the framework ...
(please note that i'm not against the js-sdk, go ahead, this is a nice initiative indeed, i just can't see the business case for not reusing existent infrastructure cause it works for all your needs and eventually both worlds would benefiting from it UI and java-sdk users cause you where extending it with additional capabilities they may also need)
For the sake of simplicity, I guess it's best to start with SDK that has no dependencies whatsoever.
so why won't you rewrite the engine in Java-script? your js-sdk eventually will be depending on it, this way you'll have control over it (and it's features) as well ;-)
After all, there's no common dependency (aside from running Engine to provide XSD & RSDL) between Java & Python SDK too, if I understand correctly.
In other words, building on top of something existing (just because we can do that) isn't always appropriate/flexible/efficient, it always depends on given context and requirements.
it would be true, if your requirements would make existing infrastructure inappropriate.
As proposed on the meeting, I think it's best to aim for JavaScript SDK as the lowest common denominator for *any* web application that wants to work with REST API. oVirt GWT-based UI can simply bind to JavaScript SDK, i.e. Java/GWT code that just overlays objects and functions provided by JavaScript SDK. Another reason is ease of maintenance - I'd rather see JavaScript SDK's code generation process to be independent of any other SDK (people responsible for maintaining JavaScript SDK should have full control over generated code).
what do you mean by "people should have full control over generated code"?
It's related to "coupling from SDK perspective" I mentioned above: "the only way I can impose changes in my SDK is through supported customizations as you control original (Java SDK) sources"
if you need additional functionality in java-sdk, you could do the following:
1. submit a patch to java-sdk 2. build new java-sdk locally and use it along with new feature you've added 3. make UI depending on next version of java-sdk (which includes your new feature)
we (and all other SW projects) doing that day by day in engine,api,etc.
(as i mentioned this would also benefit java-sdk users with additional features they might find useful as well)
(by "people" I meant "JavaScript SDK developers")
Full control means ability to change generated sources in whatever way desired, but assuming the idea of reusing/customizing existing SDK code, aspect of full control is lost in favor of reusing existing code.
i disagree on this one, you have all control you need over java-sdk at any time as it one of indoor projects.
And of course, this assumes that existing code (Java SDK) provides everything we need, which might or might not be the case.
So I just vote for simplicity, generate JavaScript SDK the way like other SDKs (Java/Python) - not trying to reuse anything, just grab XSD & RSDL and generate sources.
the purpose of code generation is to ease maintenance, i.e you/maintainer should not write the feature once it available in api, just run CodeGen and you'll get it for free, but this is zero control over code.
+1
I agree with you on this.
--
Q: What about functionality currently used by oVirt UI but not supported by REST API? (asked by Einav) [For example, fetching VM entity over GWT RPC also returns related data such as Cluster name.]
A: Based on discussion I've had with other colleagues after yesterday's review call, I don't think that separate support-like backend layer is a good idea. Instead, this is the kind of functionality that could be placed in oVirt.js library. Logical operations like "get VMs and related data" would be exposed through oVirt.js (callback-based) API and ultimately realized as multiple physical requests to REST API via JavaScript Binding.
oVirt.js client would be completely oblivious to the fact that multiple physical requests are dispatched. In fact, since HTTP communication is asynchronous in nature, oVirt.js client wouldn't even notice any difference in terms of API consumption. This assumes JavaScript SDK would use callback-based (non-blocking) API instead of blocking one - after all, blocking API on top of non-blocking implementation sounds pretty much like leaky abstraction [1].
For example:
sdk.getVmsWithExtraData( callbackToGetExtraDataForGivenVm, // might cause extra physical requests to REST API callbackFiredWhenAllDataIsReady // update client only when all data is ready )
actually this the main bottleneck in moving UI to work on top of REST, and most interesting/complex part of this project,
Agreed, it's because UI "got used to" using internal backend interface concepts (actions, queries etc.) in the first place.. So we'll have to emulate what we used to use to prevent regressions, maybe improve/refactor in future.
you should think of very wise polling mechanism cause callbacks is a nice thing on paper, but behind the scene it all about polling:
- the entity/s till action got accomplished - add to this updating different grids - running multiple actions - showing events - and obviously much more
IMHO polling is just a workaround and indicates lack of proper notification solution.
Apparently, oVirt web UI isn't some CLI program for which HTTP request/response style is sufficient. oVirt web UI is dynamic, interactive web application that displays/updates data in real time. This is, in my opinion, quite a big difference.
I don't think callbacks are just a nice thing on paper. Callbacks are needed because the underlying communcation is async in nature:
Vojtech, you've got all wrong, i told that you *do need* callbacks, but implementing them only sounds easy, while actually it will be a quite complicated task.
- caller invokes API function and provides callback to execute when operation completes -> API is non-blocking - polling attempts to detect change (i.e. operation completed) and notify the caller, so it's also some sort of callback -> this is more complicated compared to simple callback
and don't forget that every polled entity should be marshalled from xml to the javascript entity so at the end, "callbacks" mechanism will be extremely CPU consuming.
First of all, I don't understand how callback mechanism can be CPU consuming, can you please provide some explanation or use case?
of course, you'll have to do per call-back call:
s/"call-back call"/"polling request", e.g: 1 polling task == N * (#1+#2+#3) [where N is amount of requests you need to perform till desired state is achieved]
1. request/response to the server 2. decompression of data from gzip 3. object mapping (in 99% of cases)
note you'll have a lot of callback consumers that monitoring resource state, waiting for new events.
Does Java SDK provide ability to poll Engine in order to get recent updates, and if yes, why?
i was kinda hoping that you'll add it, but you've chosen to write your own sdk ;-)
Finally, polling makes things stateful, whereas SDK code should be stateless instead. If client wants to get recent updates, it should just use (stateless) SDK code to achieve this goal.
sdk cannot be stateful by definition simply because server is stateless, (also pooling != keeping state, being stateful means that you save data for request on server side)
[1] http://en.wikipedia.org/wiki/Leaky_abstraction
--
Last but not least, where to maintain JavaScript SDK projects: low-level JavaScript Binding + high-level oVirt.js library.
I agree that conceptually both above mentioned projects should go into dedicated "ovirt-engine-sdk-js" git repository and have their own build/release process. However, for now, we're just making baby steps so let's keep things simple and prototype these projects as part of "ovirt-engine" git repository.
... we can complicate things anytime, but we should know that any complex system that works has inevitably evolved from simple system that works ... (quote from http://en.wikipedia.org/wiki/Gall%27s_law)
Regards, Vojtech _______________________________________________ Engine-devel mailing list Engine-devel@ovirt.org http://lists.ovirt.org/mailman/listinfo/engine-devel
--
Michael Pasternak RedHat, ENG-Virtualization R&D
-- Michael Pasternak RedHat, ENG-Virtualization R&D
On 11/29/2013 11:59 AM, Michael Pasternak wrote:
On 11/29/2013 11:45 AM, Michael Pasternak wrote:
On 11/28/2013 09:22 PM, Vojtech Szocs wrote:
----- Original Message -----
From: "Michael Pasternak" <mpastern@redhat.com> To: "Vojtech Szocs" <vszocs@redhat.com> Cc: "engine-devel" <engine-devel@ovirt.org> Sent: Sunday, November 24, 2013 9:07:01 AM Subject: Re: [Engine-devel] Using REST API in web UI - review call summary
Hi Vojtech,
First of all it was a good "presentation" of requirements + suggested solutions - well done!,
Thank you :)
few comments/questions inline.
On 11/21/2013 11:18 PM, Vojtech Szocs wrote:
Hi guys,
this is a summary of yesterday's review call, I'll try to highlight important Q/A and things we agreed on. Feel free to add anything in case I've missed something.
--
Q: Why don't we simply try to use existing Java SDK and adapt it for GWT apps? (asked by Michael & Gilad)
A: This might be a viable option to consider if we wanted to skip JavaScript-based SDK altogether and target Java/GWT code directly; we could simply take Java SDK and customize its abstractions where necessary, i.e. using HTTP transport layer implementation that works with GWT. In any case, this would mean coupling ourselves to Java SDK (which has its own release cycle) and I think this would complicate things for us.
not sure i buy this one :), this is the purpose of any sdk, including the one you about to write, people that will use it, will be "coupling" to it ...
Of course, but by saying "coupling ourselves to Java SDK" I meant SDK perspective, not client perspective:
of course, but you told something different, that you want js-sdk to be aware of the client, and this is actually why you taking this path.
- someone else (you) maintains Java SDK and therefore controls generated sources (JAR or RPM isn't relevant here) - another guy (me) maintains (fictional) Java/GWT SDK that relies on Java SDK + some (supported) customizations - the only way I can impose changes in my SDK is through supported customizations as you control original (Java SDK) sources, i.e. the whole code generation process is driven by your SDK, so my SDK is coupled to your SDK's build/release cycle
that's how things working in software, you always depending on the certain version of the component you're working against, as it expose set of features you need, i don't think that having control over framework features, justifying rewriting the framework ...
(please note that i'm not against the js-sdk, go ahead, this is a nice initiative indeed, i just can't see the business case for not reusing existent infrastructure cause it works for all your needs and eventually both worlds would benefiting from it UI and java-sdk users cause you where extending it with additional capabilities they may also need)
For the sake of simplicity, I guess it's best to start with SDK that has no dependencies whatsoever.
so why won't you rewrite the engine in Java-script? your js-sdk eventually will be depending on it, this way you'll have control over it (and it's features) as well ;-)
After all, there's no common dependency (aside from running Engine to provide XSD & RSDL) between Java & Python SDK too, if I understand correctly.
In other words, building on top of something existing (just because we can do that) isn't always appropriate/flexible/efficient, it always depends on given context and requirements.
it would be true, if your requirements would make existing infrastructure inappropriate.
As proposed on the meeting, I think it's best to aim for JavaScript SDK as the lowest common denominator for *any* web application that wants to work with REST API. oVirt GWT-based UI can simply bind to JavaScript SDK, i.e. Java/GWT code that just overlays objects and functions provided by JavaScript SDK. Another reason is ease of maintenance - I'd rather see JavaScript SDK's code generation process to be independent of any other SDK (people responsible for maintaining JavaScript SDK should have full control over generated code).
what do you mean by "people should have full control over generated code"?
It's related to "coupling from SDK perspective" I mentioned above: "the only way I can impose changes in my SDK is through supported customizations as you control original (Java SDK) sources"
if you need additional functionality in java-sdk, you could do the following:
1. submit a patch to java-sdk 2. build new java-sdk locally and use it along with new feature you've added 3. make UI depending on next version of java-sdk (which includes your new feature)
we (and all other SW projects) doing that day by day in engine,api,etc.
(as i mentioned this would also benefit java-sdk users with additional features they might find useful as well)
(by "people" I meant "JavaScript SDK developers")
Full control means ability to change generated sources in whatever way desired, but assuming the idea of reusing/customizing existing SDK code, aspect of full control is lost in favor of reusing existing code.
i disagree on this one, you have all control you need over java-sdk at any time as it one of indoor projects.
And of course, this assumes that existing code (Java SDK) provides everything we need, which might or might not be the case.
So I just vote for simplicity, generate JavaScript SDK the way like other SDKs (Java/Python) - not trying to reuse anything, just grab XSD & RSDL and generate sources.
the purpose of code generation is to ease maintenance, i.e you/maintainer should not write the feature once it available in api, just run CodeGen and you'll get it for free, but this is zero control over code.
+1
I agree with you on this.
--
Q: What about functionality currently used by oVirt UI but not supported by REST API? (asked by Einav) [For example, fetching VM entity over GWT RPC also returns related data such as Cluster name.]
A: Based on discussion I've had with other colleagues after yesterday's review call, I don't think that separate support-like backend layer is a good idea. Instead, this is the kind of functionality that could be placed in oVirt.js library. Logical operations like "get VMs and related data" would be exposed through oVirt.js (callback-based) API and ultimately realized as multiple physical requests to REST API via JavaScript Binding.
oVirt.js client would be completely oblivious to the fact that multiple physical requests are dispatched. In fact, since HTTP communication is asynchronous in nature, oVirt.js client wouldn't even notice any difference in terms of API consumption. This assumes JavaScript SDK would use callback-based (non-blocking) API instead of blocking one - after all, blocking API on top of non-blocking implementation sounds pretty much like leaky abstraction [1].
For example:
sdk.getVmsWithExtraData( callbackToGetExtraDataForGivenVm, // might cause extra physical requests to REST API callbackFiredWhenAllDataIsReady // update client only when all data is ready )
actually this the main bottleneck in moving UI to work on top of REST, and most interesting/complex part of this project,
Agreed, it's because UI "got used to" using internal backend interface concepts (actions, queries etc.) in the first place.. So we'll have to emulate what we used to use to prevent regressions, maybe improve/refactor in future.
you should think of very wise polling mechanism cause callbacks is a nice thing on paper, but behind the scene it all about polling:
- the entity/s till action got accomplished - add to this updating different grids - running multiple actions - showing events - and obviously much more
IMHO polling is just a workaround and indicates lack of proper notification solution.
Apparently, oVirt web UI isn't some CLI program for which HTTP request/response style is sufficient. oVirt web UI is dynamic, interactive web application that displays/updates data in real time. This is, in my opinion, quite a big difference.
I don't think callbacks are just a nice thing on paper. Callbacks are needed because the underlying communcation is async in nature:
Vojtech, you've got all wrong, i told that you *do need* callbacks, but implementing them only sounds easy, while actually it will be a quite complicated task.
- caller invokes API function and provides callback to execute when operation completes -> API is non-blocking - polling attempts to detect change (i.e. operation completed) and notify the caller, so it's also some sort of callback -> this is more complicated compared to simple callback
and don't forget that every polled entity should be marshalled from xml to the javascript entity so at the end, "callbacks" mechanism will be extremely CPU consuming.
First of all, I don't understand how callback mechanism can be CPU consuming, can you please provide some explanation or use case?
of course, you'll have to do per call-back call:
s/"call-back call"/"polling request", e.g:
1 polling task == N * (#1+#2+#3)
[where N is amount of requests you need to perform till desired state is achieved]
i have a solution for this!, will get back to you/publish it when i have a mature design.
1. request/response to the server 2. decompression of data from gzip 3. object mapping (in 99% of cases)
note you'll have a lot of callback consumers that monitoring resource state, waiting for new events.
Does Java SDK provide ability to poll Engine in order to get recent updates, and if yes, why?
i was kinda hoping that you'll add it, but you've chosen to write your own sdk ;-)
Finally, polling makes things stateful, whereas SDK code should be stateless instead. If client wants to get recent updates, it should just use (stateless) SDK code to achieve this goal.
sdk cannot be stateful by definition simply because server is stateless, (also pooling != keeping state, being stateful means that you save data for request on server side)
[1] http://en.wikipedia.org/wiki/Leaky_abstraction
--
Last but not least, where to maintain JavaScript SDK projects: low-level JavaScript Binding + high-level oVirt.js library.
I agree that conceptually both above mentioned projects should go into dedicated "ovirt-engine-sdk-js" git repository and have their own build/release process. However, for now, we're just making baby steps so let's keep things simple and prototype these projects as part of "ovirt-engine" git repository.
... we can complicate things anytime, but we should know that any complex system that works has inevitably evolved from simple system that works ... (quote from http://en.wikipedia.org/wiki/Gall%27s_law)
Regards, Vojtech _______________________________________________ Engine-devel mailing list Engine-devel@ovirt.org http://lists.ovirt.org/mailman/listinfo/engine-devel
--
Michael Pasternak RedHat, ENG-Virtualization R&D
-- Michael Pasternak RedHat, ENG-Virtualization R&D
Hi guys, I apologize for my extremely late reply in this discussion.. (please find my comments inline) ----- Original Message -----
From: "Michael Pasternak" <mpastern@redhat.com> To: "Vojtech Szocs" <vszocs@redhat.com> Cc: "engine-devel" <engine-devel@ovirt.org> Sent: Sunday, December 1, 2013 8:02:12 AM Subject: Re: [Engine-devel] Using REST API in web UI - review call summary
On 11/29/2013 11:59 AM, Michael Pasternak wrote:
On 11/29/2013 11:45 AM, Michael Pasternak wrote:
On 11/28/2013 09:22 PM, Vojtech Szocs wrote:
----- Original Message -----
From: "Michael Pasternak" <mpastern@redhat.com> To: "Vojtech Szocs" <vszocs@redhat.com> Cc: "engine-devel" <engine-devel@ovirt.org> Sent: Sunday, November 24, 2013 9:07:01 AM Subject: Re: [Engine-devel] Using REST API in web UI - review call summary
Hi Vojtech,
First of all it was a good "presentation" of requirements + suggested solutions - well done!,
Thank you :)
few comments/questions inline.
On 11/21/2013 11:18 PM, Vojtech Szocs wrote:
Hi guys,
this is a summary of yesterday's review call, I'll try to highlight important Q/A and things we agreed on. Feel free to add anything in case I've missed something.
--
Q: Why don't we simply try to use existing Java SDK and adapt it for GWT apps? (asked by Michael & Gilad)
A: This might be a viable option to consider if we wanted to skip JavaScript-based SDK altogether and target Java/GWT code directly; we could simply take Java SDK and customize its abstractions where necessary, i.e. using HTTP transport layer implementation that works with GWT. In any case, this would mean coupling ourselves to Java SDK (which has its own release cycle) and I think this would complicate things for us.
not sure i buy this one :), this is the purpose of any sdk, including the one you about to write, people that will use it, will be "coupling" to it ...
Of course, but by saying "coupling ourselves to Java SDK" I meant SDK perspective, not client perspective:
of course, but you told something different, that you want js-sdk to be aware of the client, and this is actually why you taking this path.
Yes, this SDK targets JavaScript/browser runtime and its code should therefore be as "close" to it as possible. There are some conceptual gaps between world of Java vs. JavaScript in terms of language, coding practices and patterns, available APIs, etc. (at least) for the first iteration, I wanted to avoid dealing with any sort of discrepancies between Java vs. JavaScript + extra overhead when dealing with Java SDK; my goal is to prototype initial SDK code by hand, verify it in GUI code, then consider code generation, etc. (so-called "baby steps", complexity can always be added..)
- someone else (you) maintains Java SDK and therefore controls generated sources (JAR or RPM isn't relevant here) - another guy (me) maintains (fictional) Java/GWT SDK that relies on Java SDK + some (supported) customizations - the only way I can impose changes in my SDK is through supported customizations as you control original (Java SDK) sources, i.e. the whole code generation process is driven by your SDK, so my SDK is coupled to your SDK's build/release cycle
that's how things working in software, you always depending on the certain version of the component you're working against, as it expose set of features you need, i don't think that having control over framework features, justifying rewriting the framework ...
I agree with your point. For me, however, putting Java SDK on "starting position" for JavaScript SDK in its initial/prototype phase is just unnecessary complexity. I think that lowest common denominator for Java vs. JavaScript SDK is REST API definition, i.e. definition of the interface. On one hand, building JavaScript SDK on top of Java SDK would mean that Java SDK might get improvements over time as JavaScript SDK evolves. On the other hand, it imposes coupling between these two, which might complicate things in certain situations. For now, I'd like to keep things simple.
(please note that i'm not against the js-sdk, go ahead, this is a nice initiative indeed, i just can't see the business case for not reusing existent infrastructure cause it works for all your needs and eventually both worlds would benefiting from it UI and java-sdk users cause you where extending it with additional capabilities they may also need)
My motivation is avoiding any extra complexity during initial/prototype phase. But I see your point about Java SDK benefiting when used as the base component.
For the sake of simplicity, I guess it's best to start with SDK that has no dependencies whatsoever.
so why won't you rewrite the engine in Java-script? your js-sdk eventually will be depending on it, this way you'll have control over it (and it's features) as well ;-)
No need to rewrite Engine in JavaScript :) there is already a well-defined API, my concern was about being flexible for JavaScript SDK development.
After all, there's no common dependency (aside from running Engine to provide XSD & RSDL) between Java & Python SDK too, if I understand correctly.
In other words, building on top of something existing (just because we can do that) isn't always appropriate/flexible/efficient, it always depends on given context and requirements.
it would be true, if your requirements would make existing infrastructure inappropriate.
Well, I can't tell right now if Java SDK offers everything we "need" in JavaScript SDK because we need to do some prototyping first..
As proposed on the meeting, I think it's best to aim for JavaScript SDK as the lowest common denominator for *any* web application that wants to work with REST API. oVirt GWT-based UI can simply bind to JavaScript SDK, i.e. Java/GWT code that just overlays objects and functions provided by JavaScript SDK. Another reason is ease of maintenance - I'd rather see JavaScript SDK's code generation process to be independent of any other SDK (people responsible for maintaining JavaScript SDK should have full control over generated code).
what do you mean by "people should have full control over generated code"?
It's related to "coupling from SDK perspective" I mentioned above: "the only way I can impose changes in my SDK is through supported customizations as you control original (Java SDK) sources"
if you need additional functionality in java-sdk, you could do the following:
1. submit a patch to java-sdk 2. build new java-sdk locally and use it along with new feature you've added 3. make UI depending on next version of java-sdk (which includes your new feature)
we (and all other SW projects) doing that day by day in engine,api,etc.
(as i mentioned this would also benefit java-sdk users with additional features they might find useful as well)
Yes, I see your point, and agree with that.
(by "people" I meant "JavaScript SDK developers")
Full control means ability to change generated sources in whatever way desired, but assuming the idea of reusing/customizing existing SDK code, aspect of full control is lost in favor of reusing existing code.
i disagree on this one, you have all control you need over java-sdk at any time as it one of indoor projects.
I meant "control" as being "maintainer" of something, sure you can propose patches to Java SDK too, but there can be problematic cases like some feature essential in JavaScript SDK but completely meaningless in Java SDK, etc.
And of course, this assumes that existing code (Java SDK) provides everything we need, which might or might not be the case.
So I just vote for simplicity, generate JavaScript SDK the way like other SDKs (Java/Python) - not trying to reuse anything, just grab XSD & RSDL and generate sources.
the purpose of code generation is to ease maintenance, i.e you/maintainer should not write the feature once it available in api, just run CodeGen and you'll get it for free, but this is zero control over code.
+1
I agree with you on this.
--
Q: What about functionality currently used by oVirt UI but not supported by REST API? (asked by Einav) [For example, fetching VM entity over GWT RPC also returns related data such as Cluster name.]
A: Based on discussion I've had with other colleagues after yesterday's review call, I don't think that separate support-like backend layer is a good idea. Instead, this is the kind of functionality that could be placed in oVirt.js library. Logical operations like "get VMs and related data" would be exposed through oVirt.js (callback-based) API and ultimately realized as multiple physical requests to REST API via JavaScript Binding.
oVirt.js client would be completely oblivious to the fact that multiple physical requests are dispatched. In fact, since HTTP communication is asynchronous in nature, oVirt.js client wouldn't even notice any difference in terms of API consumption. This assumes JavaScript SDK would use callback-based (non-blocking) API instead of blocking one - after all, blocking API on top of non-blocking implementation sounds pretty much like leaky abstraction [1].
For example:
sdk.getVmsWithExtraData( callbackToGetExtraDataForGivenVm, // might cause extra physical requests to REST API callbackFiredWhenAllDataIsReady // update client only when all data is ready )
actually this the main bottleneck in moving UI to work on top of REST, and most interesting/complex part of this project,
Agreed, it's because UI "got used to" using internal backend interface concepts (actions, queries etc.) in the first place.. So we'll have to emulate what we used to use to prevent regressions, maybe improve/refactor in future.
you should think of very wise polling mechanism cause callbacks is a nice thing on paper, but behind the scene it all about polling:
- the entity/s till action got accomplished - add to this updating different grids - running multiple actions - showing events - and obviously much more
IMHO polling is just a workaround and indicates lack of proper notification solution.
Apparently, oVirt web UI isn't some CLI program for which HTTP request/response style is sufficient. oVirt web UI is dynamic, interactive web application that displays/updates data in real time. This is, in my opinion, quite a big difference.
I don't think callbacks are just a nice thing on paper. Callbacks are needed because the underlying communcation is async in nature:
Vojtech, you've got all wrong, i told that you *do need* callbacks, but implementing them only sounds easy, while actually it will be a quite complicated task.
Sorry, I misunderstood. Sure, it can be complicated, we just need to find (reasonably) efficient way to do things :)
- caller invokes API function and provides callback to execute when operation completes -> API is non-blocking - polling attempts to detect change (i.e. operation completed) and notify the caller, so it's also some sort of callback -> this is more complicated compared to simple callback
and don't forget that every polled entity should be marshalled from xml to the javascript entity so at the end, "callbacks" mechanism will be extremely CPU consuming.
First of all, I don't understand how callback mechanism can be CPU consuming, can you please provide some explanation or use case?
of course, you'll have to do per call-back call:
s/"call-back call"/"polling request", e.g:
1 polling task == N * (#1+#2+#3)
[where N is amount of requests you need to perform till desired state is achieved]
Yeah, I call this "async callback chain", but if API doesn't support "atomic" operations based on GUI requirements, we just have to deal with it, one way or another.
i have a solution for this!, will get back to you/publish it when i have a mature design.
1. request/response to the server 2. decompression of data from gzip 3. object mapping (in 99% of cases)
note you'll have a lot of callback consumers that monitoring resource state, waiting for new events.
With callbacks, consumers just provide function to invoke upon given event :) in other words, no need to monitor/check for events.
Does Java SDK provide ability to poll Engine in order to get recent updates, and if yes, why?
i was kinda hoping that you'll add it, but you've chosen to write your own sdk ;-)
Finally, polling makes things stateful, whereas SDK code should be stateless instead. If client wants to get recent updates, it should just use (stateless) SDK code to achieve this goal.
sdk cannot be stateful by definition simply because server is stateless, (also pooling != keeping state, being stateful means that you save data for request on server side)
Hm, I'm wondering what others think, but for me, stateless > stateful. Keeping any sort of state causes extra complexity due to access/management of such shared state.. I'd rather avoid introducing state into SDK if possible, and delegate the responsibility of keeping state to SDK consumer (client) code.
[1] http://en.wikipedia.org/wiki/Leaky_abstraction
--
Last but not least, where to maintain JavaScript SDK projects: low-level JavaScript Binding + high-level oVirt.js library.
I agree that conceptually both above mentioned projects should go into dedicated "ovirt-engine-sdk-js" git repository and have their own build/release process. However, for now, we're just making baby steps so let's keep things simple and prototype these projects as part of "ovirt-engine" git repository.
... we can complicate things anytime, but we should know that any complex system that works has inevitably evolved from simple system that works ... (quote from http://en.wikipedia.org/wiki/Gall%27s_law)
Regards, Vojtech _______________________________________________ Engine-devel mailing list Engine-devel@ovirt.org http://lists.ovirt.org/mailman/listinfo/engine-devel
--
Michael Pasternak RedHat, ENG-Virtualization R&D
--
Michael Pasternak RedHat, ENG-Virtualization R&D
participants (4)
-
Antoni Segura Puimedon -
Itamar Heim -
Michael Pasternak -
Vojtech Szocs