2:20 a.m.
Hi All,
for quite some time I have been more or less involved in development of
various UIs for oVirt based entirely on the oVirt's REST API ranging from
the quite mature moVirt [1] through some cockpit extensions to a young and
experimental user portal replacement [2].
One issue we hit over and over again is the missing data aggregation. In
the 3.x era we used to use in moVirt the detail=something
api to get the disks and nics of the VM, something like:
GET /ovirt-engine/api/vms
Accept: application/json; detail=disks
This allowed us to store this data in local database leading to great user
experience. Since this feature has been removed in 4.x API [3]
we needed to retire to a different solution. When the VM detail is selected
by the user, start loading the disks and nics and hope the user
will not be fast enough to see the delay. The UX is slightly worse bug
kinda acceptable.
We hit this issue harder in the new user portal [2], because we already
have the VM cached and show the whole VM in one screen. So, if you pick it,
you will get it's details immediately.
But, since you don't have all the details, we need to do an additional call
(two actually) to load this data and they start to appear later.
So, something which would be very fast and smooth starts to feel sluggish.
Recently, we hit this issue again which forced us to sacrifice the UX even
more - it is the "console in use" feature of user portal.
The use case is this:
- if the console is already taken by some user, there are complications if
other current user tryes to take it as well (will avoid details about
settings and permissins involved, but long story short, the user will
probably not be allowed to connect to it. The "probably" is the key here
since we can not do any intelligent decision in advance, we can only warn
the user that the console is taken).
- in the current GWT user portal, if the VM's console is taken, it is shown
on the VM's "box" that "console is taken". This was a highly
requested
feature
- to get this information using the current REST API, we need to go to the
/vms/<vmid>/sessions subcollection. To get this for all VMs, it would be
doing N queries per poll which we can not afford
- so the current PR [4] will probably end up to only check it on the
attempt to connect to the console warning the user. Maybe it will be also
shown in Vm details. But the UX in case the user will look for a VM which
has free console will suffer significantly (e.g. try one by one until some
opens or look at details one by one to see if the warning appears (with a
delay))
I understand that embedding the details of the VM to the response comes
with a cost, namely:
- performance hit
- complexity of the API code
- the "cleanness" of REST suffers
But I think we should seriously consider to provide some option to data
aggregation.
I know this has been discussed many times with no result, but I think it is
time to bring this topic up again. I'll try to summarize the (failed)
attempts tried so far:
- the detail=<something> parameter with ad-hoc embedding of data. This has
been there and removed in 4.0 [3]
- the DoctorREST project - e.g. a proxy above the current api. The idea was
to create a service which will be independent of the engine itself, will
locally poll the engine's REST, store all data in local (mongo)DB and
provide a rich api with aggregations and projections and push
notifications. This polling of everything to get the data to DoctorREST
proved to be pretty costy, so also a more invasive approach of pushing data
from engine to doctor has been discused [5]. None of this two approaches
have been accepted (too complicated, too invasive).
- writing some custom ad-hoc servlet serving only a purpose of one frontend
- this is actually there for the dashboard, but it is not a generic
solution for the other frontends and we really should not develop custom
"APIs" for every frontend
- there were some other proposals discussed (some 3th party solutions etc)
but I think none of them made it even to a PoC
So, now I would try again and try small to get at least some benefit. I see
2 paths we could try:
1: embed something which burns us immediatly, e.g. the /sessions into VMs.
I really liked the ;detail=sessions approach, could we move it back?
2: add some tiny service which would just accept a list of queries, execute
them locally (but using real HTTP requests) and return in one bulk. A naive
implementation just to give a sense of what I mean of this would be a shell
script getting list of strings like "
https://localhost/ovirt-engine/api/vms/123/sessions" iterate over them and
do a curl request for each, mangle the results into one string and return
(credits for this idea to msivak). Easy to implement, possibility to add
also projections later to save some bandwidth. But the API would anyway be
hammered by bunch of queries, only the network roundtrip would be saved.
3: any other simple approaches?
I honestly prefer the first approach. It is not beautiful, it is not
REST-ful, but it is easy to implement, very pragmatic and useful.
What do you think?
Thank you and sorry for the long mail :)
Tomas
[1]: https://github.com/oVirt/moVirt
[2]: https://github.com/oVirt/ovirt-web-ui
[3]: https://gerrit.ovirt.org/#/c/61260
[4]: https://github.com/oVirt/ovirt-web-ui/pull/106/
[5]: https://gerrit.ovirt.org/#/c/45233/
2:54 a.m.
for quite some time I have been more or less involved in development
of
various UIs for oVirt based entirely on the oVirt's REST API ranging from
the quite mature moVirt [1] through some cockpit extensions to a young and
experimental user portal replacement [2].
oVirt optimizer has the same issue..
2: add some tiny service which would just accept a list of queries,
execute
them locally (but using real HTTP requests) and return in one bulk. A naive
implementation just to give a sense of what I mean of this would be a shell
script getting list of strings like
"https://localhost/ovirt-engine/api/vms/123/sessions" iterate over them and
do a curl request for each, mangle the results into one string and return
(credits for this idea to msivak). Easy to implement, possibility to add
also projections later to save some bandwidth. But the API would anyway be
hammered by bunch of queries, only the network roundtrip would be saved.
The biggest cost for (especially mobile) clients is the cost of
establishing new SSL connection. SSL is also pretty expensive on the
server side.
So running the aggregation service on the ovirt-engine machine (behind
Apache) means the client will do a single SSL request with list of N
urls and the local "reverse-proxy" will perform single authentication
and N plain HTTP requests (or even better - AJP). It won't remove any
time from the actual command run time, but it will reduce protocol
overhead.
I think this is the simplest first step that requires almost no change
to existing infrastructure.
--
Martin Sivak
SLA / oVirt
On Fri, Mar 24, 2017 at 10:20 AM, Tomas Jelinek <tjelinek(a)redhat.com> wrote:
> Hi All,
>
for quite some time I have been more or less involved in development
of
various UIs for oVirt based entirely on the oVirt's REST API ranging from
the quite mature moVirt [1] through some cockpit extensions to a young and
experimental user portal replacement [2].
>
> One issue we hit over and over again is the missing data aggregation. In the
> 3.x era we used to use in moVirt the detail=something
> api to get the disks and nics of the VM, something like:
>
> GET /ovirt-engine/api/vms
> Accept: application/json; detail=disks
>
> This allowed us to store this data in local database leading to great user
> experience. Since this feature has been removed in 4.x API [3]
> we needed to retire to a different solution. When the VM detail is selected
> by the user, start loading the disks and nics and hope the user
> will not be fast enough to see the delay. The UX is slightly worse bug kinda
> acceptable.
>
> We hit this issue harder in the new user portal [2], because we already have
> the VM cached and show the whole VM in one screen. So, if you pick it, you
> will get it's details immediately.
> But, since you don't have all the details, we need to do an additional call
> (two actually) to load this data and they start to appear later.
> So, something which would be very fast and smooth starts to feel sluggish.
>
> Recently, we hit this issue again which forced us to sacrifice the UX even
> more - it is the "console in use" feature of user portal.
> The use case is this:
> - if the console is already taken by some user, there are complications if
> other current user tryes to take it as well (will avoid details about
> settings and permissins involved, but long story short, the user will
> probably not be allowed to connect to it. The "probably" is the key here
> since we can not do any intelligent decision in advance, we can only warn
> the user that the console is taken).
> - in the current GWT user portal, if the VM's console is taken, it is shown
> on the VM's "box" that "console is taken". This was a highly
requested
> feature
> - to get this information using the current REST API, we need to go to the
> /vms/<vmid>/sessions subcollection. To get this for all VMs, it would be
> doing N queries per poll which we can not afford
> - so the current PR [4] will probably end up to only check it on the attempt
> to connect to the console warning the user. Maybe it will be also shown in
> Vm details. But the UX in case the user will look for a VM which has free
> console will suffer significantly (e.g. try one by one until some opens or
> look at details one by one to see if the warning appears (with a delay))
>
> I understand that embedding the details of the VM to the response comes with
> a cost, namely:
> - performance hit
> - complexity of the API code
> - the "cleanness" of REST suffers
>
> But I think we should seriously consider to provide some option to data
> aggregation.
>
> I know this has been discussed many times with no result, but I think it is
> time to bring this topic up again. I'll try to summarize the (failed)
> attempts tried so far:
> - the detail=<something> parameter with ad-hoc embedding of data. This has
> been there and removed in 4.0 [3]
> - the DoctorREST project - e.g. a proxy above the current api. The idea was
> to create a service which will be independent of the engine itself, will
> locally poll the engine's REST, store all data in local (mongo)DB and
> provide a rich api with aggregations and projections and push notifications.
> This polling of everything to get the data to DoctorREST proved to be pretty
> costy, so also a more invasive approach of pushing data from engine to
> doctor has been discused [5]. None of this two approaches have been accepted
> (too complicated, too invasive).
> - writing some custom ad-hoc servlet serving only a purpose of one frontend
> - this is actually there for the dashboard, but it is not a generic solution
> for the other frontends and we really should not develop custom "APIs" for
> every frontend
> - there were some other proposals discussed (some 3th party solutions etc)
> but I think none of them made it even to a PoC
>
> So, now I would try again and try small to get at least some benefit. I see
> 2 paths we could try:
> 1: embed something which burns us immediatly, e.g. the /sessions into VMs. I
> really liked the ;detail=sessions approach, could we move it back?
2: add some tiny service which would just accept a list of queries,
execute
them locally (but using real HTTP requests) and return in one bulk. A naive
implementation just to give a sense of what I mean of this would be a shell
script getting list of strings like
"https://localhost/ovirt-engine/api/vms/123/sessions" iterate over them and
do a curl request for each, mangle the results into one string and return
(credits for this idea to msivak). Easy to implement, possibility to add
also projections later to save some bandwidth. But the API would anyway be
hammered by bunch of queries, only the network roundtrip would be saved.
> 3:
any other simple approaches?
>
> I honestly prefer the first approach. It is not beautiful, it is not
> REST-ful, but it is easy to implement, very pragmatic and useful.
> What do you think?
>
> Thank you and sorry for the long mail :)
> Tomas
>
> [1]: https://github.com/oVirt/moVirt
> [2]: https://github.com/oVirt/ovirt-web-ui
> [3]: https://gerrit.ovirt.org/#/c/61260
> [4]: https://github.com/oVirt/ovirt-web-ui/pull/106/
> [5]: https://gerrit.ovirt.org/#/c/45233/
>
>
> _______________________________________________
> Devel mailing list
> Devel(a)ovirt.org
> http://lists.ovirt.org/mailman/listinfo/devel
7:47 a.m.
I feel like every REST API I've ever worked with has had the aggregation +
projection problem. It's like we're trying to use REST as a replacement for
SQL -- but the logic that executes the "SQL" lives in a browser now, and it
used to live on a server close to the DB. And REST isn't expressive for
selecting data like SQL is.
There must be some industry solution to this "I want to do SQL over REST"
problem.
On Fri, Mar 24, 2017 at 5:54 AM, Martin Sivak <msivak(a)redhat.com> wrote:
> for quite some time I have been more or less involved in
development of
> various UIs for oVirt based entirely on the oVirt's REST API ranging from
> the quite mature moVirt [1] through some cockpit extensions to a young
and
> experimental user portal replacement [2].
oVirt optimizer has the same issue..
> 2: add some tiny service which would just accept a list of queries,
execute
> them locally (but using real HTTP requests) and return in one bulk. A
naive
> implementation just to give a sense of what I mean of this would be a
shell
> script getting list of strings like
> "https://localhost/ovirt-engine/api/vms/123/sessions" iterate over them
and
> do a curl request for each, mangle the results into one string and return
> (credits for this idea to msivak). Easy to implement, possibility to add
> also projections later to save some bandwidth. But the API would anyway
be
> hammered by bunch of queries, only the network roundtrip would be saved.
The biggest cost for (especially mobile) clients is the cost of
establishing new SSL connection. SSL is also pretty expensive on the
server side.
So running the aggregation service on the ovirt-engine machine (behind
Apache) means the client will do a single SSL request with list of N
urls and the local "reverse-proxy" will perform single authentication
and N plain HTTP requests (or even better - AJP). It won't remove any
time from the actual command run time, but it will reduce protocol
overhead.
I think this is the simplest first step that requires almost no change
to existing infrastructure.
--
Martin Sivak
SLA / oVirt
On Fri, Mar 24, 2017 at 10:20 AM, Tomas Jelinek <tjelinek(a)redhat.com>
wrote:
> Hi All,
>
> for quite some time I have been more or less involved in development of
> various UIs for oVirt based entirely on the oVirt's REST API ranging from
> the quite mature moVirt [1] through some cockpit extensions to a young
and
> experimental user portal replacement [2].
>
> One issue we hit over and over again is the missing data aggregation. In
the
> 3.x era we used to use in moVirt the detail=something
> api to get the disks and nics of the VM, something like:
>
> GET /ovirt-engine/api/vms
> Accept: application/json; detail=disks
>
> This allowed us to store this data in local database leading to great
user
> experience. Since this feature has been removed in 4.x API [3]
> we needed to retire to a different solution. When the VM detail is
selected
> by the user, start loading the disks and nics and hope the user
> will not be fast enough to see the delay. The UX is slightly worse bug
kinda
> acceptable.
>
> We hit this issue harder in the new user portal [2], because we already
have
> the VM cached and show the whole VM in one screen. So, if you pick it,
you
> will get it's details immediately.
> But, since you don't have all the details, we need to do an additional
call
> (two actually) to load this data and they start to appear later.
> So, something which would be very fast and smooth starts to feel
sluggish.
>
> Recently, we hit this issue again which forced us to sacrifice the UX
even
> more - it is the "console in use" feature of user portal.
> The use case is this:
> - if the console is already taken by some user, there are complications
if
> other current user tryes to take it as well (will avoid details about
> settings and permissins involved, but long story short, the user will
> probably not be allowed to connect to it. The "probably" is the key here
> since we can not do any intelligent decision in advance, we can only warn
> the user that the console is taken).
> - in the current GWT user portal, if the VM's console is taken, it is
shown
> on the VM's "box" that "console is taken". This was a highly
requested
> feature
> - to get this information using the current REST API, we need to go to
the
> /vms/<vmid>/sessions subcollection. To get this for all VMs, it would be
> doing N queries per poll which we can not afford
> - so the current PR [4] will probably end up to only check it on the
attempt
> to connect to the console warning the user. Maybe it will be also shown
in
> Vm details. But the UX in case the user will look for a VM which has free
> console will suffer significantly (e.g. try one by one until some opens
or
> look at details one by one to see if the warning appears (with a delay))
>
> I understand that embedding the details of the VM to the response comes
with
> a cost, namely:
> - performance hit
> - complexity of the API code
> - the "cleanness" of REST suffers
>
> But I think we should seriously consider to provide some option to data
> aggregation.
>
> I know this has been discussed many times with no result, but I think it
is
> time to bring this topic up again. I'll try to summarize the (failed)
> attempts tried so far:
> - the detail=<something> parameter with ad-hoc embedding of data. This
has
> been there and removed in 4.0 [3]
> - the DoctorREST project - e.g. a proxy above the current api. The idea
was
> to create a service which will be independent of the engine itself, will
> locally poll the engine's REST, store all data in local (mongo)DB and
> provide a rich api with aggregations and projections and push
notifications.
> This polling of everything to get the data to DoctorREST proved to be
pretty
> costy, so also a more invasive approach of pushing data from engine to
> doctor has been discused [5]. None of this two approaches have been
accepted
> (too complicated, too invasive).
> - writing some custom ad-hoc servlet serving only a purpose of one
frontend
> - this is actually there for the dashboard, but it is not a generic
solution
> for the other frontends and we really should not develop custom "APIs"
for
> every frontend
> - there were some other proposals discussed (some 3th party solutions
etc)
> but I think none of them made it even to a PoC
>
> So, now I would try again and try small to get at least some benefit. I
see
> 2 paths we could try:
> 1: embed something which burns us immediatly, e.g. the /sessions into
VMs. I
> really liked the ;detail=sessions approach, could we move it back?
> 2: add some tiny service which would just accept a list of queries,
execute
> them locally (but using real HTTP requests) and return in one bulk. A
naive
> implementation just to give a sense of what I mean of this would be a
shell
> script getting list of strings like
> "https://localhost/ovirt-engine/api/vms/123/sessions" iterate over them
and
> do a curl request for each, mangle the results into one string and return
> (credits for this idea to msivak). Easy to implement, possibility to add
> also projections later to save some bandwidth. But the API would anyway
be
> hammered by bunch of queries, only the network roundtrip would be saved.
> 3: any other simple approaches?
>
> I honestly prefer the first approach. It is not beautiful, it is not
> REST-ful, but it is easy to implement, very pragmatic and useful.
> What do you think?
>
> Thank you and sorry for the long mail :)
> Tomas
>
> [1]: https://github.com/oVirt/moVirt
> [2]: https://github.com/oVirt/ovirt-web-ui
> [3]: https://gerrit.ovirt.org/#/c/61260
> [4]: https://github.com/oVirt/ovirt-web-ui/pull/106/
> [5]: https://gerrit.ovirt.org/#/c/45233/
>
>
> _______________________________________________
> Devel mailing list
> Devel(a)ovirt.org
> http://lists.ovirt.org/mailman/listinfo/devel
_______________________________________________
Devel mailing list
Devel(a)ovirt.org
http://lists.ovirt.org/mailman/listinfo/devel
--
Greg Sheremeta, MBA
Red Hat, Inc.
Sr. Software Engineer
gshereme(a)redhat.com
7:58 a.m.
I feel like every REST API I've ever worked with has had the
aggregation +
projection problem. It's like we're trying to use REST as a replacement for
SQL -- but the logic that executes the "SQL" lives in a browser now, and it
used to live on a server close to the DB. And REST isn't expressive for
selecting data like SQL is.
The current industry solution I know about is called API gateway..
most of the big players have internal API with lots of low level stuff
and then couple of external API gateways tailored to what the client
needs.
http://microservices.io/patterns/apigateway.html (check the backend
for frontend section)
This trend is also visible when you think about services that offer
API gateway management and billing like
https://aws.amazon.com/api-gateway/ or our very own
https://www.3scale.net/
Martin
On Fri, Mar 24, 2017 at 3:47 PM, Greg Sheremeta <gshereme(a)redhat.com> wrote:
I feel like every REST API I've ever worked with has had the
aggregation +
projection problem. It's like we're trying to use REST as a replacement for
SQL -- but the logic that executes the "SQL" lives in a browser now, and it
used to live on a server close to the DB. And REST isn't expressive for
selecting data like SQL is.
>
> There must be some industry solution to this "I want to do SQL over REST"
> problem.
>
> On Fri, Mar 24, 2017 at 5:54 AM, Martin Sivak <msivak(a)redhat.com> wrote:
>>
>> > for quite some time I have been more or less involved in development of
>> > various UIs for oVirt based entirely on the oVirt's REST API ranging
>> > from
>> > the quite mature moVirt [1] through some cockpit extensions to a young
>> > and
>> > experimental user portal replacement [2].
>>
>> oVirt optimizer has the same issue..
>>
>> > 2: add some tiny service which would just accept a list of queries,
>> > execute
>> > them locally (but using real HTTP requests) and return in one bulk. A
>> > naive
>> > implementation just to give a sense of what I mean of this would be a
>> > shell
>> > script getting list of strings like
>> > "https://localhost/ovirt-engine/api/vms/123/sessions" iterate over
them
>> > and
>> > do a curl request for each, mangle the results into one string and
>> > return
>> > (credits for this idea to msivak). Easy to implement, possibility to add
>> > also projections later to save some bandwidth. But the API would anyway
>> > be
>> > hammered by bunch of queries, only the network roundtrip would be saved.
>>
>> The biggest cost for (especially mobile) clients is the cost of
>> establishing new SSL connection. SSL is also pretty expensive on the
>> server side.
>>
>> So running the aggregation service on the ovirt-engine machine (behind
>> Apache) means the client will do a single SSL request with list of N
>> urls and the local "reverse-proxy" will perform single authentication
>> and N plain HTTP requests (or even better - AJP). It won't remove any
>> time from the actual command run time, but it will reduce protocol
>> overhead.
>>
>> I think this is the simplest first step that requires almost no change
>> to existing infrastructure.
>>
>> --
>> Martin Sivak
>> SLA / oVirt
>>
>> On Fri, Mar 24, 2017 at 10:20 AM, Tomas Jelinek <tjelinek(a)redhat.com>
>> wrote:
>> > Hi All,
>> >
>> > for quite some time I have been more or less involved in development of
>> > various UIs for oVirt based entirely on the oVirt's REST API ranging
>> > from
>> > the quite mature moVirt [1] through some cockpit extensions to a young
>> > and
>> > experimental user portal replacement [2].
>> >
>> > One issue we hit over and over again is the missing data aggregation. In
>> > the
>> > 3.x era we used to use in moVirt the detail=something
>> > api to get the disks and nics of the VM, something like:
>> >
>> > GET /ovirt-engine/api/vms
>> > Accept: application/json; detail=disks
>> >
>> > This allowed us to store this data in local database leading to great
>> > user
>> > experience. Since this feature has been removed in 4.x API [3]
>> > we needed to retire to a different solution. When the VM detail is
>> > selected
>> > by the user, start loading the disks and nics and hope the user
>> > will not be fast enough to see the delay. The UX is slightly worse bug
>> > kinda
>> > acceptable.
>> >
>> > We hit this issue harder in the new user portal [2], because we already
>> > have
>> > the VM cached and show the whole VM in one screen. So, if you pick it,
>> > you
>> > will get it's details immediately.
>> > But, since you don't have all the details, we need to do an additional
>> > call
>> > (two actually) to load this data and they start to appear later.
>> > So, something which would be very fast and smooth starts to feel
>> > sluggish.
>> >
>> > Recently, we hit this issue again which forced us to sacrifice the UX
>> > even
>> > more - it is the "console in use" feature of user portal.
>> > The use case is this:
>> > - if the console is already taken by some user, there are complications
>> > if
>> > other current user tryes to take it as well (will avoid details about
>> > settings and permissins involved, but long story short, the user will
>> > probably not be allowed to connect to it. The "probably" is the
key here
>> > since we can not do any intelligent decision in advance, we can only
>> > warn
>> > the user that the console is taken).
>> > - in the current GWT user portal, if the VM's console is taken, it is
>> > shown
>> > on the VM's "box" that "console is taken". This was
a highly requested
>> > feature
>> > - to get this information using the current REST API, we need to go to
>> > the
>> > /vms/<vmid>/sessions subcollection. To get this for all VMs, it would
be
>> > doing N queries per poll which we can not afford
>> > - so the current PR [4] will probably end up to only check it on the
>> > attempt
>> > to connect to the console warning the user. Maybe it will be also shown
>> > in
>> > Vm details. But the UX in case the user will look for a VM which has
>> > free
>> > console will suffer significantly (e.g. try one by one until some opens
>> > or
>> > look at details one by one to see if the warning appears (with a delay))
>> >
>> > I understand that embedding the details of the VM to the response comes
>> > with
>> > a cost, namely:
>> > - performance hit
>> > - complexity of the API code
>> > - the "cleanness" of REST suffers
>> >
>> > But I think we should seriously consider to provide some option to data
>> > aggregation.
>> >
>> > I know this has been discussed many times with no result, but I think it
>> > is
>> > time to bring this topic up again. I'll try to summarize the (failed)
>> > attempts tried so far:
>> > - the detail=<something> parameter with ad-hoc embedding of data.
This
>> > has
>> > been there and removed in 4.0 [3]
>> > - the DoctorREST project - e.g. a proxy above the current api. The idea
>> > was
>> > to create a service which will be independent of the engine itself, will
>> > locally poll the engine's REST, store all data in local (mongo)DB and
>> > provide a rich api with aggregations and projections and push
>> > notifications.
>> > This polling of everything to get the data to DoctorREST proved to be
>> > pretty
>> > costy, so also a more invasive approach of pushing data from engine to
>> > doctor has been discused [5]. None of this two approaches have been
>> > accepted
>> > (too complicated, too invasive).
>> > - writing some custom ad-hoc servlet serving only a purpose of one
>> > frontend
>> > - this is actually there for the dashboard, but it is not a generic
>> > solution
>> > for the other frontends and we really should not develop custom
"APIs"
>> > for
>> > every frontend
>> > - there were some other proposals discussed (some 3th party solutions
>> > etc)
>> > but I think none of them made it even to a PoC
>> >
>> > So, now I would try again and try small to get at least some benefit. I
>> > see
>> > 2 paths we could try:
>> > 1: embed something which burns us immediatly, e.g. the /sessions into
>> > VMs. I
>> > really liked the ;detail=sessions approach, could we move it back?
>> > 2: add some tiny service which would just accept a list of queries,
>> > execute
>> > them locally (but using real HTTP requests) and return in one bulk. A
>> > naive
>> > implementation just to give a sense of what I mean of this would be a
>> > shell
>> > script getting list of strings like
>> > "https://localhost/ovirt-engine/api/vms/123/sessions" iterate over
them
>> > and
>> > do a curl request for each, mangle the results into one string and
>> > return
>> > (credits for this idea to msivak). Easy to implement, possibility to add
>> > also projections later to save some bandwidth. But the API would anyway
>> > be
>> > hammered by bunch of queries, only the network roundtrip would be saved.
>> > 3: any other simple approaches?
>> >
>> > I honestly prefer the first approach. It is not beautiful, it is not
>> > REST-ful, but it is easy to implement, very pragmatic and useful.
>> > What do you think?
>> >
>> > Thank you and sorry for the long mail :)
>> > Tomas
>> >
>> > [1]: https://github.com/oVirt/moVirt
>> > [2]: https://github.com/oVirt/ovirt-web-ui
>> > [3]: https://gerrit.ovirt.org/#/c/61260
>> > [4]: https://github.com/oVirt/ovirt-web-ui/pull/106/
>> > [5]: https://gerrit.ovirt.org/#/c/45233/
>> >
>> >
>> > _______________________________________________
>> > Devel mailing list
>> > Devel(a)ovirt.org
>> > http://lists.ovirt.org/mailman/listinfo/devel
>> _______________________________________________
>> Devel mailing list
>> Devel(a)ovirt.org
>> http://lists.ovirt.org/mailman/listinfo/devel
>
>
>
>
> --
> Greg Sheremeta, MBA
> Red Hat, Inc.
> Sr. Software Engineer
> gshereme(a)redhat.com
8:22 a.m.
On Fri, Mar 24, 2017 at 3:58 PM, Martin Sivak <msivak(a)redhat.com> wrote:
> I feel like every REST API I've ever worked with has had the
aggregation
+
> projection problem. It's like we're trying to use REST as a replacement
for
> SQL -- but the logic that executes the "SQL" lives in a browser now, and
it
> used to live on a server close to the DB. And REST isn't expressive for
> selecting data like SQL is.
The current industry solution I know about is called API gateway..
most of the big players have internal API with lots of low level stuff
and then couple of external API gateways tailored to what the client
needs.
http://microservices.io/patterns/apigateway.html (check the backend
for frontend section)
This trend is also visible when you think about services that offer
API gateway management and billing like
https://aws.amazon.com/api-gateway/ or our very own
https://www.3scale.net/
right, but the api gateway solves 2 problems:
1: if you have a microservice architecture it is hard for frontend to talk
to 20 different moving services. So the gateway hides this complexity
behind it. This is not the problem we have.
2: you can have more api gateways (e.g. more apis) tailored for every
frontend. I don't think we need this - the current API serves us pretty
well in every FE Im involved in. The only thing which I miss is the data
aggregation.
So I would go back to the original topic of this thread - do some small
change which has a chance to be merged to the project and helps us where it
hurts.
Martin
On Fri, Mar 24, 2017 at 3:47 PM, Greg Sheremeta <gshereme(a)redhat.com>
wrote:
> I feel like every REST API I've ever worked with has had the aggregation
+
> projection problem. It's like we're trying to use REST as a replacement
for
> SQL -- but the logic that executes the "SQL" lives in a browser now, and
it
> used to live on a server close to the DB. And REST isn't expressive for
> selecting data like SQL is.
>
> There must be some industry solution to this "I want to do SQL over REST"
> problem.
>
> On Fri, Mar 24, 2017 at 5:54 AM, Martin Sivak <msivak(a)redhat.com> wrote:
>>
>> > for quite some time I have been more or less involved in development
of
>> > various UIs for oVirt based entirely on the oVirt's REST API ranging
>> > from
>> > the quite mature moVirt [1] through some cockpit extensions to a young
>> > and
>> > experimental user portal replacement [2].
>>
>> oVirt optimizer has the same issue..
>>
>> > 2: add some tiny service which would just accept a list of queries,
>> > execute
>> > them locally (but using real HTTP requests) and return in one bulk. A
>> > naive
>> > implementation just to give a sense of what I mean of this would be a
>> > shell
>> > script getting list of strings like
>> > "https://localhost/ovirt-engine/api/vms/123/sessions" iterate
over
them
>> > and
>> > do a curl request for each, mangle the results into one string and
>> > return
>> > (credits for this idea to msivak). Easy to implement, possibility to
add
>> > also projections later to save some bandwidth. But the API would
anyway
>> > be
>> > hammered by bunch of queries, only the network roundtrip would be
saved.
>>
>> The biggest cost for (especially mobile) clients is the cost of
>> establishing new SSL connection. SSL is also pretty expensive on the
>> server side.
>>
>> So running the aggregation service on the ovirt-engine machine (behind
>> Apache) means the client will do a single SSL request with list of N
>> urls and the local "reverse-proxy" will perform single authentication
>> and N plain HTTP requests (or even better - AJP). It won't remove any
>> time from the actual command run time, but it will reduce protocol
>> overhead.
>>
>> I think this is the simplest first step that requires almost no change
>> to existing infrastructure.
>>
>> --
>> Martin Sivak
>> SLA / oVirt
>>
>> On Fri, Mar 24, 2017 at 10:20 AM, Tomas Jelinek <tjelinek(a)redhat.com>
>> wrote:
>> > Hi All,
>> >
>> > for quite some time I have been more or less involved in development
of
>> > various UIs for oVirt based entirely on the oVirt's REST API ranging
>> > from
>> > the quite mature moVirt [1] through some cockpit extensions to a young
>> > and
>> > experimental user portal replacement [2].
>> >
>> > One issue we hit over and over again is the missing data aggregation.
In
>> > the
>> > 3.x era we used to use in moVirt the detail=something
>> > api to get the disks and nics of the VM, something like:
>> >
>> > GET /ovirt-engine/api/vms
>> > Accept: application/json; detail=disks
>> >
>> > This allowed us to store this data in local database leading to great
>> > user
>> > experience. Since this feature has been removed in 4.x API [3]
>> > we needed to retire to a different solution. When the VM detail is
>> > selected
>> > by the user, start loading the disks and nics and hope the user
>> > will not be fast enough to see the delay. The UX is slightly worse bug
>> > kinda
>> > acceptable.
>> >
>> > We hit this issue harder in the new user portal [2], because we
already
>> > have
>> > the VM cached and show the whole VM in one screen. So, if you pick it,
>> > you
>> > will get it's details immediately.
>> > But, since you don't have all the details, we need to do an additional
>> > call
>> > (two actually) to load this data and they start to appear later.
>> > So, something which would be very fast and smooth starts to feel
>> > sluggish.
>> >
>> > Recently, we hit this issue again which forced us to sacrifice the UX
>> > even
>> > more - it is the "console in use" feature of user portal.
>> > The use case is this:
>> > - if the console is already taken by some user, there are
complications
>> > if
>> > other current user tryes to take it as well (will avoid details about
>> > settings and permissins involved, but long story short, the user will
>> > probably not be allowed to connect to it. The "probably" is the
key
here
>> > since we can not do any intelligent decision in advance, we can only
>> > warn
>> > the user that the console is taken).
>> > - in the current GWT user portal, if the VM's console is taken, it is
>> > shown
>> > on the VM's "box" that "console is taken". This was
a highly requested
>> > feature
>> > - to get this information using the current REST API, we need to go to
>> > the
>> > /vms/<vmid>/sessions subcollection. To get this for all VMs, it
would
be
>> > doing N queries per poll which we can not afford
>> > - so the current PR [4] will probably end up to only check it on the
>> > attempt
>> > to connect to the console warning the user. Maybe it will be also
shown
>> > in
>> > Vm details. But the UX in case the user will look for a VM which has
>> > free
>> > console will suffer significantly (e.g. try one by one until some
opens
>> > or
>> > look at details one by one to see if the warning appears (with a
delay))
>> >
>> > I understand that embedding the details of the VM to the response
comes
>> > with
>> > a cost, namely:
>> > - performance hit
>> > - complexity of the API code
>> > - the "cleanness" of REST suffers
>> >
>> > But I think we should seriously consider to provide some option to
data
>> > aggregation.
>> >
>> > I know this has been discussed many times with no result, but I think
it
>> > is
>> > time to bring this topic up again. I'll try to summarize the (failed)
>> > attempts tried so far:
>> > - the detail=<something> parameter with ad-hoc embedding of data.
This
>> > has
>> > been there and removed in 4.0 [3]
>> > - the DoctorREST project - e.g. a proxy above the current api. The
idea
>> > was
>> > to create a service which will be independent of the engine itself,
will
>> > locally poll the engine's REST, store all data in local (mongo)DB and
>> > provide a rich api with aggregations and projections and push
>> > notifications.
>> > This polling of everything to get the data to DoctorREST proved to be
>> > pretty
>> > costy, so also a more invasive approach of pushing data from engine to
>> > doctor has been discused [5]. None of this two approaches have been
>> > accepted
>> > (too complicated, too invasive).
>> > - writing some custom ad-hoc servlet serving only a purpose of one
>> > frontend
>> > - this is actually there for the dashboard, but it is not a generic
>> > solution
>> > for the other frontends and we really should not develop custom
"APIs"
>> > for
>> > every frontend
>> > - there were some other proposals discussed (some 3th party solutions
>> > etc)
>> > but I think none of them made it even to a PoC
>> >
>> > So, now I would try again and try small to get at least some benefit.
I
>> > see
>> > 2 paths we could try:
>> > 1: embed something which burns us immediatly, e.g. the /sessions into
>> > VMs. I
>> > really liked the ;detail=sessions approach, could we move it back?
>> > 2: add some tiny service which would just accept a list of queries,
>> > execute
>> > them locally (but using real HTTP requests) and return in one bulk. A
>> > naive
>> > implementation just to give a sense of what I mean of this would be a
>> > shell
>> > script getting list of strings like
>> > "https://localhost/ovirt-engine/api/vms/123/sessions" iterate
over
them
>> > and
>> > do a curl request for each, mangle the results into one string and
>> > return
>> > (credits for this idea to msivak). Easy to implement, possibility to
add
>> > also projections later to save some bandwidth. But the API would
anyway
>> > be
>> > hammered by bunch of queries, only the network roundtrip would be
saved.
>> > 3: any other simple approaches?
>> >
>> > I honestly prefer the first approach. It is not beautiful, it is not
>> > REST-ful, but it is easy to implement, very pragmatic and useful.
>> > What do you think?
>> >
>> > Thank you and sorry for the long mail :)
>> > Tomas
>> >
>> > [1]: https://github.com/oVirt/moVirt
>> > [2]: https://github.com/oVirt/ovirt-web-ui
>> > [3]: https://gerrit.ovirt.org/#/c/61260
>> > [4]: https://github.com/oVirt/ovirt-web-ui/pull/106/
>> > [5]: https://gerrit.ovirt.org/#/c/45233/
>> >
>> >
>> > _______________________________________________
>> > Devel mailing list
>> > Devel(a)ovirt.org
>> > http://lists.ovirt.org/mailman/listinfo/devel
>> _______________________________________________
>> Devel mailing list
>> Devel(a)ovirt.org
>> http://lists.ovirt.org/mailman/listinfo/devel
>
>
>
>
> --
> Greg Sheremeta, MBA
> Red Hat, Inc.
> Sr. Software Engineer
> gshereme(a)redhat.com
8:43 a.m.
2: you can have more api gateways (e.g. more apis) tailored for
every
frontend. I don't think we need this - the current API serves us pretty well
in every FE Im involved in. The only thing which I miss is the data
aggregation.
So it does not serve us well. Aggregation of data is one the usual
points of using the gateway.
Yes microservices are affected by this indeed, but so are we because
implementing the aggregation directly in the current engine API layer
is hard.
So I would go back to the original topic of this thread - do some
small
change which has a chance to be merged to the project and helps us where it
hurts.
Can a simple HTTP/2 to HTTP/AJP gateway be the simplest solution? Our
Apache might even have a module for it already.
That way you can multiplex all the REST calls using a single tcp
connection (and a single SSL negotiation).
A custom SSO enabled service like that might be even better as it
would be able to skip the authentication
layers too and that would lower the engine load. But I am not sure it
is possible with the current codebase.
Martin
On Fri, Mar 24, 2017 at 4:22 PM, Tomas Jelinek <tjelinek(a)redhat.com> wrote:
>
>
> On Fri, Mar 24, 2017 at 3:58 PM, Martin Sivak <msivak(a)redhat.com> wrote:
>>
>> > I feel like every REST API I've ever worked with has had the
aggregation
>> > +
>> > projection problem. It's like we're trying to use REST as a
replacement
>> > for
>> > SQL -- but the logic that executes the "SQL" lives in a browser
now, and
>> > it
>> > used to live on a server close to the DB. And REST isn't expressive for
>> > selecting data like SQL is.
>>
>> The current industry solution I know about is called API gateway..
>> most of the big players have internal API with lots of low level stuff
>> and then couple of external API gateways tailored to what the client
>> needs.
>>
>> http://microservices.io/patterns/apigateway.html (check the backend
>> for frontend section)
>>
>> This trend is also visible when you think about services that offer
>> API gateway management and billing like
>> https://aws.amazon.com/api-gateway/ or our very own
>> https://www.3scale.net/
>
>
> right, but the api gateway solves 2 problems:
>
> 1: if you have a microservice architecture it is hard for frontend to talk
> to 20 different moving services. So the gateway hides this complexity behind
> it. This is not the problem we have.
>
2: you can have more api gateways (e.g. more apis) tailored for
every
frontend. I don't think we need this - the current API serves us pretty well
in every FE Im involved in. The only thing which I miss is the data
aggregation.
>
So I would go back to the original topic of this thread - do some
small
change which has a chance to be merged to the project and helps us where it
hurts.
>
>>
>>
>>
>> Martin
>>
>> On Fri, Mar 24, 2017 at 3:47 PM, Greg Sheremeta <gshereme(a)redhat.com>
>> wrote:
>> > I feel like every REST API I've ever worked with has had the
aggregation
>> > +
>> > projection problem. It's like we're trying to use REST as a
replacement
>> > for
>> > SQL -- but the logic that executes the "SQL" lives in a browser
now, and
>> > it
>> > used to live on a server close to the DB. And REST isn't expressive for
>> > selecting data like SQL is.
>> >
>> > There must be some industry solution to this "I want to do SQL over
>> > REST"
>> > problem.
>> >
>> > On Fri, Mar 24, 2017 at 5:54 AM, Martin Sivak <msivak(a)redhat.com>
wrote:
>> >>
>> >> > for quite some time I have been more or less involved in
development
>> >> > of
>> >> > various UIs for oVirt based entirely on the oVirt's REST API
ranging
>> >> > from
>> >> > the quite mature moVirt [1] through some cockpit extensions to a
>> >> > young
>> >> > and
>> >> > experimental user portal replacement [2].
>> >>
>> >> oVirt optimizer has the same issue..
>> >>
>> >> > 2: add some tiny service which would just accept a list of
queries,
>> >> > execute
>> >> > them locally (but using real HTTP requests) and return in one bulk.
A
>> >> > naive
>> >> > implementation just to give a sense of what I mean of this would be
a
>> >> > shell
>> >> > script getting list of strings like
>> >> > "https://localhost/ovirt-engine/api/vms/123/sessions"
iterate over
>> >> > them
>> >> > and
>> >> > do a curl request for each, mangle the results into one string and
>> >> > return
>> >> > (credits for this idea to msivak). Easy to implement, possibility
to
>> >> > add
>> >> > also projections later to save some bandwidth. But the API would
>> >> > anyway
>> >> > be
>> >> > hammered by bunch of queries, only the network roundtrip would be
>> >> > saved.
>> >>
>> >> The biggest cost for (especially mobile) clients is the cost of
>> >> establishing new SSL connection. SSL is also pretty expensive on the
>> >> server side.
>> >>
>> >> So running the aggregation service on the ovirt-engine machine (behind
>> >> Apache) means the client will do a single SSL request with list of N
>> >> urls and the local "reverse-proxy" will perform single
authentication
>> >> and N plain HTTP requests (or even better - AJP). It won't remove
any
>> >> time from the actual command run time, but it will reduce protocol
>> >> overhead.
>> >>
>> >> I think this is the simplest first step that requires almost no change
>> >> to existing infrastructure.
>> >>
>> >> --
>> >> Martin Sivak
>> >> SLA / oVirt
>> >>
>> >> On Fri, Mar 24, 2017 at 10:20 AM, Tomas Jelinek
<tjelinek(a)redhat.com>
>> >> wrote:
>> >> > Hi All,
>> >> >
>> >> > for quite some time I have been more or less involved in
development
>> >> > of
>> >> > various UIs for oVirt based entirely on the oVirt's REST API
ranging
>> >> > from
>> >> > the quite mature moVirt [1] through some cockpit extensions to a
>> >> > young
>> >> > and
>> >> > experimental user portal replacement [2].
>> >> >
>> >> > One issue we hit over and over again is the missing data
aggregation.
>> >> > In
>> >> > the
>> >> > 3.x era we used to use in moVirt the detail=something
>> >> > api to get the disks and nics of the VM, something like:
>> >> >
>> >> > GET /ovirt-engine/api/vms
>> >> > Accept: application/json; detail=disks
>> >> >
>> >> > This allowed us to store this data in local database leading to
great
>> >> > user
>> >> > experience. Since this feature has been removed in 4.x API [3]
>> >> > we needed to retire to a different solution. When the VM detail is
>> >> > selected
>> >> > by the user, start loading the disks and nics and hope the user
>> >> > will not be fast enough to see the delay. The UX is slightly worse
>> >> > bug
>> >> > kinda
>> >> > acceptable.
>> >> >
>> >> > We hit this issue harder in the new user portal [2], because we
>> >> > already
>> >> > have
>> >> > the VM cached and show the whole VM in one screen. So, if you pick
>> >> > it,
>> >> > you
>> >> > will get it's details immediately.
>> >> > But, since you don't have all the details, we need to do an
>> >> > additional
>> >> > call
>> >> > (two actually) to load this data and they start to appear later.
>> >> > So, something which would be very fast and smooth starts to feel
>> >> > sluggish.
>> >> >
>> >> > Recently, we hit this issue again which forced us to sacrifice the
UX
>> >> > even
>> >> > more - it is the "console in use" feature of user
portal.
>> >> > The use case is this:
>> >> > - if the console is already taken by some user, there are
>> >> > complications
>> >> > if
>> >> > other current user tryes to take it as well (will avoid details
about
>> >> > settings and permissins involved, but long story short, the user
will
>> >> > probably not be allowed to connect to it. The "probably"
is the key
>> >> > here
>> >> > since we can not do any intelligent decision in advance, we can
only
>> >> > warn
>> >> > the user that the console is taken).
>> >> > - in the current GWT user portal, if the VM's console is taken,
it is
>> >> > shown
>> >> > on the VM's "box" that "console is taken".
This was a highly
>> >> > requested
>> >> > feature
>> >> > - to get this information using the current REST API, we need to
go
>> >> > to
>> >> > the
>> >> > /vms/<vmid>/sessions subcollection. To get this for all VMs,
it would
>> >> > be
>> >> > doing N queries per poll which we can not afford
>> >> > - so the current PR [4] will probably end up to only check it on
the
>> >> > attempt
>> >> > to connect to the console warning the user. Maybe it will be also
>> >> > shown
>> >> > in
>> >> > Vm details. But the UX in case the user will look for a VM which
has
>> >> > free
>> >> > console will suffer significantly (e.g. try one by one until some
>> >> > opens
>> >> > or
>> >> > look at details one by one to see if the warning appears (with a
>> >> > delay))
>> >> >
>> >> > I understand that embedding the details of the VM to the response
>> >> > comes
>> >> > with
>> >> > a cost, namely:
>> >> > - performance hit
>> >> > - complexity of the API code
>> >> > - the "cleanness" of REST suffers
>> >> >
>> >> > But I think we should seriously consider to provide some option to
>> >> > data
>> >> > aggregation.
>> >> >
>> >> > I know this has been discussed many times with no result, but I
think
>> >> > it
>> >> > is
>> >> > time to bring this topic up again. I'll try to summarize the
(failed)
>> >> > attempts tried so far:
>> >> > - the detail=<something> parameter with ad-hoc embedding of
data.
>> >> > This
>> >> > has
>> >> > been there and removed in 4.0 [3]
>> >> > - the DoctorREST project - e.g. a proxy above the current api. The
>> >> > idea
>> >> > was
>> >> > to create a service which will be independent of the engine
itself,
>> >> > will
>> >> > locally poll the engine's REST, store all data in local
(mongo)DB and
>> >> > provide a rich api with aggregations and projections and push
>> >> > notifications.
>> >> > This polling of everything to get the data to DoctorREST proved to
be
>> >> > pretty
>> >> > costy, so also a more invasive approach of pushing data from
engine
>> >> > to
>> >> > doctor has been discused [5]. None of this two approaches have
been
>> >> > accepted
>> >> > (too complicated, too invasive).
>> >> > - writing some custom ad-hoc servlet serving only a purpose of one
>> >> > frontend
>> >> > - this is actually there for the dashboard, but it is not a
generic
>> >> > solution
>> >> > for the other frontends and we really should not develop custom
>> >> > "APIs"
>> >> > for
>> >> > every frontend
>> >> > - there were some other proposals discussed (some 3th party
solutions
>> >> > etc)
>> >> > but I think none of them made it even to a PoC
>> >> >
>> >> > So, now I would try again and try small to get at least some
benefit.
>> >> > I
>> >> > see
>> >> > 2 paths we could try:
>> >> > 1: embed something which burns us immediatly, e.g. the /sessions
into
>> >> > VMs. I
>> >> > really liked the ;detail=sessions approach, could we move it back?
>> >> > 2: add some tiny service which would just accept a list of
queries,
>> >> > execute
>> >> > them locally (but using real HTTP requests) and return in one bulk.
A
>> >> > naive
>> >> > implementation just to give a sense of what I mean of this would be
a
>> >> > shell
>> >> > script getting list of strings like
>> >> > "https://localhost/ovirt-engine/api/vms/123/sessions"
iterate over
>> >> > them
>> >> > and
>> >> > do a curl request for each, mangle the results into one string and
>> >> > return
>> >> > (credits for this idea to msivak). Easy to implement, possibility
to
>> >> > add
>> >> > also projections later to save some bandwidth. But the API would
>> >> > anyway
>> >> > be
>> >> > hammered by bunch of queries, only the network roundtrip would be
>> >> > saved.
>> >> > 3: any other simple approaches?
>> >> >
>> >> > I honestly prefer the first approach. It is not beautiful, it is
not
>> >> > REST-ful, but it is easy to implement, very pragmatic and useful.
>> >> > What do you think?
>> >> >
>> >> > Thank you and sorry for the long mail :)
>> >> > Tomas
>> >> >
>> >> > [1]: https://github.com/oVirt/moVirt
>> >> > [2]: https://github.com/oVirt/ovirt-web-ui
>> >> > [3]: https://gerrit.ovirt.org/#/c/61260
>> >> > [4]: https://github.com/oVirt/ovirt-web-ui/pull/106/
>> >> > [5]: https://gerrit.ovirt.org/#/c/45233/
>> >> >
>> >> >
>> >> > _______________________________________________
>> >> > Devel mailing list
>> >> > Devel(a)ovirt.org
>> >> > http://lists.ovirt.org/mailman/listinfo/devel
>> >> _______________________________________________
>> >> Devel mailing list
>> >> Devel(a)ovirt.org
>> >> http://lists.ovirt.org/mailman/listinfo/devel
>> >
>> >
>> >
>> >
>> > --
>> > Greg Sheremeta, MBA
>> > Red Hat, Inc.
>> > Sr. Software Engineer
>> > gshereme(a)redhat.com
>
>
9:30 a.m.
On Fri, Mar 24, 2017 at 6:43 PM, Martin Sivak <msivak(a)redhat.com> wrote:
> 2: you can have more api gateways (e.g. more apis) tailored for
every
> frontend. I don't think we need this - the current API serves us pretty
well
> in every FE Im involved in. The only thing which I miss is the data
> aggregation.
So it does not serve us well. Aggregation of data is one the usual
points of using the gateway.
Yes microservices are affected by this indeed, but so are we because
implementing the aggregation directly in the current engine API layer
is hard.
> So I would go back to the original topic of this thread - do some small
> change which has a chance to be merged to the project and helps us where
it
> hurts.
I'm wondering if very specific additional REST API calls can suffice.
For example, a 'Get VM + disks + NIC' API call seems reasonable to add for
the various clients who commonly need it.
Can a simple HTTP/2 to HTTP/AJP gateway be the simplest solution?
Our
Apache might even have a module for it already.
Current Apache used has only experimental module for it.
Undertow is supposed to have a better support. I wonder when/if we can drop
Apache...
Y.
That way you can multiplex all the REST calls using a single tcp
connection (and a single SSL negotiation).
A custom SSO enabled service like that might be even better as it
would be able to skip the authentication
layers too and that would lower the engine load. But I am not sure it
is possible with the current codebase.
Martin
On Fri, Mar 24, 2017 at 4:22 PM, Tomas Jelinek <tjelinek(a)redhat.com>
wrote:
>
>
> On Fri, Mar 24, 2017 at 3:58 PM, Martin Sivak <msivak(a)redhat.com> wrote:
>>
>> > I feel like every REST API I've ever worked with has had the
aggregation
>> > +
>> > projection problem. It's like we're trying to use REST as a
replacement
>> > for
>> > SQL -- but the logic that executes the "SQL" lives in a browser
now,
and
>> > it
>> > used to live on a server close to the DB. And REST isn't expressive
for
>> > selecting data like SQL is.
>>
>> The current industry solution I know about is called API gateway..
>> most of the big players have internal API with lots of low level stuff
>> and then couple of external API gateways tailored to what the client
>> needs.
>>
>> http://microservices.io/patterns/apigateway.html (check the backend
>> for frontend section)
>>
>> This trend is also visible when you think about services that offer
>> API gateway management and billing like
>> https://aws.amazon.com/api-gateway/ or our very own
>> https://www.3scale.net/
>
>
> right, but the api gateway solves 2 problems:
>
> 1: if you have a microservice architecture it is hard for frontend to
talk
> to 20 different moving services. So the gateway hides this complexity
behind
> it. This is not the problem we have.
>
> 2: you can have more api gateways (e.g. more apis) tailored for every
> frontend. I don't think we need this - the current API serves us pretty
well
> in every FE Im involved in. The only thing which I miss is the data
> aggregation.
>
> So I would go back to the original topic of this thread - do some small
> change which has a chance to be merged to the project and helps us where
it
> hurts.
>
>>
>>
>>
>> Martin
>>
>> On Fri, Mar 24, 2017 at 3:47 PM, Greg Sheremeta <gshereme(a)redhat.com>
>> wrote:
>> > I feel like every REST API I've ever worked with has had the
aggregation
>> > +
>> > projection problem. It's like we're trying to use REST as a
replacement
>> > for
>> > SQL -- but the logic that executes the "SQL" lives in a browser
now,
and
>> > it
>> > used to live on a server close to the DB. And REST isn't expressive
for
>> > selecting data like SQL is.
>> >
>> > There must be some industry solution to this "I want to do SQL over
>> > REST"
>> > problem.
>> >
>> > On Fri, Mar 24, 2017 at 5:54 AM, Martin Sivak <msivak(a)redhat.com>
wrote:
>> >>
>> >> > for quite some time I have been more or less involved in
development
>> >> > of
>> >> > various UIs for oVirt based entirely on the oVirt's REST API
ranging
>> >> > from
>> >> > the quite mature moVirt [1] through some cockpit extensions to a
>> >> > young
>> >> > and
>> >> > experimental user portal replacement [2].
>> >>
>> >> oVirt optimizer has the same issue..
>> >>
>> >> > 2: add some tiny service which would just accept a list of
queries,
>> >> > execute
>> >> > them locally (but using real HTTP requests) and return in one
bulk. A
>> >> > naive
>> >> > implementation just to give a sense of what I mean of this would
be a
>> >> > shell
>> >> > script getting list of strings like
>> >> > "https://localhost/ovirt-engine/api/vms/123/sessions"
iterate over
>> >> > them
>> >> > and
>> >> > do a curl request for each, mangle the results into one string
and
>> >> > return
>> >> > (credits for this idea to msivak). Easy to implement, possibility
to
>> >> > add
>> >> > also projections later to save some bandwidth. But the API would
>> >> > anyway
>> >> > be
>> >> > hammered by bunch of queries, only the network roundtrip would be
>> >> > saved.
>> >>
>> >> The biggest cost for (especially mobile) clients is the cost of
>> >> establishing new SSL connection. SSL is also pretty expensive on the
>> >> server side.
>> >>
>> >> So running the aggregation service on the ovirt-engine machine
(behind
>> >> Apache) means the client will do a single SSL request with list of N
>> >> urls and the local "reverse-proxy" will perform single
authentication
>> >> and N plain HTTP requests (or even better - AJP). It won't remove
any
>> >> time from the actual command run time, but it will reduce protocol
>> >> overhead.
>> >>
>> >> I think this is the simplest first step that requires almost no
change
>> >> to existing infrastructure.
>> >>
>> >> --
>> >> Martin Sivak
>> >> SLA / oVirt
>> >>
>> >> On Fri, Mar 24, 2017 at 10:20 AM, Tomas Jelinek
<tjelinek(a)redhat.com
>
>> >> wrote:
>> >> > Hi All,
>> >> >
>> >> > for quite some time I have been more or less involved in
development
>> >> > of
>> >> > various UIs for oVirt based entirely on the oVirt's REST API
ranging
>> >> > from
>> >> > the quite mature moVirt [1] through some cockpit extensions to a
>> >> > young
>> >> > and
>> >> > experimental user portal replacement [2].
>> >> >
>> >> > One issue we hit over and over again is the missing data
aggregation.
>> >> > In
>> >> > the
>> >> > 3.x era we used to use in moVirt the detail=something
>> >> > api to get the disks and nics of the VM, something like:
>> >> >
>> >> > GET /ovirt-engine/api/vms
>> >> > Accept: application/json; detail=disks
>> >> >
>> >> > This allowed us to store this data in local database leading to
great
>> >> > user
>> >> > experience. Since this feature has been removed in 4.x API [3]
>> >> > we needed to retire to a different solution. When the VM detail
is
>> >> > selected
>> >> > by the user, start loading the disks and nics and hope the user
>> >> > will not be fast enough to see the delay. The UX is slightly
worse
>> >> > bug
>> >> > kinda
>> >> > acceptable.
>> >> >
>> >> > We hit this issue harder in the new user portal [2], because we
>> >> > already
>> >> > have
>> >> > the VM cached and show the whole VM in one screen. So, if you
pick
>> >> > it,
>> >> > you
>> >> > will get it's details immediately.
>> >> > But, since you don't have all the details, we need to do an
>> >> > additional
>> >> > call
>> >> > (two actually) to load this data and they start to appear later.
>> >> > So, something which would be very fast and smooth starts to feel
>> >> > sluggish.
>> >> >
>> >> > Recently, we hit this issue again which forced us to sacrifice
the
UX
>> >> > even
>> >> > more - it is the "console in use" feature of user
portal.
>> >> > The use case is this:
>> >> > - if the console is already taken by some user, there are
>> >> > complications
>> >> > if
>> >> > other current user tryes to take it as well (will avoid details
about
>> >> > settings and permissins involved, but long story short, the user
will
>> >> > probably not be allowed to connect to it. The "probably"
is the key
>> >> > here
>> >> > since we can not do any intelligent decision in advance, we can
only
>> >> > warn
>> >> > the user that the console is taken).
>> >> > - in the current GWT user portal, if the VM's console is
taken, it
is
>> >> > shown
>> >> > on the VM's "box" that "console is taken".
This was a highly
>> >> > requested
>> >> > feature
>> >> > - to get this information using the current REST API, we need to
go
>> >> > to
>> >> > the
>> >> > /vms/<vmid>/sessions subcollection. To get this for all VMs,
it
would
>> >> > be
>> >> > doing N queries per poll which we can not afford
>> >> > - so the current PR [4] will probably end up to only check it on
the
>> >> > attempt
>> >> > to connect to the console warning the user. Maybe it will be also
>> >> > shown
>> >> > in
>> >> > Vm details. But the UX in case the user will look for a VM which
has
>> >> > free
>> >> > console will suffer significantly (e.g. try one by one until some
>> >> > opens
>> >> > or
>> >> > look at details one by one to see if the warning appears (with a
>> >> > delay))
>> >> >
>> >> > I understand that embedding the details of the VM to the response
>> >> > comes
>> >> > with
>> >> > a cost, namely:
>> >> > - performance hit
>> >> > - complexity of the API code
>> >> > - the "cleanness" of REST suffers
>> >> >
>> >> > But I think we should seriously consider to provide some option
to
>> >> > data
>> >> > aggregation.
>> >> >
>> >> > I know this has been discussed many times with no result, but I
think
>> >> > it
>> >> > is
>> >> > time to bring this topic up again. I'll try to summarize the
(failed)
>> >> > attempts tried so far:
>> >> > - the detail=<something> parameter with ad-hoc embedding of
data.
>> >> > This
>> >> > has
>> >> > been there and removed in 4.0 [3]
>> >> > - the DoctorREST project - e.g. a proxy above the current api.
The
>> >> > idea
>> >> > was
>> >> > to create a service which will be independent of the engine
itself,
>> >> > will
>> >> > locally poll the engine's REST, store all data in local
(mongo)DB
and
>> >> > provide a rich api with aggregations and projections and push
>> >> > notifications.
>> >> > This polling of everything to get the data to DoctorREST proved
to
be
>> >> > pretty
>> >> > costy, so also a more invasive approach of pushing data from
engine
>> >> > to
>> >> > doctor has been discused [5]. None of this two approaches have
been
>> >> > accepted
>> >> > (too complicated, too invasive).
>> >> > - writing some custom ad-hoc servlet serving only a purpose of
one
>> >> > frontend
>> >> > - this is actually there for the dashboard, but it is not a
generic
>> >> > solution
>> >> > for the other frontends and we really should not develop custom
>> >> > "APIs"
>> >> > for
>> >> > every frontend
>> >> > - there were some other proposals discussed (some 3th party
solutions
>> >> > etc)
>> >> > but I think none of them made it even to a PoC
>> >> >
>> >> > So, now I would try again and try small to get at least some
benefit.
>> >> > I
>> >> > see
>> >> > 2 paths we could try:
>> >> > 1: embed something which burns us immediatly, e.g. the /sessions
into
>> >> > VMs. I
>> >> > really liked the ;detail=sessions approach, could we move it
back?
>> >> > 2: add some tiny service which would just accept a list of
queries,
>> >> > execute
>> >> > them locally (but using real HTTP requests) and return in one
bulk. A
>> >> > naive
>> >> > implementation just to give a sense of what I mean of this would
be a
>> >> > shell
>> >> > script getting list of strings like
>> >> > "https://localhost/ovirt-engine/api/vms/123/sessions"
iterate over
>> >> > them
>> >> > and
>> >> > do a curl request for each, mangle the results into one string
and
>> >> > return
>> >> > (credits for this idea to msivak). Easy to implement, possibility
to
>> >> > add
>> >> > also projections later to save some bandwidth. But the API would
>> >> > anyway
>> >> > be
>> >> > hammered by bunch of queries, only the network roundtrip would be
>> >> > saved.
>> >> > 3: any other simple approaches?
>> >> >
>> >> > I honestly prefer the first approach. It is not beautiful, it is
not
>> >> > REST-ful, but it is easy to implement, very pragmatic and useful.
>> >> > What do you think?
>> >> >
>> >> > Thank you and sorry for the long mail :)
>> >> > Tomas
>> >> >
>> >> > [1]: https://github.com/oVirt/moVirt
>> >> > [2]: https://github.com/oVirt/ovirt-web-ui
>> >> > [3]: https://gerrit.ovirt.org/#/c/61260
>> >> > [4]: https://github.com/oVirt/ovirt-web-ui/pull/106/
>> >> > [5]: https://gerrit.ovirt.org/#/c/45233/
>> >> >
>> >> >
>> >> > _______________________________________________
>> >> > Devel mailing list
>> >> > Devel(a)ovirt.org
>> >> > http://lists.ovirt.org/mailman/listinfo/devel
>> >> _______________________________________________
>> >> Devel mailing list
>> >> Devel(a)ovirt.org
>> >> http://lists.ovirt.org/mailman/listinfo/devel
>> >
>> >
>> >
>> >
>> > --
>> > Greg Sheremeta, MBA
>> > Red Hat, Inc.
>> > Sr. Software Engineer
>> > gshereme(a)redhat.com
>
>
_______________________________________________
Devel mailing list
Devel(a)ovirt.org
http://lists.ovirt.org/mailman/listinfo/devel
10:57 a.m.
Current Apache used has only experimental module for it.
Undertow is supposed to have a better support. I wonder when/if we can drop
Apache...
The last info I have about that from mperina is that we need Apache
for kerberos support atm.
Martin
On Fri, Mar 24, 2017 at 5:30 PM, Yaniv Kaul <ykaul(a)redhat.com> wrote:
>
>
> On Fri, Mar 24, 2017 at 6:43 PM, Martin Sivak <msivak(a)redhat.com> wrote:
>>
>> > 2: you can have more api gateways (e.g. more apis) tailored for every
>> > frontend. I don't think we need this - the current API serves us pretty
>> > well
>> > in every FE Im involved in. The only thing which I miss is the data
>> > aggregation.
>>
>> So it does not serve us well. Aggregation of data is one the usual
>> points of using the gateway.
>> Yes microservices are affected by this indeed, but so are we because
>> implementing the aggregation directly in the current engine API layer
>> is hard.
>>
>> > So I would go back to the original topic of this thread - do some small
>> > change which has a chance to be merged to the project and helps us where
>> > it
>> > hurts.
>
>
> I'm wondering if very specific additional REST API calls can suffice.
> For example, a 'Get VM + disks + NIC' API call seems reasonable to add for
> the various clients who commonly need it.
>
>>
>> Can a simple HTTP/2 to HTTP/AJP gateway be the simplest solution? Our
>> Apache might even have a module for it already.
>
>
Current Apache used has only experimental module for it.
Undertow is supposed to have a better support. I wonder when/if we can drop
Apache...
> Y.
>
>>
>> That way you can multiplex all the REST calls using a single tcp
>> connection (and a single SSL negotiation).
>>
>> A custom SSO enabled service like that might be even better as it
>> would be able to skip the authentication
>> layers too and that would lower the engine load. But I am not sure it
>> is possible with the current codebase.
>>
>> Martin
>>
>> On Fri, Mar 24, 2017 at 4:22 PM, Tomas Jelinek <tjelinek(a)redhat.com>
>> wrote:
>> >
>> >
>> > On Fri, Mar 24, 2017 at 3:58 PM, Martin Sivak <msivak(a)redhat.com>
wrote:
>> >>
>> >> > I feel like every REST API I've ever worked with has had the
>> >> > aggregation
>> >> > +
>> >> > projection problem. It's like we're trying to use REST as
a
>> >> > replacement
>> >> > for
>> >> > SQL -- but the logic that executes the "SQL" lives in a
browser now,
>> >> > and
>> >> > it
>> >> > used to live on a server close to the DB. And REST isn't
expressive
>> >> > for
>> >> > selecting data like SQL is.
>> >>
>> >> The current industry solution I know about is called API gateway..
>> >> most of the big players have internal API with lots of low level stuff
>> >> and then couple of external API gateways tailored to what the client
>> >> needs.
>> >>
>> >> http://microservices.io/patterns/apigateway.html (check the backend
>> >> for frontend section)
>> >>
>> >> This trend is also visible when you think about services that offer
>> >> API gateway management and billing like
>> >> https://aws.amazon.com/api-gateway/ or our very own
>> >> https://www.3scale.net/
>> >
>> >
>> > right, but the api gateway solves 2 problems:
>> >
>> > 1: if you have a microservice architecture it is hard for frontend to
>> > talk
>> > to 20 different moving services. So the gateway hides this complexity
>> > behind
>> > it. This is not the problem we have.
>> >
>> > 2: you can have more api gateways (e.g. more apis) tailored for every
>> > frontend. I don't think we need this - the current API serves us pretty
>> > well
>> > in every FE Im involved in. The only thing which I miss is the data
>> > aggregation.
>> >
>> > So I would go back to the original topic of this thread - do some small
>> > change which has a chance to be merged to the project and helps us where
>> > it
>> > hurts.
>> >
>> >>
>> >>
>> >>
>> >> Martin
>> >>
>> >> On Fri, Mar 24, 2017 at 3:47 PM, Greg Sheremeta
<gshereme(a)redhat.com>
>> >> wrote:
>> >> > I feel like every REST API I've ever worked with has had the
>> >> > aggregation
>> >> > +
>> >> > projection problem. It's like we're trying to use REST as
a
>> >> > replacement
>> >> > for
>> >> > SQL -- but the logic that executes the "SQL" lives in a
browser now,
>> >> > and
>> >> > it
>> >> > used to live on a server close to the DB. And REST isn't
expressive
>> >> > for
>> >> > selecting data like SQL is.
>> >> >
>> >> > There must be some industry solution to this "I want to do SQL
over
>> >> > REST"
>> >> > problem.
>> >> >
>> >> > On Fri, Mar 24, 2017 at 5:54 AM, Martin Sivak
<msivak(a)redhat.com>
>> >> > wrote:
>> >> >>
>> >> >> > for quite some time I have been more or less involved in
>> >> >> > development
>> >> >> > of
>> >> >> > various UIs for oVirt based entirely on the oVirt's
REST API
>> >> >> > ranging
>> >> >> > from
>> >> >> > the quite mature moVirt [1] through some cockpit
extensions to a
>> >> >> > young
>> >> >> > and
>> >> >> > experimental user portal replacement [2].
>> >> >>
>> >> >> oVirt optimizer has the same issue..
>> >> >>
>> >> >> > 2: add some tiny service which would just accept a list
of
>> >> >> > queries,
>> >> >> > execute
>> >> >> > them locally (but using real HTTP requests) and return in
one
>> >> >> > bulk. A
>> >> >> > naive
>> >> >> > implementation just to give a sense of what I mean of this
would
>> >> >> > be a
>> >> >> > shell
>> >> >> > script getting list of strings like
>> >> >> >
"https://localhost/ovirt-engine/api/vms/123/sessions" iterate over
>> >> >> > them
>> >> >> > and
>> >> >> > do a curl request for each, mangle the results into one
string and
>> >> >> > return
>> >> >> > (credits for this idea to msivak). Easy to implement,
possibility
>> >> >> > to
>> >> >> > add
>> >> >> > also projections later to save some bandwidth. But the API
would
>> >> >> > anyway
>> >> >> > be
>> >> >> > hammered by bunch of queries, only the network roundtrip
would be
>> >> >> > saved.
>> >> >>
>> >> >> The biggest cost for (especially mobile) clients is the cost
of
>> >> >> establishing new SSL connection. SSL is also pretty expensive
on the
>> >> >> server side.
>> >> >>
>> >> >> So running the aggregation service on the ovirt-engine machine
>> >> >> (behind
>> >> >> Apache) means the client will do a single SSL request with list
of N
>> >> >> urls and the local "reverse-proxy" will perform
single
>> >> >> authentication
>> >> >> and N plain HTTP requests (or even better - AJP). It won't
remove
>> >> >> any
>> >> >> time from the actual command run time, but it will reduce
protocol
>> >> >> overhead.
>> >> >>
>> >> >> I think this is the simplest first step that requires almost
no
>> >> >> change
>> >> >> to existing infrastructure.
>> >> >>
>> >> >> --
>> >> >> Martin Sivak
>> >> >> SLA / oVirt
>> >> >>
>> >> >> On Fri, Mar 24, 2017 at 10:20 AM, Tomas Jelinek
>> >> >> <tjelinek(a)redhat.com>
>> >> >> wrote:
>> >> >> > Hi All,
>> >> >> >
>> >> >> > for quite some time I have been more or less involved in
>> >> >> > development
>> >> >> > of
>> >> >> > various UIs for oVirt based entirely on the oVirt's
REST API
>> >> >> > ranging
>> >> >> > from
>> >> >> > the quite mature moVirt [1] through some cockpit
extensions to a
>> >> >> > young
>> >> >> > and
>> >> >> > experimental user portal replacement [2].
>> >> >> >
>> >> >> > One issue we hit over and over again is the missing data
>> >> >> > aggregation.
>> >> >> > In
>> >> >> > the
>> >> >> > 3.x era we used to use in moVirt the detail=something
>> >> >> > api to get the disks and nics of the VM, something like:
>> >> >> >
>> >> >> > GET /ovirt-engine/api/vms
>> >> >> > Accept: application/json; detail=disks
>> >> >> >
>> >> >> > This allowed us to store this data in local database
leading to
>> >> >> > great
>> >> >> > user
>> >> >> > experience. Since this feature has been removed in 4.x API
[3]
>> >> >> > we needed to retire to a different solution. When the VM
detail is
>> >> >> > selected
>> >> >> > by the user, start loading the disks and nics and hope the
user
>> >> >> > will not be fast enough to see the delay. The UX is
slightly worse
>> >> >> > bug
>> >> >> > kinda
>> >> >> > acceptable.
>> >> >> >
>> >> >> > We hit this issue harder in the new user portal [2],
because we
>> >> >> > already
>> >> >> > have
>> >> >> > the VM cached and show the whole VM in one screen. So, if
you pick
>> >> >> > it,
>> >> >> > you
>> >> >> > will get it's details immediately.
>> >> >> > But, since you don't have all the details, we need to
do an
>> >> >> > additional
>> >> >> > call
>> >> >> > (two actually) to load this data and they start to appear
later.
>> >> >> > So, something which would be very fast and smooth starts
to feel
>> >> >> > sluggish.
>> >> >> >
>> >> >> > Recently, we hit this issue again which forced us to
sacrifice the
>> >> >> > UX
>> >> >> > even
>> >> >> > more - it is the "console in use" feature of
user portal.
>> >> >> > The use case is this:
>> >> >> > - if the console is already taken by some user, there are
>> >> >> > complications
>> >> >> > if
>> >> >> > other current user tryes to take it as well (will avoid
details
>> >> >> > about
>> >> >> > settings and permissins involved, but long story short,
the user
>> >> >> > will
>> >> >> > probably not be allowed to connect to it. The
"probably" is the
>> >> >> > key
>> >> >> > here
>> >> >> > since we can not do any intelligent decision in advance,
we can
>> >> >> > only
>> >> >> > warn
>> >> >> > the user that the console is taken).
>> >> >> > - in the current GWT user portal, if the VM's console
is taken, it
>> >> >> > is
>> >> >> > shown
>> >> >> > on the VM's "box" that "console is
taken". This was a highly
>> >> >> > requested
>> >> >> > feature
>> >> >> > - to get this information using the current REST API, we
need to
>> >> >> > go
>> >> >> > to
>> >> >> > the
>> >> >> > /vms/<vmid>/sessions subcollection. To get this for
all VMs, it
>> >> >> > would
>> >> >> > be
>> >> >> > doing N queries per poll which we can not afford
>> >> >> > - so the current PR [4] will probably end up to only check
it on
>> >> >> > the
>> >> >> > attempt
>> >> >> > to connect to the console warning the user. Maybe it will
be also
>> >> >> > shown
>> >> >> > in
>> >> >> > Vm details. But the UX in case the user will look for a VM
which
>> >> >> > has
>> >> >> > free
>> >> >> > console will suffer significantly (e.g. try one by one
until some
>> >> >> > opens
>> >> >> > or
>> >> >> > look at details one by one to see if the warning appears
(with a
>> >> >> > delay))
>> >> >> >
>> >> >> > I understand that embedding the details of the VM to the
response
>> >> >> > comes
>> >> >> > with
>> >> >> > a cost, namely:
>> >> >> > - performance hit
>> >> >> > - complexity of the API code
>> >> >> > - the "cleanness" of REST suffers
>> >> >> >
>> >> >> > But I think we should seriously consider to provide some
option to
>> >> >> > data
>> >> >> > aggregation.
>> >> >> >
>> >> >> > I know this has been discussed many times with no result,
but I
>> >> >> > think
>> >> >> > it
>> >> >> > is
>> >> >> > time to bring this topic up again. I'll try to
summarize the
>> >> >> > (failed)
>> >> >> > attempts tried so far:
>> >> >> > - the detail=<something> parameter with ad-hoc
embedding of data.
>> >> >> > This
>> >> >> > has
>> >> >> > been there and removed in 4.0 [3]
>> >> >> > - the DoctorREST project - e.g. a proxy above the current
api. The
>> >> >> > idea
>> >> >> > was
>> >> >> > to create a service which will be independent of the
engine
>> >> >> > itself,
>> >> >> > will
>> >> >> > locally poll the engine's REST, store all data in
local (mongo)DB
>> >> >> > and
>> >> >> > provide a rich api with aggregations and projections and
push
>> >> >> > notifications.
>> >> >> > This polling of everything to get the data to DoctorREST
proved to
>> >> >> > be
>> >> >> > pretty
>> >> >> > costy, so also a more invasive approach of pushing data
from
>> >> >> > engine
>> >> >> > to
>> >> >> > doctor has been discused [5]. None of this two approaches
have
>> >> >> > been
>> >> >> > accepted
>> >> >> > (too complicated, too invasive).
>> >> >> > - writing some custom ad-hoc servlet serving only a
purpose of one
>> >> >> > frontend
>> >> >> > - this is actually there for the dashboard, but it is not
a
>> >> >> > generic
>> >> >> > solution
>> >> >> > for the other frontends and we really should not develop
custom
>> >> >> > "APIs"
>> >> >> > for
>> >> >> > every frontend
>> >> >> > - there were some other proposals discussed (some 3th
party
>> >> >> > solutions
>> >> >> > etc)
>> >> >> > but I think none of them made it even to a PoC
>> >> >> >
>> >> >> > So, now I would try again and try small to get at least
some
>> >> >> > benefit.
>> >> >> > I
>> >> >> > see
>> >> >> > 2 paths we could try:
>> >> >> > 1: embed something which burns us immediatly, e.g. the
/sessions
>> >> >> > into
>> >> >> > VMs. I
>> >> >> > really liked the ;detail=sessions approach, could we move
it back?
>> >> >> > 2: add some tiny service which would just accept a list
of
>> >> >> > queries,
>> >> >> > execute
>> >> >> > them locally (but using real HTTP requests) and return in
one
>> >> >> > bulk. A
>> >> >> > naive
>> >> >> > implementation just to give a sense of what I mean of this
would
>> >> >> > be a
>> >> >> > shell
>> >> >> > script getting list of strings like
>> >> >> >
"https://localhost/ovirt-engine/api/vms/123/sessions" iterate over
>> >> >> > them
>> >> >> > and
>> >> >> > do a curl request for each, mangle the results into one
string and
>> >> >> > return
>> >> >> > (credits for this idea to msivak). Easy to implement,
possibility
>> >> >> > to
>> >> >> > add
>> >> >> > also projections later to save some bandwidth. But the API
would
>> >> >> > anyway
>> >> >> > be
>> >> >> > hammered by bunch of queries, only the network roundtrip
would be
>> >> >> > saved.
>> >> >> > 3: any other simple approaches?
>> >> >> >
>> >> >> > I honestly prefer the first approach. It is not beautiful,
it is
>> >> >> > not
>> >> >> > REST-ful, but it is easy to implement, very pragmatic and
useful.
>> >> >> > What do you think?
>> >> >> >
>> >> >> > Thank you and sorry for the long mail :)
>> >> >> > Tomas
>> >> >> >
>> >> >> > [1]: https://github.com/oVirt/moVirt
>> >> >> > [2]: https://github.com/oVirt/ovirt-web-ui
>> >> >> > [3]: https://gerrit.ovirt.org/#/c/61260
>> >> >> > [4]: https://github.com/oVirt/ovirt-web-ui/pull/106/
>> >> >> > [5]: https://gerrit.ovirt.org/#/c/45233/
>> >> >> >
>> >> >> >
>> >> >> > _______________________________________________
>> >> >> > Devel mailing list
>> >> >> > Devel(a)ovirt.org
>> >> >> > http://lists.ovirt.org/mailman/listinfo/devel
>> >> >> _______________________________________________
>> >> >> Devel mailing list
>> >> >> Devel(a)ovirt.org
>> >> >> http://lists.ovirt.org/mailman/listinfo/devel
>> >> >
>> >> >
>> >> >
>> >> >
>> >> > --
>> >> > Greg Sheremeta, MBA
>> >> > Red Hat, Inc.
>> >> > Sr. Software Engineer
>> >> > gshereme(a)redhat.com
>> >
>> >
>> _______________________________________________
>> Devel mailing list
>> Devel(a)ovirt.org
>> http://lists.ovirt.org/mailman/listinfo/devel
>
>
3:16 p.m.
On Fri, Mar 24, 2017 at 8:57 PM, Martin Sivak <msivak(a)redhat.com> wrote:
> Current Apache used has only experimental module for it.
> Undertow is supposed to have a better support. I wonder when/if we can
drop
> Apache...
The last info I have about that from mperina is that we need Apache
for kerberos support atm.
I don't think we need it - I remember reading that Undertow does support it
as well.
The only issue is that there are probably 10 people in the world who know
how to configure Undertow for Kerberos, while many do for Apache. And since
we leave it for the user to configure...
Y.
Martin
On Fri, Mar 24, 2017 at 5:30 PM, Yaniv Kaul <ykaul(a)redhat.com> wrote:
>
>
> On Fri, Mar 24, 2017 at 6:43 PM, Martin Sivak <msivak(a)redhat.com> wrote:
>>
>> > 2: you can have more api gateways (e.g. more apis) tailored for every
>> > frontend. I don't think we need this - the current API serves us
pretty
>> > well
>> > in every FE Im involved in. The only thing which I miss is the data
>> > aggregation.
>>
>> So it does not serve us well. Aggregation of data is one the usual
>> points of using the gateway.
>> Yes microservices are affected by this indeed, but so are we because
>> implementing the aggregation directly in the current engine API layer
>> is hard.
>>
>> > So I would go back to the original topic of this thread - do some
small
>> > change which has a chance to be merged to the project and helps us
where
>> > it
>> > hurts.
>
>
> I'm wondering if very specific additional REST API calls can suffice.
> For example, a 'Get VM + disks + NIC' API call seems reasonable to add
for
> the various clients who commonly need it.
>
>>
>> Can a simple HTTP/2 to HTTP/AJP gateway be the simplest solution? Our
>> Apache might even have a module for it already.
>
>
> Current Apache used has only experimental module for it.
> Undertow is supposed to have a better support. I wonder when/if we can
drop
> Apache...
> Y.
>
>>
>> That way you can multiplex all the REST calls using a single tcp
>> connection (and a single SSL negotiation).
>>
>> A custom SSO enabled service like that might be even better as it
>> would be able to skip the authentication
>> layers too and that would lower the engine load. But I am not sure it
>> is possible with the current codebase.
>>
>> Martin
>>
>> On Fri, Mar 24, 2017 at 4:22 PM, Tomas Jelinek <tjelinek(a)redhat.com>
>> wrote:
>> >
>> >
>> > On Fri, Mar 24, 2017 at 3:58 PM, Martin Sivak <msivak(a)redhat.com>
wrote:
>> >>
>> >> > I feel like every REST API I've ever worked with has had the
>> >> > aggregation
>> >> > +
>> >> > projection problem. It's like we're trying to use REST as
a
>> >> > replacement
>> >> > for
>> >> > SQL -- but the logic that executes the "SQL" lives in a
browser
now,
>> >> > and
>> >> > it
>> >> > used to live on a server close to the DB. And REST isn't
expressive
>> >> > for
>> >> > selecting data like SQL is.
>> >>
>> >> The current industry solution I know about is called API gateway..
>> >> most of the big players have internal API with lots of low level
stuff
>> >> and then couple of external API gateways tailored to what the client
>> >> needs.
>> >>
>> >> http://microservices.io/patterns/apigateway.html (check the backend
>> >> for frontend section)
>> >>
>> >> This trend is also visible when you think about services that offer
>> >> API gateway management and billing like
>> >> https://aws.amazon.com/api-gateway/ or our very own
>> >> https://www.3scale.net/
>> >
>> >
>> > right, but the api gateway solves 2 problems:
>> >
>> > 1: if you have a microservice architecture it is hard for frontend to
>> > talk
>> > to 20 different moving services. So the gateway hides this complexity
>> > behind
>> > it. This is not the problem we have.
>> >
>> > 2: you can have more api gateways (e.g. more apis) tailored for every
>> > frontend. I don't think we need this - the current API serves us
pretty
>> > well
>> > in every FE Im involved in. The only thing which I miss is the data
>> > aggregation.
>> >
>> > So I would go back to the original topic of this thread - do some
small
>> > change which has a chance to be merged to the project and helps us
where
>> > it
>> > hurts.
>> >
>> >>
>> >>
>> >>
>> >> Martin
>> >>
>> >> On Fri, Mar 24, 2017 at 3:47 PM, Greg Sheremeta
<gshereme(a)redhat.com
>
>> >> wrote:
>> >> > I feel like every REST API I've ever worked with has had the
>> >> > aggregation
>> >> > +
>> >> > projection problem. It's like we're trying to use REST as
a
>> >> > replacement
>> >> > for
>> >> > SQL -- but the logic that executes the "SQL" lives in a
browser
now,
>> >> > and
>> >> > it
>> >> > used to live on a server close to the DB. And REST isn't
expressive
>> >> > for
>> >> > selecting data like SQL is.
>> >> >
>> >> > There must be some industry solution to this "I want to do
SQL over
>> >> > REST"
>> >> > problem.
>> >> >
>> >> > On Fri, Mar 24, 2017 at 5:54 AM, Martin Sivak
<msivak(a)redhat.com>
>> >> > wrote:
>> >> >>
>> >> >> > for quite some time I have been more or less involved in
>> >> >> > development
>> >> >> > of
>> >> >> > various UIs for oVirt based entirely on the oVirt's
REST API
>> >> >> > ranging
>> >> >> > from
>> >> >> > the quite mature moVirt [1] through some cockpit
extensions to a
>> >> >> > young
>> >> >> > and
>> >> >> > experimental user portal replacement [2].
>> >> >>
>> >> >> oVirt optimizer has the same issue..
>> >> >>
>> >> >> > 2: add some tiny service which would just accept a list
of
>> >> >> > queries,
>> >> >> > execute
>> >> >> > them locally (but using real HTTP requests) and return in
one
>> >> >> > bulk. A
>> >> >> > naive
>> >> >> > implementation just to give a sense of what I mean of
this would
>> >> >> > be a
>> >> >> > shell
>> >> >> > script getting list of strings like
>> >> >> >
"https://localhost/ovirt-engine/api/vms/123/sessions" iterate
over
>> >> >> > them
>> >> >> > and
>> >> >> > do a curl request for each, mangle the results into one
string
and
>> >> >> > return
>> >> >> > (credits for this idea to msivak). Easy to implement,
possibility
>> >> >> > to
>> >> >> > add
>> >> >> > also projections later to save some bandwidth. But the
API would
>> >> >> > anyway
>> >> >> > be
>> >> >> > hammered by bunch of queries, only the network roundtrip
would
be
>> >> >> > saved.
>> >> >>
>> >> >> The biggest cost for (especially mobile) clients is the cost
of
>> >> >> establishing new SSL connection. SSL is also pretty expensive
on
the
>> >> >> server side.
>> >> >>
>> >> >> So running the aggregation service on the ovirt-engine
machine
>> >> >> (behind
>> >> >> Apache) means the client will do a single SSL request with
list
of N
>> >> >> urls and the local "reverse-proxy" will perform
single
>> >> >> authentication
>> >> >> and N plain HTTP requests (or even better - AJP). It won't
remove
>> >> >> any
>> >> >> time from the actual command run time, but it will reduce
protocol
>> >> >> overhead.
>> >> >>
>> >> >> I think this is the simplest first step that requires almost
no
>> >> >> change
>> >> >> to existing infrastructure.
>> >> >>
>> >> >> --
>> >> >> Martin Sivak
>> >> >> SLA / oVirt
>> >> >>
>> >> >> On Fri, Mar 24, 2017 at 10:20 AM, Tomas Jelinek
>> >> >> <tjelinek(a)redhat.com>
>> >> >> wrote:
>> >> >> > Hi All,
>> >> >> >
>> >> >> > for quite some time I have been more or less involved in
>> >> >> > development
>> >> >> > of
>> >> >> > various UIs for oVirt based entirely on the oVirt's
REST API
>> >> >> > ranging
>> >> >> > from
>> >> >> > the quite mature moVirt [1] through some cockpit
extensions to a
>> >> >> > young
>> >> >> > and
>> >> >> > experimental user portal replacement [2].
>> >> >> >
>> >> >> > One issue we hit over and over again is the missing data
>> >> >> > aggregation.
>> >> >> > In
>> >> >> > the
>> >> >> > 3.x era we used to use in moVirt the detail=something
>> >> >> > api to get the disks and nics of the VM, something like:
>> >> >> >
>> >> >> > GET /ovirt-engine/api/vms
>> >> >> > Accept: application/json; detail=disks
>> >> >> >
>> >> >> > This allowed us to store this data in local database
leading to
>> >> >> > great
>> >> >> > user
>> >> >> > experience. Since this feature has been removed in 4.x
API [3]
>> >> >> > we needed to retire to a different solution. When the VM
detail
is
>> >> >> > selected
>> >> >> > by the user, start loading the disks and nics and hope
the user
>> >> >> > will not be fast enough to see the delay. The UX is
slightly
worse
>> >> >> > bug
>> >> >> > kinda
>> >> >> > acceptable.
>> >> >> >
>> >> >> > We hit this issue harder in the new user portal [2],
because we
>> >> >> > already
>> >> >> > have
>> >> >> > the VM cached and show the whole VM in one screen. So, if
you
pick
>> >> >> > it,
>> >> >> > you
>> >> >> > will get it's details immediately.
>> >> >> > But, since you don't have all the details, we need to
do an
>> >> >> > additional
>> >> >> > call
>> >> >> > (two actually) to load this data and they start to appear
later.
>> >> >> > So, something which would be very fast and smooth starts
to feel
>> >> >> > sluggish.
>> >> >> >
>> >> >> > Recently, we hit this issue again which forced us to
sacrifice
the
>> >> >> > UX
>> >> >> > even
>> >> >> > more - it is the "console in use" feature of
user portal.
>> >> >> > The use case is this:
>> >> >> > - if the console is already taken by some user, there
are
>> >> >> > complications
>> >> >> > if
>> >> >> > other current user tryes to take it as well (will avoid
details
>> >> >> > about
>> >> >> > settings and permissins involved, but long story short,
the user
>> >> >> > will
>> >> >> > probably not be allowed to connect to it. The
"probably" is the
>> >> >> > key
>> >> >> > here
>> >> >> > since we can not do any intelligent decision in advance,
we can
>> >> >> > only
>> >> >> > warn
>> >> >> > the user that the console is taken).
>> >> >> > - in the current GWT user portal, if the VM's console
is taken,
it
>> >> >> > is
>> >> >> > shown
>> >> >> > on the VM's "box" that "console is
taken". This was a highly
>> >> >> > requested
>> >> >> > feature
>> >> >> > - to get this information using the current REST API, we
need to
>> >> >> > go
>> >> >> > to
>> >> >> > the
>> >> >> > /vms/<vmid>/sessions subcollection. To get this for
all VMs, it
>> >> >> > would
>> >> >> > be
>> >> >> > doing N queries per poll which we can not afford
>> >> >> > - so the current PR [4] will probably end up to only
check it on
>> >> >> > the
>> >> >> > attempt
>> >> >> > to connect to the console warning the user. Maybe it will
be
also
>> >> >> > shown
>> >> >> > in
>> >> >> > Vm details. But the UX in case the user will look for a
VM which
>> >> >> > has
>> >> >> > free
>> >> >> > console will suffer significantly (e.g. try one by one
until
some
>> >> >> > opens
>> >> >> > or
>> >> >> > look at details one by one to see if the warning appears
(with a
>> >> >> > delay))
>> >> >> >
>> >> >> > I understand that embedding the details of the VM to the
response
>> >> >> > comes
>> >> >> > with
>> >> >> > a cost, namely:
>> >> >> > - performance hit
>> >> >> > - complexity of the API code
>> >> >> > - the "cleanness" of REST suffers
>> >> >> >
>> >> >> > But I think we should seriously consider to provide some
option
to
>> >> >> > data
>> >> >> > aggregation.
>> >> >> >
>> >> >> > I know this has been discussed many times with no result,
but I
>> >> >> > think
>> >> >> > it
>> >> >> > is
>> >> >> > time to bring this topic up again. I'll try to
summarize the
>> >> >> > (failed)
>> >> >> > attempts tried so far:
>> >> >> > - the detail=<something> parameter with ad-hoc
embedding of
data.
>> >> >> > This
>> >> >> > has
>> >> >> > been there and removed in 4.0 [3]
>> >> >> > - the DoctorREST project - e.g. a proxy above the current
api.
The
>> >> >> > idea
>> >> >> > was
>> >> >> > to create a service which will be independent of the
engine
>> >> >> > itself,
>> >> >> > will
>> >> >> > locally poll the engine's REST, store all data in
local
(mongo)DB
>> >> >> > and
>> >> >> > provide a rich api with aggregations and projections and
push
>> >> >> > notifications.
>> >> >> > This polling of everything to get the data to DoctorREST
proved
to
>> >> >> > be
>> >> >> > pretty
>> >> >> > costy, so also a more invasive approach of pushing data
from
>> >> >> > engine
>> >> >> > to
>> >> >> > doctor has been discused [5]. None of this two approaches
have
>> >> >> > been
>> >> >> > accepted
>> >> >> > (too complicated, too invasive).
>> >> >> > - writing some custom ad-hoc servlet serving only a
purpose of
one
>> >> >> > frontend
>> >> >> > - this is actually there for the dashboard, but it is not
a
>> >> >> > generic
>> >> >> > solution
>> >> >> > for the other frontends and we really should not develop
custom
>> >> >> > "APIs"
>> >> >> > for
>> >> >> > every frontend
>> >> >> > - there were some other proposals discussed (some 3th
party
>> >> >> > solutions
>> >> >> > etc)
>> >> >> > but I think none of them made it even to a PoC
>> >> >> >
>> >> >> > So, now I would try again and try small to get at least
some
>> >> >> > benefit.
>> >> >> > I
>> >> >> > see
>> >> >> > 2 paths we could try:
>> >> >> > 1: embed something which burns us immediatly, e.g. the
/sessions
>> >> >> > into
>> >> >> > VMs. I
>> >> >> > really liked the ;detail=sessions approach, could we move
it
back?
>> >> >> > 2: add some tiny service which would just accept a list
of
>> >> >> > queries,
>> >> >> > execute
>> >> >> > them locally (but using real HTTP requests) and return in
one
>> >> >> > bulk. A
>> >> >> > naive
>> >> >> > implementation just to give a sense of what I mean of
this would
>> >> >> > be a
>> >> >> > shell
>> >> >> > script getting list of strings like
>> >> >> >
"https://localhost/ovirt-engine/api/vms/123/sessions" iterate
over
>> >> >> > them
>> >> >> > and
>> >> >> > do a curl request for each, mangle the results into one
string
and
>> >> >> > return
>> >> >> > (credits for this idea to msivak). Easy to implement,
possibility
>> >> >> > to
>> >> >> > add
>> >> >> > also projections later to save some bandwidth. But the
API would
>> >> >> > anyway
>> >> >> > be
>> >> >> > hammered by bunch of queries, only the network roundtrip
would
be
>> >> >> > saved.
>> >> >> > 3: any other simple approaches?
>> >> >> >
>> >> >> > I honestly prefer the first approach. It is not
beautiful, it is
>> >> >> > not
>> >> >> > REST-ful, but it is easy to implement, very pragmatic
and
useful.
>> >> >> > What do you think?
>> >> >> >
>> >> >> > Thank you and sorry for the long mail :)
>> >> >> > Tomas
>> >> >> >
>> >> >> > [1]: https://github.com/oVirt/moVirt
>> >> >> > [2]: https://github.com/oVirt/ovirt-web-ui
>> >> >> > [3]: https://gerrit.ovirt.org/#/c/61260
>> >> >> > [4]: https://github.com/oVirt/ovirt-web-ui/pull/106/
>> >> >> > [5]: https://gerrit.ovirt.org/#/c/45233/
>> >> >> >
>> >> >> >
>> >> >> > _______________________________________________
>> >> >> > Devel mailing list
>> >> >> > Devel(a)ovirt.org
>> >> >> > http://lists.ovirt.org/mailman/listinfo/devel
>> >> >> _______________________________________________
>> >> >> Devel mailing list
>> >> >> Devel(a)ovirt.org
>> >> >> http://lists.ovirt.org/mailman/listinfo/devel
>> >> >
>> >> >
>> >> >
>> >> >
>> >> > --
>> >> > Greg Sheremeta, MBA
>> >> > Red Hat, Inc.
>> >> > Sr. Software Engineer
>> >> > gshereme(a)redhat.com
>> >
>> >
>> _______________________________________________
>> Devel mailing list
>> Devel(a)ovirt.org
>> http://lists.ovirt.org/mailman/listinfo/devel
>
>
2:21 a.m.
Top posting, sorry.
There are a few things I'd like to clarify, regarding this subject:
1. Data aggregation, as requested now by Tomas, and by other people in
the past.
We used to have that 'detail' parameter, to aggregate certain very
specific types of data, in particular to aggregate VM disks and NICs. We
removed that in version 4 of the API because the implementation was
extremely inefficient, from the engine point of view. An innocent
request like this:
GET /ovirt-engine/api/vms?detail=+disks,+nics
Would generate, with the implementation we used to have, 1 query for the
VMs and then as many queries for disks and NICs as VMs in the system. In
our scale test environments, for example, with approx 4000 VMs and 10000
disks, that would take more than 20 hours to execute.
In addition, we didn't have in the past any mechanism to make this
available in a generic one, because there was no knowledge in the API of
what are 'details'.
In version 4 of the API we introduced a formal (kind of) specification
of the API (a.k.a. the model), and int includes knowledge about what are
'links'. For example, the specification of the VM type contains this:
@Link DiskAttachment[] diskAttachments();
@Link Nic[] nics();
With this information we are now in a position where we can implement
this in a generic way.
We intend to implement this using a mechanism similar to the existing
'detail' parameter:
GET /ovirt-engine/api/vms/123?follow=disk_attachments,nics
The naive implementation of this is to let the API call itself. For
example, when the user requests to follow the 'disk_attachments' detail
the API can just call itself to get that:
GET /ovirt-engine/api/vms/123/disk_attachments
However, we can't use that naive approach, if we do we end with the
1+C*N query problem described before. We need to use specific
implementations for certain frequent use cases, like VMs+disks+nics, and
that needs work in the API and in the backend.
Tomas, if you want to help moving this forward, please open a RFE and
makes sure it gets attention.
2. Reuse of TLS sessions.
The part of creating TLS sessions that is expensive is the generation of
the shared session key. That can be avoided if both the server and the
client are careful and reuse the session, using the session cache
mechanism built-in into TLS itself. The web servers that we use (Apache
and Undertow) do implement this mechanism, and so do most of our
clients. Make sure that your client uses it as well. In Java this is
achieved re-using the SSLContext. We already do that for the engine to
VDSM communciation for example. In JavaScript the browser already takes
care of this.
3. Parallelism and latency.
A typical problem that we have is that we send many request to the
server. For example, to retrieve user sessions for a set of VMs we tend
to send many requests like this:
GET /ovirt-engine/api/vms/1/sessions
GET /ovirt/engine/api/vms/2/sessions
GET /ovirt-engine/api/vms/3/sessions
...
And we do that in a synchronous way: send one, wait for the result, send
another one, wait for the result, etc. This means that we don't take
advantage of the parallelism of the server and that we add to each
request the network round trip time. So if we have N requests, we have
to wait at least N*RTT.
The web servers that we use support multiple connections, and the
protocol that we use, HTTP, supports pipe-lining. This means that you
can send multiple requests in parallel, and that you can send multiple
requests without waiting for the response. To give you an idea of the
improvement that can be achieved, we recently added asynchronous request
support to the Ruby SDK, with multiple connections and pipe-lining. In
our scale testing environment that reduced the time to collect a
complete inventory from approx 30 min to approx 2 min. Here you have an
example:
https://github.com/oVirt/ovirt-engine-sdk-ruby/blob/master/sdk/examples/a...
So make sure that you take advantage of that in your clients. Sadly
pipe-lining is disabled by default in most browsers, so this isn't
helpful for JavaScript applications.
4. HTTP/2 support.
The application server that we use, WildFly, supports HTTP/2, including
ALPN, out of the box, since version 10.1. We need a mechanism to enable it:
core: Add support for enabling HTTP/2
https://gerrit.ovirt.org/74621
And then we need to get Apache out of the way, for API traffic, at
least. I think that is something we can do in the context of the engine
"podification" effort.
However, note that HTTP/2 won't have that big impact in performance for
applications that continue to use a synchronous/serial style of
interaction with the API.
On 03/24/2017 11:16 PM, Yaniv Kaul wrote:
On Fri, Mar 24, 2017 at 8:57 PM, Martin Sivak <msivak(a)redhat.com
<mailto:msivak@redhat.com>> wrote:
> Current Apache used has only experimental module for it.
> Undertow is supposed to have a better support. I wonder when/if we can drop
> Apache...
The last info I have about that from mperina is that we need Apache
for kerberos support atm.
I don't think we need it - I remember reading that Undertow does support
it as well.
The only issue is that there are probably 10 people in the world who
know how to configure Undertow for Kerberos, while many do for Apache.
And since we leave it for the user to configure...
Y.
Martin
On Fri, Mar 24, 2017 at 5:30 PM, Yaniv Kaul <ykaul(a)redhat.com
<mailto:ykaul@redhat.com>> wrote:
>
>
> On Fri, Mar 24, 2017 at 6:43 PM, Martin Sivak <msivak(a)redhat.com
<mailto:msivak@redhat.com>> wrote:
>>
>> > 2: you can have more api gateways (e.g. more apis) tailored for
every
>> > frontend. I don't think we need this - the current API serves
us pretty
>> > well
>> > in every FE Im involved in. The only thing which I miss is the data
>> > aggregation.
>>
>> So it does not serve us well. Aggregation of data is one the usual
>> points of using the gateway.
>> Yes microservices are affected by this indeed, but so are we because
>> implementing the aggregation directly in the current engine API layer
>> is hard.
>>
>> > So I would go back to the original topic of this thread - do
some small
>> > change which has a chance to be merged to the project and helps
us where
>> > it
>> > hurts.
>
>
> I'm wondering if very specific additional REST API calls can suffice.
> For example, a 'Get VM + disks + NIC' API call seems reasonable to
add for
> the various clients who commonly need it.
>
>>
>> Can a simple HTTP/2 to HTTP/AJP gateway be the simplest solution? Our
>> Apache might even have a module for it already.
>
>
> Current Apache used has only experimental module for it.
> Undertow is supposed to have a better support. I wonder when/if we
can drop
> Apache...
> Y.
>
>>
>> That way you can multiplex all the REST calls using a single tcp
>> connection (and a single SSL negotiation).
>>
>> A custom SSO enabled service like that might be even better as it
>> would be able to skip the authentication
>> layers too and that would lower the engine load. But I am not sure it
>> is possible with the current codebase.
>>
>> Martin
>>
>> On Fri, Mar 24, 2017 at 4:22 PM, Tomas Jelinek
<tjelinek(a)redhat.com <mailto:tjelinek@redhat.com>>
>> wrote:
>> >
>> >
>> > On Fri, Mar 24, 2017 at 3:58 PM, Martin Sivak
<msivak(a)redhat.com <mailto:msivak@redhat.com>> wrote:
>> >>
>> >> > I feel like every REST API I've ever worked with has had
the
>> >> > aggregation
>> >> > +
>> >> > projection problem. It's like we're trying to use REST
as a
>> >> > replacement
>> >> > for
>> >> > SQL -- but the logic that executes the "SQL" lives
in a
browser now,
>> >> > and
>> >> > it
>> >> > used to live on a server close to the DB. And REST isn't
expressive
>> >> > for
>> >> > selecting data like SQL is.
>> >>
>> >> The current industry solution I know about is called API gateway..
>> >> most of the big players have internal API with lots of low
level stuff
>> >> and then couple of external API gateways tailored to what the
client
>> >> needs.
>> >>
>> >> http://microservices.io/patterns/apigateway.html
<http://microservices.io/patterns/apigateway.html> (check the backend
>> >> for frontend section)
>> >>
>> >> This trend is also visible when you think about services that
offer
>> >> API gateway management and billing like
>> >> https://aws.amazon.com/api-gateway/
<https://aws.amazon.com/api-gateway/> or our very own
>> >> https://www.3scale.net/
>> >
>> >
>> > right, but the api gateway solves 2 problems:
>> >
>> > 1: if you have a microservice architecture it is hard for
frontend to
>> > talk
>> > to 20 different moving services. So the gateway hides this
complexity
>> > behind
>> > it. This is not the problem we have.
>> >
>> > 2: you can have more api gateways (e.g. more apis) tailored for
every
>> > frontend. I don't think we need this - the current API serves
us pretty
>> > well
>> > in every FE Im involved in. The only thing which I miss is the data
>> > aggregation.
>> >
>> > So I would go back to the original topic of this thread - do
some small
>> > change which has a chance to be merged to the project and helps
us where
>> > it
>> > hurts.
>> >
>> >>
>> >>
>> >>
>> >> Martin
>> >>
>> >> On Fri, Mar 24, 2017 at 3:47 PM, Greg Sheremeta
<gshereme(a)redhat.com <mailto:gshereme@redhat.com>>
>> >> wrote:
>> >> > I feel like every REST API I've ever worked with has had
the
>> >> > aggregation
>> >> > +
>> >> > projection problem. It's like we're trying to use REST
as a
>> >> > replacement
>> >> > for
>> >> > SQL -- but the logic that executes the "SQL" lives
in a
browser now,
>> >> > and
>> >> > it
>> >> > used to live on a server close to the DB. And REST isn't
expressive
>> >> > for
>> >> > selecting data like SQL is.
>> >> >
>> >> > There must be some industry solution to this "I want to
do
SQL over
>> >> > REST"
>> >> > problem.
>> >> >
>> >> > On Fri, Mar 24, 2017 at 5:54 AM, Martin Sivak
<msivak(a)redhat.com <mailto:msivak@redhat.com>>
>> >> > wrote:
>> >> >>
>> >> >> > for quite some time I have been more or less involved
in
>> >> >> > development
>> >> >> > of
>> >> >> > various UIs for oVirt based entirely on the
oVirt's REST API
>> >> >> > ranging
>> >> >> > from
>> >> >> > the quite mature moVirt [1] through some cockpit
extensions to a
>> >> >> > young
>> >> >> > and
>> >> >> > experimental user portal replacement [2].
>> >> >>
>> >> >> oVirt optimizer has the same issue..
>> >> >>
>> >> >> > 2: add some tiny service which would just accept a
list of
>> >> >> > queries,
>> >> >> > execute
>> >> >> > them locally (but using real HTTP requests) and
return in one
>> >> >> > bulk. A
>> >> >> > naive
>> >> >> > implementation just to give a sense of what I mean
of
this would
>> >> >> > be a
>> >> >> > shell
>> >> >> > script getting list of strings like
>> >> >> >
"https://localhost/ovirt-engine/api/vms/123/sessions
<https://localhost/ovirt-engine/api/vms/123/sessions>" iterate over
>> >> >> > them
>> >> >> > and
>> >> >> > do a curl request for each, mangle the results into
one
string and
>> >> >> > return
>> >> >> > (credits for this idea to msivak). Easy to
implement,
possibility
>> >> >> > to
>> >> >> > add
>> >> >> > also projections later to save some bandwidth. But
the
API would
>> >> >> > anyway
>> >> >> > be
>> >> >> > hammered by bunch of queries, only the network
roundtrip
would be
>> >> >> > saved.
>> >> >>
>> >> >> The biggest cost for (especially mobile) clients is the
cost of
>> >> >> establishing new SSL connection. SSL is also pretty
expensive on the
>> >> >> server side.
>> >> >>
>> >> >> So running the aggregation service on the ovirt-engine
machine
>> >> >> (behind
>> >> >> Apache) means the client will do a single SSL request
with
list of N
>> >> >> urls and the local "reverse-proxy" will perform
single
>> >> >> authentication
>> >> >> and N plain HTTP requests (or even better - AJP). It
won't
remove
>> >> >> any
>> >> >> time from the actual command run time, but it will reduce
protocol
>> >> >> overhead.
>> >> >>
>> >> >> I think this is the simplest first step that requires
almost no
>> >> >> change
>> >> >> to existing infrastructure.
>> >> >>
>> >> >> --
>> >> >> Martin Sivak
>> >> >> SLA / oVirt
>> >> >>
>> >> >> On Fri, Mar 24, 2017 at 10:20 AM, Tomas Jelinek
>> >> >> <tjelinek(a)redhat.com
<mailto:tjelinek@redhat.com>>
>> >> >> wrote:
>> >> >> > Hi All,
>> >> >> >
>> >> >> > for quite some time I have been more or less involved
in
>> >> >> > development
>> >> >> > of
>> >> >> > various UIs for oVirt based entirely on the
oVirt's REST API
>> >> >> > ranging
>> >> >> > from
>> >> >> > the quite mature moVirt [1] through some cockpit
extensions to a
>> >> >> > young
>> >> >> > and
>> >> >> > experimental user portal replacement [2].
>> >> >> >
>> >> >> > One issue we hit over and over again is the missing
data
>> >> >> > aggregation.
>> >> >> > In
>> >> >> > the
>> >> >> > 3.x era we used to use in moVirt the
detail=something
>> >> >> > api to get the disks and nics of the VM, something
like:
>> >> >> >
>> >> >> > GET /ovirt-engine/api/vms
>> >> >> > Accept: application/json; detail=disks
>> >> >> >
>> >> >> > This allowed us to store this data in local database
leading to
>> >> >> > great
>> >> >> > user
>> >> >> > experience. Since this feature has been removed in
4.x
API [3]
>> >> >> > we needed to retire to a different solution. When the
VM
detail is
>> >> >> > selected
>> >> >> > by the user, start loading the disks and nics and
hope
the user
>> >> >> > will not be fast enough to see the delay. The UX is
slightly worse
>> >> >> > bug
>> >> >> > kinda
>> >> >> > acceptable.
>> >> >> >
>> >> >> > We hit this issue harder in the new user portal [2],
because we
>> >> >> > already
>> >> >> > have
>> >> >> > the VM cached and show the whole VM in one screen.
So, if
you pick
>> >> >> > it,
>> >> >> > you
>> >> >> > will get it's details immediately.
>> >> >> > But, since you don't have all the details, we
need to do an
>> >> >> > additional
>> >> >> > call
>> >> >> > (two actually) to load this data and they start to
appear
later.
>> >> >> > So, something which would be very fast and smooth
starts
to feel
>> >> >> > sluggish.
>> >> >> >
>> >> >> > Recently, we hit this issue again which forced us to
sacrifice the
>> >> >> > UX
>> >> >> > even
>> >> >> > more - it is the "console in use" feature
of user portal.
>> >> >> > The use case is this:
>> >> >> > - if the console is already taken by some user, there
are
>> >> >> > complications
>> >> >> > if
>> >> >> > other current user tryes to take it as well (will
avoid
details
>> >> >> > about
>> >> >> > settings and permissins involved, but long story
short,
the user
>> >> >> > will
>> >> >> > probably not be allowed to connect to it. The
"probably"
is the
>> >> >> > key
>> >> >> > here
>> >> >> > since we can not do any intelligent decision in
advance,
we can
>> >> >> > only
>> >> >> > warn
>> >> >> > the user that the console is taken).
>> >> >> > - in the current GWT user portal, if the VM's
console is
taken, it
>> >> >> > is
>> >> >> > shown
>> >> >> > on the VM's "box" that "console is
taken". This was a highly
>> >> >> > requested
>> >> >> > feature
>> >> >> > - to get this information using the current REST API,
we
need to
>> >> >> > go
>> >> >> > to
>> >> >> > the
>> >> >> > /vms/<vmid>/sessions subcollection. To get this
for all
VMs, it
>> >> >> > would
>> >> >> > be
>> >> >> > doing N queries per poll which we can not afford
>> >> >> > - so the current PR [4] will probably end up to only
check it on
>> >> >> > the
>> >> >> > attempt
>> >> >> > to connect to the console warning the user. Maybe it
will
be also
>> >> >> > shown
>> >> >> > in
>> >> >> > Vm details. But the UX in case the user will look for
a
VM which
>> >> >> > has
>> >> >> > free
>> >> >> > console will suffer significantly (e.g. try one by
one
until some
>> >> >> > opens
>> >> >> > or
>> >> >> > look at details one by one to see if the warning
appears
(with a
>> >> >> > delay))
>> >> >> >
>> >> >> > I understand that embedding the details of the VM to
the
response
>> >> >> > comes
>> >> >> > with
>> >> >> > a cost, namely:
>> >> >> > - performance hit
>> >> >> > - complexity of the API code
>> >> >> > - the "cleanness" of REST suffers
>> >> >> >
>> >> >> > But I think we should seriously consider to provide
some
option to
>> >> >> > data
>> >> >> > aggregation.
>> >> >> >
>> >> >> > I know this has been discussed many times with no
result,
but I
>> >> >> > think
>> >> >> > it
>> >> >> > is
>> >> >> > time to bring this topic up again. I'll try to
summarize the
>> >> >> > (failed)
>> >> >> > attempts tried so far:
>> >> >> > - the detail=<something> parameter with ad-hoc
embedding
of data.
>> >> >> > This
>> >> >> > has
>> >> >> > been there and removed in 4.0 [3]
>> >> >> > - the DoctorREST project - e.g. a proxy above the
current
api. The
>> >> >> > idea
>> >> >> > was
>> >> >> > to create a service which will be independent of the
engine
>> >> >> > itself,
>> >> >> > will
>> >> >> > locally poll the engine's REST, store all data in
local
(mongo)DB
>> >> >> > and
>> >> >> > provide a rich api with aggregations and projections
and push
>> >> >> > notifications.
>> >> >> > This polling of everything to get the data to
DoctorREST
proved to
>> >> >> > be
>> >> >> > pretty
>> >> >> > costy, so also a more invasive approach of pushing
data from
>> >> >> > engine
>> >> >> > to
>> >> >> > doctor has been discused [5]. None of this two
approaches
have
>> >> >> > been
>> >> >> > accepted
>> >> >> > (too complicated, too invasive).
>> >> >> > - writing some custom ad-hoc servlet serving only a
purpose of one
>> >> >> > frontend
>> >> >> > - this is actually there for the dashboard, but it is
not a
>> >> >> > generic
>> >> >> > solution
>> >> >> > for the other frontends and we really should not
develop
custom
>> >> >> > "APIs"
>> >> >> > for
>> >> >> > every frontend
>> >> >> > - there were some other proposals discussed (some 3th
party
>> >> >> > solutions
>> >> >> > etc)
>> >> >> > but I think none of them made it even to a PoC
>> >> >> >
>> >> >> > So, now I would try again and try small to get at
least some
>> >> >> > benefit.
>> >> >> > I
>> >> >> > see
>> >> >> > 2 paths we could try:
>> >> >> > 1: embed something which burns us immediatly, e.g.
the
/sessions
>> >> >> > into
>> >> >> > VMs. I
>> >> >> > really liked the ;detail=sessions approach, could we
move
it back?
>> >> >> > 2: add some tiny service which would just accept a
list of
>> >> >> > queries,
>> >> >> > execute
>> >> >> > them locally (but using real HTTP requests) and
return in one
>> >> >> > bulk. A
>> >> >> > naive
>> >> >> > implementation just to give a sense of what I mean
of
this would
>> >> >> > be a
>> >> >> > shell
>> >> >> > script getting list of strings like
>> >> >> >
"https://localhost/ovirt-engine/api/vms/123/sessions
<https://localhost/ovirt-engine/api/vms/123/sessions>" iterate over
>> >> >> > them
>> >> >> > and
>> >> >> > do a curl request for each, mangle the results into
one
string and
>> >> >> > return
>> >> >> > (credits for this idea to msivak). Easy to
implement,
possibility
>> >> >> > to
>> >> >> > add
>> >> >> > also projections later to save some bandwidth. But
the
API would
>> >> >> > anyway
>> >> >> > be
>> >> >> > hammered by bunch of queries, only the network
roundtrip
would be
>> >> >> > saved.
>> >> >> > 3: any other simple approaches?
>> >> >> >
>> >> >> > I honestly prefer the first approach. It is not
beautiful, it is
>> >> >> > not
>> >> >> > REST-ful, but it is easy to implement, very pragmatic
and
useful.
>> >> >> > What do you think?
>> >> >> >
>> >> >> > Thank you and sorry for the long mail :)
>> >> >> > Tomas
>> >> >> >
>> >> >> > [1]: https://github.com/oVirt/moVirt
<https://github.com/oVirt/moVirt>
>> >> >> > [2]: https://github.com/oVirt/ovirt-web-ui
<https://github.com/oVirt/ovirt-web-ui>
>> >> >> > [3]: https://gerrit.ovirt.org/#/c/61260
<https://gerrit.ovirt.org/#/c/61260>
>> >> >> > [4]: https://github.com/oVirt/ovirt-web-ui/pull/106/
<https://github.com/oVirt/ovirt-web-ui/pull/106/>
>> >> >> > [5]: https://gerrit.ovirt.org/#/c/45233/
<https://gerrit.ovirt.org/#/c/45233/>
>> >> >> >
>> >> >> >
>> >> >> > _______________________________________________
>> >> >> > Devel mailing list
>> >> >> > Devel(a)ovirt.org <mailto:Devel@ovirt.org>
>> >> >> > http://lists.ovirt.org/mailman/listinfo/devel
<http://lists.ovirt.org/mailman/listinfo/devel>
>> >> >> _______________________________________________
>> >> >> Devel mailing list
>> >> >> Devel(a)ovirt.org <mailto:Devel@ovirt.org>
>> >> >> http://lists.ovirt.org/mailman/listinfo/devel
<http://lists.ovirt.org/mailman/listinfo/devel>
>> >> >
>> >> >
>> >> >
>> >> >
>> >> > --
>> >> > Greg Sheremeta, MBA
>> >> > Red Hat, Inc.
>> >> > Sr. Software Engineer
>> >> > gshereme(a)redhat.com <mailto:gshereme@redhat.com>
>> >
>> >
>> _______________________________________________
>> Devel mailing list
>> Devel(a)ovirt.org <mailto:Devel@ovirt.org>
>> http://lists.ovirt.org/mailman/listinfo/devel
<http://lists.ovirt.org/mailman/listinfo/devel>
>
>
4:03 a.m.
On Mon, Mar 27, 2017 at 11:21 AM, Juan Hernández <jhernand(a)redhat.com>
wrote:
Top posting, sorry.
There are a few things I'd like to clarify, regarding this subject:
1. Data aggregation, as requested now by Tomas, and by other people in
the past.
We used to have that 'detail' parameter, to aggregate certain very
specific types of data, in particular to aggregate VM disks and NICs. We
removed that in version 4 of the API because the implementation was
extremely inefficient, from the engine point of view. An innocent
request like this:
GET /ovirt-engine/api/vms?detail=+disks,+nics
Would generate, with the implementation we used to have, 1 query for the
VMs and then as many queries for disks and NICs as VMs in the system. In
our scale test environments, for example, with approx 4000 VMs and 10000
disks, that would take more than 20 hours to execute.
In addition, we didn't have in the past any mechanism to make this
available in a generic one, because there was no knowledge in the API of
what are 'details'.
In version 4 of the API we introduced a formal (kind of) specification
of the API (a.k.a. the model), and int includes knowledge about what are
'links'. For example, the specification of the VM type contains this:
@Link DiskAttachment[] diskAttachments();
@Link Nic[] nics();
With this information we are now in a position where we can implement
this in a generic way.
We intend to implement this using a mechanism similar to the existing
'detail' parameter:
GET /ovirt-engine/api/vms/123?follow=disk_attachments,nics
The naive implementation of this is to let the API call itself. For
example, when the user requests to follow the 'disk_attachments' detail
the API can just call itself to get that:
GET /ovirt-engine/api/vms/123/disk_attachments
However, we can't use that naive approach, if we do we end with the
1+C*N query problem described before. We need to use specific
implementations for certain frequent use cases, like VMs+disks+nics, and
that needs work in the API and in the backend.
Tomas, if you want to help moving this forward, please open a RFE and
makes sure it gets attention.
This sounds pretty good! I will open, but since we are talking already here
I'll just use the opportunity to clarify the topic more and than I'll open
the BZ.
What I can imagine is the GetAllVmsQuery will accept in params also the
list of details it should provide. Than, the GetAllVmsQuery will implement
the efficient way of retrieving this info as well.
So, from the API perspective, it will be about taking the
?follow=<something> part and passing it to the backend query params.
What you think?
2. Reuse of TLS sessions.
The part of creating TLS sessions that is expensive is the generation of
the shared session key. That can be avoided if both the server and the
client are careful and reuse the session, using the session cache
mechanism built-in into TLS itself. The web servers that we use (Apache
and Undertow) do implement this mechanism, and so do most of our
clients. Make sure that your client uses it as well. In Java this is
achieved re-using the SSLContext. We already do that for the engine to
VDSM communciation for example. In JavaScript the browser already takes
care of this.
3. Parallelism and latency.
A typical problem that we have is that we send many request to the
server. For example, to retrieve user sessions for a set of VMs we tend
to send many requests like this:
GET /ovirt-engine/api/vms/1/sessions
GET /ovirt/engine/api/vms/2/sessions
GET /ovirt-engine/api/vms/3/sessions
...
And we do that in a synchronous way: send one, wait for the result, send
another one, wait for the result, etc. This means that we don't take
advantage of the parallelism of the server and that we add to each
request the network round trip time. So if we have N requests, we have
to wait at least N*RTT.
The web servers that we use support multiple connections, and the
protocol that we use, HTTP, supports pipe-lining. This means that you
can send multiple requests in parallel, and that you can send multiple
requests without waiting for the response. To give you an idea of the
improvement that can be achieved, we recently added asynchronous request
support to the Ruby SDK, with multiple connections and pipe-lining. In
our scale testing environment that reduced the time to collect a
complete inventory from approx 30 min to approx 2 min. Here you have an
example:
https://github.com/oVirt/ovirt-engine-sdk-ruby/blob/master/sdk/examples/
asynchronous_inventory.rb
So make sure that you take advantage of that in your clients. Sadly
pipe-lining is disabled by default in most browsers, so this isn't
helpful for JavaScript applications.
But we can try what we can do in moVirt about this:
https://github.com/oVirt/moVirt/issues/260
4. HTTP/2 support.
The application server that we use, WildFly, supports HTTP/2, including
ALPN, out of the box, since version 10.1. We need a mechanism to enable it:
core: Add support for enabling HTTP/2
https://gerrit.ovirt.org/74621
And then we need to get Apache out of the way, for API traffic, at
least. I think that is something we can do in the context of the engine
"podification" effort.
However, note that HTTP/2 won't have that big impact in performance for
applications that continue to use a synchronous/serial style of
interaction with the API.
On 03/24/2017 11:16 PM, Yaniv Kaul wrote:
>
>
> On Fri, Mar 24, 2017 at 8:57 PM, Martin Sivak <msivak(a)redhat.com
> <mailto:msivak@redhat.com>> wrote:
>
> > Current Apache used has only experimental module for it.
> > Undertow is supposed to have a better support. I wonder when/if we
can drop
> > Apache...
>
> The last info I have about that from mperina is that we need Apache
> for kerberos support atm.
>
>
> I don't think we need it - I remember reading that Undertow does support
> it as well.
> The only issue is that there are probably 10 people in the world who
> know how to configure Undertow for Kerberos, while many do for Apache.
> And since we leave it for the user to configure...
> Y.
>
>
>
> Martin
>
> On Fri, Mar 24, 2017 at 5:30 PM, Yaniv Kaul <ykaul(a)redhat.com
> <mailto:ykaul@redhat.com>> wrote:
> >
> >
> > On Fri, Mar 24, 2017 at 6:43 PM, Martin Sivak <msivak(a)redhat.com
> <mailto:msivak@redhat.com>> wrote:
> >>
> >> > 2: you can have more api gateways (e.g. more apis) tailored for
> every
> >> > frontend. I don't think we need this - the current API serves
> us pretty
> >> > well
> >> > in every FE Im involved in. The only thing which I miss is the
data
> >> > aggregation.
> >>
> >> So it does not serve us well. Aggregation of data is one the usual
> >> points of using the gateway.
> >> Yes microservices are affected by this indeed, but so are we
because
> >> implementing the aggregation directly in the current engine API
layer
> >> is hard.
> >>
> >> > So I would go back to the original topic of this thread - do
> some small
> >> > change which has a chance to be merged to the project and helps
> us where
> >> > it
> >> > hurts.
> >
> >
> > I'm wondering if very specific additional REST API calls can
suffice.
> > For example, a 'Get VM + disks + NIC' API call seems reasonable to
> add for
> > the various clients who commonly need it.
> >
> >>
> >> Can a simple HTTP/2 to HTTP/AJP gateway be the simplest solution?
Our
> >> Apache might even have a module for it already.
> >
> >
> > Current Apache used has only experimental module for it.
> > Undertow is supposed to have a better support. I wonder when/if we
> can drop
> > Apache...
> > Y.
> >
> >>
> >> That way you can multiplex all the REST calls using a single tcp
> >> connection (and a single SSL negotiation).
> >>
> >> A custom SSO enabled service like that might be even better as it
> >> would be able to skip the authentication
> >> layers too and that would lower the engine load. But I am not
sure it
> >> is possible with the current codebase.
> >>
> >> Martin
> >>
> >> On Fri, Mar 24, 2017 at 4:22 PM, Tomas Jelinek
> <tjelinek(a)redhat.com <mailto:tjelinek@redhat.com>>
> >> wrote:
> >> >
> >> >
> >> > On Fri, Mar 24, 2017 at 3:58 PM, Martin Sivak
> <msivak(a)redhat.com <mailto:msivak@redhat.com>> wrote:
> >> >>
> >> >> > I feel like every REST API I've ever worked with has
had the
> >> >> > aggregation
> >> >> > +
> >> >> > projection problem. It's like we're trying to use
REST as a
> >> >> > replacement
> >> >> > for
> >> >> > SQL -- but the logic that executes the "SQL"
lives in a
> browser now,
> >> >> > and
> >> >> > it
> >> >> > used to live on a server close to the DB. And REST
isn't
> expressive
> >> >> > for
> >> >> > selecting data like SQL is.
> >> >>
> >> >> The current industry solution I know about is called API
gateway..
> >> >> most of the big players have internal API with lots of low
> level stuff
> >> >> and then couple of external API gateways tailored to what the
> client
> >> >> needs.
> >> >>
> >> >> http://microservices.io/patterns/apigateway.html
> <http://microservices.io/patterns/apigateway.html> (check the
backend
> >> >> for frontend section)
> >> >>
> >> >> This trend is also visible when you think about services that
> offer
> >> >> API gateway management and billing like
> >> >> https://aws.amazon.com/api-gateway/
> <https://aws.amazon.com/api-gateway/> or our very own
> >> >> https://www.3scale.net/
> >> >
> >> >
> >> > right, but the api gateway solves 2 problems:
> >> >
> >> > 1: if you have a microservice architecture it is hard for
> frontend to
> >> > talk
> >> > to 20 different moving services. So the gateway hides this
> complexity
> >> > behind
> >> > it. This is not the problem we have.
> >> >
> >> > 2: you can have more api gateways (e.g. more apis) tailored for
> every
> >> > frontend. I don't think we need this - the current API serves
> us pretty
> >> > well
> >> > in every FE Im involved in. The only thing which I miss is the
data
> >> > aggregation.
> >> >
> >> > So I would go back to the original topic of this thread - do
> some small
> >> > change which has a chance to be merged to the project and helps
> us where
> >> > it
> >> > hurts.
> >> >
> >> >>
> >> >>
> >> >>
> >> >> Martin
> >> >>
> >> >> On Fri, Mar 24, 2017 at 3:47 PM, Greg Sheremeta
> <gshereme(a)redhat.com <mailto:gshereme@redhat.com>>
> >> >> wrote:
> >> >> > I feel like every REST API I've ever worked with has
had the
> >> >> > aggregation
> >> >> > +
> >> >> > projection problem. It's like we're trying to use
REST as a
> >> >> > replacement
> >> >> > for
> >> >> > SQL -- but the logic that executes the "SQL"
lives in a
> browser now,
> >> >> > and
> >> >> > it
> >> >> > used to live on a server close to the DB. And REST
isn't
> expressive
> >> >> > for
> >> >> > selecting data like SQL is.
> >> >> >
> >> >> > There must be some industry solution to this "I want
to do
> SQL over
> >> >> > REST"
> >> >> > problem.
> >> >> >
> >> >> > On Fri, Mar 24, 2017 at 5:54 AM, Martin Sivak
> <msivak(a)redhat.com <mailto:msivak@redhat.com>>
> >> >> > wrote:
> >> >> >>
> >> >> >> > for quite some time I have been more or less
involved in
> >> >> >> > development
> >> >> >> > of
> >> >> >> > various UIs for oVirt based entirely on the
oVirt's REST
API
> >> >> >> > ranging
> >> >> >> > from
> >> >> >> > the quite mature moVirt [1] through some
cockpit
> extensions to a
> >> >> >> > young
> >> >> >> > and
> >> >> >> > experimental user portal replacement [2].
> >> >> >>
> >> >> >> oVirt optimizer has the same issue..
> >> >> >>
> >> >> >> > 2: add some tiny service which would just accept
a list of
> >> >> >> > queries,
> >> >> >> > execute
> >> >> >> > them locally (but using real HTTP requests) and
return in
one
> >> >> >> > bulk. A
> >> >> >> > naive
> >> >> >> > implementation just to give a sense of what I
mean of
> this would
> >> >> >> > be a
> >> >> >> > shell
> >> >> >> > script getting list of strings like
> >> >> >> >
"https://localhost/ovirt-engine/api/vms/123/sessions
> <https://localhost/ovirt-engine/api/vms/123/sessions>" iterate over
> >> >> >> > them
> >> >> >> > and
> >> >> >> > do a curl request for each, mangle the results
into one
> string and
> >> >> >> > return
> >> >> >> > (credits for this idea to msivak). Easy to
implement,
> possibility
> >> >> >> > to
> >> >> >> > add
> >> >> >> > also projections later to save some bandwidth.
But the
> API would
> >> >> >> > anyway
> >> >> >> > be
> >> >> >> > hammered by bunch of queries, only the network
roundtrip
> would be
> >> >> >> > saved.
> >> >> >>
> >> >> >> The biggest cost for (especially mobile) clients is
the
cost of
> >> >> >> establishing new SSL connection. SSL is also pretty
> expensive on the
> >> >> >> server side.
> >> >> >>
> >> >> >> So running the aggregation service on the
ovirt-engine
machine
> >> >> >> (behind
> >> >> >> Apache) means the client will do a single SSL request
with
> list of N
> >> >> >> urls and the local "reverse-proxy" will
perform single
> >> >> >> authentication
> >> >> >> and N plain HTTP requests (or even better - AJP). It
won't
> remove
> >> >> >> any
> >> >> >> time from the actual command run time, but it will
reduce
> protocol
> >> >> >> overhead.
> >> >> >>
> >> >> >> I think this is the simplest first step that
requires
almost no
> >> >> >> change
> >> >> >> to existing infrastructure.
> >> >> >>
> >> >> >> --
> >> >> >> Martin Sivak
> >> >> >> SLA / oVirt
> >> >> >>
> >> >> >> On Fri, Mar 24, 2017 at 10:20 AM, Tomas Jelinek
> >> >> >> <tjelinek(a)redhat.com
<mailto:tjelinek@redhat.com>>
> >> >> >> wrote:
> >> >> >> > Hi All,
> >> >> >> >
> >> >> >> > for quite some time I have been more or less
involved in
> >> >> >> > development
> >> >> >> > of
> >> >> >> > various UIs for oVirt based entirely on the
oVirt's REST
API
> >> >> >> > ranging
> >> >> >> > from
> >> >> >> > the quite mature moVirt [1] through some
cockpit
> extensions to a
> >> >> >> > young
> >> >> >> > and
> >> >> >> > experimental user portal replacement [2].
> >> >> >> >
> >> >> >> > One issue we hit over and over again is the
missing data
> >> >> >> > aggregation.
> >> >> >> > In
> >> >> >> > the
> >> >> >> > 3.x era we used to use in moVirt the
detail=something
> >> >> >> > api to get the disks and nics of the VM,
something like:
> >> >> >> >
> >> >> >> > GET /ovirt-engine/api/vms
> >> >> >> > Accept: application/json; detail=disks
> >> >> >> >
> >> >> >> > This allowed us to store this data in local
database
> leading to
> >> >> >> > great
> >> >> >> > user
> >> >> >> > experience. Since this feature has been removed
in 4.x
> API [3]
> >> >> >> > we needed to retire to a different solution.
When the VM
> detail is
> >> >> >> > selected
> >> >> >> > by the user, start loading the disks and nics
and hope
> the user
> >> >> >> > will not be fast enough to see the delay. The UX
is
> slightly worse
> >> >> >> > bug
> >> >> >> > kinda
> >> >> >> > acceptable.
> >> >> >> >
> >> >> >> > We hit this issue harder in the new user portal
[2],
> because we
> >> >> >> > already
> >> >> >> > have
> >> >> >> > the VM cached and show the whole VM in one
screen. So, if
> you pick
> >> >> >> > it,
> >> >> >> > you
> >> >> >> > will get it's details immediately.
> >> >> >> > But, since you don't have all the details,
we need to do
an
> >> >> >> > additional
> >> >> >> > call
> >> >> >> > (two actually) to load this data and they start
to appear
> later.
> >> >> >> > So, something which would be very fast and
smooth starts
> to feel
> >> >> >> > sluggish.
> >> >> >> >
> >> >> >> > Recently, we hit this issue again which forced
us to
> sacrifice the
> >> >> >> > UX
> >> >> >> > even
> >> >> >> > more - it is the "console in use"
feature of user portal.
> >> >> >> > The use case is this:
> >> >> >> > - if the console is already taken by some user,
there are
> >> >> >> > complications
> >> >> >> > if
> >> >> >> > other current user tryes to take it as well
(will avoid
> details
> >> >> >> > about
> >> >> >> > settings and permissins involved, but long story
short,
> the user
> >> >> >> > will
> >> >> >> > probably not be allowed to connect to it. The
"probably"
> is the
> >> >> >> > key
> >> >> >> > here
> >> >> >> > since we can not do any intelligent decision in
advance,
> we can
> >> >> >> > only
> >> >> >> > warn
> >> >> >> > the user that the console is taken).
> >> >> >> > - in the current GWT user portal, if the
VM's console is
> taken, it
> >> >> >> > is
> >> >> >> > shown
> >> >> >> > on the VM's "box" that
"console is taken". This was a
highly
> >> >> >> > requested
> >> >> >> > feature
> >> >> >> > - to get this information using the current REST
API, we
> need to
> >> >> >> > go
> >> >> >> > to
> >> >> >> > the
> >> >> >> > /vms/<vmid>/sessions subcollection. To get
this for all
> VMs, it
> >> >> >> > would
> >> >> >> > be
> >> >> >> > doing N queries per poll which we can not
afford
> >> >> >> > - so the current PR [4] will probably end up to
only
> check it on
> >> >> >> > the
> >> >> >> > attempt
> >> >> >> > to connect to the console warning the user.
Maybe it will
> be also
> >> >> >> > shown
> >> >> >> > in
> >> >> >> > Vm details. But the UX in case the user will
look for a
> VM which
> >> >> >> > has
> >> >> >> > free
> >> >> >> > console will suffer significantly (e.g. try one
by one
> until some
> >> >> >> > opens
> >> >> >> > or
> >> >> >> > look at details one by one to see if the warning
appears
> (with a
> >> >> >> > delay))
> >> >> >> >
> >> >> >> > I understand that embedding the details of the
VM to the
> response
> >> >> >> > comes
> >> >> >> > with
> >> >> >> > a cost, namely:
> >> >> >> > - performance hit
> >> >> >> > - complexity of the API code
> >> >> >> > - the "cleanness" of REST suffers
> >> >> >> >
> >> >> >> > But I think we should seriously consider to
provide some
> option to
> >> >> >> > data
> >> >> >> > aggregation.
> >> >> >> >
> >> >> >> > I know this has been discussed many times with
no result,
> but I
> >> >> >> > think
> >> >> >> > it
> >> >> >> > is
> >> >> >> > time to bring this topic up again. I'll try
to summarize
the
> >> >> >> > (failed)
> >> >> >> > attempts tried so far:
> >> >> >> > - the detail=<something> parameter with
ad-hoc embedding
> of data.
> >> >> >> > This
> >> >> >> > has
> >> >> >> > been there and removed in 4.0 [3]
> >> >> >> > - the DoctorREST project - e.g. a proxy above
the current
> api. The
> >> >> >> > idea
> >> >> >> > was
> >> >> >> > to create a service which will be independent of
the
engine
> >> >> >> > itself,
> >> >> >> > will
> >> >> >> > locally poll the engine's REST, store all
data in local
> (mongo)DB
> >> >> >> > and
> >> >> >> > provide a rich api with aggregations and
projections and
push
> >> >> >> > notifications.
> >> >> >> > This polling of everything to get the data to
DoctorREST
> proved to
> >> >> >> > be
> >> >> >> > pretty
> >> >> >> > costy, so also a more invasive approach of
pushing data
from
> >> >> >> > engine
> >> >> >> > to
> >> >> >> > doctor has been discused [5]. None of this two
approaches
> have
> >> >> >> > been
> >> >> >> > accepted
> >> >> >> > (too complicated, too invasive).
> >> >> >> > - writing some custom ad-hoc servlet serving
only a
> purpose of one
> >> >> >> > frontend
> >> >> >> > - this is actually there for the dashboard, but
it is not
a
> >> >> >> > generic
> >> >> >> > solution
> >> >> >> > for the other frontends and we really should not
develop
> custom
> >> >> >> > "APIs"
> >> >> >> > for
> >> >> >> > every frontend
> >> >> >> > - there were some other proposals discussed
(some 3th
party
> >> >> >> > solutions
> >> >> >> > etc)
> >> >> >> > but I think none of them made it even to a PoC
> >> >> >> >
> >> >> >> > So, now I would try again and try small to get
at least
some
> >> >> >> > benefit.
> >> >> >> > I
> >> >> >> > see
> >> >> >> > 2 paths we could try:
> >> >> >> > 1: embed something which burns us immediatly,
e.g. the
> /sessions
> >> >> >> > into
> >> >> >> > VMs. I
> >> >> >> > really liked the ;detail=sessions approach,
could we move
> it back?
> >> >> >> > 2: add some tiny service which would just accept
a list of
> >> >> >> > queries,
> >> >> >> > execute
> >> >> >> > them locally (but using real HTTP requests) and
return in
one
> >> >> >> > bulk. A
> >> >> >> > naive
> >> >> >> > implementation just to give a sense of what I
mean of
> this would
> >> >> >> > be a
> >> >> >> > shell
> >> >> >> > script getting list of strings like
> >> >> >> >
"https://localhost/ovirt-engine/api/vms/123/sessions
> <https://localhost/ovirt-engine/api/vms/123/sessions>" iterate over
> >> >> >> > them
> >> >> >> > and
> >> >> >> > do a curl request for each, mangle the results
into one
> string and
> >> >> >> > return
> >> >> >> > (credits for this idea to msivak). Easy to
implement,
> possibility
> >> >> >> > to
> >> >> >> > add
> >> >> >> > also projections later to save some bandwidth.
But the
> API would
> >> >> >> > anyway
> >> >> >> > be
> >> >> >> > hammered by bunch of queries, only the network
roundtrip
> would be
> >> >> >> > saved.
> >> >> >> > 3: any other simple approaches?
> >> >> >> >
> >> >> >> > I honestly prefer the first approach. It is not
> beautiful, it is
> >> >> >> > not
> >> >> >> > REST-ful, but it is easy to implement, very
pragmatic and
> useful.
> >> >> >> > What do you think?
> >> >> >> >
> >> >> >> > Thank you and sorry for the long mail :)
> >> >> >> > Tomas
> >> >> >> >
> >> >> >> > [1]: https://github.com/oVirt/moVirt
> <https://github.com/oVirt/moVirt>
> >> >> >> > [2]: https://github.com/oVirt/ovirt-web-ui
> <https://github.com/oVirt/ovirt-web-ui>
> >> >> >> > [3]: https://gerrit.ovirt.org/#/c/61260
> <https://gerrit.ovirt.org/#/c/61260>
> >> >> >> > [4]:
https://github.com/oVirt/ovirt-web-ui/pull/106/
> <https://github.com/oVirt/ovirt-web-ui/pull/106/>
> >> >> >> > [5]: https://gerrit.ovirt.org/#/c/45233/
> <https://gerrit.ovirt.org/#/c/45233/>
> >> >> >> >
> >> >> >> >
> >> >> >> > _______________________________________________
> >> >> >> > Devel mailing list
> >> >> >> > Devel(a)ovirt.org <mailto:Devel@ovirt.org>
> >> >> >> > http://lists.ovirt.org/mailman/listinfo/devel
> <http://lists.ovirt.org/mailman/listinfo/devel>
> >> >> >> _______________________________________________
> >> >> >> Devel mailing list
> >> >> >> Devel(a)ovirt.org <mailto:Devel@ovirt.org>
> >> >> >> http://lists.ovirt.org/mailman/listinfo/devel
> <http://lists.ovirt.org/mailman/listinfo/devel>
> >> >> >
> >> >> >
> >> >> >
> >> >> >
> >> >> > --
> >> >> > Greg Sheremeta, MBA
> >> >> > Red Hat, Inc.
> >> >> > Sr. Software Engineer
> >> >> > gshereme(a)redhat.com <mailto:gshereme@redhat.com>
> >> >
> >> >
> >> _______________________________________________
> >> Devel mailing list
> >> Devel(a)ovirt.org <mailto:Devel@ovirt.org>
> >> http://lists.ovirt.org/mailman/listinfo/devel
> <http://lists.ovirt.org/mailman/listinfo/devel>
> >
> >
>
>
4:32 a.m.
On 03/27/2017 01:03 PM, Tomas Jelinek wrote:
On Mon, Mar 27, 2017 at 11:21 AM, Juan Hernández <jhernand(a)redhat.com
<mailto:jhernand@redhat.com>> wrote:
Top posting, sorry.
There are a few things I'd like to clarify, regarding this subject:
1. Data aggregation, as requested now by Tomas, and by other people in
the past.
We used to have that 'detail' parameter, to aggregate certain very
specific types of data, in particular to aggregate VM disks and NICs. We
removed that in version 4 of the API because the implementation was
extremely inefficient, from the engine point of view. An innocent
request like this:
GET /ovirt-engine/api/vms?detail=+disks,+nics
Would generate, with the implementation we used to have, 1 query for the
VMs and then as many queries for disks and NICs as VMs in the system. In
our scale test environments, for example, with approx 4000 VMs and 10000
disks, that would take more than 20 hours to execute.
In addition, we didn't have in the past any mechanism to make this
available in a generic one, because there was no knowledge in the API of
what are 'details'.
In version 4 of the API we introduced a formal (kind of) specification
of the API (a.k.a. the model), and int includes knowledge about what are
'links'. For example, the specification of the VM type contains this:
@Link DiskAttachment[] diskAttachments();
@Link Nic[] nics();
With this information we are now in a position where we can implement
this in a generic way.
We intend to implement this using a mechanism similar to the existing
'detail' parameter:
GET /ovirt-engine/api/vms/123?follow=disk_attachments,nics
The naive implementation of this is to let the API call itself. For
example, when the user requests to follow the 'disk_attachments' detail
the API can just call itself to get that:
GET /ovirt-engine/api/vms/123/disk_attachments
However, we can't use that naive approach, if we do we end with the
1+C*N query problem described before. We need to use specific
implementations for certain frequent use cases, like VMs+disks+nics, and
that needs work in the API and in the backend.
Tomas, if you want to help moving this forward, please open a RFE and
makes sure it gets attention.
This sounds pretty good! I will open, but since we are talking already
here I'll just use the opportunity to clarify the topic more and than
I'll open the BZ.
What I can imagine is the GetAllVmsQuery will accept in params also the
list of details it should provide. Than, the GetAllVmsQuery will
implement the efficient way of retrieving this info as well.
So, from the API perspective, it will be about taking the
?follow=<something> part and passing it to the backend query params.
What you think?
Exactly, that is the point! The API by itself can't optimize database
queries, all it can do is call the backend. It is the backend that has
the opportunity and possibility to send optimized queries to the database.
For other less common things we can use the naive approach, and
implement the aggregation in the API itself. But for common use cases,
like VM+disks+nics, we need to do it in an efficient way.
2. Reuse of TLS sessions.
The part of creating TLS sessions that is expensive is the generation of
the shared session key. That can be avoided if both the server and the
client are careful and reuse the session, using the session cache
mechanism built-in into TLS itself. The web servers that we use (Apache
and Undertow) do implement this mechanism, and so do most of our
clients. Make sure that your client uses it as well. In Java this is
achieved re-using the SSLContext. We already do that for the engine to
VDSM communciation for example. In JavaScript the browser already takes
care of this.
3. Parallelism and latency.
A typical problem that we have is that we send many request to the
server. For example, to retrieve user sessions for a set of VMs we tend
to send many requests like this:
GET /ovirt-engine/api/vms/1/sessions
GET /ovirt/engine/api/vms/2/sessions
GET /ovirt-engine/api/vms/3/sessions
...
And we do that in a synchronous way: send one, wait for the result, send
another one, wait for the result, etc. This means that we don't take
advantage of the parallelism of the server and that we add to each
request the network round trip time. So if we have N requests, we have
to wait at least N*RTT.
The web servers that we use support multiple connections, and the
protocol that we use, HTTP, supports pipe-lining. This means that you
can send multiple requests in parallel, and that you can send multiple
requests without waiting for the response. To give you an idea of the
improvement that can be achieved, we recently added asynchronous request
support to the Ruby SDK, with multiple connections and pipe-lining. In
our scale testing environment that reduced the time to collect a
complete inventory from approx 30 min to approx 2 min. Here you have an
example:
https://github.com/oVirt/ovirt-engine-sdk-ruby/blob/master/sdk/examples/a...
<https://github.com/oVirt/ovirt-engine-sdk-ruby/blob/master/sdk/examples/a...
So make sure that you take advantage of that in your clients. Sadly
pipe-lining is disabled by default in most browsers, so this isn't
helpful for JavaScript applications.
But we can try what we can do in moVirt about
this: https://github.com/oVirt/moVirt/issues/260
Sure, I think there are plenty of asynchronous HTTP clients for Android,
worth trying one of them. If you are brave enough you can even consider
using the same library used in the Ruby SDK: libcurl. A bit of JNI here
and there, and you are done.
4. HTTP/2 support.
The application server that we use, WildFly, supports HTTP/2, including
ALPN, out of the box, since version 10.1. We need a mechanism to
enable it:
core: Add support for enabling HTTP/2
https://gerrit.ovirt.org/74621
And then we need to get Apache out of the way, for API traffic, at
least. I think that is something we can do in the context of the engine
"podification" effort.
However, note that HTTP/2 won't have that big impact in performance for
applications that continue to use a synchronous/serial style of
interaction with the API.
On 03/24/2017 11:16 PM, Yaniv Kaul wrote:
>
>
> On Fri, Mar 24, 2017 at 8:57 PM, Martin Sivak <msivak(a)redhat.com
<mailto:msivak@redhat.com>
> <mailto:msivak@redhat.com <mailto:msivak@redhat.com>>> wrote:
>
> > Current Apache used has only experimental module for it.
> > Undertow is supposed to have a better support. I wonder when/if we can
drop
> > Apache...
>
> The last info I have about that from mperina is that we need Apache
> for kerberos support atm.
>
>
> I don't think we need it - I remember reading that Undertow does support
> it as well.
> The only issue is that there are probably 10 people in the world who
> know how to configure Undertow for Kerberos, while many do for Apache.
> And since we leave it for the user to configure...
> Y.
>
>
>
> Martin
>
> On Fri, Mar 24, 2017 at 5:30 PM, Yaniv Kaul <ykaul(a)redhat.com
<mailto:ykaul@redhat.com>
> <mailto:ykaul@redhat.com <mailto:ykaul@redhat.com>>> wrote:
> >
> >
> > On Fri, Mar 24, 2017 at 6:43 PM, Martin Sivak <msivak(a)redhat.com
<mailto:msivak@redhat.com>
> <mailto:msivak@redhat.com <mailto:msivak@redhat.com>>>
wrote:
> >>
> >> > 2: you can have more api gateways (e.g. more apis)
tailored for
> every
> >> > frontend. I don't think we need this - the current API
serves
> us pretty
> >> > well
> >> > in every FE Im involved in. The only thing which I miss
is the data
> >> > aggregation.
> >>
> >> So it does not serve us well. Aggregation of data is one
the usual
> >> points of using the gateway.
> >> Yes microservices are affected by this indeed, but so are
we because
> >> implementing the aggregation directly in the current engine
API layer
> >> is hard.
> >>
> >> > So I would go back to the original topic of this thread - do
> some small
> >> > change which has a chance to be merged to the project and
helps
> us where
> >> > it
> >> > hurts.
> >
> >
> > I'm wondering if very specific additional REST API calls can
suffice.
> > For example, a 'Get VM + disks + NIC' API call seems
reasonable to
> add for
> > the various clients who commonly need it.
> >
> >>
> >> Can a simple HTTP/2 to HTTP/AJP gateway be the simplest
solution? Our
> >> Apache might even have a module for it already.
> >
> >
> > Current Apache used has only experimental module for it.
> > Undertow is supposed to have a better support. I wonder
when/if we
> can drop
> > Apache...
> > Y.
> >
> >>
> >> That way you can multiplex all the REST calls using a
single tcp
> >> connection (and a single SSL negotiation).
> >>
> >> A custom SSO enabled service like that might be even better
as it
> >> would be able to skip the authentication
> >> layers too and that would lower the engine load. But I am
not sure it
> >> is possible with the current codebase.
> >>
> >> Martin
> >>
> >> On Fri, Mar 24, 2017 at 4:22 PM, Tomas Jelinek
> <tjelinek(a)redhat.com <mailto:tjelinek@redhat.com>
<mailto:tjelinek@redhat.com <mailto:tjelinek@redhat.com>>>
> >> wrote:
> >> >
> >> >
> >> > On Fri, Mar 24, 2017 at 3:58 PM, Martin Sivak
> <msivak(a)redhat.com <mailto:msivak@redhat.com>
<mailto:msivak@redhat.com <mailto:msivak@redhat.com>>> wrote:
> >> >>
> >> >> > I feel like every REST API I've ever worked with
has
had the
> >> >> > aggregation
> >> >> > +
> >> >> > projection problem. It's like we're trying to
use REST
as a
> >> >> > replacement
> >> >> > for
> >> >> > SQL -- but the logic that executes the
"SQL" lives in a
> browser now,
> >> >> > and
> >> >> > it
> >> >> > used to live on a server close to the DB. And REST
isn't
> expressive
> >> >> > for
> >> >> > selecting data like SQL is.
> >> >>
> >> >> The current industry solution I know about is called API
gateway..
> >> >> most of the big players have internal API with lots of
low
> level stuff
> >> >> and then couple of external API gateways tailored to
what the
> client
> >> >> needs.
> >> >>
> >> >> http://microservices.io/patterns/apigateway.html
<http://microservices.io/patterns/apigateway.html>
> <http://microservices.io/patterns/apigateway.html
<http://microservices.io/patterns/apigateway.html>> (check the backend
> >> >> for frontend section)
> >> >>
> >> >> This trend is also visible when you think about services
that
> offer
> >> >> API gateway management and billing like
> >> >> https://aws.amazon.com/api-gateway/
<https://aws.amazon.com/api-gateway/>
> <https://aws.amazon.com/api-gateway/
<https://aws.amazon.com/api-gateway/>> or our very own
> >> >> https://www.3scale.net/
> >> >
> >> >
> >> > right, but the api gateway solves 2 problems:
> >> >
> >> > 1: if you have a microservice architecture it is hard for
> frontend to
> >> > talk
> >> > to 20 different moving services. So the gateway hides this
> complexity
> >> > behind
> >> > it. This is not the problem we have.
> >> >
> >> > 2: you can have more api gateways (e.g. more apis)
tailored for
> every
> >> > frontend. I don't think we need this - the current API
serves
> us pretty
> >> > well
> >> > in every FE Im involved in. The only thing which I miss
is the data
> >> > aggregation.
> >> >
> >> > So I would go back to the original topic of this thread - do
> some small
> >> > change which has a chance to be merged to the project and
helps
> us where
> >> > it
> >> > hurts.
> >> >
> >> >>
> >> >>
> >> >>
> >> >> Martin
> >> >>
> >> >> On Fri, Mar 24, 2017 at 3:47 PM, Greg Sheremeta
> <gshereme(a)redhat.com <mailto:gshereme@redhat.com>
<mailto:gshereme@redhat.com <mailto:gshereme@redhat.com>>>
> >> >> wrote:
> >> >> > I feel like every REST API I've ever worked with
has had the
> >> >> > aggregation
> >> >> > +
> >> >> > projection problem. It's like we're trying to
use REST as a
> >> >> > replacement
> >> >> > for
> >> >> > SQL -- but the logic that executes the
"SQL" lives in a
> browser now,
> >> >> > and
> >> >> > it
> >> >> > used to live on a server close to the DB. And REST
isn't
> expressive
> >> >> > for
> >> >> > selecting data like SQL is.
> >> >> >
> >> >> > There must be some industry solution to this "I
want to do
> SQL over
> >> >> > REST"
> >> >> > problem.
> >> >> >
> >> >> > On Fri, Mar 24, 2017 at 5:54 AM, Martin Sivak
> <msivak(a)redhat.com <mailto:msivak@redhat.com>
<mailto:msivak@redhat.com <mailto:msivak@redhat.com>>>
> >> >> > wrote:
> >> >> >>
> >> >> >> > for quite some time I have been more or
less
involved in
> >> >> >> > development
> >> >> >> > of
> >> >> >> > various UIs for oVirt based entirely on the
oVirt's
REST API
> >> >> >> > ranging
> >> >> >> > from
> >> >> >> > the quite mature moVirt [1] through some
cockpit
> extensions to a
> >> >> >> > young
> >> >> >> > and
> >> >> >> > experimental user portal replacement [2].
> >> >> >>
> >> >> >> oVirt optimizer has the same issue..
> >> >> >>
> >> >> >> > 2: add some tiny service which would just
accept a
list of
> >> >> >> > queries,
> >> >> >> > execute
> >> >> >> > them locally (but using real HTTP requests)
and
return in one
> >> >> >> > bulk. A
> >> >> >> > naive
> >> >> >> > implementation just to give a sense of what
I mean of
> this would
> >> >> >> > be a
> >> >> >> > shell
> >> >> >> > script getting list of strings like
> >> >> >> >
"https://localhost/ovirt-engine/api/vms/123/sessions
<https://localhost/ovirt-engine/api/vms/123/sessions>
> <https://localhost/ovirt-engine/api/vms/123/sessions
<https://localhost/ovirt-engine/api/vms/123/sessions>>" iterate over
> >> >> >> > them
> >> >> >> > and
> >> >> >> > do a curl request for each, mangle the
results into one
> string and
> >> >> >> > return
> >> >> >> > (credits for this idea to msivak). Easy to
implement,
> possibility
> >> >> >> > to
> >> >> >> > add
> >> >> >> > also projections later to save some
bandwidth. But the
> API would
> >> >> >> > anyway
> >> >> >> > be
> >> >> >> > hammered by bunch of queries, only the
network
roundtrip
> would be
> >> >> >> > saved.
> >> >> >>
> >> >> >> The biggest cost for (especially mobile) clients
is
the cost of
> >> >> >> establishing new SSL connection. SSL is also
pretty
> expensive on the
> >> >> >> server side.
> >> >> >>
> >> >> >> So running the aggregation service on the
ovirt-engine machine
> >> >> >> (behind
> >> >> >> Apache) means the client will do a single SSL
request
with
> list of N
> >> >> >> urls and the local "reverse-proxy" will
perform single
> >> >> >> authentication
> >> >> >> and N plain HTTP requests (or even better - AJP).
It
won't
> remove
> >> >> >> any
> >> >> >> time from the actual command run time, but it
will reduce
> protocol
> >> >> >> overhead.
> >> >> >>
> >> >> >> I think this is the simplest first step that
requires
almost no
> >> >> >> change
> >> >> >> to existing infrastructure.
> >> >> >>
> >> >> >> --
> >> >> >> Martin Sivak
> >> >> >> SLA / oVirt
> >> >> >>
> >> >> >> On Fri, Mar 24, 2017 at 10:20 AM, Tomas Jelinek
> >> >> >> <tjelinek(a)redhat.com
<mailto:tjelinek@redhat.com>
<mailto:tjelinek@redhat.com <mailto:tjelinek@redhat.com>>>
> >> >> >> wrote:
> >> >> >> > Hi All,
> >> >> >> >
> >> >> >> > for quite some time I have been more or
less
involved in
> >> >> >> > development
> >> >> >> > of
> >> >> >> > various UIs for oVirt based entirely on the
oVirt's
REST API
> >> >> >> > ranging
> >> >> >> > from
> >> >> >> > the quite mature moVirt [1] through some
cockpit
> extensions to a
> >> >> >> > young
> >> >> >> > and
> >> >> >> > experimental user portal replacement [2].
> >> >> >> >
> >> >> >> > One issue we hit over and over again is the
missing
data
> >> >> >> > aggregation.
> >> >> >> > In
> >> >> >> > the
> >> >> >> > 3.x era we used to use in moVirt the
detail=something
> >> >> >> > api to get the disks and nics of the VM,
something
like:
> >> >> >> >
> >> >> >> > GET /ovirt-engine/api/vms
> >> >> >> > Accept: application/json; detail=disks
> >> >> >> >
> >> >> >> > This allowed us to store this data in local
database
> leading to
> >> >> >> > great
> >> >> >> > user
> >> >> >> > experience. Since this feature has been
removed in 4.x
> API [3]
> >> >> >> > we needed to retire to a different solution.
When
the VM
> detail is
> >> >> >> > selected
> >> >> >> > by the user, start loading the disks and
nics and hope
> the user
> >> >> >> > will not be fast enough to see the delay.
The UX is
> slightly worse
> >> >> >> > bug
> >> >> >> > kinda
> >> >> >> > acceptable.
> >> >> >> >
> >> >> >> > We hit this issue harder in the new user
portal [2],
> because we
> >> >> >> > already
> >> >> >> > have
> >> >> >> > the VM cached and show the whole VM in one
screen.
So, if
> you pick
> >> >> >> > it,
> >> >> >> > you
> >> >> >> > will get it's details immediately.
> >> >> >> > But, since you don't have all the
details, we need
to do an
> >> >> >> > additional
> >> >> >> > call
> >> >> >> > (two actually) to load this data and they
start to
appear
> later.
> >> >> >> > So, something which would be very fast and
smooth
starts
> to feel
> >> >> >> > sluggish.
> >> >> >> >
> >> >> >> > Recently, we hit this issue again which
forced us to
> sacrifice the
> >> >> >> > UX
> >> >> >> > even
> >> >> >> > more - it is the "console in use"
feature of user
portal.
> >> >> >> > The use case is this:
> >> >> >> > - if the console is already taken by some
user,
there are
> >> >> >> > complications
> >> >> >> > if
> >> >> >> > other current user tryes to take it as well
(will avoid
> details
> >> >> >> > about
> >> >> >> > settings and permissins involved, but long
story short,
> the user
> >> >> >> > will
> >> >> >> > probably not be allowed to connect to it.
The
"probably"
> is the
> >> >> >> > key
> >> >> >> > here
> >> >> >> > since we can not do any intelligent decision
in
advance,
> we can
> >> >> >> > only
> >> >> >> > warn
> >> >> >> > the user that the console is taken).
> >> >> >> > - in the current GWT user portal, if the
VM's
console is
> taken, it
> >> >> >> > is
> >> >> >> > shown
> >> >> >> > on the VM's "box" that
"console is taken". This was
a highly
> >> >> >> > requested
> >> >> >> > feature
> >> >> >> > - to get this information using the current
REST
API, we
> need to
> >> >> >> > go
> >> >> >> > to
> >> >> >> > the
> >> >> >> > /vms/<vmid>/sessions subcollection. To
get this for all
> VMs, it
> >> >> >> > would
> >> >> >> > be
> >> >> >> > doing N queries per poll which we can not
afford
> >> >> >> > - so the current PR [4] will probably end up
to only
> check it on
> >> >> >> > the
> >> >> >> > attempt
> >> >> >> > to connect to the console warning the user.
Maybe
it will
> be also
> >> >> >> > shown
> >> >> >> > in
> >> >> >> > Vm details. But the UX in case the user will
look for a
> VM which
> >> >> >> > has
> >> >> >> > free
> >> >> >> > console will suffer significantly (e.g. try
one by one
> until some
> >> >> >> > opens
> >> >> >> > or
> >> >> >> > look at details one by one to see if the
warning
appears
> (with a
> >> >> >> > delay))
> >> >> >> >
> >> >> >> > I understand that embedding the details of
the VM
to the
> response
> >> >> >> > comes
> >> >> >> > with
> >> >> >> > a cost, namely:
> >> >> >> > - performance hit
> >> >> >> > - complexity of the API code
> >> >> >> > - the "cleanness" of REST suffers
> >> >> >> >
> >> >> >> > But I think we should seriously consider to
provide
some
> option to
> >> >> >> > data
> >> >> >> > aggregation.
> >> >> >> >
> >> >> >> > I know this has been discussed many times
with no
result,
> but I
> >> >> >> > think
> >> >> >> > it
> >> >> >> > is
> >> >> >> > time to bring this topic up again. I'll
try to
summarize the
> >> >> >> > (failed)
> >> >> >> > attempts tried so far:
> >> >> >> > - the detail=<something> parameter
with ad-hoc
embedding
> of data.
> >> >> >> > This
> >> >> >> > has
> >> >> >> > been there and removed in 4.0 [3]
> >> >> >> > - the DoctorREST project - e.g. a proxy
above the
current
> api. The
> >> >> >> > idea
> >> >> >> > was
> >> >> >> > to create a service which will be
independent of
the engine
> >> >> >> > itself,
> >> >> >> > will
> >> >> >> > locally poll the engine's REST, store
all data in local
> (mongo)DB
> >> >> >> > and
> >> >> >> > provide a rich api with aggregations and
projections and push
> >> >> >> > notifications.
> >> >> >> > This polling of everything to get the data
to
DoctorREST
> proved to
> >> >> >> > be
> >> >> >> > pretty
> >> >> >> > costy, so also a more invasive approach of
pushing
data from
> >> >> >> > engine
> >> >> >> > to
> >> >> >> > doctor has been discused [5]. None of this
two
approaches
> have
> >> >> >> > been
> >> >> >> > accepted
> >> >> >> > (too complicated, too invasive).
> >> >> >> > - writing some custom ad-hoc servlet serving
only a
> purpose of one
> >> >> >> > frontend
> >> >> >> > - this is actually there for the dashboard,
but it
is not a
> >> >> >> > generic
> >> >> >> > solution
> >> >> >> > for the other frontends and we really should
not
develop
> custom
> >> >> >> > "APIs"
> >> >> >> > for
> >> >> >> > every frontend
> >> >> >> > - there were some other proposals discussed
(some
3th party
> >> >> >> > solutions
> >> >> >> > etc)
> >> >> >> > but I think none of them made it even to a
PoC
> >> >> >> >
> >> >> >> > So, now I would try again and try small to
get at
least some
> >> >> >> > benefit.
> >> >> >> > I
> >> >> >> > see
> >> >> >> > 2 paths we could try:
> >> >> >> > 1: embed something which burns us
immediatly, e.g. the
> /sessions
> >> >> >> > into
> >> >> >> > VMs. I
> >> >> >> > really liked the ;detail=sessions approach,
could
we move
> it back?
> >> >> >> > 2: add some tiny service which would just
accept a
list of
> >> >> >> > queries,
> >> >> >> > execute
> >> >> >> > them locally (but using real HTTP requests)
and
return in one
> >> >> >> > bulk. A
> >> >> >> > naive
> >> >> >> > implementation just to give a sense of what
I mean of
> this would
> >> >> >> > be a
> >> >> >> > shell
> >> >> >> > script getting list of strings like
> >> >> >> >
"https://localhost/ovirt-engine/api/vms/123/sessions
<https://localhost/ovirt-engine/api/vms/123/sessions>
> <https://localhost/ovirt-engine/api/vms/123/sessions
<https://localhost/ovirt-engine/api/vms/123/sessions>>" iterate over
> >> >> >> > them
> >> >> >> > and
> >> >> >> > do a curl request for each, mangle the
results into one
> string and
> >> >> >> > return
> >> >> >> > (credits for this idea to msivak). Easy to
implement,
> possibility
> >> >> >> > to
> >> >> >> > add
> >> >> >> > also projections later to save some
bandwidth. But the
> API would
> >> >> >> > anyway
> >> >> >> > be
> >> >> >> > hammered by bunch of queries, only the
network
roundtrip
> would be
> >> >> >> > saved.
> >> >> >> > 3: any other simple approaches?
> >> >> >> >
> >> >> >> > I honestly prefer the first approach. It is
not
> beautiful, it is
> >> >> >> > not
> >> >> >> > REST-ful, but it is easy to implement, very
pragmatic and
> useful.
> >> >> >> > What do you think?
> >> >> >> >
> >> >> >> > Thank you and sorry for the long mail :)
> >> >> >> > Tomas
> >> >> >> >
> >> >> >> > [1]: https://github.com/oVirt/moVirt
<https://github.com/oVirt/moVirt>
> <https://github.com/oVirt/moVirt
<https://github.com/oVirt/moVirt>>
> >> >> >> > [2]: https://github.com/oVirt/ovirt-web-ui
<https://github.com/oVirt/ovirt-web-ui>
> <https://github.com/oVirt/ovirt-web-ui
<https://github.com/oVirt/ovirt-web-ui>>
> >> >> >> > [3]: https://gerrit.ovirt.org/#/c/61260
<https://gerrit.ovirt.org/#/c/61260>
> <https://gerrit.ovirt.org/#/c/61260
<https://gerrit.ovirt.org/#/c/61260>>
> >> >> >> > [4]:
https://github.com/oVirt/ovirt-web-ui/pull/106/
<https://github.com/oVirt/ovirt-web-ui/pull/106/>
> <https://github.com/oVirt/ovirt-web-ui/pull/106/
<https://github.com/oVirt/ovirt-web-ui/pull/106/>>
> >> >> >> > [5]: https://gerrit.ovirt.org/#/c/45233/
<https://gerrit.ovirt.org/#/c/45233/>
> <https://gerrit.ovirt.org/#/c/45233/
<https://gerrit.ovirt.org/#/c/45233/>>
> >> >> >> >
> >> >> >> >
> >> >> >> >
_______________________________________________
> >> >> >> > Devel mailing list
> >> >> >> > Devel(a)ovirt.org
<mailto:Devel@ovirt.org>
<mailto:Devel@ovirt.org <mailto:Devel@ovirt.org>>
> >> >> >> >
http://lists.ovirt.org/mailman/listinfo/devel
<http://lists.ovirt.org/mailman/listinfo/devel>
> <http://lists.ovirt.org/mailman/listinfo/devel
<http://lists.ovirt.org/mailman/listinfo/devel>>
> >> >> >> _______________________________________________
> >> >> >> Devel mailing list
> >> >> >> Devel(a)ovirt.org <mailto:Devel@ovirt.org>
<mailto:Devel@ovirt.org <mailto:Devel@ovirt.org>>
> >> >> >> http://lists.ovirt.org/mailman/listinfo/devel
<http://lists.ovirt.org/mailman/listinfo/devel>
> <http://lists.ovirt.org/mailman/listinfo/devel
<http://lists.ovirt.org/mailman/listinfo/devel>>
> >> >> >
> >> >> >
> >> >> >
> >> >> >
> >> >> > --
> >> >> > Greg Sheremeta, MBA
> >> >> > Red Hat, Inc.
> >> >> > Sr. Software Engineer
> >> >> > gshereme(a)redhat.com
<mailto:gshereme@redhat.com>
<mailto:gshereme@redhat.com <mailto:gshereme@redhat.com>>
> >> >
> >> >
> >> _______________________________________________
> >> Devel mailing list
> >> Devel(a)ovirt.org <mailto:Devel@ovirt.org>
<mailto:Devel@ovirt.org <mailto:Devel@ovirt.org>>
> >> http://lists.ovirt.org/mailman/listinfo/devel
<http://lists.ovirt.org/mailman/listinfo/devel>
> <http://lists.ovirt.org/mailman/listinfo/devel
<http://lists.ovirt.org/mailman/listinfo/devel>>
> >
> >
>
>
5:59 a.m.
On Mon, Mar 27, 2017 at 1:32 PM, Juan Hernández <jhernand(a)redhat.com> wrote:
On 03/27/2017 01:03 PM, Tomas Jelinek wrote:
>
>
> On Mon, Mar 27, 2017 at 11:21 AM, Juan Hernández <jhernand(a)redhat.com
> <mailto:jhernand@redhat.com>> wrote:
>
> Top posting, sorry.
>
> There are a few things I'd like to clarify, regarding this subject:
>
> 1. Data aggregation, as requested now by Tomas, and by other people
in
> the past.
>
> We used to have that 'detail' parameter, to aggregate certain very
> specific types of data, in particular to aggregate VM disks and
NICs. We
> removed that in version 4 of the API because the implementation was
> extremely inefficient, from the engine point of view. An innocent
> request like this:
>
> GET /ovirt-engine/api/vms?detail=+disks,+nics
>
> Would generate, with the implementation we used to have, 1 query for
the
> VMs and then as many queries for disks and NICs as VMs in the
system. In
> our scale test environments, for example, with approx 4000 VMs and
10000
> disks, that would take more than 20 hours to execute.
>
> In addition, we didn't have in the past any mechanism to make this
> available in a generic one, because there was no knowledge in the
API of
> what are 'details'.
>
> In version 4 of the API we introduced a formal (kind of)
specification
> of the API (a.k.a. the model), and int includes knowledge about what
are
> 'links'. For example, the specification of the VM type contains this:
>
> @Link DiskAttachment[] diskAttachments();
> @Link Nic[] nics();
>
> With this information we are now in a position where we can implement
> this in a generic way.
>
> We intend to implement this using a mechanism similar to the existing
> 'detail' parameter:
>
> GET /ovirt-engine/api/vms/123?follow=disk_attachments,nics
>
> The naive implementation of this is to let the API call itself. For
> example, when the user requests to follow the 'disk_attachments'
detail
> the API can just call itself to get that:
>
> GET /ovirt-engine/api/vms/123/disk_attachments
>
> However, we can't use that naive approach, if we do we end with the
> 1+C*N query problem described before. We need to use specific
> implementations for certain frequent use cases, like VMs+disks+nics,
and
> that needs work in the API and in the backend.
>
> Tomas, if you want to help moving this forward, please open a RFE and
> makes sure it gets attention.
ok, opened: https://bugzilla.redhat.com/show_bug.cgi?id=1436206
Will try to get it done soon.
>
>
> This sounds pretty good! I will open, but since we are talking already
> here I'll just use the opportunity to clarify the topic more and than
> I'll open the BZ.
>
> What I can imagine is the GetAllVmsQuery will accept in params also the
> list of details it should provide. Than, the GetAllVmsQuery will
> implement the efficient way of retrieving this info as well.
>
> So, from the API perspective, it will be about taking the
> ?follow=<something> part and passing it to the backend query params.
>
> What you think?
>
Exactly, that is the point! The API by itself can't optimize database
queries, all it can do is call the backend. It is the backend that has
the opportunity and possibility to send optimized queries to the database.
For other less common things we can use the naive approach, and
implement the aggregation in the API itself. But for common use cases,
like VM+disks+nics, we need to do it in an efficient way.
>
>
> 2. Reuse of TLS sessions.
>
> The part of creating TLS sessions that is expensive is the
generation of
> the shared session key. That can be avoided if both the server and
the
> client are careful and reuse the session, using the session cache
> mechanism built-in into TLS itself. The web servers that we use
(Apache
> and Undertow) do implement this mechanism, and so do most of our
> clients. Make sure that your client uses it as well. In Java this is
> achieved re-using the SSLContext. We already do that for the engine
to
> VDSM communciation for example. In JavaScript the browser already
takes
> care of this.
>
> 3. Parallelism and latency.
>
> A typical problem that we have is that we send many request to the
> server. For example, to retrieve user sessions for a set of VMs we
tend
> to send many requests like this:
>
> GET /ovirt-engine/api/vms/1/sessions
> GET /ovirt/engine/api/vms/2/sessions
> GET /ovirt-engine/api/vms/3/sessions
> ...
>
> And we do that in a synchronous way: send one, wait for the result,
send
> another one, wait for the result, etc. This means that we don't take
> advantage of the parallelism of the server and that we add to each
> request the network round trip time. So if we have N requests, we
have
> to wait at least N*RTT.
>
> The web servers that we use support multiple connections, and the
> protocol that we use, HTTP, supports pipe-lining. This means that you
> can send multiple requests in parallel, and that you can send
multiple
> requests without waiting for the response. To give you an idea of the
> improvement that can be achieved, we recently added asynchronous
request
> support to the Ruby SDK, with multiple connections and pipe-lining.
In
> our scale testing environment that reduced the time to collect a
> complete inventory from approx 30 min to approx 2 min. Here you have
an
> example:
>
>
> https://github.com/oVirt/ovirt-engine-sdk-ruby/blob/
master/sdk/examples/asynchronous_inventory.rb
> <https://github.com/oVirt/ovirt-engine-sdk-ruby/blob/
master/sdk/examples/asynchronous_inventory.rb>
>
> So make sure that you take advantage of that in your clients. Sadly
> pipe-lining is disabled by default in most browsers, so this isn't
> helpful for JavaScript applications.
>
>
> But we can try what we can do in moVirt about
> this: https://github.com/oVirt/moVirt/issues/260
>
Sure, I think there are plenty of asynchronous HTTP clients for Android,
worth trying one of them. If you are brave enough you can even consider
using the same library used in the Ruby SDK: libcurl. A bit of JNI here
and there, and you are done.
>
>
> 4. HTTP/2 support.
>
> The application server that we use, WildFly, supports HTTP/2,
including
> ALPN, out of the box, since version 10.1. We need a mechanism to
> enable it:
>
> core: Add support for enabling HTTP/2
> https://gerrit.ovirt.org/74621
>
> And then we need to get Apache out of the way, for API traffic, at
> least. I think that is something we can do in the context of the
engine
> "podification" effort.
>
> However, note that HTTP/2 won't have that big impact in performance
for
> applications that continue to use a synchronous/serial style of
> interaction with the API.
>
> On 03/24/2017 11:16 PM, Yaniv Kaul wrote:
> >
> >
> > On Fri, Mar 24, 2017 at 8:57 PM, Martin Sivak <msivak(a)redhat.com
<mailto:msivak@redhat.com>
> > <mailto:msivak@redhat.com <mailto:msivak@redhat.com>>>
wrote:
> >
> > > Current Apache used has only experimental module for it.
> > > Undertow is supposed to have a better support. I wonder
when/if we can drop
> > > Apache...
> >
> > The last info I have about that from mperina is that we need
Apache
> > for kerberos support atm.
> >
> >
> > I don't think we need it - I remember reading that Undertow does
support
> > it as well.
> > The only issue is that there are probably 10 people in the world
who
> > know how to configure Undertow for Kerberos, while many do for
Apache.
> > And since we leave it for the user to configure...
> > Y.
> >
> >
> >
> > Martin
> >
> > On Fri, Mar 24, 2017 at 5:30 PM, Yaniv Kaul <ykaul(a)redhat.com
<mailto:ykaul@redhat.com>
> > <mailto:ykaul@redhat.com <mailto:ykaul@redhat.com>>>
wrote:
> > >
> > >
> > > On Fri, Mar 24, 2017 at 6:43 PM, Martin Sivak <
msivak(a)redhat.com <mailto:msivak@redhat.com>
> > <mailto:msivak@redhat.com <mailto:msivak@redhat.com>>>
wrote:
> > >>
> > >> > 2: you can have more api gateways (e.g. more apis)
> tailored for
> > every
> > >> > frontend. I don't think we need this - the current
API
serves
> > us pretty
> > >> > well
> > >> > in every FE Im involved in. The only thing which I miss
> is the data
> > >> > aggregation.
> > >>
> > >> So it does not serve us well. Aggregation of data is one
> the usual
> > >> points of using the gateway.
> > >> Yes microservices are affected by this indeed, but so are
> we because
> > >> implementing the aggregation directly in the current engine
> API layer
> > >> is hard.
> > >>
> > >> > So I would go back to the original topic of this thread
-
do
> > some small
> > >> > change which has a chance to be merged to the project
and
> helps
> > us where
> > >> > it
> > >> > hurts.
> > >
> > >
> > > I'm wondering if very specific additional REST API calls can
> suffice.
> > > For example, a 'Get VM + disks + NIC' API call seems
> reasonable to
> > add for
> > > the various clients who commonly need it.
> > >
> > >>
> > >> Can a simple HTTP/2 to HTTP/AJP gateway be the simplest
> solution? Our
> > >> Apache might even have a module for it already.
> > >
> > >
> > > Current Apache used has only experimental module for it.
> > > Undertow is supposed to have a better support. I wonder
> when/if we
> > can drop
> > > Apache...
> > > Y.
> > >
> > >>
> > >> That way you can multiplex all the REST calls using a
> single tcp
> > >> connection (and a single SSL negotiation).
> > >>
> > >> A custom SSO enabled service like that might be even better
> as it
> > >> would be able to skip the authentication
> > >> layers too and that would lower the engine load. But I am
> not sure it
> > >> is possible with the current codebase.
> > >>
> > >> Martin
> > >>
> > >> On Fri, Mar 24, 2017 at 4:22 PM, Tomas Jelinek
> > <tjelinek(a)redhat.com <mailto:tjelinek@redhat.com>
> <mailto:tjelinek@redhat.com <mailto:tjelinek@redhat.com>>>
> > >> wrote:
> > >> >
> > >> >
> > >> > On Fri, Mar 24, 2017 at 3:58 PM, Martin Sivak
> > <msivak(a)redhat.com <mailto:msivak@redhat.com>
> <mailto:msivak@redhat.com <mailto:msivak@redhat.com>>> wrote:
> > >> >>
> > >> >> > I feel like every REST API I've ever worked
with has
> had the
> > >> >> > aggregation
> > >> >> > +
> > >> >> > projection problem. It's like we're
trying to use REST
> as a
> > >> >> > replacement
> > >> >> > for
> > >> >> > SQL -- but the logic that executes the
"SQL" lives in a
> > browser now,
> > >> >> > and
> > >> >> > it
> > >> >> > used to live on a server close to the DB. And
REST
isn't
> > expressive
> > >> >> > for
> > >> >> > selecting data like SQL is.
> > >> >>
> > >> >> The current industry solution I know about is called
API
> gateway..
> > >> >> most of the big players have internal API with lots
of
low
> > level stuff
> > >> >> and then couple of external API gateways tailored to
> what the
> > client
> > >> >> needs.
> > >> >>
> > >> >> http://microservices.io/patterns/apigateway.html
> <http://microservices.io/patterns/apigateway.html>
> > <http://microservices.io/patterns/apigateway.html
> <http://microservices.io/patterns/apigateway.html>> (check the
backend
> > >> >> for frontend section)
> > >> >>
> > >> >> This trend is also visible when you think about
services
> that
> > offer
> > >> >> API gateway management and billing like
> > >> >> https://aws.amazon.com/api-gateway/
> <https://aws.amazon.com/api-gateway/>
> > <https://aws.amazon.com/api-gateway/
> <https://aws.amazon.com/api-gateway/>> or our very own
> > >> >> https://www.3scale.net/
> > >> >
> > >> >
> > >> > right, but the api gateway solves 2 problems:
> > >> >
> > >> > 1: if you have a microservice architecture it is hard
for
> > frontend to
> > >> > talk
> > >> > to 20 different moving services. So the gateway hides
this
> > complexity
> > >> > behind
> > >> > it. This is not the problem we have.
> > >> >
> > >> > 2: you can have more api gateways (e.g. more apis)
> tailored for
> > every
> > >> > frontend. I don't think we need this - the current
API
serves
> > us pretty
> > >> > well
> > >> > in every FE Im involved in. The only thing which I miss
> is the data
> > >> > aggregation.
> > >> >
> > >> > So I would go back to the original topic of this thread
-
do
> > some small
> > >> > change which has a chance to be merged to the project
and
> helps
> > us where
> > >> > it
> > >> > hurts.
> > >> >
> > >> >>
> > >> >>
> > >> >>
> > >> >> Martin
> > >> >>
> > >> >> On Fri, Mar 24, 2017 at 3:47 PM, Greg Sheremeta
> > <gshereme(a)redhat.com <mailto:gshereme@redhat.com>
> <mailto:gshereme@redhat.com <mailto:gshereme@redhat.com>>>
> > >> >> wrote:
> > >> >> > I feel like every REST API I've ever worked
with has
had the
> > >> >> > aggregation
> > >> >> > +
> > >> >> > projection problem. It's like we're
trying to use REST
as a
> > >> >> > replacement
> > >> >> > for
> > >> >> > SQL -- but the logic that executes the
"SQL" lives in a
> > browser now,
> > >> >> > and
> > >> >> > it
> > >> >> > used to live on a server close to the DB. And
REST
isn't
> > expressive
> > >> >> > for
> > >> >> > selecting data like SQL is.
> > >> >> >
> > >> >> > There must be some industry solution to this
"I want
to do
> > SQL over
> > >> >> > REST"
> > >> >> > problem.
> > >> >> >
> > >> >> > On Fri, Mar 24, 2017 at 5:54 AM, Martin Sivak
> > <msivak(a)redhat.com <mailto:msivak@redhat.com>
> <mailto:msivak@redhat.com <mailto:msivak@redhat.com>>>
> > >> >> > wrote:
> > >> >> >>
> > >> >> >> > for quite some time I have been more or
less
> involved in
> > >> >> >> > development
> > >> >> >> > of
> > >> >> >> > various UIs for oVirt based entirely on
the oVirt's
> REST API
> > >> >> >> > ranging
> > >> >> >> > from
> > >> >> >> > the quite mature moVirt [1] through
some cockpit
> > extensions to a
> > >> >> >> > young
> > >> >> >> > and
> > >> >> >> > experimental user portal replacement
[2].
> > >> >> >>
> > >> >> >> oVirt optimizer has the same issue..
> > >> >> >>
> > >> >> >> > 2: add some tiny service which would
just accept a
> list of
> > >> >> >> > queries,
> > >> >> >> > execute
> > >> >> >> > them locally (but using real HTTP
requests) and
> return in one
> > >> >> >> > bulk. A
> > >> >> >> > naive
> > >> >> >> > implementation just to give a sense of
what I mean
of
> > this would
> > >> >> >> > be a
> > >> >> >> > shell
> > >> >> >> > script getting list of strings like
> > >> >> >> >
> "https://localhost/ovirt-engine/api/vms/123/sessions
> <https://localhost/ovirt-engine/api/vms/123/sessions>
> > <https://localhost/ovirt-engine/api/vms/123/sessions
> <https://localhost/ovirt-engine/api/vms/123/sessions>>" iterate
over
> > >> >> >> > them
> > >> >> >> > and
> > >> >> >> > do a curl request for each, mangle the
results into
one
> > string and
> > >> >> >> > return
> > >> >> >> > (credits for this idea to msivak). Easy
to
implement,
> > possibility
> > >> >> >> > to
> > >> >> >> > add
> > >> >> >> > also projections later to save some
bandwidth. But
the
> > API would
> > >> >> >> > anyway
> > >> >> >> > be
> > >> >> >> > hammered by bunch of queries, only the
network
> roundtrip
> > would be
> > >> >> >> > saved.
> > >> >> >>
> > >> >> >> The biggest cost for (especially mobile)
clients is
> the cost of
> > >> >> >> establishing new SSL connection. SSL is also
pretty
> > expensive on the
> > >> >> >> server side.
> > >> >> >>
> > >> >> >> So running the aggregation service on the
> ovirt-engine machine
> > >> >> >> (behind
> > >> >> >> Apache) means the client will do a single
SSL request
> with
> > list of N
> > >> >> >> urls and the local "reverse-proxy"
will perform single
> > >> >> >> authentication
> > >> >> >> and N plain HTTP requests (or even better -
AJP). It
> won't
> > remove
> > >> >> >> any
> > >> >> >> time from the actual command run time, but
it will
reduce
> > protocol
> > >> >> >> overhead.
> > >> >> >>
> > >> >> >> I think this is the simplest first step that
requires
> almost no
> > >> >> >> change
> > >> >> >> to existing infrastructure.
> > >> >> >>
> > >> >> >> --
> > >> >> >> Martin Sivak
> > >> >> >> SLA / oVirt
> > >> >> >>
> > >> >> >> On Fri, Mar 24, 2017 at 10:20 AM, Tomas
Jelinek
> > >> >> >> <tjelinek(a)redhat.com
<mailto:tjelinek@redhat.com>
> <mailto:tjelinek@redhat.com <mailto:tjelinek@redhat.com>>>
> > >> >> >> wrote:
> > >> >> >> > Hi All,
> > >> >> >> >
> > >> >> >> > for quite some time I have been more or
less
> involved in
> > >> >> >> > development
> > >> >> >> > of
> > >> >> >> > various UIs for oVirt based entirely on
the oVirt's
> REST API
> > >> >> >> > ranging
> > >> >> >> > from
> > >> >> >> > the quite mature moVirt [1] through
some cockpit
> > extensions to a
> > >> >> >> > young
> > >> >> >> > and
> > >> >> >> > experimental user portal replacement
[2].
> > >> >> >> >
> > >> >> >> > One issue we hit over and over again is
the missing
> data
> > >> >> >> > aggregation.
> > >> >> >> > In
> > >> >> >> > the
> > >> >> >> > 3.x era we used to use in moVirt the
detail=something
> > >> >> >> > api to get the disks and nics of the
VM, something
> like:
> > >> >> >> >
> > >> >> >> > GET /ovirt-engine/api/vms
> > >> >> >> > Accept: application/json; detail=disks
> > >> >> >> >
> > >> >> >> > This allowed us to store this data in
local database
> > leading to
> > >> >> >> > great
> > >> >> >> > user
> > >> >> >> > experience. Since this feature has been
removed in
4.x
> > API [3]
> > >> >> >> > we needed to retire to a different
solution. When
> the VM
> > detail is
> > >> >> >> > selected
> > >> >> >> > by the user, start loading the disks
and nics and
hope
> > the user
> > >> >> >> > will not be fast enough to see the
delay. The UX is
> > slightly worse
> > >> >> >> > bug
> > >> >> >> > kinda
> > >> >> >> > acceptable.
> > >> >> >> >
> > >> >> >> > We hit this issue harder in the new
user portal [2],
> > because we
> > >> >> >> > already
> > >> >> >> > have
> > >> >> >> > the VM cached and show the whole VM in
one screen.
> So, if
> > you pick
> > >> >> >> > it,
> > >> >> >> > you
> > >> >> >> > will get it's details immediately.
> > >> >> >> > But, since you don't have all the
details, we need
> to do an
> > >> >> >> > additional
> > >> >> >> > call
> > >> >> >> > (two actually) to load this data and
they start to
> appear
> > later.
> > >> >> >> > So, something which would be very fast
and smooth
> starts
> > to feel
> > >> >> >> > sluggish.
> > >> >> >> >
> > >> >> >> > Recently, we hit this issue again which
forced us to
> > sacrifice the
> > >> >> >> > UX
> > >> >> >> > even
> > >> >> >> > more - it is the "console in
use" feature of user
> portal.
> > >> >> >> > The use case is this:
> > >> >> >> > - if the console is already taken by
some user,
> there are
> > >> >> >> > complications
> > >> >> >> > if
> > >> >> >> > other current user tryes to take it as
well (will
avoid
> > details
> > >> >> >> > about
> > >> >> >> > settings and permissins involved, but
long story
short,
> > the user
> > >> >> >> > will
> > >> >> >> > probably not be allowed to connect to
it. The
> "probably"
> > is the
> > >> >> >> > key
> > >> >> >> > here
> > >> >> >> > since we can not do any intelligent
decision in
> advance,
> > we can
> > >> >> >> > only
> > >> >> >> > warn
> > >> >> >> > the user that the console is taken).
> > >> >> >> > - in the current GWT user portal, if
the VM's
> console is
> > taken, it
> > >> >> >> > is
> > >> >> >> > shown
> > >> >> >> > on the VM's "box" that
"console is taken". This was
> a highly
> > >> >> >> > requested
> > >> >> >> > feature
> > >> >> >> > - to get this information using the
current REST
> API, we
> > need to
> > >> >> >> > go
> > >> >> >> > to
> > >> >> >> > the
> > >> >> >> > /vms/<vmid>/sessions
subcollection. To get this for
all
> > VMs, it
> > >> >> >> > would
> > >> >> >> > be
> > >> >> >> > doing N queries per poll which we can
not afford
> > >> >> >> > - so the current PR [4] will probably
end up to only
> > check it on
> > >> >> >> > the
> > >> >> >> > attempt
> > >> >> >> > to connect to the console warning the
user. Maybe
> it will
> > be also
> > >> >> >> > shown
> > >> >> >> > in
> > >> >> >> > Vm details. But the UX in case the user
will look
for a
> > VM which
> > >> >> >> > has
> > >> >> >> > free
> > >> >> >> > console will suffer significantly (e.g.
try one by
one
> > until some
> > >> >> >> > opens
> > >> >> >> > or
> > >> >> >> > look at details one by one to see if
the warning
> appears
> > (with a
> > >> >> >> > delay))
> > >> >> >> >
> > >> >> >> > I understand that embedding the details
of the VM
> to the
> > response
> > >> >> >> > comes
> > >> >> >> > with
> > >> >> >> > a cost, namely:
> > >> >> >> > - performance hit
> > >> >> >> > - complexity of the API code
> > >> >> >> > - the "cleanness" of REST
suffers
> > >> >> >> >
> > >> >> >> > But I think we should seriously
consider to provide
> some
> > option to
> > >> >> >> > data
> > >> >> >> > aggregation.
> > >> >> >> >
> > >> >> >> > I know this has been discussed many
times with no
> result,
> > but I
> > >> >> >> > think
> > >> >> >> > it
> > >> >> >> > is
> > >> >> >> > time to bring this topic up again.
I'll try to
> summarize the
> > >> >> >> > (failed)
> > >> >> >> > attempts tried so far:
> > >> >> >> > - the detail=<something>
parameter with ad-hoc
> embedding
> > of data.
> > >> >> >> > This
> > >> >> >> > has
> > >> >> >> > been there and removed in 4.0 [3]
> > >> >> >> > - the DoctorREST project - e.g. a proxy
above the
> current
> > api. The
> > >> >> >> > idea
> > >> >> >> > was
> > >> >> >> > to create a service which will be
independent of
> the engine
> > >> >> >> > itself,
> > >> >> >> > will
> > >> >> >> > locally poll the engine's REST,
store all data in
local
> > (mongo)DB
> > >> >> >> > and
> > >> >> >> > provide a rich api with aggregations
and
> projections and push
> > >> >> >> > notifications.
> > >> >> >> > This polling of everything to get the
data to
> DoctorREST
> > proved to
> > >> >> >> > be
> > >> >> >> > pretty
> > >> >> >> > costy, so also a more invasive approach
of pushing
> data from
> > >> >> >> > engine
> > >> >> >> > to
> > >> >> >> > doctor has been discused [5]. None of
this two
> approaches
> > have
> > >> >> >> > been
> > >> >> >> > accepted
> > >> >> >> > (too complicated, too invasive).
> > >> >> >> > - writing some custom ad-hoc servlet
serving only a
> > purpose of one
> > >> >> >> > frontend
> > >> >> >> > - this is actually there for the
dashboard, but it
> is not a
> > >> >> >> > generic
> > >> >> >> > solution
> > >> >> >> > for the other frontends and we really
should not
> develop
> > custom
> > >> >> >> > "APIs"
> > >> >> >> > for
> > >> >> >> > every frontend
> > >> >> >> > - there were some other proposals
discussed (some
> 3th party
> > >> >> >> > solutions
> > >> >> >> > etc)
> > >> >> >> > but I think none of them made it even
to a PoC
> > >> >> >> >
> > >> >> >> > So, now I would try again and try small
to get at
> least some
> > >> >> >> > benefit.
> > >> >> >> > I
> > >> >> >> > see
> > >> >> >> > 2 paths we could try:
> > >> >> >> > 1: embed something which burns us
immediatly, e.g.
the
> > /sessions
> > >> >> >> > into
> > >> >> >> > VMs. I
> > >> >> >> > really liked the ;detail=sessions
approach, could
> we move
> > it back?
> > >> >> >> > 2: add some tiny service which would
just accept a
> list of
> > >> >> >> > queries,
> > >> >> >> > execute
> > >> >> >> > them locally (but using real HTTP
requests) and
> return in one
> > >> >> >> > bulk. A
> > >> >> >> > naive
> > >> >> >> > implementation just to give a sense of
what I mean
of
> > this would
> > >> >> >> > be a
> > >> >> >> > shell
> > >> >> >> > script getting list of strings like
> > >> >> >> >
> "https://localhost/ovirt-engine/api/vms/123/sessions
> <https://localhost/ovirt-engine/api/vms/123/sessions>
> > <https://localhost/ovirt-engine/api/vms/123/sessions
> <https://localhost/ovirt-engine/api/vms/123/sessions>>" iterate
over
> > >> >> >> > them
> > >> >> >> > and
> > >> >> >> > do a curl request for each, mangle the
results into
one
> > string and
> > >> >> >> > return
> > >> >> >> > (credits for this idea to msivak). Easy
to
implement,
> > possibility
> > >> >> >> > to
> > >> >> >> > add
> > >> >> >> > also projections later to save some
bandwidth. But
the
> > API would
> > >> >> >> > anyway
> > >> >> >> > be
> > >> >> >> > hammered by bunch of queries, only the
network
> roundtrip
> > would be
> > >> >> >> > saved.
> > >> >> >> > 3: any other simple approaches?
> > >> >> >> >
> > >> >> >> > I honestly prefer the first approach.
It is not
> > beautiful, it is
> > >> >> >> > not
> > >> >> >> > REST-ful, but it is easy to implement,
very
> pragmatic and
> > useful.
> > >> >> >> > What do you think?
> > >> >> >> >
> > >> >> >> > Thank you and sorry for the long mail
:)
> > >> >> >> > Tomas
> > >> >> >> >
> > >> >> >> > [1]: https://github.com/oVirt/moVirt
> <https://github.com/oVirt/moVirt>
> > <https://github.com/oVirt/moVirt
> <https://github.com/oVirt/moVirt>>
> > >> >> >> > [2]:
https://github.com/oVirt/ovirt-web-ui
> <https://github.com/oVirt/ovirt-web-ui>
> > <https://github.com/oVirt/ovirt-web-ui
> <https://github.com/oVirt/ovirt-web-ui>>
> > >> >> >> > [3]:
https://gerrit.ovirt.org/#/c/61260
> <https://gerrit.ovirt.org/#/c/61260>
> > <https://gerrit.ovirt.org/#/c/61260
> <https://gerrit.ovirt.org/#/c/61260>>
> > >> >> >> > [4]:
> https://github.com/oVirt/ovirt-web-ui/pull/106/
> <https://github.com/oVirt/ovirt-web-ui/pull/106/>
> > <https://github.com/oVirt/ovirt-web-ui/pull/106/
> <https://github.com/oVirt/ovirt-web-ui/pull/106/>>
> > >> >> >> > [5]:
https://gerrit.ovirt.org/#/c/45233/
> <https://gerrit.ovirt.org/#/c/45233/>
> > <https://gerrit.ovirt.org/#/c/45233/
> <https://gerrit.ovirt.org/#/c/45233/>>
> > >> >> >> >
> > >> >> >> >
> > >> >> >> >
_______________________________________________
> > >> >> >> > Devel mailing list
> > >> >> >> > Devel(a)ovirt.org
<mailto:Devel@ovirt.org>
> <mailto:Devel@ovirt.org <mailto:Devel@ovirt.org>>
> > >> >> >> >
http://lists.ovirt.org/mailman/listinfo/devel
> <http://lists.ovirt.org/mailman/listinfo/devel>
> > <http://lists.ovirt.org/mailman/listinfo/devel
> <http://lists.ovirt.org/mailman/listinfo/devel>>
> > >> >> >>
_______________________________________________
> > >> >> >> Devel mailing list
> > >> >> >> Devel(a)ovirt.org
<mailto:Devel@ovirt.org>
> <mailto:Devel@ovirt.org <mailto:Devel@ovirt.org>>
> > >> >> >>
http://lists.ovirt.org/mailman/listinfo/devel
> <http://lists.ovirt.org/mailman/listinfo/devel>
> > <http://lists.ovirt.org/mailman/listinfo/devel
> <http://lists.ovirt.org/mailman/listinfo/devel>>
> > >> >> >
> > >> >> >
> > >> >> >
> > >> >> >
> > >> >> > --
> > >> >> > Greg Sheremeta, MBA
> > >> >> > Red Hat, Inc.
> > >> >> > Sr. Software Engineer
> > >> >> > gshereme(a)redhat.com
<mailto:gshereme@redhat.com>
> <mailto:gshereme@redhat.com <mailto:gshereme@redhat.com>>
> > >> >
> > >> >
> > >> _______________________________________________
> > >> Devel mailing list
> > >> Devel(a)ovirt.org <mailto:Devel@ovirt.org>
> <mailto:Devel@ovirt.org <mailto:Devel@ovirt.org>>
> > >> http://lists.ovirt.org/mailman/listinfo/devel
> <http://lists.ovirt.org/mailman/listinfo/devel>
> > <http://lists.ovirt.org/mailman/listinfo/devel
> <http://lists.ovirt.org/mailman/listinfo/devel>>
> > >
> > >
> >
> >
>
>
7:49 a.m.
On Mon, Mar 27, 2017 at 8:59 AM, Tomas Jelinek <tjelinek(a)redhat.com> wrote:
On Mon, Mar 27, 2017 at 1:32 PM, Juan Hernández <jhernand(a)redhat.com>
wrote:
> On 03/27/2017 01:03 PM, Tomas Jelinek wrote:
> >
> >
> > On Mon, Mar 27, 2017 at 11:21 AM, Juan Hernández <jhernand(a)redhat.com
> > <mailto:jhernand@redhat.com>> wrote:
> >
> > Top posting, sorry.
> >
> > There are a few things I'd like to clarify, regarding this subject:
> >
> > 1. Data aggregation, as requested now by Tomas, and by other people
> in
> > the past.
> >
> > We used to have that 'detail' parameter, to aggregate certain very
> > specific types of data, in particular to aggregate VM disks and
> NICs. We
> > removed that in version 4 of the API because the implementation was
> > extremely inefficient, from the engine point of view. An innocent
> > request like this:
> >
> > GET /ovirt-engine/api/vms?detail=+disks,+nics
> >
> > Would generate, with the implementation we used to have, 1 query
> for the
> > VMs and then as many queries for disks and NICs as VMs in the
> system. In
> > our scale test environments, for example, with approx 4000 VMs and
> 10000
> > disks, that would take more than 20 hours to execute.
> >
> > In addition, we didn't have in the past any mechanism to make this
> > available in a generic one, because there was no knowledge in the
> API of
> > what are 'details'.
> >
> > In version 4 of the API we introduced a formal (kind of)
> specification
> > of the API (a.k.a. the model), and int includes knowledge about
> what are
> > 'links'. For example, the specification of the VM type contains
> this:
> >
> > @Link DiskAttachment[] diskAttachments();
> > @Link Nic[] nics();
> >
> > With this information we are now in a position where we can
> implement
> > this in a generic way.
> >
> > We intend to implement this using a mechanism similar to the
> existing
> > 'detail' parameter:
> >
> > GET /ovirt-engine/api/vms/123?follow=disk_attachments,nics
> >
> > The naive implementation of this is to let the API call itself. For
> > example, when the user requests to follow the 'disk_attachments'
> detail
> > the API can just call itself to get that:
> >
> > GET /ovirt-engine/api/vms/123/disk_attachments
> >
> > However, we can't use that naive approach, if we do we end with the
> > 1+C*N query problem described before. We need to use specific
> > implementations for certain frequent use cases, like
> VMs+disks+nics, and
> > that needs work in the API and in the backend.
> >
> > Tomas, if you want to help moving this forward, please open a RFE
> and
> > makes sure it gets attention.
>
ok, opened: https://bugzilla.redhat.com/show_bug.cgi?id=1436206
Will try to get it done soon.
Forgive me if this is radical, but has anyone thought of / discussed using
a NoSQL alternative to our very normalized SQL db as a way to avoid the
problem of aggregating details? Using mongodb as an example, embed some of
the smaller objects, and there's no cost of aggregation there. IIRC, Doctor
REST uses mongo under the hood.
https://docs.mongodb.com/manual/tutorial/model-embedded-one-to-many-relat...
Greg
> >
> >
> > This sounds pretty good! I will open, but since we are talking already
> > here I'll just use the opportunity to clarify the topic more and than
> > I'll open the BZ.
> >
> > What I can imagine is the GetAllVmsQuery will accept in params also the
> > list of details it should provide. Than, the GetAllVmsQuery will
> > implement the efficient way of retrieving this info as well.
> >
> > So, from the API perspective, it will be about taking the
> > ?follow=<something> part and passing it to the backend query params.
> >
> > What you think?
> >
>
> Exactly, that is the point! The API by itself can't optimize database
> queries, all it can do is call the backend. It is the backend that has
> the opportunity and possibility to send optimized queries to the database.
>
> For other less common things we can use the naive approach, and
> implement the aggregation in the API itself. But for common use cases,
> like VM+disks+nics, we need to do it in an efficient way.
>
> >
> >
> > 2. Reuse of TLS sessions.
> >
> > The part of creating TLS sessions that is expensive is the
> generation of
> > the shared session key. That can be avoided if both the server and
> the
> > client are careful and reuse the session, using the session cache
> > mechanism built-in into TLS itself. The web servers that we use
> (Apache
> > and Undertow) do implement this mechanism, and so do most of our
> > clients. Make sure that your client uses it as well. In Java this is
> > achieved re-using the SSLContext. We already do that for the engine
> to
> > VDSM communciation for example. In JavaScript the browser already
> takes
> > care of this.
> >
> > 3. Parallelism and latency.
> >
> > A typical problem that we have is that we send many request to the
> > server. For example, to retrieve user sessions for a set of VMs we
> tend
> > to send many requests like this:
> >
> > GET /ovirt-engine/api/vms/1/sessions
> > GET /ovirt/engine/api/vms/2/sessions
> > GET /ovirt-engine/api/vms/3/sessions
> > ...
> >
> > And we do that in a synchronous way: send one, wait for the result,
> send
> > another one, wait for the result, etc. This means that we don't take
> > advantage of the parallelism of the server and that we add to each
> > request the network round trip time. So if we have N requests, we
> have
> > to wait at least N*RTT.
> >
> > The web servers that we use support multiple connections, and the
> > protocol that we use, HTTP, supports pipe-lining. This means that
> you
> > can send multiple requests in parallel, and that you can send
> multiple
> > requests without waiting for the response. To give you an idea of
> the
> > improvement that can be achieved, we recently added asynchronous
> request
> > support to the Ruby SDK, with multiple connections and pipe-lining.
> In
> > our scale testing environment that reduced the time to collect a
> > complete inventory from approx 30 min to approx 2 min. Here you
> have an
> > example:
> >
> >
> > https://github.com/oVirt/ovirt-engine-sdk-ruby/blob/master/
> sdk/examples/asynchronous_inventory.rb
> > <https://github.com/oVirt/ovirt-engine-sdk-ruby/blob/master
> /sdk/examples/asynchronous_inventory.rb>
> >
> > So make sure that you take advantage of that in your clients. Sadly
> > pipe-lining is disabled by default in most browsers, so this isn't
> > helpful for JavaScript applications.
> >
> >
> > But we can try what we can do in moVirt about
> > this: https://github.com/oVirt/moVirt/issues/260
> >
>
> Sure, I think there are plenty of asynchronous HTTP clients for Android,
> worth trying one of them. If you are brave enough you can even consider
> using the same library used in the Ruby SDK: libcurl. A bit of JNI here
> and there, and you are done.
>
> >
> >
> > 4. HTTP/2 support.
> >
> > The application server that we use, WildFly, supports HTTP/2,
> including
> > ALPN, out of the box, since version 10.1. We need a mechanism to
> > enable it:
> >
> > core: Add support for enabling HTTP/2
> > https://gerrit.ovirt.org/74621
> >
> > And then we need to get Apache out of the way, for API traffic, at
> > least. I think that is something we can do in the context of the
> engine
> > "podification" effort.
> >
> > However, note that HTTP/2 won't have that big impact in performance
> for
> > applications that continue to use a synchronous/serial style of
> > interaction with the API.
> >
> > On 03/24/2017 11:16 PM, Yaniv Kaul wrote:
> > >
> > >
> > > On Fri, Mar 24, 2017 at 8:57 PM, Martin Sivak <msivak(a)redhat.com
> <mailto:msivak@redhat.com>
> > > <mailto:msivak@redhat.com <mailto:msivak@redhat.com>>>
wrote:
> > >
> > > > Current Apache used has only experimental module for it.
> > > > Undertow is supposed to have a better support. I wonder
> when/if we can drop
> > > > Apache...
> > >
> > > The last info I have about that from mperina is that we need
> Apache
> > > for kerberos support atm.
> > >
> > >
> > > I don't think we need it - I remember reading that Undertow does
> support
> > > it as well.
> > > The only issue is that there are probably 10 people in the world
> who
> > > know how to configure Undertow for Kerberos, while many do for
> Apache.
> > > And since we leave it for the user to configure...
> > > Y.
> > >
> > >
> > >
> > > Martin
> > >
> > > On Fri, Mar 24, 2017 at 5:30 PM, Yaniv Kaul <ykaul(a)redhat.com
> <mailto:ykaul@redhat.com>
> > > <mailto:ykaul@redhat.com <mailto:ykaul@redhat.com>>>
wrote:
> > > >
> > > >
> > > > On Fri, Mar 24, 2017 at 6:43 PM, Martin Sivak <
> msivak(a)redhat.com <mailto:msivak@redhat.com>
> > > <mailto:msivak@redhat.com
<mailto:msivak@redhat.com>>> wrote:
> > > >>
> > > >> > 2: you can have more api gateways (e.g. more apis)
> > tailored for
> > > every
> > > >> > frontend. I don't think we need this - the
current API
> serves
> > > us pretty
> > > >> > well
> > > >> > in every FE Im involved in. The only thing which I
miss
> > is the data
> > > >> > aggregation.
> > > >>
> > > >> So it does not serve us well. Aggregation of data is one
> > the usual
> > > >> points of using the gateway.
> > > >> Yes microservices are affected by this indeed, but so are
> > we because
> > > >> implementing the aggregation directly in the current
engine
> > API layer
> > > >> is hard.
> > > >>
> > > >> > So I would go back to the original topic of this
thread
> - do
> > > some small
> > > >> > change which has a chance to be merged to the project
and
> > helps
> > > us where
> > > >> > it
> > > >> > hurts.
> > > >
> > > >
> > > > I'm wondering if very specific additional REST API calls
can
> > suffice.
> > > > For example, a 'Get VM + disks + NIC' API call seems
> > reasonable to
> > > add for
> > > > the various clients who commonly need it.
> > > >
> > > >>
> > > >> Can a simple HTTP/2 to HTTP/AJP gateway be the simplest
> > solution? Our
> > > >> Apache might even have a module for it already.
> > > >
> > > >
> > > > Current Apache used has only experimental module for it.
> > > > Undertow is supposed to have a better support. I wonder
> > when/if we
> > > can drop
> > > > Apache...
> > > > Y.
> > > >
> > > >>
> > > >> That way you can multiplex all the REST calls using a
> > single tcp
> > > >> connection (and a single SSL negotiation).
> > > >>
> > > >> A custom SSO enabled service like that might be even
better
> > as it
> > > >> would be able to skip the authentication
> > > >> layers too and that would lower the engine load. But I am
> > not sure it
> > > >> is possible with the current codebase.
> > > >>
> > > >> Martin
> > > >>
> > > >> On Fri, Mar 24, 2017 at 4:22 PM, Tomas Jelinek
> > > <tjelinek(a)redhat.com <mailto:tjelinek@redhat.com>
> > <mailto:tjelinek@redhat.com <mailto:tjelinek@redhat.com>>>
> > > >> wrote:
> > > >> >
> > > >> >
> > > >> > On Fri, Mar 24, 2017 at 3:58 PM, Martin Sivak
> > > <msivak(a)redhat.com <mailto:msivak@redhat.com>
> > <mailto:msivak@redhat.com <mailto:msivak@redhat.com>>>
wrote:
> > > >> >>
> > > >> >> > I feel like every REST API I've ever
worked with has
> > had the
> > > >> >> > aggregation
> > > >> >> > +
> > > >> >> > projection problem. It's like we're
trying to use REST
> > as a
> > > >> >> > replacement
> > > >> >> > for
> > > >> >> > SQL -- but the logic that executes the
"SQL" lives in
> a
> > > browser now,
> > > >> >> > and
> > > >> >> > it
> > > >> >> > used to live on a server close to the DB.
And REST
> isn't
> > > expressive
> > > >> >> > for
> > > >> >> > selecting data like SQL is.
> > > >> >>
> > > >> >> The current industry solution I know about is
called API
> > gateway..
> > > >> >> most of the big players have internal API with
lots of
> low
> > > level stuff
> > > >> >> and then couple of external API gateways tailored
to
> > what the
> > > client
> > > >> >> needs.
> > > >> >>
> > > >> >> http://microservices.io/patterns/apigateway.html
> > <http://microservices.io/patterns/apigateway.html>
> > > <http://microservices.io/patterns/apigateway.html
> > <http://microservices.io/patterns/apigateway.html>> (check the
> backend
> > > >> >> for frontend section)
> > > >> >>
> > > >> >> This trend is also visible when you think about
services
> > that
> > > offer
> > > >> >> API gateway management and billing like
> > > >> >> https://aws.amazon.com/api-gateway/
> > <https://aws.amazon.com/api-gateway/>
> > > <https://aws.amazon.com/api-gateway/
> > <https://aws.amazon.com/api-gateway/>> or our very own
> > > >> >> https://www.3scale.net/
> > > >> >
> > > >> >
> > > >> > right, but the api gateway solves 2 problems:
> > > >> >
> > > >> > 1: if you have a microservice architecture it is hard
for
> > > frontend to
> > > >> > talk
> > > >> > to 20 different moving services. So the gateway
hides
> this
> > > complexity
> > > >> > behind
> > > >> > it. This is not the problem we have.
> > > >> >
> > > >> > 2: you can have more api gateways (e.g. more apis)
> > tailored for
> > > every
> > > >> > frontend. I don't think we need this - the
current API
> serves
> > > us pretty
> > > >> > well
> > > >> > in every FE Im involved in. The only thing which I
miss
> > is the data
> > > >> > aggregation.
> > > >> >
> > > >> > So I would go back to the original topic of this
thread
> - do
> > > some small
> > > >> > change which has a chance to be merged to the project
and
> > helps
> > > us where
> > > >> > it
> > > >> > hurts.
> > > >> >
> > > >> >>
> > > >> >>
> > > >> >>
> > > >> >> Martin
> > > >> >>
> > > >> >> On Fri, Mar 24, 2017 at 3:47 PM, Greg Sheremeta
> > > <gshereme(a)redhat.com <mailto:gshereme@redhat.com>
> > <mailto:gshereme@redhat.com <mailto:gshereme@redhat.com>>>
> > > >> >> wrote:
> > > >> >> > I feel like every REST API I've ever
worked with has
> had the
> > > >> >> > aggregation
> > > >> >> > +
> > > >> >> > projection problem. It's like we're
trying to use
> REST as a
> > > >> >> > replacement
> > > >> >> > for
> > > >> >> > SQL -- but the logic that executes the
"SQL" lives in
> a
> > > browser now,
> > > >> >> > and
> > > >> >> > it
> > > >> >> > used to live on a server close to the DB.
And REST
> isn't
> > > expressive
> > > >> >> > for
> > > >> >> > selecting data like SQL is.
> > > >> >> >
> > > >> >> > There must be some industry solution to this
"I want
> to do
> > > SQL over
> > > >> >> > REST"
> > > >> >> > problem.
> > > >> >> >
> > > >> >> > On Fri, Mar 24, 2017 at 5:54 AM, Martin
Sivak
> > > <msivak(a)redhat.com <mailto:msivak@redhat.com>
> > <mailto:msivak@redhat.com <mailto:msivak@redhat.com>>>
> > > >> >> > wrote:
> > > >> >> >>
> > > >> >> >> > for quite some time I have been
more or less
> > involved in
> > > >> >> >> > development
> > > >> >> >> > of
> > > >> >> >> > various UIs for oVirt based
entirely on the oVirt's
> > REST API
> > > >> >> >> > ranging
> > > >> >> >> > from
> > > >> >> >> > the quite mature moVirt [1] through
some cockpit
> > > extensions to a
> > > >> >> >> > young
> > > >> >> >> > and
> > > >> >> >> > experimental user portal
replacement [2].
> > > >> >> >>
> > > >> >> >> oVirt optimizer has the same issue..
> > > >> >> >>
> > > >> >> >> > 2: add some tiny service which
would just accept a
> > list of
> > > >> >> >> > queries,
> > > >> >> >> > execute
> > > >> >> >> > them locally (but using real HTTP
requests) and
> > return in one
> > > >> >> >> > bulk. A
> > > >> >> >> > naive
> > > >> >> >> > implementation just to give a sense
of what I mean
> of
> > > this would
> > > >> >> >> > be a
> > > >> >> >> > shell
> > > >> >> >> > script getting list of strings
like
> > > >> >> >> >
> > "https://localhost/ovirt-engine/api/vms/123/sessions
> > <https://localhost/ovirt-engine/api/vms/123/sessions>
> > > <https://localhost/ovirt-engine/api/vms/123/sessions
> > <https://localhost/ovirt-engine/api/vms/123/sessions>>"
iterate
> over
> > > >> >> >> > them
> > > >> >> >> > and
> > > >> >> >> > do a curl request for each, mangle
the results
> into one
> > > string and
> > > >> >> >> > return
> > > >> >> >> > (credits for this idea to msivak).
Easy to
> implement,
> > > possibility
> > > >> >> >> > to
> > > >> >> >> > add
> > > >> >> >> > also projections later to save some
bandwidth. But
> the
> > > API would
> > > >> >> >> > anyway
> > > >> >> >> > be
> > > >> >> >> > hammered by bunch of queries, only
the network
> > roundtrip
> > > would be
> > > >> >> >> > saved.
> > > >> >> >>
> > > >> >> >> The biggest cost for (especially mobile)
clients is
> > the cost of
> > > >> >> >> establishing new SSL connection. SSL is
also pretty
> > > expensive on the
> > > >> >> >> server side.
> > > >> >> >>
> > > >> >> >> So running the aggregation service on
the
> > ovirt-engine machine
> > > >> >> >> (behind
> > > >> >> >> Apache) means the client will do a
single SSL request
> > with
> > > list of N
> > > >> >> >> urls and the local
"reverse-proxy" will perform
> single
> > > >> >> >> authentication
> > > >> >> >> and N plain HTTP requests (or even
better - AJP). It
> > won't
> > > remove
> > > >> >> >> any
> > > >> >> >> time from the actual command run time,
but it will
> reduce
> > > protocol
> > > >> >> >> overhead.
> > > >> >> >>
> > > >> >> >> I think this is the simplest first step
that requires
> > almost no
> > > >> >> >> change
> > > >> >> >> to existing infrastructure.
> > > >> >> >>
> > > >> >> >> --
> > > >> >> >> Martin Sivak
> > > >> >> >> SLA / oVirt
> > > >> >> >>
> > > >> >> >> On Fri, Mar 24, 2017 at 10:20 AM, Tomas
Jelinek
> > > >> >> >> <tjelinek(a)redhat.com
<mailto:tjelinek@redhat.com>
> > <mailto:tjelinek@redhat.com <mailto:tjelinek@redhat.com>>>
> > > >> >> >> wrote:
> > > >> >> >> > Hi All,
> > > >> >> >> >
> > > >> >> >> > for quite some time I have been
more or less
> > involved in
> > > >> >> >> > development
> > > >> >> >> > of
> > > >> >> >> > various UIs for oVirt based
entirely on the oVirt's
> > REST API
> > > >> >> >> > ranging
> > > >> >> >> > from
> > > >> >> >> > the quite mature moVirt [1] through
some cockpit
> > > extensions to a
> > > >> >> >> > young
> > > >> >> >> > and
> > > >> >> >> > experimental user portal
replacement [2].
> > > >> >> >> >
> > > >> >> >> > One issue we hit over and over
again is the missing
> > data
> > > >> >> >> > aggregation.
> > > >> >> >> > In
> > > >> >> >> > the
> > > >> >> >> > 3.x era we used to use in moVirt
the
> detail=something
> > > >> >> >> > api to get the disks and nics of
the VM, something
> > like:
> > > >> >> >> >
> > > >> >> >> > GET /ovirt-engine/api/vms
> > > >> >> >> > Accept: application/json;
detail=disks
> > > >> >> >> >
> > > >> >> >> > This allowed us to store this data
in local
> database
> > > leading to
> > > >> >> >> > great
> > > >> >> >> > user
> > > >> >> >> > experience. Since this feature has
been removed in
> 4.x
> > > API [3]
> > > >> >> >> > we needed to retire to a different
solution. When
> > the VM
> > > detail is
> > > >> >> >> > selected
> > > >> >> >> > by the user, start loading the
disks and nics and
> hope
> > > the user
> > > >> >> >> > will not be fast enough to see the
delay. The UX is
> > > slightly worse
> > > >> >> >> > bug
> > > >> >> >> > kinda
> > > >> >> >> > acceptable.
> > > >> >> >> >
> > > >> >> >> > We hit this issue harder in the new
user portal
> [2],
> > > because we
> > > >> >> >> > already
> > > >> >> >> > have
> > > >> >> >> > the VM cached and show the whole VM
in one screen.
> > So, if
> > > you pick
> > > >> >> >> > it,
> > > >> >> >> > you
> > > >> >> >> > will get it's details
immediately.
> > > >> >> >> > But, since you don't have all
the details, we need
> > to do an
> > > >> >> >> > additional
> > > >> >> >> > call
> > > >> >> >> > (two actually) to load this data
and they start to
> > appear
> > > later.
> > > >> >> >> > So, something which would be very
fast and smooth
> > starts
> > > to feel
> > > >> >> >> > sluggish.
> > > >> >> >> >
> > > >> >> >> > Recently, we hit this issue again
which forced us
> to
> > > sacrifice the
> > > >> >> >> > UX
> > > >> >> >> > even
> > > >> >> >> > more - it is the "console in
use" feature of user
> > portal.
> > > >> >> >> > The use case is this:
> > > >> >> >> > - if the console is already taken
by some user,
> > there are
> > > >> >> >> > complications
> > > >> >> >> > if
> > > >> >> >> > other current user tryes to take it
as well (will
> avoid
> > > details
> > > >> >> >> > about
> > > >> >> >> > settings and permissins involved,
but long story
> short,
> > > the user
> > > >> >> >> > will
> > > >> >> >> > probably not be allowed to connect
to it. The
> > "probably"
> > > is the
> > > >> >> >> > key
> > > >> >> >> > here
> > > >> >> >> > since we can not do any intelligent
decision in
> > advance,
> > > we can
> > > >> >> >> > only
> > > >> >> >> > warn
> > > >> >> >> > the user that the console is
taken).
> > > >> >> >> > - in the current GWT user portal,
if the VM's
> > console is
> > > taken, it
> > > >> >> >> > is
> > > >> >> >> > shown
> > > >> >> >> > on the VM's "box"
that "console is taken". This was
> > a highly
> > > >> >> >> > requested
> > > >> >> >> > feature
> > > >> >> >> > - to get this information using the
current REST
> > API, we
> > > need to
> > > >> >> >> > go
> > > >> >> >> > to
> > > >> >> >> > the
> > > >> >> >> > /vms/<vmid>/sessions
subcollection. To get this
> for all
> > > VMs, it
> > > >> >> >> > would
> > > >> >> >> > be
> > > >> >> >> > doing N queries per poll which we
can not afford
> > > >> >> >> > - so the current PR [4] will
probably end up to
> only
> > > check it on
> > > >> >> >> > the
> > > >> >> >> > attempt
> > > >> >> >> > to connect to the console warning
the user. Maybe
> > it will
> > > be also
> > > >> >> >> > shown
> > > >> >> >> > in
> > > >> >> >> > Vm details. But the UX in case the
user will look
> for a
> > > VM which
> > > >> >> >> > has
> > > >> >> >> > free
> > > >> >> >> > console will suffer significantly
(e.g. try one by
> one
> > > until some
> > > >> >> >> > opens
> > > >> >> >> > or
> > > >> >> >> > look at details one by one to see
if the warning
> > appears
> > > (with a
> > > >> >> >> > delay))
> > > >> >> >> >
> > > >> >> >> > I understand that embedding the
details of the VM
> > to the
> > > response
> > > >> >> >> > comes
> > > >> >> >> > with
> > > >> >> >> > a cost, namely:
> > > >> >> >> > - performance hit
> > > >> >> >> > - complexity of the API code
> > > >> >> >> > - the "cleanness" of REST
suffers
> > > >> >> >> >
> > > >> >> >> > But I think we should seriously
consider to provide
> > some
> > > option to
> > > >> >> >> > data
> > > >> >> >> > aggregation.
> > > >> >> >> >
> > > >> >> >> > I know this has been discussed many
times with no
> > result,
> > > but I
> > > >> >> >> > think
> > > >> >> >> > it
> > > >> >> >> > is
> > > >> >> >> > time to bring this topic up again.
I'll try to
> > summarize the
> > > >> >> >> > (failed)
> > > >> >> >> > attempts tried so far:
> > > >> >> >> > - the detail=<something>
parameter with ad-hoc
> > embedding
> > > of data.
> > > >> >> >> > This
> > > >> >> >> > has
> > > >> >> >> > been there and removed in 4.0 [3]
> > > >> >> >> > - the DoctorREST project - e.g. a
proxy above the
> > current
> > > api. The
> > > >> >> >> > idea
> > > >> >> >> > was
> > > >> >> >> > to create a service which will be
independent of
> > the engine
> > > >> >> >> > itself,
> > > >> >> >> > will
> > > >> >> >> > locally poll the engine's REST,
store all data in
> local
> > > (mongo)DB
> > > >> >> >> > and
> > > >> >> >> > provide a rich api with
aggregations and
> > projections and push
> > > >> >> >> > notifications.
> > > >> >> >> > This polling of everything to get
the data to
> > DoctorREST
> > > proved to
> > > >> >> >> > be
> > > >> >> >> > pretty
> > > >> >> >> > costy, so also a more invasive
approach of pushing
> > data from
> > > >> >> >> > engine
> > > >> >> >> > to
> > > >> >> >> > doctor has been discused [5]. None
of this two
> > approaches
> > > have
> > > >> >> >> > been
> > > >> >> >> > accepted
> > > >> >> >> > (too complicated, too invasive).
> > > >> >> >> > - writing some custom ad-hoc
servlet serving only a
> > > purpose of one
> > > >> >> >> > frontend
> > > >> >> >> > - this is actually there for the
dashboard, but it
> > is not a
> > > >> >> >> > generic
> > > >> >> >> > solution
> > > >> >> >> > for the other frontends and we
really should not
> > develop
> > > custom
> > > >> >> >> > "APIs"
> > > >> >> >> > for
> > > >> >> >> > every frontend
> > > >> >> >> > - there were some other proposals
discussed (some
> > 3th party
> > > >> >> >> > solutions
> > > >> >> >> > etc)
> > > >> >> >> > but I think none of them made it
even to a PoC
> > > >> >> >> >
> > > >> >> >> > So, now I would try again and try
small to get at
> > least some
> > > >> >> >> > benefit.
> > > >> >> >> > I
> > > >> >> >> > see
> > > >> >> >> > 2 paths we could try:
> > > >> >> >> > 1: embed something which burns us
immediatly, e.g.
> the
> > > /sessions
> > > >> >> >> > into
> > > >> >> >> > VMs. I
> > > >> >> >> > really liked the ;detail=sessions
approach, could
> > we move
> > > it back?
> > > >> >> >> > 2: add some tiny service which
would just accept a
> > list of
> > > >> >> >> > queries,
> > > >> >> >> > execute
> > > >> >> >> > them locally (but using real HTTP
requests) and
> > return in one
> > > >> >> >> > bulk. A
> > > >> >> >> > naive
> > > >> >> >> > implementation just to give a sense
of what I mean
> of
> > > this would
> > > >> >> >> > be a
> > > >> >> >> > shell
> > > >> >> >> > script getting list of strings
like
> > > >> >> >> >
> > "https://localhost/ovirt-engine/api/vms/123/sessions
> > <https://localhost/ovirt-engine/api/vms/123/sessions>
> > > <https://localhost/ovirt-engine/api/vms/123/sessions
> > <https://localhost/ovirt-engine/api/vms/123/sessions>>"
iterate
> over
> > > >> >> >> > them
> > > >> >> >> > and
> > > >> >> >> > do a curl request for each, mangle
the results
> into one
> > > string and
> > > >> >> >> > return
> > > >> >> >> > (credits for this idea to msivak).
Easy to
> implement,
> > > possibility
> > > >> >> >> > to
> > > >> >> >> > add
> > > >> >> >> > also projections later to save some
bandwidth. But
> the
> > > API would
> > > >> >> >> > anyway
> > > >> >> >> > be
> > > >> >> >> > hammered by bunch of queries, only
the network
> > roundtrip
> > > would be
> > > >> >> >> > saved.
> > > >> >> >> > 3: any other simple approaches?
> > > >> >> >> >
> > > >> >> >> > I honestly prefer the first
approach. It is not
> > > beautiful, it is
> > > >> >> >> > not
> > > >> >> >> > REST-ful, but it is easy to
implement, very
> > pragmatic and
> > > useful.
> > > >> >> >> > What do you think?
> > > >> >> >> >
> > > >> >> >> > Thank you and sorry for the long
mail :)
> > > >> >> >> > Tomas
> > > >> >> >> >
> > > >> >> >> > [1]:
https://github.com/oVirt/moVirt
> > <https://github.com/oVirt/moVirt>
> > > <https://github.com/oVirt/moVirt
> > <https://github.com/oVirt/moVirt>>
> > > >> >> >> > [2]:
https://github.com/oVirt/ovirt-web-ui
> > <https://github.com/oVirt/ovirt-web-ui>
> > > <https://github.com/oVirt/ovirt-web-ui
> > <https://github.com/oVirt/ovirt-web-ui>>
> > > >> >> >> > [3]:
https://gerrit.ovirt.org/#/c/61260
> > <https://gerrit.ovirt.org/#/c/61260>
> > > <https://gerrit.ovirt.org/#/c/61260
> > <https://gerrit.ovirt.org/#/c/61260>>
> > > >> >> >> > [4]:
> > https://github.com/oVirt/ovirt-web-ui/pull/106/
> > <https://github.com/oVirt/ovirt-web-ui/pull/106/>
> > > <https://github.com/oVirt/ovirt-web-ui/pull/106/
> > <https://github.com/oVirt/ovirt-web-ui/pull/106/>>
> > > >> >> >> > [5]:
https://gerrit.ovirt.org/#/c/45233/
> > <https://gerrit.ovirt.org/#/c/45233/>
> > > <https://gerrit.ovirt.org/#/c/45233/
> > <https://gerrit.ovirt.org/#/c/45233/>>
> > > >> >> >> >
> > > >> >> >> >
> > > >> >> >> >
_______________________________________________
> > > >> >> >> > Devel mailing list
> > > >> >> >> > Devel(a)ovirt.org
<mailto:Devel@ovirt.org>
> > <mailto:Devel@ovirt.org <mailto:Devel@ovirt.org>>
> > > >> >> >> >
http://lists.ovirt.org/mailman/listinfo/devel
> > <http://lists.ovirt.org/mailman/listinfo/devel>
> > > <http://lists.ovirt.org/mailman/listinfo/devel
> > <http://lists.ovirt.org/mailman/listinfo/devel>>
> > > >> >> >>
_______________________________________________
> > > >> >> >> Devel mailing list
> > > >> >> >> Devel(a)ovirt.org
<mailto:Devel@ovirt.org>
> > <mailto:Devel@ovirt.org <mailto:Devel@ovirt.org>>
> > > >> >> >>
http://lists.ovirt.org/mailman/listinfo/devel
> > <http://lists.ovirt.org/mailman/listinfo/devel>
> > > <http://lists.ovirt.org/mailman/listinfo/devel
> > <http://lists.ovirt.org/mailman/listinfo/devel>>
> > > >> >> >
> > > >> >> >
> > > >> >> >
> > > >> >> >
> > > >> >> > --
> > > >> >> > Greg Sheremeta, MBA
> > > >> >> > Red Hat, Inc.
> > > >> >> > Sr. Software Engineer
> > > >> >> > gshereme(a)redhat.com
<mailto:gshereme@redhat.com>
> > <mailto:gshereme@redhat.com <mailto:gshereme@redhat.com>>
> > > >> >
> > > >> >
> > > >> _______________________________________________
> > > >> Devel mailing list
> > > >> Devel(a)ovirt.org <mailto:Devel@ovirt.org>
> > <mailto:Devel@ovirt.org <mailto:Devel@ovirt.org>>
> > > >> http://lists.ovirt.org/mailman/listinfo/devel
> > <http://lists.ovirt.org/mailman/listinfo/devel>
> > > <http://lists.ovirt.org/mailman/listinfo/devel
> > <http://lists.ovirt.org/mailman/listinfo/devel>>
> > > >
> > > >
> > >
> > >
> >
> >
>
>
_______________________________________________
Devel mailing list
Devel(a)ovirt.org
http://lists.ovirt.org/mailman/listinfo/devel
--
Greg Sheremeta, MBA
Red Hat, Inc.
Sr. Software Engineer
gshereme(a)redhat.com
8:41 a.m.
Hi!
Forgive me if this is radical, but has anyone thought of / discussed
using a
NoSQL alternative to our very normalized SQL db as a way to avoid the
problem of aggregating details? Using mongodb as an example, embed some of
the smaller objects, and there's no cost of aggregation there. IIRC, Doctor
REST uses mongo under the hood.
https://docs.mongodb.com/manual/tutorial/model-embedded-one-to-many-relat...
What about data integrity that is critical to us?
Shmuel
8:43 a.m.
On Mon, Mar 27, 2017 at 4:49 PM, Greg Sheremeta <gshereme(a)redhat.com> wrote:
On Mon, Mar 27, 2017 at 8:59 AM, Tomas Jelinek
<tjelinek(a)redhat.com>
wrote:
>
>
> On Mon, Mar 27, 2017 at 1:32 PM, Juan Hernández <jhernand(a)redhat.com>
> wrote:
>
>> On 03/27/2017 01:03 PM, Tomas Jelinek wrote:
>> >
>> >
>> > On Mon, Mar 27, 2017 at 11:21 AM, Juan Hernández <jhernand(a)redhat.com
>> > <mailto:jhernand@redhat.com>> wrote:
>> >
>> > Top posting, sorry.
>> >
>> > There are a few things I'd like to clarify, regarding this subject:
>> >
>> > 1. Data aggregation, as requested now by Tomas, and by other
>> people in
>> > the past.
>> >
>> > We used to have that 'detail' parameter, to aggregate certain
very
>> > specific types of data, in particular to aggregate VM disks and
>> NICs. We
>> > removed that in version 4 of the API because the implementation was
>> > extremely inefficient, from the engine point of view. An innocent
>> > request like this:
>> >
>> > GET /ovirt-engine/api/vms?detail=+disks,+nics
>> >
>> > Would generate, with the implementation we used to have, 1 query
>> for the
>> > VMs and then as many queries for disks and NICs as VMs in the
>> system. In
>> > our scale test environments, for example, with approx 4000 VMs and
>> 10000
>> > disks, that would take more than 20 hours to execute.
>> >
>> > In addition, we didn't have in the past any mechanism to make this
>> > available in a generic one, because there was no knowledge in the
>> API of
>> > what are 'details'.
>> >
>> > In version 4 of the API we introduced a formal (kind of)
>> specification
>> > of the API (a.k.a. the model), and int includes knowledge about
>> what are
>> > 'links'. For example, the specification of the VM type contains
>> this:
>> >
>> > @Link DiskAttachment[] diskAttachments();
>> > @Link Nic[] nics();
>> >
>> > With this information we are now in a position where we can
>> implement
>> > this in a generic way.
>> >
>> > We intend to implement this using a mechanism similar to the
>> existing
>> > 'detail' parameter:
>> >
>> > GET /ovirt-engine/api/vms/123?follow=disk_attachments,nics
>> >
>> > The naive implementation of this is to let the API call itself. For
>> > example, when the user requests to follow the
'disk_attachments'
>> detail
>> > the API can just call itself to get that:
>> >
>> > GET /ovirt-engine/api/vms/123/disk_attachments
>> >
>> > However, we can't use that naive approach, if we do we end with the
>> > 1+C*N query problem described before. We need to use specific
>> > implementations for certain frequent use cases, like
>> VMs+disks+nics, and
>> > that needs work in the API and in the backend.
>> >
>> > Tomas, if you want to help moving this forward, please open a RFE
>> and
>> > makes sure it gets attention.
>>
>
> ok, opened: https://bugzilla.redhat.com/show_bug.cgi?id=1436206
> Will try to get it done soon.
>
Forgive me if this is radical, but has anyone thought of / discussed using
a NoSQL alternative to our very normalized SQL db as a way to avoid the
problem of aggregating details? Using mongodb as an example, embed some of
the smaller objects, and there's no cost of aggregation there. IIRC, Doctor
REST uses mongo under the hood.
https://docs.mongodb.com/manual/tutorial/model-embedded-one-to-many-
relationships-between-documents/
Greg, I really appreciate your enthusiasm for NoSQL technologies but we
have to distinguish between the functional requirements for cache of
schemaless frontend-optimized projections (as in the Dr. Rest case) and the
main application data store, supporting transactional business logic. I
don't believe the current architecture could survive the weak
eventual-consistency guarantees of not fully transactional DB (such as
Postgres is) :-)
Best regards,
Martin
Greg
>
>
>> >
>> >
>> > This sounds pretty good! I will open, but since we are talking already
>> > here I'll just use the opportunity to clarify the topic more and than
>> > I'll open the BZ.
>> >
>> > What I can imagine is the GetAllVmsQuery will accept in params also the
>> > list of details it should provide. Than, the GetAllVmsQuery will
>> > implement the efficient way of retrieving this info as well.
>> >
>> > So, from the API perspective, it will be about taking the
>> > ?follow=<something> part and passing it to the backend query params.
>> >
>> > What you think?
>> >
>>
>> Exactly, that is the point! The API by itself can't optimize database
>> queries, all it can do is call the backend. It is the backend that has
>> the opportunity and possibility to send optimized queries to the
>> database.
>>
>> For other less common things we can use the naive approach, and
>> implement the aggregation in the API itself. But for common use cases,
>> like VM+disks+nics, we need to do it in an efficient way.
>>
>> >
>> >
>> > 2. Reuse of TLS sessions.
>> >
>> > The part of creating TLS sessions that is expensive is the
>> generation of
>> > the shared session key. That can be avoided if both the server and
>> the
>> > client are careful and reuse the session, using the session cache
>> > mechanism built-in into TLS itself. The web servers that we use
>> (Apache
>> > and Undertow) do implement this mechanism, and so do most of our
>> > clients. Make sure that your client uses it as well. In Java this
>> is
>> > achieved re-using the SSLContext. We already do that for the
>> engine to
>> > VDSM communciation for example. In JavaScript the browser already
>> takes
>> > care of this.
>> >
>> > 3. Parallelism and latency.
>> >
>> > A typical problem that we have is that we send many request to the
>> > server. For example, to retrieve user sessions for a set of VMs we
>> tend
>> > to send many requests like this:
>> >
>> > GET /ovirt-engine/api/vms/1/sessions
>> > GET /ovirt/engine/api/vms/2/sessions
>> > GET /ovirt-engine/api/vms/3/sessions
>> > ...
>> >
>> > And we do that in a synchronous way: send one, wait for the
>> result, send
>> > another one, wait for the result, etc. This means that we don't
>> take
>> > advantage of the parallelism of the server and that we add to each
>> > request the network round trip time. So if we have N requests, we
>> have
>> > to wait at least N*RTT.
>> >
>> > The web servers that we use support multiple connections, and the
>> > protocol that we use, HTTP, supports pipe-lining. This means that
>> you
>> > can send multiple requests in parallel, and that you can send
>> multiple
>> > requests without waiting for the response. To give you an idea of
>> the
>> > improvement that can be achieved, we recently added asynchronous
>> request
>> > support to the Ruby SDK, with multiple connections and
>> pipe-lining. In
>> > our scale testing environment that reduced the time to collect a
>> > complete inventory from approx 30 min to approx 2 min. Here you
>> have an
>> > example:
>> >
>> >
>> > https://github.com/oVirt/ovirt-engine-sdk-ruby/blob/master/
>> sdk/examples/asynchronous_inventory.rb
>> > <https://github.com/oVirt/ovirt-engine-sdk-ruby/blob/master
>> /sdk/examples/asynchronous_inventory.rb>
>> >
>> > So make sure that you take advantage of that in your clients. Sadly
>> > pipe-lining is disabled by default in most browsers, so this isn't
>> > helpful for JavaScript applications.
>> >
>> >
>> > But we can try what we can do in moVirt about
>> > this: https://github.com/oVirt/moVirt/issues/260
>> >
>>
>> Sure, I think there are plenty of asynchronous HTTP clients for Android,
>> worth trying one of them. If you are brave enough you can even consider
>> using the same library used in the Ruby SDK: libcurl. A bit of JNI here
>> and there, and you are done.
>>
>> >
>> >
>> > 4. HTTP/2 support.
>> >
>> > The application server that we use, WildFly, supports HTTP/2,
>> including
>> > ALPN, out of the box, since version 10.1. We need a mechanism to
>> > enable it:
>> >
>> > core: Add support for enabling HTTP/2
>> > https://gerrit.ovirt.org/74621
>> >
>> > And then we need to get Apache out of the way, for API traffic, at
>> > least. I think that is something we can do in the context of the
>> engine
>> > "podification" effort.
>> >
>> > However, note that HTTP/2 won't have that big impact in
>> performance for
>> > applications that continue to use a synchronous/serial style of
>> > interaction with the API.
>> >
>> > On 03/24/2017 11:16 PM, Yaniv Kaul wrote:
>> > >
>> > >
>> > > On Fri, Mar 24, 2017 at 8:57 PM, Martin Sivak
<msivak(a)redhat.com
>> <mailto:msivak@redhat.com>
>> > > <mailto:msivak@redhat.com
<mailto:msivak@redhat.com>>> wrote:
>> > >
>> > > > Current Apache used has only experimental module for it.
>> > > > Undertow is supposed to have a better support. I wonder
>> when/if we can drop
>> > > > Apache...
>> > >
>> > > The last info I have about that from mperina is that we need
>> Apache
>> > > for kerberos support atm.
>> > >
>> > >
>> > > I don't think we need it - I remember reading that Undertow
does
>> support
>> > > it as well.
>> > > The only issue is that there are probably 10 people in the world
>> who
>> > > know how to configure Undertow for Kerberos, while many do for
>> Apache.
>> > > And since we leave it for the user to configure...
>> > > Y.
>> > >
>> > >
>> > >
>> > > Martin
>> > >
>> > > On Fri, Mar 24, 2017 at 5:30 PM, Yaniv Kaul <
>> ykaul(a)redhat.com <mailto:ykaul@redhat.com>
>> > > <mailto:ykaul@redhat.com
<mailto:ykaul@redhat.com>>> wrote:
>> > > >
>> > > >
>> > > > On Fri, Mar 24, 2017 at 6:43 PM, Martin Sivak <
>> msivak(a)redhat.com <mailto:msivak@redhat.com>
>> > > <mailto:msivak@redhat.com
<mailto:msivak@redhat.com>>>
>> wrote:
>> > > >>
>> > > >> > 2: you can have more api gateways (e.g. more
apis)
>> > tailored for
>> > > every
>> > > >> > frontend. I don't think we need this - the
current API
>> serves
>> > > us pretty
>> > > >> > well
>> > > >> > in every FE Im involved in. The only thing which
I miss
>> > is the data
>> > > >> > aggregation.
>> > > >>
>> > > >> So it does not serve us well. Aggregation of data is
one
>> > the usual
>> > > >> points of using the gateway.
>> > > >> Yes microservices are affected by this indeed, but so
are
>> > we because
>> > > >> implementing the aggregation directly in the current
>> engine
>> > API layer
>> > > >> is hard.
>> > > >>
>> > > >> > So I would go back to the original topic of this
thread
>> - do
>> > > some small
>> > > >> > change which has a chance to be merged to the
project
>> and
>> > helps
>> > > us where
>> > > >> > it
>> > > >> > hurts.
>> > > >
>> > > >
>> > > > I'm wondering if very specific additional REST API
calls
>> can
>> > suffice.
>> > > > For example, a 'Get VM + disks + NIC' API call
seems
>> > reasonable to
>> > > add for
>> > > > the various clients who commonly need it.
>> > > >
>> > > >>
>> > > >> Can a simple HTTP/2 to HTTP/AJP gateway be the
simplest
>> > solution? Our
>> > > >> Apache might even have a module for it already.
>> > > >
>> > > >
>> > > > Current Apache used has only experimental module for it.
>> > > > Undertow is supposed to have a better support. I wonder
>> > when/if we
>> > > can drop
>> > > > Apache...
>> > > > Y.
>> > > >
>> > > >>
>> > > >> That way you can multiplex all the REST calls using a
>> > single tcp
>> > > >> connection (and a single SSL negotiation).
>> > > >>
>> > > >> A custom SSO enabled service like that might be even
>> better
>> > as it
>> > > >> would be able to skip the authentication
>> > > >> layers too and that would lower the engine load. But I
am
>> > not sure it
>> > > >> is possible with the current codebase.
>> > > >>
>> > > >> Martin
>> > > >>
>> > > >> On Fri, Mar 24, 2017 at 4:22 PM, Tomas Jelinek
>> > > <tjelinek(a)redhat.com <mailto:tjelinek@redhat.com>
>> > <mailto:tjelinek@redhat.com
<mailto:tjelinek@redhat.com>>>
>> > > >> wrote:
>> > > >> >
>> > > >> >
>> > > >> > On Fri, Mar 24, 2017 at 3:58 PM, Martin Sivak
>> > > <msivak(a)redhat.com <mailto:msivak@redhat.com>
>> > <mailto:msivak@redhat.com <mailto:msivak@redhat.com>>>
wrote:
>> > > >> >>
>> > > >> >> > I feel like every REST API I've ever
worked with has
>> > had the
>> > > >> >> > aggregation
>> > > >> >> > +
>> > > >> >> > projection problem. It's like
we're trying to use
>> REST
>> > as a
>> > > >> >> > replacement
>> > > >> >> > for
>> > > >> >> > SQL -- but the logic that executes the
"SQL" lives
>> in a
>> > > browser now,
>> > > >> >> > and
>> > > >> >> > it
>> > > >> >> > used to live on a server close to the
DB. And REST
>> isn't
>> > > expressive
>> > > >> >> > for
>> > > >> >> > selecting data like SQL is.
>> > > >> >>
>> > > >> >> The current industry solution I know about is
called
>> API
>> > gateway..
>> > > >> >> most of the big players have internal API
with lots of
>> low
>> > > level stuff
>> > > >> >> and then couple of external API gateways
tailored to
>> > what the
>> > > client
>> > > >> >> needs.
>> > > >> >>
>> > > >> >>
http://microservices.io/patterns/apigateway.html
>> > <http://microservices.io/patterns/apigateway.html>
>> > > <http://microservices.io/patterns/apigateway.html
>> > <http://microservices.io/patterns/apigateway.html>> (check the
>> backend
>> > > >> >> for frontend section)
>> > > >> >>
>> > > >> >> This trend is also visible when you think
about
>> services
>> > that
>> > > offer
>> > > >> >> API gateway management and billing like
>> > > >> >> https://aws.amazon.com/api-gateway/
>> > <https://aws.amazon.com/api-gateway/>
>> > > <https://aws.amazon.com/api-gateway/
>> > <https://aws.amazon.com/api-gateway/>> or our very own
>> > > >> >> https://www.3scale.net/
>> > > >> >
>> > > >> >
>> > > >> > right, but the api gateway solves 2 problems:
>> > > >> >
>> > > >> > 1: if you have a microservice architecture it is
hard
>> for
>> > > frontend to
>> > > >> > talk
>> > > >> > to 20 different moving services. So the gateway
hides
>> this
>> > > complexity
>> > > >> > behind
>> > > >> > it. This is not the problem we have.
>> > > >> >
>> > > >> > 2: you can have more api gateways (e.g. more
apis)
>> > tailored for
>> > > every
>> > > >> > frontend. I don't think we need this - the
current API
>> serves
>> > > us pretty
>> > > >> > well
>> > > >> > in every FE Im involved in. The only thing which
I miss
>> > is the data
>> > > >> > aggregation.
>> > > >> >
>> > > >> > So I would go back to the original topic of this
thread
>> - do
>> > > some small
>> > > >> > change which has a chance to be merged to the
project
>> and
>> > helps
>> > > us where
>> > > >> > it
>> > > >> > hurts.
>> > > >> >
>> > > >> >>
>> > > >> >>
>> > > >> >>
>> > > >> >> Martin
>> > > >> >>
>> > > >> >> On Fri, Mar 24, 2017 at 3:47 PM, Greg
Sheremeta
>> > > <gshereme(a)redhat.com <mailto:gshereme@redhat.com>
>> > <mailto:gshereme@redhat.com
<mailto:gshereme@redhat.com>>>
>> > > >> >> wrote:
>> > > >> >> > I feel like every REST API I've ever
worked with has
>> had the
>> > > >> >> > aggregation
>> > > >> >> > +
>> > > >> >> > projection problem. It's like
we're trying to use
>> REST as a
>> > > >> >> > replacement
>> > > >> >> > for
>> > > >> >> > SQL -- but the logic that executes the
"SQL" lives
>> in a
>> > > browser now,
>> > > >> >> > and
>> > > >> >> > it
>> > > >> >> > used to live on a server close to the
DB. And REST
>> isn't
>> > > expressive
>> > > >> >> > for
>> > > >> >> > selecting data like SQL is.
>> > > >> >> >
>> > > >> >> > There must be some industry solution to
this "I want
>> to do
>> > > SQL over
>> > > >> >> > REST"
>> > > >> >> > problem.
>> > > >> >> >
>> > > >> >> > On Fri, Mar 24, 2017 at 5:54 AM, Martin
Sivak
>> > > <msivak(a)redhat.com <mailto:msivak@redhat.com>
>> > <mailto:msivak@redhat.com <mailto:msivak@redhat.com>>>
>> > > >> >> > wrote:
>> > > >> >> >>
>> > > >> >> >> > for quite some time I have been
more or less
>> > involved in
>> > > >> >> >> > development
>> > > >> >> >> > of
>> > > >> >> >> > various UIs for oVirt based
entirely on the
>> oVirt's
>> > REST API
>> > > >> >> >> > ranging
>> > > >> >> >> > from
>> > > >> >> >> > the quite mature moVirt [1]
through some cockpit
>> > > extensions to a
>> > > >> >> >> > young
>> > > >> >> >> > and
>> > > >> >> >> > experimental user portal
replacement [2].
>> > > >> >> >>
>> > > >> >> >> oVirt optimizer has the same
issue..
>> > > >> >> >>
>> > > >> >> >> > 2: add some tiny service which
would just accept a
>> > list of
>> > > >> >> >> > queries,
>> > > >> >> >> > execute
>> > > >> >> >> > them locally (but using real
HTTP requests) and
>> > return in one
>> > > >> >> >> > bulk. A
>> > > >> >> >> > naive
>> > > >> >> >> > implementation just to give a
sense of what I
>> mean of
>> > > this would
>> > > >> >> >> > be a
>> > > >> >> >> > shell
>> > > >> >> >> > script getting list of strings
like
>> > > >> >> >> >
>> > "https://localhost/ovirt-engine/api/vms/123/sessions
>> > <https://localhost/ovirt-engine/api/vms/123/sessions>
>> > > <https://localhost/ovirt-engine/api/vms/123/sessions
>> > <https://localhost/ovirt-engine/api/vms/123/sessions>>"
iterate
>> over
>> > > >> >> >> > them
>> > > >> >> >> > and
>> > > >> >> >> > do a curl request for each,
mangle the results
>> into one
>> > > string and
>> > > >> >> >> > return
>> > > >> >> >> > (credits for this idea to
msivak). Easy to
>> implement,
>> > > possibility
>> > > >> >> >> > to
>> > > >> >> >> > add
>> > > >> >> >> > also projections later to save
some bandwidth.
>> But the
>> > > API would
>> > > >> >> >> > anyway
>> > > >> >> >> > be
>> > > >> >> >> > hammered by bunch of queries,
only the network
>> > roundtrip
>> > > would be
>> > > >> >> >> > saved.
>> > > >> >> >>
>> > > >> >> >> The biggest cost for (especially
mobile) clients is
>> > the cost of
>> > > >> >> >> establishing new SSL connection. SSL
is also pretty
>> > > expensive on the
>> > > >> >> >> server side.
>> > > >> >> >>
>> > > >> >> >> So running the aggregation service
on the
>> > ovirt-engine machine
>> > > >> >> >> (behind
>> > > >> >> >> Apache) means the client will do a
single SSL
>> request
>> > with
>> > > list of N
>> > > >> >> >> urls and the local
"reverse-proxy" will perform
>> single
>> > > >> >> >> authentication
>> > > >> >> >> and N plain HTTP requests (or even
better - AJP). It
>> > won't
>> > > remove
>> > > >> >> >> any
>> > > >> >> >> time from the actual command run
time, but it will
>> reduce
>> > > protocol
>> > > >> >> >> overhead.
>> > > >> >> >>
>> > > >> >> >> I think this is the simplest first
step that
>> requires
>> > almost no
>> > > >> >> >> change
>> > > >> >> >> to existing infrastructure.
>> > > >> >> >>
>> > > >> >> >> --
>> > > >> >> >> Martin Sivak
>> > > >> >> >> SLA / oVirt
>> > > >> >> >>
>> > > >> >> >> On Fri, Mar 24, 2017 at 10:20 AM,
Tomas Jelinek
>> > > >> >> >> <tjelinek(a)redhat.com
<mailto:tjelinek@redhat.com>
>> > <mailto:tjelinek@redhat.com
<mailto:tjelinek@redhat.com>>>
>> > > >> >> >> wrote:
>> > > >> >> >> > Hi All,
>> > > >> >> >> >
>> > > >> >> >> > for quite some time I have been
more or less
>> > involved in
>> > > >> >> >> > development
>> > > >> >> >> > of
>> > > >> >> >> > various UIs for oVirt based
entirely on the
>> oVirt's
>> > REST API
>> > > >> >> >> > ranging
>> > > >> >> >> > from
>> > > >> >> >> > the quite mature moVirt [1]
through some cockpit
>> > > extensions to a
>> > > >> >> >> > young
>> > > >> >> >> > and
>> > > >> >> >> > experimental user portal
replacement [2].
>> > > >> >> >> >
>> > > >> >> >> > One issue we hit over and over
again is the
>> missing
>> > data
>> > > >> >> >> > aggregation.
>> > > >> >> >> > In
>> > > >> >> >> > the
>> > > >> >> >> > 3.x era we used to use in
moVirt the
>> detail=something
>> > > >> >> >> > api to get the disks and nics
of the VM, something
>> > like:
>> > > >> >> >> >
>> > > >> >> >> > GET /ovirt-engine/api/vms
>> > > >> >> >> > Accept: application/json;
detail=disks
>> > > >> >> >> >
>> > > >> >> >> > This allowed us to store this
data in local
>> database
>> > > leading to
>> > > >> >> >> > great
>> > > >> >> >> > user
>> > > >> >> >> > experience. Since this feature
has been removed
>> in 4.x
>> > > API [3]
>> > > >> >> >> > we needed to retire to a
different solution. When
>> > the VM
>> > > detail is
>> > > >> >> >> > selected
>> > > >> >> >> > by the user, start loading the
disks and nics and
>> hope
>> > > the user
>> > > >> >> >> > will not be fast enough to see
the delay. The UX
>> is
>> > > slightly worse
>> > > >> >> >> > bug
>> > > >> >> >> > kinda
>> > > >> >> >> > acceptable.
>> > > >> >> >> >
>> > > >> >> >> > We hit this issue harder in the
new user portal
>> [2],
>> > > because we
>> > > >> >> >> > already
>> > > >> >> >> > have
>> > > >> >> >> > the VM cached and show the
whole VM in one screen.
>> > So, if
>> > > you pick
>> > > >> >> >> > it,
>> > > >> >> >> > you
>> > > >> >> >> > will get it's details
immediately.
>> > > >> >> >> > But, since you don't have
all the details, we need
>> > to do an
>> > > >> >> >> > additional
>> > > >> >> >> > call
>> > > >> >> >> > (two actually) to load this
data and they start to
>> > appear
>> > > later.
>> > > >> >> >> > So, something which would be
very fast and smooth
>> > starts
>> > > to feel
>> > > >> >> >> > sluggish.
>> > > >> >> >> >
>> > > >> >> >> > Recently, we hit this issue
again which forced us
>> to
>> > > sacrifice the
>> > > >> >> >> > UX
>> > > >> >> >> > even
>> > > >> >> >> > more - it is the "console
in use" feature of user
>> > portal.
>> > > >> >> >> > The use case is this:
>> > > >> >> >> > - if the console is already
taken by some user,
>> > there are
>> > > >> >> >> > complications
>> > > >> >> >> > if
>> > > >> >> >> > other current user tryes to
take it as well (will
>> avoid
>> > > details
>> > > >> >> >> > about
>> > > >> >> >> > settings and permissins
involved, but long story
>> short,
>> > > the user
>> > > >> >> >> > will
>> > > >> >> >> > probably not be allowed to
connect to it. The
>> > "probably"
>> > > is the
>> > > >> >> >> > key
>> > > >> >> >> > here
>> > > >> >> >> > since we can not do any
intelligent decision in
>> > advance,
>> > > we can
>> > > >> >> >> > only
>> > > >> >> >> > warn
>> > > >> >> >> > the user that the console is
taken).
>> > > >> >> >> > - in the current GWT user
portal, if the VM's
>> > console is
>> > > taken, it
>> > > >> >> >> > is
>> > > >> >> >> > shown
>> > > >> >> >> > on the VM's "box"
that "console is taken". This
>> was
>> > a highly
>> > > >> >> >> > requested
>> > > >> >> >> > feature
>> > > >> >> >> > - to get this information using
the current REST
>> > API, we
>> > > need to
>> > > >> >> >> > go
>> > > >> >> >> > to
>> > > >> >> >> > the
>> > > >> >> >> > /vms/<vmid>/sessions
subcollection. To get this
>> for all
>> > > VMs, it
>> > > >> >> >> > would
>> > > >> >> >> > be
>> > > >> >> >> > doing N queries per poll which
we can not afford
>> > > >> >> >> > - so the current PR [4] will
probably end up to
>> only
>> > > check it on
>> > > >> >> >> > the
>> > > >> >> >> > attempt
>> > > >> >> >> > to connect to the console
warning the user. Maybe
>> > it will
>> > > be also
>> > > >> >> >> > shown
>> > > >> >> >> > in
>> > > >> >> >> > Vm details. But the UX in case
the user will look
>> for a
>> > > VM which
>> > > >> >> >> > has
>> > > >> >> >> > free
>> > > >> >> >> > console will suffer
significantly (e.g. try one
>> by one
>> > > until some
>> > > >> >> >> > opens
>> > > >> >> >> > or
>> > > >> >> >> > look at details one by one to
see if the warning
>> > appears
>> > > (with a
>> > > >> >> >> > delay))
>> > > >> >> >> >
>> > > >> >> >> > I understand that embedding the
details of the VM
>> > to the
>> > > response
>> > > >> >> >> > comes
>> > > >> >> >> > with
>> > > >> >> >> > a cost, namely:
>> > > >> >> >> > - performance hit
>> > > >> >> >> > - complexity of the API code
>> > > >> >> >> > - the "cleanness" of
REST suffers
>> > > >> >> >> >
>> > > >> >> >> > But I think we should seriously
consider to
>> provide
>> > some
>> > > option to
>> > > >> >> >> > data
>> > > >> >> >> > aggregation.
>> > > >> >> >> >
>> > > >> >> >> > I know this has been discussed
many times with no
>> > result,
>> > > but I
>> > > >> >> >> > think
>> > > >> >> >> > it
>> > > >> >> >> > is
>> > > >> >> >> > time to bring this topic up
again. I'll try to
>> > summarize the
>> > > >> >> >> > (failed)
>> > > >> >> >> > attempts tried so far:
>> > > >> >> >> > - the detail=<something>
parameter with ad-hoc
>> > embedding
>> > > of data.
>> > > >> >> >> > This
>> > > >> >> >> > has
>> > > >> >> >> > been there and removed in 4.0
[3]
>> > > >> >> >> > - the DoctorREST project - e.g.
a proxy above the
>> > current
>> > > api. The
>> > > >> >> >> > idea
>> > > >> >> >> > was
>> > > >> >> >> > to create a service which will
be independent of
>> > the engine
>> > > >> >> >> > itself,
>> > > >> >> >> > will
>> > > >> >> >> > locally poll the engine's
REST, store all data in
>> local
>> > > (mongo)DB
>> > > >> >> >> > and
>> > > >> >> >> > provide a rich api with
aggregations and
>> > projections and push
>> > > >> >> >> > notifications.
>> > > >> >> >> > This polling of everything to
get the data to
>> > DoctorREST
>> > > proved to
>> > > >> >> >> > be
>> > > >> >> >> > pretty
>> > > >> >> >> > costy, so also a more invasive
approach of pushing
>> > data from
>> > > >> >> >> > engine
>> > > >> >> >> > to
>> > > >> >> >> > doctor has been discused [5].
None of this two
>> > approaches
>> > > have
>> > > >> >> >> > been
>> > > >> >> >> > accepted
>> > > >> >> >> > (too complicated, too
invasive).
>> > > >> >> >> > - writing some custom ad-hoc
servlet serving only
>> a
>> > > purpose of one
>> > > >> >> >> > frontend
>> > > >> >> >> > - this is actually there for
the dashboard, but it
>> > is not a
>> > > >> >> >> > generic
>> > > >> >> >> > solution
>> > > >> >> >> > for the other frontends and we
really should not
>> > develop
>> > > custom
>> > > >> >> >> > "APIs"
>> > > >> >> >> > for
>> > > >> >> >> > every frontend
>> > > >> >> >> > - there were some other
proposals discussed (some
>> > 3th party
>> > > >> >> >> > solutions
>> > > >> >> >> > etc)
>> > > >> >> >> > but I think none of them made
it even to a PoC
>> > > >> >> >> >
>> > > >> >> >> > So, now I would try again and
try small to get at
>> > least some
>> > > >> >> >> > benefit.
>> > > >> >> >> > I
>> > > >> >> >> > see
>> > > >> >> >> > 2 paths we could try:
>> > > >> >> >> > 1: embed something which burns
us immediatly,
>> e.g. the
>> > > /sessions
>> > > >> >> >> > into
>> > > >> >> >> > VMs. I
>> > > >> >> >> > really liked the
;detail=sessions approach, could
>> > we move
>> > > it back?
>> > > >> >> >> > 2: add some tiny service which
would just accept a
>> > list of
>> > > >> >> >> > queries,
>> > > >> >> >> > execute
>> > > >> >> >> > them locally (but using real
HTTP requests) and
>> > return in one
>> > > >> >> >> > bulk. A
>> > > >> >> >> > naive
>> > > >> >> >> > implementation just to give a
sense of what I
>> mean of
>> > > this would
>> > > >> >> >> > be a
>> > > >> >> >> > shell
>> > > >> >> >> > script getting list of strings
like
>> > > >> >> >> >
>> > "https://localhost/ovirt-engine/api/vms/123/sessions
>> > <https://localhost/ovirt-engine/api/vms/123/sessions>
>> > > <https://localhost/ovirt-engine/api/vms/123/sessions
>> > <https://localhost/ovirt-engine/api/vms/123/sessions>>"
iterate
>> over
>> > > >> >> >> > them
>> > > >> >> >> > and
>> > > >> >> >> > do a curl request for each,
mangle the results
>> into one
>> > > string and
>> > > >> >> >> > return
>> > > >> >> >> > (credits for this idea to
msivak). Easy to
>> implement,
>> > > possibility
>> > > >> >> >> > to
>> > > >> >> >> > add
>> > > >> >> >> > also projections later to save
some bandwidth.
>> But the
>> > > API would
>> > > >> >> >> > anyway
>> > > >> >> >> > be
>> > > >> >> >> > hammered by bunch of queries,
only the network
>> > roundtrip
>> > > would be
>> > > >> >> >> > saved.
>> > > >> >> >> > 3: any other simple
approaches?
>> > > >> >> >> >
>> > > >> >> >> > I honestly prefer the first
approach. It is not
>> > > beautiful, it is
>> > > >> >> >> > not
>> > > >> >> >> > REST-ful, but it is easy to
implement, very
>> > pragmatic and
>> > > useful.
>> > > >> >> >> > What do you think?
>> > > >> >> >> >
>> > > >> >> >> > Thank you and sorry for the
long mail :)
>> > > >> >> >> > Tomas
>> > > >> >> >> >
>> > > >> >> >> > [1]:
https://github.com/oVirt/moVirt
>> > <https://github.com/oVirt/moVirt>
>> > > <https://github.com/oVirt/moVirt
>> > <https://github.com/oVirt/moVirt>>
>> > > >> >> >> > [2]:
https://github.com/oVirt/ovirt-web-ui
>> > <https://github.com/oVirt/ovirt-web-ui>
>> > > <https://github.com/oVirt/ovirt-web-ui
>> > <https://github.com/oVirt/ovirt-web-ui>>
>> > > >> >> >> > [3]:
https://gerrit.ovirt.org/#/c/61260
>> > <https://gerrit.ovirt.org/#/c/61260>
>> > > <https://gerrit.ovirt.org/#/c/61260
>> > <https://gerrit.ovirt.org/#/c/61260>>
>> > > >> >> >> > [4]:
>> > https://github.com/oVirt/ovirt-web-ui/pull/106/
>> > <https://github.com/oVirt/ovirt-web-ui/pull/106/>
>> > > <https://github.com/oVirt/ovirt-web-ui/pull/106/
>> > <https://github.com/oVirt/ovirt-web-ui/pull/106/>>
>> > > >> >> >> > [5]:
https://gerrit.ovirt.org/#/c/45233/
>> > <https://gerrit.ovirt.org/#/c/45233/>
>> > > <https://gerrit.ovirt.org/#/c/45233/
>> > <https://gerrit.ovirt.org/#/c/45233/>>
>> > > >> >> >> >
>> > > >> >> >> >
>> > > >> >> >> >
_______________________________________________
>> > > >> >> >> > Devel mailing list
>> > > >> >> >> > Devel(a)ovirt.org
<mailto:Devel@ovirt.org>
>> > <mailto:Devel@ovirt.org <mailto:Devel@ovirt.org>>
>> > > >> >> >> >
http://lists.ovirt.org/mailman/listinfo/devel
>> > <http://lists.ovirt.org/mailman/listinfo/devel>
>> > > <http://lists.ovirt.org/mailman/listinfo/devel
>> > <http://lists.ovirt.org/mailman/listinfo/devel>>
>> > > >> >> >>
_______________________________________________
>> > > >> >> >> Devel mailing list
>> > > >> >> >> Devel(a)ovirt.org
<mailto:Devel@ovirt.org>
>> > <mailto:Devel@ovirt.org <mailto:Devel@ovirt.org>>
>> > > >> >> >>
http://lists.ovirt.org/mailman/listinfo/devel
>> > <http://lists.ovirt.org/mailman/listinfo/devel>
>> > > <http://lists.ovirt.org/mailman/listinfo/devel
>> > <http://lists.ovirt.org/mailman/listinfo/devel>>
>> > > >> >> >
>> > > >> >> >
>> > > >> >> >
>> > > >> >> >
>> > > >> >> > --
>> > > >> >> > Greg Sheremeta, MBA
>> > > >> >> > Red Hat, Inc.
>> > > >> >> > Sr. Software Engineer
>> > > >> >> > gshereme(a)redhat.com
<mailto:gshereme@redhat.com>
>> > <mailto:gshereme@redhat.com <mailto:gshereme@redhat.com>>
>> > > >> >
>> > > >> >
>> > > >> _______________________________________________
>> > > >> Devel mailing list
>> > > >> Devel(a)ovirt.org <mailto:Devel@ovirt.org>
>> > <mailto:Devel@ovirt.org <mailto:Devel@ovirt.org>>
>> > > >> http://lists.ovirt.org/mailman/listinfo/devel
>> > <http://lists.ovirt.org/mailman/listinfo/devel>
>> > > <http://lists.ovirt.org/mailman/listinfo/devel
>> > <http://lists.ovirt.org/mailman/listinfo/devel>>
>> > > >
>> > > >
>> > >
>> > >
>> >
>> >
>>
>>
>
> _______________________________________________
> Devel mailing list
> Devel(a)ovirt.org
> http://lists.ovirt.org/mailman/listinfo/devel
>
--
Greg Sheremeta, MBA
Red Hat, Inc.
Sr. Software Engineer
gshereme(a)redhat.com
2587
days inactive
2590
days old
15 comments
7 participants
participants (7)
-
Greg Sheremeta -
Juan Hernández -
Martin Betak -
Martin Sivak -
Shmuel Melamud -
Tomas Jelinek -
Yaniv Kaul