[ovirt-devel] [ATN] Introduction of RESTAPI metamodel
Juan Hernández
jhernand at redhat.com
Tue Oct 27 09:47:40 UTC 2015
On 10/27/2015 10:16 AM, Roman Mohr wrote:
>
>
> On Mon, Oct 26, 2015 at 5:32 PM, Juan Hernández <jhernand at redhat.com
> <mailto:jhernand at redhat.com>> wrote:
>
> On 10/26/2015 04:56 PM, Roman Mohr wrote:
> > Hi Juan,
> >
> > The way to specify the contract look pretty clean and nice.
> > I would love to read a few words about the big picture. What is the
> > final scenario?
> >
>
> The motivation for this change is that currently we don't have a central
> place where the RESTAPI is specified, rather we have several different
> places, using several different technologies:
>
> * XML schema for the data model.
> * JAX-RS for part of the operational model (without the parameters).
> * rsdl_metadata.yaml for the parameters of the operational model.
>
> This makes it difficult to infer information about the model. For
> example, the generators of the SDKs have to download the XML schema, and
> the RSDL (which is generated from the JAX-RS interfaces using reflection
> and combining it with the information from the rsdl_metadata.yaml file)
> and then they have to do their own computations to extract what they
> need.
>
> Same happens with the CLI: it has to extract the information it needs
> from the Python code generated for the Python SDK, yet another level of
> indirection.
>
>
> You are right, that definitely needs to be cleaned up. I just want to
> discuss a few points below with you.
>
>
>
> We are also lacking a comprehensive reference documentation of the
> RESTAPI. What we currently have has been written by hand, and gets out
> of sync very quickly, and we don't even notice.
>
>
> Did you also consider swagger? It is made for exactly that purpose.
> I created a demo in [1] which uses resteasy, weld, hibernate-validator
> and swagger to demonstrate how to do DRY with jaxrs.
> Would be great to hear you thoughts on that.
>
> And there is the great swagger-ui [8] to display the documentation in a
> more human readable way.
>
Yes, I considered Swagger, and rejected it because it is JSON centric,
and I think JSON isn't as good as Java to represent the contracts of our
RESTAPI.
In addition we need to do these changes in a smooth way, without causing
big changes in the middle. For example, in the first step we need to
preserve the JAX-RS interfaces as they are today, to avoid massive
changes to all the resource implementations. This could be done with
Swagger, but would require custom code generators. With less effort we
can do our own.
Swagger UI is certainly great. I did test it and it is really good. We
may be able to copy some concepts.
>
>
> To solve these issues I intend to have the specification of the RESTAPI
> only in one place, and using only one technology. I decided to use Java
> interfaces for that. Note however that they are just the support for the
> information, like paper is the support for ink. I decided to use Java
> because it is easy to create, modify and re-factor using tools familiar
> to most of us.
>
> These source of these interfaces is analysed (using QDox, currently) and
> a "model" of the RESTAPI is generated in memory. This model is
> independent of the supporting Java source, and easy to consume. For
> example, imagine that you want to list all the types available in the
> model and for each one display its documentation:
>
> Model model = ...;
> for (Type type : model.getTypes()) {
> Name name = type.getName();
> String doc = type.getDoc();
> System.out.println(name + ": " + doc);
> }
>
> Something like this, but more elaborate, will be part of a web
> application that provides comprehensive reference documentation,
> assuming that we dedicate the time to write documentation comments in
> the specification.
>
> I intend to use this model also to do simplify the generators of the
> SDKs and the CLI.
>
> In addition these are some of the things that I would like to change in
> the near future (for 4.0):
>
> * Move the specification of the parameters of operations out of the
> rsdl_metadata.yaml file and into the model. For example:
>
> @Service
> public VmService {
> /**
> * The operation to add a virtual machine.
> */
> interface Add {
> /**
> * The representation of the virtual machine is received
> * as parameter, and the representation of the created
> * virtual machine is returned as result.
> */
> @In @Out Vm vm();
>
> /**
> * In the future, we will be able to specify other
> * parameters here.
> */
> @In Boolean force();
>
> /**
> * Even with default values.
> */
> @In default Boolean force() { return true; }
>
> /**
> * And we will be able to specify constraints, which
> * will replace the rsdl_metadata.yaml file.
> */
> @Constraint
> default boolean vmNameMustNotBeNull() {
> return vm().name() != null;
> }
> }
> }
>
> * Enforce the constraints automatically. If the constraints are in the
> model, then we can just check them and reject requests before delivering
> them to the application. Currently we do this manually (and often
> forget) with calls to "validate(...)" methods.
>
>
>
> Did you consider just annotating the DTOs with JSR-303 annotations and
> integrate a validator with jax-rs?
> See [2] for an example.
>
This is a great way to implement a system, but the goal here isn't to
implement it, rather to specify it. Using annotations in this way won't
help the generators of the SDKs, for example, to figure out what
parameters are required, mandatory, etc.
>
>
> * Generate the Java classes directly from the model. Instead of Model ->
> XML Schema -> Java, we can do Model -> Java. This will allow us to solve
> some of the XJC compiler limitations, like the horrible way we handle
> arrays today.
>
>
> Swagger [3] is a rest documentation specification. There is also a maven
> plugin [4] and you can create clients for example with [5].
>
>
>
> * Replace JAX-RS with a simpler infrastructure that supports better
> streaming and CDI injection.
>
>
>
> With resteasy-cdi you have pretty good injection support for resteasy.
> Run the demo in [1] to see it in action and look at the file at [6].
>
Resteasy-CDI isn't standard, it only works with Resteasy. If we rely on
it then we re tied to Resteasy for ever.
>
>
> * Add support for multiple versions of the API, using the "Version"
> header, and generating different Java classes for entities and services.
> For example, if we have versions 4 and 5 of the model as separate
> artifacts, then we can generate "V4Vm" and "V5Vm" entity classes, and
> "V4VmService" and "V5VmService" service classes. These can be used
> simultaneously in the server, so we can have in the same engine
> implementations for multiple versions.
>
>
> There are also many ways to do that. Here [7] is a pretty clean way to
> do it with jax-rs and you will have everything related in one resource.
>
Yes, there are many ways. In my opinion it is better to use the HTTP
"Version" header, and to forward requests to different resource
implementations without requiring different URLs or different content types.
>
>
> The final picture isn't completely defined yet.
>
> Regards,
> Juan Hernandez
>
> > On Mon, Oct 26, 2015 at 4:03 PM, Juan Hernández <jhernand at redhat.com <mailto:jhernand at redhat.com>
> > <mailto:jhernand at redhat.com <mailto:jhernand at redhat.com>>> wrote:
> >
> > Hello,
> >
> > I will soon merge the following patches that introduce a new
> way to
> > specify the contracts of the RESTAPI:
> >
> > restapi: Introduce metamodel
> > https://gerrit.ovirt.org/45852
> >
> > restapi: Use metamodel
> > https://gerrit.ovirt.org/46478
> >
> > restapi: Generate JAX-RS interfaces from model
> > https://gerrit.ovirt.org/47337
> >
> >
>
> > Looks pretty much like we are replacing one way of annotating things
> > with another way of specifying things.
> > Could you elaborate what the benefit of that way of description is?
> >
> > How would I customize endpoints with e.g. @Gzip annotations? Would
> I at
> > the end still have my JAX-RS annotates resource classes?
> >
> >
> > These patches introduce a new "metamodel" concept, and move
> the current
> > specification of the RESTAPI based on XML schema and JAX-RS
> interfaces
> > to a new "model" built on the new metamodel.
> >
> >
> > What does this mean for you in practical terms? Currently when
> you want
> > to introduce or modify one of the data types used by the
> RESTAPI you
> > start by modifying the XML schema. Once the patches are merged
> the XML
> > schema will never be touched, as it will be automatically
> generated from
> > the "model". For example, imagine that you need to add a new
> "color"
> > attribute to the "VM" entity. To do so with the new model you
> will have
> > to modify the following file, which is the specification of
> the "Vm"
> > entity, written as a Java interface:
> >
> >
> >
> https://gerrit.ovirt.org/#/c/46478/16/backend/manager/modules/restapi/model/src/main/java/types/Vm.java
> >
> > In that interface you will have to add a line like this:
> >
> > String color();
> >
> > Note that this Java interface is just the specification of the
> entity,
> > it won't be used at all during runtime. Instead of that the
> XML schema
> > will be generated from it, and then Java will be generated
> from the XML
> > schema, as we do today (this will change in the future, but
> not yet).
> >
> > Same for the services. If you want to add a new "paint" action
> to the
> > "Vm" resource then you won't modify the JAX-RS interfaces,
> instead of
> > that you will modify the following file, which is the
> specification of
> > the "Vm" service, written as a Java interface:
> >
> >
> >
> https://gerrit.ovirt.org/#/c/47337/6/backend/manager/modules/restapi/model/src/main/java/services/VmService.java
> >
> > In that interface you will need to add a sub-interface
> representing the
> > action:
> >
> > interface Paint {
> > }
> >
> > The JAX-RS interface will be generated from that. Currently these
> > sub-interfaces are empty. In the future they will contain the
> > specifications of the parameters (currently in the
> rsdl_metadata.yml
> > file).
> >
> >
> >
> > These changes will currently affect only the specification of the
> > RESTAPI, not the implementation, so in in the
> "Backend*Resource" classes
> > things won't change yet.
> >
> >
> > Currently I do not really understand where we are going here. Are we
> > trying to get rid of rdsl?
> >
> > So basically two questions:
> >
> > 1) What is the final goal?
> > 2) What speaks agains using Hibernate validator on Daos in combination
> > with JAX-RS annotated resources (and just removing all interfaces, as
> > far as I can see we only have one implementation per endpoint) and
> > creating all schemas and clients through SWAGGER tooling?
> >
> >
> > If you have doubts, please let me know.
> >
> > Regards,
> > Juan Hernandez
> >
> > --
> > Dirección Comercial: C/Jose Bardasano Baos, 9, Edif. Gorbea 3,
> planta
> > 3ºD, 28016 Madrid, Spain
> > Inscrita en el Reg. Mercantil de Madrid – C.I.F. B82657941 -
> Red Hat
> > S.L.
> > _______________________________________________
> > Devel mailing list
> > Devel at ovirt.org <mailto:Devel at ovirt.org>
> <mailto:Devel at ovirt.org <mailto:Devel at ovirt.org>>
> > http://lists.ovirt.org/mailman/listinfo/devel
> >
> >
> > Thanks,
> >
> > Roman
>
>
> --
> Dirección Comercial: C/Jose Bardasano Baos, 9, Edif. Gorbea 3, planta
> 3ºD, 28016 Madrid, Spain
> Inscrita en el Reg. Mercantil de Madrid – C.I.F. B82657941 - Red Hat
> S.L.
>
>
>
> I don't know if it is the right thing to do to invent something new
> here. I personally would prefer to thread a path which is very common on
> the java community.
> I would love follow the DRY principle regarding to the stack and the
> code and would just use the great community projects there.
>
> It would also completely eliminate any custom magic. The JAX-RS and CDI
> magic is pretty standard and easy to understand.
> From my perspective, real JAX-RS resoures have the advantage of
>
> * being very easy to understand (there is magic, but the connection to
> the real endpoint is pretty clear)
> * being easy to customize suff, like adding @GZip to an annotation
> * describing pretty clearly the connection between the generated rest
> interface and the internal services
>
> Finally writing hand crafted tests is also much easier.
>
> What are your thoughts about that?
>
> Best Regards,
> Roman
>
>
> [1] https://github.com/rmohr/jetty-maven-cdi-demo
> [2]
> https://github.com/rmohr/jetty-maven-cdi-demo/blob/master/src/main/java/rmohr/examples/cdi/MyDto.java
> [3] http://swagger.io/
> [4] https://github.com/kongchen/swagger-maven-plugin
> [5] https://github.com/swagger-api/swagger-codegen
> [6]
> https://github.com/rmohr/jetty-maven-cdi-demo/blob/master/src/main/java/rmohr/examples/cdi/RestSubResource.java
> [7]
> http://maxenglander.com/2013/04/23/basic-restful-api-versioning-in-jersey.html
> [8] https://github.com/swagger-api/swagger-ui
>
--
Dirección Comercial: C/Jose Bardasano Baos, 9, Edif. Gorbea 3, planta
3ºD, 28016 Madrid, Spain
Inscrita en el Reg. Mercantil de Madrid – C.I.F. B82657941 - Red Hat S.L.
More information about the Devel
mailing list