[ovirt-devel] [ATN] Introduction of RESTAPI metamodel
Roman Mohr
rmohr at redhat.com
Tue Oct 27 10:28:20 UTC 2015
On Tue, Oct 27, 2015 at 10:47 AM, Juan Hernández <jhernand at redhat.com>
wrote:
> On 10/27/2015 10:16 AM, Roman Mohr wrote:
> >
> >
> > On Mon, Oct 26, 2015 at 5:32 PM, Juan Hernández <jhernand at redhat.com
> > <mailto:jhernand at redhat.com>> wrote:
> >
> > On 10/26/2015 04:56 PM, Roman Mohr wrote:
> > > Hi Juan,
> > >
> > > The way to specify the contract look pretty clean and nice.
> > > I would love to read a few words about the big picture. What is the
> > > final scenario?
> > >
> >
> > The motivation for this change is that currently we don't have a
> central
> > place where the RESTAPI is specified, rather we have several
> different
> > places, using several different technologies:
> >
> > * XML schema for the data model.
> > * JAX-RS for part of the operational model (without the parameters).
> > * rsdl_metadata.yaml for the parameters of the operational model.
> >
> > This makes it difficult to infer information about the model. For
> > example, the generators of the SDKs have to download the XML schema,
> and
> > the RSDL (which is generated from the JAX-RS interfaces using
> reflection
> > and combining it with the information from the rsdl_metadata.yaml
> file)
> > and then they have to do their own computations to extract what they
> > need.
> >
> > Same happens with the CLI: it has to extract the information it needs
> > from the Python code generated for the Python SDK, yet another level
> of
> > indirection.
> >
> >
> > You are right, that definitely needs to be cleaned up. I just want to
> > discuss a few points below with you.
> >
> >
> >
> > We are also lacking a comprehensive reference documentation of the
> > RESTAPI. What we currently have has been written by hand, and gets
> out
> > of sync very quickly, and we don't even notice.
> >
> >
> > Did you also consider swagger? It is made for exactly that purpose.
> > I created a demo in [1] which uses resteasy, weld, hibernate-validator
> > and swagger to demonstrate how to do DRY with jaxrs.
> > Would be great to hear you thoughts on that.
> >
> > And there is the great swagger-ui [8] to display the documentation in a
> > more human readable way.
> >
>
> Yes, I considered Swagger, and rejected it because it is JSON centric,
> and I think JSON isn't as good as Java to represent the contracts of our
> RESTAPI.
>
You just write plain jax-rs, swagger just creates a description out of it.
So the source defining the contract is pure java (jax-rs with some swagger
annotations for description, etc.).
Or am I missing the point here?
>
> In addition we need to do these changes in a smooth way, without causing
> big changes in the middle. For example, in the first step we need to
> preserve the JAX-RS interfaces as they are today, to avoid massive
> changes to all the resource implementations. This could be done with
>
Swagger, but would require custom code generators. With less effort we
> can do our own.
>
This is of course generally a difficult task. But I do not know why it
would be more difficult to write a custom swagger reader (if we even have
to, it can read the interfaces as well) .
They are pretty streight forward. Just look at [9], this contains the wole
jax-rs specific code to generate the swagger documentation.
But yes, I don't know every detail here of the engine and can't clearly say
that integrating that would just streight forward (my feeling tells me that
it would not be too hard). I am just under the impression that we would
benefit from that. Just reduces custom magic to a minimum.
>
> Swagger UI is certainly great. I did test it and it is really good. We
> may be able to copy some concepts.
>
> >
> >
> > To solve these issues I intend to have the specification of the
> RESTAPI
> > only in one place, and using only one technology. I decided to use
> Java
> > interfaces for that. Note however that they are just the support for
> the
> > information, like paper is the support for ink. I decided to use Java
> > because it is easy to create, modify and re-factor using tools
> familiar
> > to most of us.
> >
> > These source of these interfaces is analysed (using QDox, currently)
> and
> > a "model" of the RESTAPI is generated in memory. This model is
> > independent of the supporting Java source, and easy to consume. For
> > example, imagine that you want to list all the types available in the
> > model and for each one display its documentation:
> >
> > Model model = ...;
> > for (Type type : model.getTypes()) {
> > Name name = type.getName();
> > String doc = type.getDoc();
> > System.out.println(name + ": " + doc);
> > }
> >
> > Something like this, but more elaborate, will be part of a web
> > application that provides comprehensive reference documentation,
> > assuming that we dedicate the time to write documentation comments in
> > the specification.
> >
> > I intend to use this model also to do simplify the generators of the
> > SDKs and the CLI.
> >
> > In addition these are some of the things that I would like to change
> in
> > the near future (for 4.0):
> >
> > * Move the specification of the parameters of operations out of the
> > rsdl_metadata.yaml file and into the model. For example:
> >
> > @Service
> > public VmService {
> > /**
> > * The operation to add a virtual machine.
> > */
> > interface Add {
> > /**
> > * The representation of the virtual machine is received
> > * as parameter, and the representation of the created
> > * virtual machine is returned as result.
> > */
> > @In @Out Vm vm();
> >
> > /**
> > * In the future, we will be able to specify other
> > * parameters here.
> > */
> > @In Boolean force();
> >
> > /**
> > * Even with default values.
> > */
> > @In default Boolean force() { return true; }
> >
> > /**
> > * And we will be able to specify constraints, which
> > * will replace the rsdl_metadata.yaml file.
> > */
> > @Constraint
> > default boolean vmNameMustNotBeNull() {
> > return vm().name() != null;
> > }
> > }
> > }
> >
> > * Enforce the constraints automatically. If the constraints are in
> the
> > model, then we can just check them and reject requests before
> delivering
> > them to the application. Currently we do this manually (and often
> > forget) with calls to "validate(...)" methods.
> >
> >
> >
> > Did you consider just annotating the DTOs with JSR-303 annotations and
> > integrate a validator with jax-rs?
> > See [2] for an example.
> >
>
> This is a great way to implement a system, but the goal here isn't to
> implement it, rather to specify it. Using annotations in this way won't
> help the generators of the SDKs, for example, to figure out what
> parameters are required, mandatory, etc.
>
>
Swagger understands them. From my example project, swagger created that
description:
type: "string"
minLength: 10
maxLength: 100
out of
@Size(min=10, max=100) # jsr-303
private String description;
and so does swagger-codegen which can generate clients in java, python, ...
>
> >
> > * Generate the Java classes directly from the model. Instead of
> Model ->
> > XML Schema -> Java, we can do Model -> Java. This will allow us to
> solve
> > some of the XJC compiler limitations, like the horrible way we handle
> > arrays today.
> >
> >
> > Swagger [3] is a rest documentation specification. There is also a maven
> > plugin [4] and you can create clients for example with [5].
> >
> >
> >
> > * Replace JAX-RS with a simpler infrastructure that supports better
> > streaming and CDI injection.
> >
> >
> >
> > With resteasy-cdi you have pretty good injection support for resteasy.
> > Run the demo in [1] to see it in action and look at the file at [6].
> >
>
> Resteasy-CDI isn't standard, it only works with Resteasy. If we rely on
> it then we re tied to Resteasy for ever.
>
>
Even jersey has support for that (I think it is called jeryse-gf-cdi), but
why would we want switch? I don't think that jboss will drop resteasy and
it also works fine outside of full blown containers. I don't think that
this is an argument.
> >
> >
> > * Add support for multiple versions of the API, using the "Version"
> > header, and generating different Java classes for entities and
> services.
> > For example, if we have versions 4 and 5 of the model as separate
> > artifacts, then we can generate "V4Vm" and "V5Vm" entity classes, and
> > "V4VmService" and "V5VmService" service classes. These can be used
> > simultaneously in the server, so we can have in the same engine
> > implementations for multiple versions.
> >
> >
> > There are also many ways to do that. Here [7] is a pretty clean way to
> > do it with jax-rs and you will have everything related in one resource.
> >
>
> Yes, there are many ways. In my opinion it is better to use the HTTP
> "Version" header, and to forward requests to different resource
> implementations without requiring different URLs or different content
> types.
>
> Have no strong opinion there, just seemed to be a good choice regarding to
versioning limitations in jax-rs and our use of jax-rs subresources.
> >
> >
> > The final picture isn't completely defined yet.
> >
> > Regards,
> > Juan Hernandez
> >
> > > On Mon, Oct 26, 2015 at 4:03 PM, Juan Hernández <
> jhernand at redhat.com <mailto:jhernand at redhat.com>
> > > <mailto:jhernand at redhat.com <mailto:jhernand at redhat.com>>> wrote:
> > >
> > > Hello,
> > >
> > > I will soon merge the following patches that introduce a new
> > way to
> > > specify the contracts of the RESTAPI:
> > >
> > > restapi: Introduce metamodel
> > > https://gerrit.ovirt.org/45852
> > >
> > > restapi: Use metamodel
> > > https://gerrit.ovirt.org/46478
> > >
> > > restapi: Generate JAX-RS interfaces from model
> > > https://gerrit.ovirt.org/47337
> > >
> > >
> >
> > > Looks pretty much like we are replacing one way of annotating
> things
> > > with another way of specifying things.
> > > Could you elaborate what the benefit of that way of description is?
> > >
> > > How would I customize endpoints with e.g. @Gzip annotations? Would
> > I at
> > > the end still have my JAX-RS annotates resource classes?
> > >
> > >
> > > These patches introduce a new "metamodel" concept, and move
> > the current
> > > specification of the RESTAPI based on XML schema and JAX-RS
> > interfaces
> > > to a new "model" built on the new metamodel.
> > >
> > >
> > > What does this mean for you in practical terms? Currently when
> > you want
> > > to introduce or modify one of the data types used by the
> > RESTAPI you
> > > start by modifying the XML schema. Once the patches are merged
> > the XML
> > > schema will never be touched, as it will be automatically
> > generated from
> > > the "model". For example, imagine that you need to add a new
> > "color"
> > > attribute to the "VM" entity. To do so with the new model you
> > will have
> > > to modify the following file, which is the specification of
> > the "Vm"
> > > entity, written as a Java interface:
> > >
> > >
> > >
> >
> https://gerrit.ovirt.org/#/c/46478/16/backend/manager/modules/restapi/model/src/main/java/types/Vm.java
> > >
> > > In that interface you will have to add a line like this:
> > >
> > > String color();
> > >
> > > Note that this Java interface is just the specification of the
> > entity,
> > > it won't be used at all during runtime. Instead of that the
> > XML schema
> > > will be generated from it, and then Java will be generated
> > from the XML
> > > schema, as we do today (this will change in the future, but
> > not yet).
> > >
> > > Same for the services. If you want to add a new "paint" action
> > to the
> > > "Vm" resource then you won't modify the JAX-RS interfaces,
> > instead of
> > > that you will modify the following file, which is the
> > specification of
> > > the "Vm" service, written as a Java interface:
> > >
> > >
> > >
> >
> https://gerrit.ovirt.org/#/c/47337/6/backend/manager/modules/restapi/model/src/main/java/services/VmService.java
> > >
> > > In that interface you will need to add a sub-interface
> > representing the
> > > action:
> > >
> > > interface Paint {
> > > }
> > >
> > > The JAX-RS interface will be generated from that. Currently
> these
> > > sub-interfaces are empty. In the future they will contain the
> > > specifications of the parameters (currently in the
> > rsdl_metadata.yml
> > > file).
> > >
> > >
> > >
> > > These changes will currently affect only the specification of
> the
> > > RESTAPI, not the implementation, so in in the
> > "Backend*Resource" classes
> > > things won't change yet.
> > >
> > >
> > > Currently I do not really understand where we are going here. Are
> we
> > > trying to get rid of rdsl?
> > >
> > > So basically two questions:
> > >
> > > 1) What is the final goal?
> > > 2) What speaks agains using Hibernate validator on Daos in
> combination
> > > with JAX-RS annotated resources (and just removing all interfaces,
> as
> > > far as I can see we only have one implementation per endpoint) and
> > > creating all schemas and clients through SWAGGER tooling?
> > >
> > >
> > > If you have doubts, please let me know.
> > >
> > > Regards,
> > > Juan Hernandez
> > >
> > > --
> > > Dirección Comercial: C/Jose Bardasano Baos, 9, Edif. Gorbea 3,
> > planta
> > > 3ºD, 28016 Madrid, Spain
> > > Inscrita en el Reg. Mercantil de Madrid – C.I.F. B82657941 -
> > Red Hat
> > > S.L.
> > > _______________________________________________
> > > Devel mailing list
> > > Devel at ovirt.org <mailto:Devel at ovirt.org>
> > <mailto:Devel at ovirt.org <mailto:Devel at ovirt.org>>
> > > http://lists.ovirt.org/mailman/listinfo/devel
> > >
> > >
> > > Thanks,
> > >
> > > Roman
> >
> >
> > --
> > Dirección Comercial: C/Jose Bardasano Baos, 9, Edif. Gorbea 3, planta
> > 3ºD, 28016 Madrid, Spain
> > Inscrita en el Reg. Mercantil de Madrid – C.I.F. B82657941 - Red Hat
> > S.L.
> >
> >
> >
> > I don't know if it is the right thing to do to invent something new
> > here. I personally would prefer to thread a path which is very common on
> > the java community.
> > I would love follow the DRY principle regarding to the stack and the
> > code and would just use the great community projects there.
> >
> > It would also completely eliminate any custom magic. The JAX-RS and CDI
> > magic is pretty standard and easy to understand.
> > From my perspective, real JAX-RS resoures have the advantage of
> >
> > * being very easy to understand (there is magic, but the connection to
> > the real endpoint is pretty clear)
> > * being easy to customize suff, like adding @GZip to an annotation
> > * describing pretty clearly the connection between the generated rest
> > interface and the internal services
> >
> > Finally writing hand crafted tests is also much easier.
> >
> > What are your thoughts about that?
> >
> > Best Regards,
> > Roman
> >
> >
> > [1] https://github.com/rmohr/jetty-maven-cdi-demo
> > [2]
> >
> https://github.com/rmohr/jetty-maven-cdi-demo/blob/master/src/main/java/rmohr/examples/cdi/MyDto.java
> > [3] http://swagger.io/
> > [4] https://github.com/kongchen/swagger-maven-plugin
> > [5] https://github.com/swagger-api/swagger-codegen
> > [6]
> >
> https://github.com/rmohr/jetty-maven-cdi-demo/blob/master/src/main/java/rmohr/examples/cdi/RestSubResource.java
> > [7]
> >
> http://maxenglander.com/2013/04/23/basic-restful-api-versioning-in-jersey.html
> > [8] https://github.com/swagger-api/swagger-ui
> >
>
[9]
https://github.com/swagger-api/swagger-core/blob/master/modules/swagger-jaxrs/src/main/java/io/swagger/jaxrs/Reader.java
>
>
>
> --
> Dirección Comercial: C/Jose Bardasano Baos, 9, Edif. Gorbea 3, planta
> 3ºD, 28016 Madrid, Spain
> Inscrita en el Reg. Mercantil de Madrid – C.I.F. B82657941 - Red Hat S.L.
>
-------------- next part --------------
An HTML attachment was scrubbed...
URL: <http://lists.ovirt.org/pipermail/devel/attachments/20151027/b995752d/attachment.html>
More information about the Devel
mailing list