<div dir="ltr"><br><div class="gmail_extra"><br><div class="gmail_quote">On Tue, Oct 27, 2015 at 10:47 AM, Juan Hernández <span dir="ltr"><<a href="mailto:jhernand@redhat.com" target="_blank">jhernand@redhat.com</a>></span> wrote:<br><blockquote class="gmail_quote" style="margin:0px 0px 0px 0.8ex;border-left:1px solid rgb(204,204,204);padding-left:1ex"><span class="">On 10/27/2015 10:16 AM, Roman Mohr wrote:<br>
><br>
><br>
> On Mon, Oct 26, 2015 at 5:32 PM, Juan Hernández <<a href="mailto:jhernand@redhat.com">jhernand@redhat.com</a><br>
</span><div><div class="h5">> <mailto:<a href="mailto:jhernand@redhat.com">jhernand@redhat.com</a>>> wrote:<br>
><br>
> On 10/26/2015 04:56 PM, Roman Mohr wrote:<br>
> > Hi Juan,<br>
> ><br>
> > The way to specify the contract look pretty clean and nice.<br>
> > I would love to read a few words about the big picture. What is the<br>
> > final scenario?<br>
> ><br>
><br>
> The motivation for this change is that currently we don't have a central<br>
> place where the RESTAPI is specified, rather we have several different<br>
> places, using several different technologies:<br>
><br>
> * XML schema for the data model.<br>
> * JAX-RS for part of the operational model (without the parameters).<br>
> * rsdl_metadata.yaml for the parameters of the operational model.<br>
><br>
> This makes it difficult to infer information about the model. For<br>
> example, the generators of the SDKs have to download the XML schema, and<br>
> the RSDL (which is generated from the JAX-RS interfaces using reflection<br>
> and combining it with the information from the rsdl_metadata.yaml file)<br>
> and then they have to do their own computations to extract what they<br>
> need.<br>
><br>
> Same happens with the CLI: it has to extract the information it needs<br>
> from the Python code generated for the Python SDK, yet another level of<br>
> indirection.<br>
><br>
><br>
> You are right, that definitely needs to be cleaned up. I just want to<br>
> discuss a few points below with you.<br>
><br>
><br>
><br>
> We are also lacking a comprehensive reference documentation of the<br>
> RESTAPI. What we currently have has been written by hand, and gets out<br>
> of sync very quickly, and we don't even notice.<br>
><br>
><br>
> Did you also consider swagger? It is made for exactly that purpose.<br>
> I created a demo in [1] which uses resteasy, weld, hibernate-validator<br>
> and swagger to demonstrate how to do DRY with jaxrs.<br>
> Would be great to hear you thoughts on that.<br>
><br>
> And there is the great swagger-ui [8] to display the documentation in a<br>
> more human readable way.<br>
><br>
<br>
</div></div>Yes, I considered Swagger, and rejected it because it is JSON centric,<br>
and I think JSON isn't as good as Java to represent the contracts of our<br>
RESTAPI.<br></blockquote><br></div><div class="gmail_quote">You just write plain jax-rs, swagger just creates a description out of it. So the source defining the contract is pure java (jax-rs with some swagger annotations for description, etc.).<br></div><div class="gmail_quote">Or am I missing the point here?<br></div><div class="gmail_quote"><div> </div><blockquote class="gmail_quote" style="margin:0px 0px 0px 0.8ex;border-left:1px solid rgb(204,204,204);padding-left:1ex">
<br>
In addition we need to do these changes in a smooth way, without causing<br>
big changes in the middle. For example, in the first step we need to<br>
preserve the JAX-RS interfaces as they are today, to avoid massive<br>
changes to all the resource implementations. This could be done with<br></blockquote><blockquote class="gmail_quote" style="margin:0px 0px 0px 0.8ex;border-left:1px solid rgb(204,204,204);padding-left:1ex">
Swagger, but would require custom code generators. With less effort we<br>
can do our own.<br></blockquote><div><br></div><div>This is of course generally a difficult task. But I do not know why it would be more difficult to write a custom swagger reader (if we even have to, it can read the interfaces as well) .<br></div><div>They are pretty streight forward. Just look at [9], this contains the wole jax-rs specific code to generate the swagger documentation.<br><br></div><div>But yes, I don't know every detail here of the engine and can't clearly say that integrating that would just streight forward (my feeling tells me that it would not be too hard). I am just under the impression that we would benefit from that. Just reduces custom magic to a minimum.<br></div><div> </div><blockquote class="gmail_quote" style="margin:0px 0px 0px 0.8ex;border-left:1px solid rgb(204,204,204);padding-left:1ex">
<br>
Swagger UI is certainly great. I did test it and it is really good. We<br>
may be able to copy some concepts.<br>
<div><div class="h5"><br>
><br>
><br>
> To solve these issues I intend to have the specification of the RESTAPI<br>
> only in one place, and using only one technology. I decided to use Java<br>
> interfaces for that. Note however that they are just the support for the<br>
> information, like paper is the support for ink. I decided to use Java<br>
> because it is easy to create, modify and re-factor using tools familiar<br>
> to most of us.<br>
><br>
> These source of these interfaces is analysed (using QDox, currently) and<br>
> a "model" of the RESTAPI is generated in memory. This model is<br>
> independent of the supporting Java source, and easy to consume. For<br>
> example, imagine that you want to list all the types available in the<br>
> model and for each one display its documentation:<br>
><br>
> Model model = ...;<br>
> for (Type type : model.getTypes()) {<br>
> Name name = type.getName();<br>
> String doc = type.getDoc();<br>
> System.out.println(name + ": " + doc);<br>
> }<br>
><br>
> Something like this, but more elaborate, will be part of a web<br>
> application that provides comprehensive reference documentation,<br>
> assuming that we dedicate the time to write documentation comments in<br>
> the specification.<br>
><br>
> I intend to use this model also to do simplify the generators of the<br>
> SDKs and the CLI.<br>
><br>
> In addition these are some of the things that I would like to change in<br>
> the near future (for 4.0):<br>
><br>
> * Move the specification of the parameters of operations out of the<br>
> rsdl_metadata.yaml file and into the model. For example:<br>
><br>
> @Service<br>
> public VmService {<br>
> /**<br>
> * The operation to add a virtual machine.<br>
> */<br>
> interface Add {<br>
> /**<br>
> * The representation of the virtual machine is received<br>
> * as parameter, and the representation of the created<br>
> * virtual machine is returned as result.<br>
> */<br>
> @In @Out Vm vm();<br>
><br>
> /**<br>
> * In the future, we will be able to specify other<br>
> * parameters here.<br>
> */<br>
> @In Boolean force();<br>
><br>
> /**<br>
> * Even with default values.<br>
> */<br>
> @In default Boolean force() { return true; }<br>
><br>
> /**<br>
> * And we will be able to specify constraints, which<br>
> * will replace the rsdl_metadata.yaml file.<br>
> */<br>
> @Constraint<br>
> default boolean vmNameMustNotBeNull() {<br>
> return vm().name() != null;<br>
> }<br>
> }<br>
> }<br>
><br>
> * Enforce the constraints automatically. If the constraints are in the<br>
> model, then we can just check them and reject requests before delivering<br>
> them to the application. Currently we do this manually (and often<br>
> forget) with calls to "validate(...)" methods.<br>
><br>
><br>
><br>
> Did you consider just annotating the DTOs with JSR-303 annotations and<br>
> integrate a validator with jax-rs?<br>
> See [2] for an example.<br>
><br>
<br>
</div></div>This is a great way to implement a system, but the goal here isn't to<br>
implement it, rather to specify it. Using annotations in this way won't<br>
help the generators of the SDKs, for example, to figure out what<br>
parameters are required, mandatory, etc.<br>
<span class=""><br></span></blockquote><div><br></div><div>Swagger understands them. From my example project, swagger created that<br><br> description:<br> type: "string"<br> minLength: 10<br> maxLength: 100<br> <br></div><div>out of <br><br></div><div> @Size(min=10, max=100) # jsr-303<br></div><div> private String description;<br></div><div><br></div><div>and so does swagger-codegen which can generate clients in java, python, ...<br></div><div><br></div><blockquote class="gmail_quote" style="margin:0px 0px 0px 0.8ex;border-left:1px solid rgb(204,204,204);padding-left:1ex"><span class="">
><br>
><br>
> * Generate the Java classes directly from the model. Instead of Model -><br>
> XML Schema -> Java, we can do Model -> Java. This will allow us to solve<br>
> some of the XJC compiler limitations, like the horrible way we handle<br>
> arrays today.<br>
><br>
><br>
> Swagger [3] is a rest documentation specification. There is also a maven<br>
> plugin [4] and you can create clients for example with [5].<br>
><br>
><br>
><br>
> * Replace JAX-RS with a simpler infrastructure that supports better<br>
> streaming and CDI injection.<br>
><br>
><br>
><br>
> With resteasy-cdi you have pretty good injection support for resteasy.<br>
> Run the demo in [1] to see it in action and look at the file at [6].<br>
><br>
<br>
</span>Resteasy-CDI isn't standard, it only works with Resteasy. If we rely on<br>
it then we re tied to Resteasy for ever.<br>
<span class=""><br></span></blockquote><div><br></div><div>Even jersey has support for that (I think it is called jeryse-gf-cdi), but why would we want switch? I don't think that jboss will drop resteasy and it also works fine outside of full blown containers. I don't think that this is an argument.<br> </div><blockquote class="gmail_quote" style="margin:0px 0px 0px 0.8ex;border-left:1px solid rgb(204,204,204);padding-left:1ex"><span class="">
><br>
><br>
> * Add support for multiple versions of the API, using the "Version"<br>
> header, and generating different Java classes for entities and services.<br>
> For example, if we have versions 4 and 5 of the model as separate<br>
> artifacts, then we can generate "V4Vm" and "V5Vm" entity classes, and<br>
> "V4VmService" and "V5VmService" service classes. These can be used<br>
> simultaneously in the server, so we can have in the same engine<br>
> implementations for multiple versions.<br>
><br>
><br>
> There are also many ways to do that. Here [7] is a pretty clean way to<br>
> do it with jax-rs and you will have everything related in one resource.<br>
><br>
<br>
</span>Yes, there are many ways. In my opinion it is better to use the HTTP<br>
"Version" header, and to forward requests to different resource<br>
implementations without requiring different URLs or different content types.<br>
<span class=""><br></span></blockquote><div>Have no strong opinion there, just seemed to be a good choice regarding to versioning limitations in jax-rs and our use of jax-rs subresources.<br> </div><blockquote class="gmail_quote" style="margin:0px 0px 0px 0.8ex;border-left:1px solid rgb(204,204,204);padding-left:1ex"><span class="">
><br>
><br>
> The final picture isn't completely defined yet.<br>
><br>
> Regards,<br>
> Juan Hernandez<br>
><br>
> > On Mon, Oct 26, 2015 at 4:03 PM, Juan Hernández <<a href="mailto:jhernand@redhat.com">jhernand@redhat.com</a> <mailto:<a href="mailto:jhernand@redhat.com">jhernand@redhat.com</a>><br>
</span><div><div class="h5">> > <mailto:<a href="mailto:jhernand@redhat.com">jhernand@redhat.com</a> <mailto:<a href="mailto:jhernand@redhat.com">jhernand@redhat.com</a>>>> wrote:<br>
> ><br>
> > Hello,<br>
> ><br>
> > I will soon merge the following patches that introduce a new<br>
> way to<br>
> > specify the contracts of the RESTAPI:<br>
> ><br>
> > restapi: Introduce metamodel<br>
> > <a href="https://gerrit.ovirt.org/45852" rel="noreferrer" target="_blank">https://gerrit.ovirt.org/45852</a><br>
> ><br>
> > restapi: Use metamodel<br>
> > <a href="https://gerrit.ovirt.org/46478" rel="noreferrer" target="_blank">https://gerrit.ovirt.org/46478</a><br>
> ><br>
> > restapi: Generate JAX-RS interfaces from model<br>
> > <a href="https://gerrit.ovirt.org/47337" rel="noreferrer" target="_blank">https://gerrit.ovirt.org/47337</a><br>
> ><br>
> ><br>
><br>
> > Looks pretty much like we are replacing one way of annotating things<br>
> > with another way of specifying things.<br>
> > Could you elaborate what the benefit of that way of description is?<br>
> ><br>
> > How would I customize endpoints with e.g. @Gzip annotations? Would<br>
> I at<br>
> > the end still have my JAX-RS annotates resource classes?<br>
> ><br>
> ><br>
> > These patches introduce a new "metamodel" concept, and move<br>
> the current<br>
> > specification of the RESTAPI based on XML schema and JAX-RS<br>
> interfaces<br>
> > to a new "model" built on the new metamodel.<br>
> ><br>
> ><br>
> > What does this mean for you in practical terms? Currently when<br>
> you want<br>
> > to introduce or modify one of the data types used by the<br>
> RESTAPI you<br>
> > start by modifying the XML schema. Once the patches are merged<br>
> the XML<br>
> > schema will never be touched, as it will be automatically<br>
> generated from<br>
> > the "model". For example, imagine that you need to add a new<br>
> "color"<br>
> > attribute to the "VM" entity. To do so with the new model you<br>
> will have<br>
> > to modify the following file, which is the specification of<br>
> the "Vm"<br>
> > entity, written as a Java interface:<br>
> ><br>
> ><br>
> ><br>
> <a href="https://gerrit.ovirt.org/#/c/46478/16/backend/manager/modules/restapi/model/src/main/java/types/Vm.java" rel="noreferrer" target="_blank">https://gerrit.ovirt.org/#/c/46478/16/backend/manager/modules/restapi/model/src/main/java/types/Vm.java</a><br>
> ><br>
> > In that interface you will have to add a line like this:<br>
> ><br>
> > String color();<br>
> ><br>
> > Note that this Java interface is just the specification of the<br>
> entity,<br>
> > it won't be used at all during runtime. Instead of that the<br>
> XML schema<br>
> > will be generated from it, and then Java will be generated<br>
> from the XML<br>
> > schema, as we do today (this will change in the future, but<br>
> not yet).<br>
> ><br>
> > Same for the services. If you want to add a new "paint" action<br>
> to the<br>
> > "Vm" resource then you won't modify the JAX-RS interfaces,<br>
> instead of<br>
> > that you will modify the following file, which is the<br>
> specification of<br>
> > the "Vm" service, written as a Java interface:<br>
> ><br>
> ><br>
> ><br>
> <a href="https://gerrit.ovirt.org/#/c/47337/6/backend/manager/modules/restapi/model/src/main/java/services/VmService.java" rel="noreferrer" target="_blank">https://gerrit.ovirt.org/#/c/47337/6/backend/manager/modules/restapi/model/src/main/java/services/VmService.java</a><br>
> ><br>
> > In that interface you will need to add a sub-interface<br>
> representing the<br>
> > action:<br>
> ><br>
> > interface Paint {<br>
> > }<br>
> ><br>
> > The JAX-RS interface will be generated from that. Currently these<br>
> > sub-interfaces are empty. In the future they will contain the<br>
> > specifications of the parameters (currently in the<br>
> rsdl_metadata.yml<br>
> > file).<br>
> ><br>
> ><br>
> ><br>
> > These changes will currently affect only the specification of the<br>
> > RESTAPI, not the implementation, so in in the<br>
> "Backend*Resource" classes<br>
> > things won't change yet.<br>
> ><br>
> ><br>
> > Currently I do not really understand where we are going here. Are we<br>
> > trying to get rid of rdsl?<br>
> ><br>
> > So basically two questions:<br>
> ><br>
> > 1) What is the final goal?<br>
> > 2) What speaks agains using Hibernate validator on Daos in combination<br>
> > with JAX-RS annotated resources (and just removing all interfaces, as<br>
> > far as I can see we only have one implementation per endpoint) and<br>
> > creating all schemas and clients through SWAGGER tooling?<br>
> ><br>
> ><br>
> > If you have doubts, please let me know.<br>
> ><br>
> > Regards,<br>
> > Juan Hernandez<br>
> ><br>
> > --<br>
> > Dirección Comercial: C/Jose Bardasano Baos, 9, Edif. Gorbea 3,<br>
> planta<br>
> > 3ºD, 28016 Madrid, Spain<br>
> > Inscrita en el Reg. Mercantil de Madrid – C.I.F. B82657941 -<br>
> Red Hat<br>
> > S.L.<br>
> > _______________________________________________<br>
> > Devel mailing list<br>
> > <a href="mailto:Devel@ovirt.org">Devel@ovirt.org</a> <mailto:<a href="mailto:Devel@ovirt.org">Devel@ovirt.org</a>><br>
</div></div>> <mailto:<a href="mailto:Devel@ovirt.org">Devel@ovirt.org</a> <mailto:<a href="mailto:Devel@ovirt.org">Devel@ovirt.org</a>>><br>
<div class=""><div class="h5">> > <a href="http://lists.ovirt.org/mailman/listinfo/devel" rel="noreferrer" target="_blank">http://lists.ovirt.org/mailman/listinfo/devel</a><br>
> ><br>
> ><br>
> > Thanks,<br>
> ><br>
> > Roman<br>
><br>
><br>
> --<br>
> Dirección Comercial: C/Jose Bardasano Baos, 9, Edif. Gorbea 3, planta<br>
> 3ºD, 28016 Madrid, Spain<br>
> Inscrita en el Reg. Mercantil de Madrid – C.I.F. B82657941 - Red Hat<br>
> S.L.<br>
><br>
><br>
><br>
> I don't know if it is the right thing to do to invent something new<br>
> here. I personally would prefer to thread a path which is very common on<br>
> the java community.<br>
> I would love follow the DRY principle regarding to the stack and the<br>
> code and would just use the great community projects there.<br>
><br>
> It would also completely eliminate any custom magic. The JAX-RS and CDI<br>
> magic is pretty standard and easy to understand.<br>
> From my perspective, real JAX-RS resoures have the advantage of<br>
><br>
> * being very easy to understand (there is magic, but the connection to<br>
> the real endpoint is pretty clear)<br>
> * being easy to customize suff, like adding @GZip to an annotation<br>
> * describing pretty clearly the connection between the generated rest<br>
> interface and the internal services<br>
><br>
> Finally writing hand crafted tests is also much easier.<br>
><br>
> What are your thoughts about that?<br>
><br>
> Best Regards,<br>
> Roman<br>
><br>
><br>
> [1] <a href="https://github.com/rmohr/jetty-maven-cdi-demo" rel="noreferrer" target="_blank">https://github.com/rmohr/jetty-maven-cdi-demo</a><br>
> [2]<br>
> <a href="https://github.com/rmohr/jetty-maven-cdi-demo/blob/master/src/main/java/rmohr/examples/cdi/MyDto.java" rel="noreferrer" target="_blank">https://github.com/rmohr/jetty-maven-cdi-demo/blob/master/src/main/java/rmohr/examples/cdi/MyDto.java</a><br>
> [3] <a href="http://swagger.io/" rel="noreferrer" target="_blank">http://swagger.io/</a><br>
> [4] <a href="https://github.com/kongchen/swagger-maven-plugin" rel="noreferrer" target="_blank">https://github.com/kongchen/swagger-maven-plugin</a><br>
> [5] <a href="https://github.com/swagger-api/swagger-codegen" rel="noreferrer" target="_blank">https://github.com/swagger-api/swagger-codegen</a><br>
> [6]<br>
> <a href="https://github.com/rmohr/jetty-maven-cdi-demo/blob/master/src/main/java/rmohr/examples/cdi/RestSubResource.java" rel="noreferrer" target="_blank">https://github.com/rmohr/jetty-maven-cdi-demo/blob/master/src/main/java/rmohr/examples/cdi/RestSubResource.java</a><br>
> [7]<br>
> <a href="http://maxenglander.com/2013/04/23/basic-restful-api-versioning-in-jersey.html" rel="noreferrer" target="_blank">http://maxenglander.com/2013/04/23/basic-restful-api-versioning-in-jersey.html</a><br>
> [8] <a href="https://github.com/swagger-api/swagger-ui" rel="noreferrer" target="_blank">https://github.com/swagger-api/swagger-ui</a><br>
><br></div></div></blockquote><div>[9] <a href="https://github.com/swagger-api/swagger-core/blob/master/modules/swagger-jaxrs/src/main/java/io/swagger/jaxrs/Reader.java">https://github.com/swagger-api/swagger-core/blob/master/modules/swagger-jaxrs/src/main/java/io/swagger/jaxrs/Reader.java</a> <br></div><blockquote class="gmail_quote" style="margin:0px 0px 0px 0.8ex;border-left:1px solid rgb(204,204,204);padding-left:1ex"><div class=""><div class="h5">
<br>
<br>
<br>
</div></div><div class=""><div class="h5">--<br>
Dirección Comercial: C/Jose Bardasano Baos, 9, Edif. Gorbea 3, planta<br>
3ºD, 28016 Madrid, Spain<br>
Inscrita en el Reg. Mercantil de Madrid – C.I.F. B82657941 - Red Hat S.L.<br>
</div></div></blockquote></div><br></div></div>