Consumability of vdsm via libvdsm.so

Tue Aug 14 18:57:52 UTC 2012

On Tue, Aug 14, 2012 at 02:39:11PM -0400, Saggi Mizrahi wrote:
> Well, not by much other then that: * The bindings could sit out of tree with
> less strict support guidelines (TBD).  * We support HTTP but the bindings
> don't use it (unless someone really thinks there is a reason to).  * You
> could. if you so desire. Write against the protocol directly (optimizations?).
> 
> We were really close with what we had, the only problems that people seemed to
> have is not committing to a protocol, and forcing everyone to work with the
> bindings.  This means we commit to the protocol and make the bindings optional
> (I don't see why someone would want to not use them, but I know that
> Not-Invented-Here Syndrome is quite common).
> 
> I feel like it would make everyone happy and it's an evolution of what we
> already started work on.

Ok.  In that case I don't mind committing to a stable protocol as long as we
still do bindings for C and Python in the current style because I think we have
gotten that part right.  We'll generate native bindings for Java then.  Since
JSON is very easy to sloppily extend in incompatible ways (especially for Java/C
bindings), how do we plan to enforce strict data types at the protocol level?

Could someone explain to me how the P2P (ZeroMQ/AMQP) stuff fits into this
design as well?

> 
> ----- Original Message -----
> > From: "Adam Litke" <agl at us.ibm.com> To: "Saggi Mizrahi"
> > <smizrahi at redhat.com> Cc: arch at ovirt.org, "Ayal Baron" <abaron at redhat.com>
> > Sent: Tuesday, August 14, 2012 2:11:06 PM Subject: Re: Consumability of vdsm
> > via libvdsm.so
> > 
> > On Tue, Aug 14, 2012 at 01:43:01PM -0400, Saggi Mizrahi wrote:
> > > 
> > > ----- Original Message -----
> > > > From: "Adam Litke" <agl at us.ibm.com> To: "Saggi Mizrahi"
> > > > <smizrahi at redhat.com> Cc: arch at ovirt.org, "Ayal Baron"
> > > > <abaron at redhat.com> Sent: Tuesday, August 14, 2012 12:36:47 PM Subject:
> > > > Re: Consumability of vdsm via libvdsm.so
> > > > 
> > > > On Tue, Aug 14, 2012 at 10:20:34AM -0400, Saggi Mizrahi wrote:
> > > > > I have an Idea, having json rpc for the over HTTP and 2 way json RPC
> > > > > over openstacks' messaging abstraction layer.
> > > > 
> > > > So you are proposing to disregard all of the arguments in favor of a
> > > > libvdsm and instead standardize on a protocol.  In that case, we could
> > > > just keep xmlrpc...
> > > The difference between xml-rpc and json-rpc is that json-rpc is transport
> > > agnostic, less verbose, easier to parse.
> > > > We will be back at square one having every application writing on object
> > > > model on top of the transport.  We will also need to reinvent a model
> > > > for ensuring backwards compatibility of the protocol data."
> > > As I said, I still think we need to have a schema and supply bindings,
> > > GObject for python\C\C++\Vala, and native Java (because, as Yair pointed
> > > out, JNI in Java EE is non standard).
> > > > 
> > > > > This means we have 1 actual code base, the interfaces are on the
> > > > > transport layer.  And if you choose to use messaging you (in the
> > > > > future) get 2 way communication which means we can use.  Json-RPC
> > > > > notification for events. And that we (in the future) might fit to an
> > > > > openstack cluster.
> > > > 
> > > > > I still think generating the java bindings is preferable as it will
> > > > > have to be done and having it generated straight from the schema is
> > > > > ideal.
> > > > 
> > > > In that case we will also need to generate python bindings at a minimum.
> > > > In this model, how would REST and AMQP bridges be integrated?
> > > No REST, as I said. We forgo the "everything ever can be supported" for
> > > json-rpc over different transports. HTTP\AMQP\ZMQ\Simple TCP Socket.
> > > Unlike supporting different API schemes supporting transports is fairly
> > > simple because the number of methods to implement is small and remains
> > > constant and is independent of the actual API.
> > > 
> > > The big reason to use bindings is to keep us transport agnositc.
> > > Something that REST or SOAP or XML-RPC don't give us because they are
> > > bound to HTTP.
> > > 
> > > We could also "extend" on JSON-RPC by being able to switch serializers
> > > giving us the ability to support BSON and protobuff as long as the clients
> > > adhere to the strong types in the schema.  Unlike XML-RPC that uses
> > > advanced XML features, JSON-RPC can really be any serialized object. So
> > > technically, if de\serialization becomes an issue we could put high
> > > performance serializers as configurable options for very large scale
> > > deployments.
> > > 
> > > JSON-RPC also supports extension methods that can be used to extend the
> > > protocol.
> > > 
> > > As I see it this solves all the issues that were preventing us from
> > > committing to a protocoll.  * We don't want to be bound to a single
> > > transport. (Transport agnostic) * We don't want to be bound to a single
> > > serializer (Solved with optional extension that could be added when we
> > > need it) * 2 way communication (if you use anything other then HTTP you
> > > have that) * low latency (If you use anything other then HTTP you have
> > > that too) * Removal of calls that are actually a group of calls (eg.
> > > connectStorageServers vs connectStorageServer) - Even with HTTP, JSON-RPC
> > > supports request lists which give the expected semantic.  * Simple set-up.
> > > Using ZMQ, HTTP or TCP Socket you can setup VDSM without a broker.  *
> > > Don't have a lot of big classes for bindings. Because we bind int the
> > > transport level the amount of methods per transport is minimal and
> > > constant.  * We provide the bindings. Bindings we'll be available and
> > > generated from the schema. So there is no problem with the (everyone needs
> > > to write their own bindings).  * Allow bridges. Someone can still write a
> > > lightweight REST bridge over the bindings\socket.
> > > 
> > > Is there something that this doesn't solve?
> > 
> > Thanks for the additional detail.  How is this proposal different from what
> > I have submitted to gerrit other than we now need to write an additional
> > Java bindings generator?
> > 
> > > > 
> > > > > 
> > > > > ----- Original Message -----
> > > > > > From: "Adam Litke" <agl at us.ibm.com> To: "Ayal Baron"
> > > > > > <abaron at redhat.com> Cc: "Saggi Mizrahi" <smizrahi at redhat.com>,
> > > > > > arch at ovirt.org Sent: Tuesday, August 14, 2012 10:09:28 AM Subject:
> > > > > > Re: Consumability of vdsm via libvdsm.so
> > > > > > 
> > > > > > On Mon, Aug 13, 2012 at 04:54:18PM -0400, Ayal Baron wrote:
> > > > > > > 
> > > > > > > 
> > > > > > > ----- Original Message -----
> > > > > > > > 
> > > > > > > > 
> > > > > > > > ----- Original Message -----
> > > > > > > > > From: "Yaniv Kaul" <ykaul at redhat.com> To: "Adam Litke"
> > > > > > > > > <agl at us.ibm.com> Cc: arch at ovirt.org Sent: Monday, August 13,
> > > > > > > > > 2012 11:49:21 AM Subject: Re: Consumability of vdsm via
> > > > > > > > > libvdsm.so
> > > > > > > > > 
> > > > > > > > > 
> > > > > > > > > 
> > > > > > > > > On 08/13/2012 06:21 PM, Adam Litke wrote:
> > > > > > > > > 
> > > > > > > > > 
> > > > > > > > > Hi all,
> > > > > > > > > 
> > > > > > > > > We just finished a lively discussion regarding the ongoing
> > > > > > > > > effort to stabilize the vdsm API using a C library called
> > > > > > > > > libvdsm.  There are many things that need discussion, but I
> > > > > > > > > would like to focus this thread on one in particular:
> > > > > > > > > 
> > > > > > > > > Can ovirt-engine consume libvdsm via JNI?
> > > > > > > > > 
> > > > > > > > > libvdsm provides a full-featured C interface to the vdsm API
> > > > > > > > > using GObject.  Java bindings are provided automatically by
> > > > > > > > > jGIR[1].  The library communicates with vdsmd using an
> > > > > > > > > internal transport which is not exposed to end users
> > > > > > > > > (including ovirt-engine).  I would like to learn from folks
> > > > > > > > > with deep Java knowledge if this approach is workable.  What
> > > > > > > > > are the technical challenges to integrating in this way?
> > > > > > > > > Please save discussion of AMQP and other bindings for other
> > > > > > > > > threads.
> > > > > > > > > 
> > > > > > > > > Thanks!
> > > > > > > > > 
> > > > > > > > > [1] https://live.gnome.org/JGIR How mature/maintained is JGIR?
> > > > > > > > > Last commit seems to be 2010-02-09.  The author is: Alexander
> > > > > > > > > Kurtakov <akurtako at redhat.com> His status in our
> > > > > > > > > organizational chart: Employee Type: Ex-employee
> > > > > > > > It will need some work to get it up to par with the rest of the
> > > > > > > > gobject generators
> > > > > > > 
> > > > > > > It's been dead since 2009, that doesn't seem very promising
> > > > > > > (http://www.ohloh.net/p/java-gobject-introspection)
> > > > > > > 
> > > > > > > It also states: "It also includes a custom GLib/GObject interface
> > > > > > > layer adapted from gstreamer-java." and looking at gstreamer-java
> > > > > > > yields: "An unofficial/alternative set of java bindings for the
> > > > > > > gstreamer multimedia framework"
> > > > > > > 
> > > > > > > Again, doesn't look like something you want to base your API on.
> > > > > > > Sounds to me like the 'free' java bindings come at a very high
> > > > > > > cost (bringing JGIR up to date and maintaining it).
> > > > > > > 
> > > > > > > What are the alternatives?
> > > > > > 
> > > > > > There are two other alternatives (and one bad idea) that come to
> > > > > > mind:
> > > > > > 
> > > > > > 1.) Generate our own Java bindings to libvdsm.so during the vdsm
> > > > > > build process.
> > > > > > 
> > > > > > I prefer this option from an API cleanliness POV because the
> > > > > > transport code would only be written once (in libvdsm.so).  This
> > > > > > form of generation should be simple because we are just wrapping the
> > > > > > library so it can be called via JNI.
> > > > > > 
> > > > > > 
> > > > > > 2.) Generate a native Java "library" that is equivalent to
> > > > > > libvdsm.so.
> > > > > > 
> > > > > > This gets us into the business of libvirt-style bindings generation
> > > > > > which I think is a mistake.  It opens the door to us maintaining
> > > > > > parallel implementations of "libvdsm"s (one per language).
> > > > > > 
> > > > > > 
> > > > > > -1.) Standardize the transport around JSON-RPC and make that the
> > > > > > supported interface.
> > > > > > 
> > > > > > I am only mentioning it because I am certain someone will bring it
> > > > > > up.  I think it's a bad idea and it's off-topic for this particular
> > > > > > thread anyway.
> > > > > > 
> > > > > > -- Adam Litke <agl at us.ibm.com> IBM Linux Technology Center
> > > > > > 
> > > > > > 
> > > > > 
> > > > 
> > > > -- Adam Litke <agl at us.ibm.com> IBM Linux Technology Center
> > > > 
> > > > 
> > > 
> > 
> > -- Adam Litke <agl at us.ibm.com> IBM Linux Technology Center
> > 
> > 
> 

-- 
Adam Litke <agl at us.ibm.com>
IBM Linux Technology Center