[Engine-devel] libosinfo integration

[Alon Bar-Lev]

----- Original Message -----
> From: "Juan Hernandez" <jhernand at redhat.com>
> To: "Alon Bar-Lev" <alonbl at redhat.com>
> Cc: "arch" <arch at ovirt.org>, engine-devel at ovirt.org, "Daniel P. Berrange" <berrange at redhat.com>
> Sent: Monday, September 24, 2012 9:33:38 PM
> Subject: Re: [Engine-devel] libosinfo integration
> 
> On 09/24/2012 08:59 PM, Alon Bar-Lev wrote:
> > 
> > 
> > ----- Original Message -----
> >> From: "Juan Hernandez" <jhernand at redhat.com>
> >> To: "Daniel P. Berrange" <berrange at redhat.com>
> >> Cc: "arch" <arch at ovirt.org>, engine-devel at ovirt.org
> >> Sent: Monday, September 24, 2012 6:30:43 PM
> >> Subject: Re: [Engine-devel] libosinfo integration
> >>
> >> On 09/24/2012 05:24 PM, Daniel P. Berrange wrote:
> >>> On Mon, Sep 24, 2012 at 04:51:37PM +0200, Roy Golan wrote:
> >>>> Here's a draft wiki of the libosinfo integration with the engine
> >>>> http://wiki.ovirt.org/wiki/Libosinfo
> >>>>
> >>>> please share your comments on this thread and I'll update the
> >>>> wiki
> >>>> accordingly.
> >>>
> >>> As a libosinfo maintainer I strongly discourage you from writing
> >>> a
> >>> custom
> >>> API directly against the XML. While the format is stable from the
> >>> POV
> >>> that vendors/users can create new XML docs for new/customized OS
> >>> variants
> >>> they have, we do not consider it something for applications to be
> >>> using
> >>> directly. The libosinfo library API includes alot of logic beyond
> >>> simply
> >>> being an convenient way to process the XML. In particular dealing
> >>> with
> >>> the inheritance of data across OS, searching for data across a
> >>> variety
> >>> of data sources, relying on GIO for URI resolving, live update of
> >>> the
> >>> database, and more.
> >>>
> >>> We're also not really interested in maintaining manually written
> >>> bindings
> >>> for any languages. Rather than putting effort into such bindings
> >>> for Java,
> >>> time is better spent on finishing the GOBject introspection
> >>> mapping
> >>> layer
> >>> for Java. There was a proof of concept, which was never finished
> >>> /
> >>> updated
> >>> to the final introspection spec, which can be used as a basis for
> >>> ongoing
> >>> development.
> >>>
> >>>  https://live.gnome.org/JGIR
> >>>
> >>> This work will be applicable way beyond just libosinfo, to a wide
> >>> variety of useful APIs. Of possible interest to VDSM in the
> >>> future
> >>> will
> >>> be things like the new libvirt-designer & libvirt-builder APIs
> >>> which
> >>> are going to provide a bridge between libvirt + libosinfo to
> >>> simply
> >>> life for application developers constructing XML for provisioning
> >>> guests.
> >>
> >> I think that the use of JGIR has already been discussed as a way
> >> to
> >> interface the engine with libvdsm.so:
> >>
> >> http://lists.ovirt.org/pipermail/arch/2012-August/000760.html
> >>
> >> If I remember correctly the outcome of that discussion was that it
> >> shouldn't be used because it is considered dangerous to use native
> >> libraries from Java and because JGIR itself is very outdated. I
> >> don't
> >> 100% agree with the first part, but the second part is completely
> >> true.
> >>
> >> Anyhow, as it seems to be the right way to interface with
> >> libosinfo
> >> and
> >> future libvirt capabilities, we may want to rethink and put some
> >> effort
> >> to bring it up to date.
> > 
> > I don't think Java should go native to obtain data, using JNI or
> > whatever native interface that is offered. It can totally break
> > the JVM and cause leaks and stability issues JRE would like to
> > avoid.
> 
> This is were I disagree, only a bit. It is true that using JNI
> *without
> care* can totally break the JVM, generate leaks, and bring down all
> the
> application. But if the JNI layer is developed carefully this should
> not
> be more risky than any of the other multiple native libraries that
> you
> use every day from Java without noticing.
> In addition JGIR is based on JNA (which is well maintained and
> stable)
> and doesn't include any "dangerous" native C code.
>

Well, you can call me old fashioned...

But I don't think that any none critical task should be run out of process of the production component.

It is not GObject layer I am afraid of, it is what it eventually runs.

I believe that the data provided by libosinfo is not vital for our runtime, no problem in fetching it before start.

And again, I don't think data provider should be language binding... this concept alone is making me scared, as instead of using well defined data container people will start to use mesh of components... I don't see how you can stabilize an application this way.

For the native library that you use "every day", I don't know what you referring to, I've done lsof of engine and saw no exception... which other native library we use these days?
 
> > Having language binding for data access logic seems a bit strange
> > for me, as it should be easy to stabilize a schema either using
> > XML or a set of CSVs, for all to read and use.
> > 
> > I do understand why JNI is to be used when OS specific features are
> > to be used, for example unix domain sockets, shared memory,
> > security, but not for data access.
> > 
> > The alternatives would be:
> > 
> > 1. Run a python program at ovirt-engine start-up, use the python
> > API to fetch the data we need, and generate our own tiny XML of
> > the subset or even load it to the database. We can use the output
> > safely in Java. Disadvantage is that we need to stop/start service
> > or have a "refresh" verb for reload after libosinfo update.
> > 
> > 2. When data is requested, execute a python process with cmdline
> > arguments and get result from stdout of the process. The
> > disadvantage is the overhead of clone().
> > 
> > 3. Read the XML directly on runtime, disadvantage is if XML is
> > changed.
> > 
> > 4. Use the libosinfo python API during build and distribute an
> > update to the information in versions of ovirt, disadvantage is
> > slow refresh of data, not sure how often this data is updated.
> > 
> > 5. Provide a web service that has specific XML-RPC / RESTful API,
> > that can provide data out of the libosinfo, and publish it at
> > internet. Configure ovirt to use it, disadvantage is the need for
> > internet connection.
> > 
> > 6. Configure cgi-bin locally at apache where ovirt is running,
> > which provide XML-RPC / RESTful API, and access it locally.
> > 
> > In any case I would avoid exiting the JVM to interact with data
> > provider.
> > 
> > Regards,
> > Alon.
> > 
> 
> 
> --
> Dirección Comercial: C/Jose Bardasano Baos, 9, Edif. Gorbea 3, planta
> 3ºD, 28016 Madrid, Spain
> Inscrita en el Reg. Mercantil de Madrid – C.I.F. B82657941 - Red Hat
> S.L.
>