[Engine-devel] libosinfo integration

[Alon Bar-Lev]

----- Original Message -----
> From: "Roy Golan" <rgolan at redhat.com>
> To: engine-devel at ovirt.org
> Sent: Thursday, October 11, 2012 12:22:51 PM
> Subject: Re: [Engine-devel] libosinfo integration
> 
> After considering the pros and cons and making poc's I've decided to
> take the direct XML and java pure implementation.
> http://gerrit.ovirt.org/#/c/8472/
> http://gerrit.ovirt.org/#/c/8473/
> 
> the main motivation was is that the XML loading was fast and
> standard.
> it is very easy to expose a subset of the API while leaving behind
> the
> extra library magic which we don't currently need.
> Maybe at some point the inheritance feature will come in handy and
> the
> jgir implementation will be done so we could switch to it (I'm really
> not sure whats it there which is not complete)

Great!
This is the simplest solution indeed.
Me happy :)))

> 
> 
> On 09/24/2012 10:01 PM, Alon Bar-Lev wrote:
> >
> > ----- Original Message -----
> >> From: "Juan Hernandez" <jhernand at redhat.com>
> >> To: "Alon Bar-Lev" <alonbl at redhat.com>
> >> Cc: "arch" <arch at ovirt.org>, engine-devel at ovirt.org, "Daniel P.
> >> Berrange" <berrange at redhat.com>
> >> Sent: Monday, September 24, 2012 9:33:38 PM
> >> Subject: Re: [Engine-devel] libosinfo integration
> >>
> >> On 09/24/2012 08:59 PM, Alon Bar-Lev wrote:
> >>>
> >>> ----- Original Message -----
> >>>> From: "Juan Hernandez" <jhernand at redhat.com>
> >>>> To: "Daniel P. Berrange" <berrange at redhat.com>
> >>>> Cc: "arch" <arch at ovirt.org>, engine-devel at ovirt.org
> >>>> Sent: Monday, September 24, 2012 6:30:43 PM
> >>>> Subject: Re: [Engine-devel] libosinfo integration
> >>>>
> >>>> On 09/24/2012 05:24 PM, Daniel P. Berrange wrote:
> >>>>> On Mon, Sep 24, 2012 at 04:51:37PM +0200, Roy Golan wrote:
> >>>>>> Here's a draft wiki of the libosinfo integration with the
> >>>>>> engine
> >>>>>> http://wiki.ovirt.org/wiki/Libosinfo
> >>>>>>
> >>>>>> please share your comments on this thread and I'll update the
> >>>>>> wiki
> >>>>>> accordingly.
> >>>>> As a libosinfo maintainer I strongly discourage you from
> >>>>> writing
> >>>>> a
> >>>>> custom
> >>>>> API directly against the XML. While the format is stable from
> >>>>> the
> >>>>> POV
> >>>>> that vendors/users can create new XML docs for new/customized
> >>>>> OS
> >>>>> variants
> >>>>> they have, we do not consider it something for applications to
> >>>>> be
> >>>>> using
> >>>>> directly. The libosinfo library API includes alot of logic
> >>>>> beyond
> >>>>> simply
> >>>>> being an convenient way to process the XML. In particular
> >>>>> dealing
> >>>>> with
> >>>>> the inheritance of data across OS, searching for data across a
> >>>>> variety
> >>>>> of data sources, relying on GIO for URI resolving, live update
> >>>>> of
> >>>>> the
> >>>>> database, and more.
> >>>>>
> >>>>> We're also not really interested in maintaining manually
> >>>>> written
> >>>>> bindings
> >>>>> for any languages. Rather than putting effort into such
> >>>>> bindings
> >>>>> for Java,
> >>>>> time is better spent on finishing the GOBject introspection
> >>>>> mapping
> >>>>> layer
> >>>>> for Java. There was a proof of concept, which was never
> >>>>> finished
> >>>>> /
> >>>>> updated
> >>>>> to the final introspection spec, which can be used as a basis
> >>>>> for
> >>>>> ongoing
> >>>>> development.
> >>>>>
> >>>>>   https://live.gnome.org/JGIR
> >>>>>
> >>>>> This work will be applicable way beyond just libosinfo, to a
> >>>>> wide
> >>>>> variety of useful APIs. Of possible interest to VDSM in the
> >>>>> future
> >>>>> will
> >>>>> be things like the new libvirt-designer & libvirt-builder APIs
> >>>>> which
> >>>>> are going to provide a bridge between libvirt + libosinfo to
> >>>>> simply
> >>>>> life for application developers constructing XML for
> >>>>> provisioning
> >>>>> guests.
> >>>> I think that the use of JGIR has already been discussed as a way
> >>>> to
> >>>> interface the engine with libvdsm.so:
> >>>>
> >>>> http://lists.ovirt.org/pipermail/arch/2012-August/000760.html
> >>>>
> >>>> If I remember correctly the outcome of that discussion was that
> >>>> it
> >>>> shouldn't be used because it is considered dangerous to use
> >>>> native
> >>>> libraries from Java and because JGIR itself is very outdated. I
> >>>> don't
> >>>> 100% agree with the first part, but the second part is
> >>>> completely
> >>>> true.
> >>>>
> >>>> Anyhow, as it seems to be the right way to interface with
> >>>> libosinfo
> >>>> and
> >>>> future libvirt capabilities, we may want to rethink and put some
> >>>> effort
> >>>> to bring it up to date.
> >>> I don't think Java should go native to obtain data, using JNI or
> >>> whatever native interface that is offered. It can totally break
> >>> the JVM and cause leaks and stability issues JRE would like to
> >>> avoid.
> >> This is were I disagree, only a bit. It is true that using JNI
> >> *without
> >> care* can totally break the JVM, generate leaks, and bring down
> >> all
> >> the
> >> application. But if the JNI layer is developed carefully this
> >> should
> >> not
> >> be more risky than any of the other multiple native libraries that
> >> you
> >> use every day from Java without noticing.
> >> In addition JGIR is based on JNA (which is well maintained and
> >> stable)
> >> and doesn't include any "dangerous" native C code.
> >>
> > Well, you can call me old fashioned...
> >
> > But I don't think that any none critical task should be run out of
> > process of the production component.
> >
> > It is not GObject layer I am afraid of, it is what it eventually
> > runs.
> >
> > I believe that the data provided by libosinfo is not vital for our
> > runtime, no problem in fetching it before start.
> >
> > And again, I don't think data provider should be language
> > binding... this concept alone is making me scared, as instead of
> > using well defined data container people will start to use mesh of
> > components... I don't see how you can stabilize an application
> > this way.
> >
> > For the native library that you use "every day", I don't know what
> > you referring to, I've done lsof of engine and saw no exception...
> > which other native library we use these days?
> >   
> >>> Having language binding for data access logic seems a bit strange
> >>> for me, as it should be easy to stabilize a schema either using
> >>> XML or a set of CSVs, for all to read and use.
> >>>
> >>> I do understand why JNI is to be used when OS specific features
> >>> are
> >>> to be used, for example unix domain sockets, shared memory,
> >>> security, but not for data access.
> >>>
> >>> The alternatives would be:
> >>>
> >>> 1. Run a python program at ovirt-engine start-up, use the python
> >>> API to fetch the data we need, and generate our own tiny XML of
> >>> the subset or even load it to the database. We can use the output
> >>> safely in Java. Disadvantage is that we need to stop/start
> >>> service
> >>> or have a "refresh" verb for reload after libosinfo update.
> >>>
> >>> 2. When data is requested, execute a python process with cmdline
> >>> arguments and get result from stdout of the process. The
> >>> disadvantage is the overhead of clone().
> >>>
> >>> 3. Read the XML directly on runtime, disadvantage is if XML is
> >>> changed.
> >>>
> >>> 4. Use the libosinfo python API during build and distribute an
> >>> update to the information in versions of ovirt, disadvantage is
> >>> slow refresh of data, not sure how often this data is updated.
> >>>
> >>> 5. Provide a web service that has specific XML-RPC / RESTful API,
> >>> that can provide data out of the libosinfo, and publish it at
> >>> internet. Configure ovirt to use it, disadvantage is the need for
> >>> internet connection.
> >>>
> >>> 6. Configure cgi-bin locally at apache where ovirt is running,
> >>> which provide XML-RPC / RESTful API, and access it locally.
> >>>
> >>> In any case I would avoid exiting the JVM to interact with data
> >>> provider.
> >>>
> >>> Regards,
> >>> Alon.
> >>>
> >>
> >> --
> >> Dirección Comercial: C/Jose Bardasano Baos, 9, Edif. Gorbea 3,
> >> planta
> >> 3ºD, 28016 Madrid, Spain
> >> Inscrita en el Reg. Mercantil de Madrid – C.I.F. B82657941 - Red
> >> Hat
> >> S.L.
> >>
> > _______________________________________________
> > Engine-devel mailing list
> > Engine-devel at ovirt.org
> > http://lists.ovirt.org/mailman/listinfo/engine-devel
> 
> _______________________________________________
> Engine-devel mailing list
> Engine-devel at ovirt.org
> http://lists.ovirt.org/mailman/listinfo/engine-devel
>