[Engine-devel] libosinfo integration

Thu Oct 11 10:22:51 UTC 2012

After considering the pros and cons and making poc's I've decided to 
take the direct XML and java pure implementation.
http://gerrit.ovirt.org/#/c/8472/
http://gerrit.ovirt.org/#/c/8473/

the main motivation was is that the XML loading was fast and standard. 
it is very easy to expose a subset of the API while leaving behind the 
extra library magic which we don't currently need.
Maybe at some point the inheritance feature will come in handy and the 
jgir implementation will be done so we could switch to it (I'm really 
not sure whats it there which is not complete)

On 09/24/2012 10:01 PM, Alon Bar-Lev wrote:
>
> ----- Original Message -----
>> From: "Juan Hernandez" <jhernand at redhat.com>
>> To: "Alon Bar-Lev" <alonbl at redhat.com>
>> Cc: "arch" <arch at ovirt.org>, engine-devel at ovirt.org, "Daniel P. Berrange" <berrange at redhat.com>
>> Sent: Monday, September 24, 2012 9:33:38 PM
>> Subject: Re: [Engine-devel] libosinfo integration
>>
>> On 09/24/2012 08:59 PM, Alon Bar-Lev wrote:
>>>
>>> ----- Original Message -----
>>>> From: "Juan Hernandez" <jhernand at redhat.com>
>>>> To: "Daniel P. Berrange" <berrange at redhat.com>
>>>> Cc: "arch" <arch at ovirt.org>, engine-devel at ovirt.org
>>>> Sent: Monday, September 24, 2012 6:30:43 PM
>>>> Subject: Re: [Engine-devel] libosinfo integration
>>>>
>>>> On 09/24/2012 05:24 PM, Daniel P. Berrange wrote:
>>>>> On Mon, Sep 24, 2012 at 04:51:37PM +0200, Roy Golan wrote:
>>>>>> Here's a draft wiki of the libosinfo integration with the engine
>>>>>> http://wiki.ovirt.org/wiki/Libosinfo
>>>>>>
>>>>>> please share your comments on this thread and I'll update the
>>>>>> wiki
>>>>>> accordingly.
>>>>> As a libosinfo maintainer I strongly discourage you from writing
>>>>> a
>>>>> custom
>>>>> API directly against the XML. While the format is stable from the
>>>>> POV
>>>>> that vendors/users can create new XML docs for new/customized OS
>>>>> variants
>>>>> they have, we do not consider it something for applications to be
>>>>> using
>>>>> directly. The libosinfo library API includes alot of logic beyond
>>>>> simply
>>>>> being an convenient way to process the XML. In particular dealing
>>>>> with
>>>>> the inheritance of data across OS, searching for data across a
>>>>> variety
>>>>> of data sources, relying on GIO for URI resolving, live update of
>>>>> the
>>>>> database, and more.
>>>>>
>>>>> We're also not really interested in maintaining manually written
>>>>> bindings
>>>>> for any languages. Rather than putting effort into such bindings
>>>>> for Java,
>>>>> time is better spent on finishing the GOBject introspection
>>>>> mapping
>>>>> layer
>>>>> for Java. There was a proof of concept, which was never finished
>>>>> /
>>>>> updated
>>>>> to the final introspection spec, which can be used as a basis for
>>>>> ongoing
>>>>> development.
>>>>>
>>>>>   https://live.gnome.org/JGIR
>>>>>
>>>>> This work will be applicable way beyond just libosinfo, to a wide
>>>>> variety of useful APIs. Of possible interest to VDSM in the
>>>>> future
>>>>> will
>>>>> be things like the new libvirt-designer & libvirt-builder APIs
>>>>> which
>>>>> are going to provide a bridge between libvirt + libosinfo to
>>>>> simply
>>>>> life for application developers constructing XML for provisioning
>>>>> guests.
>>>> I think that the use of JGIR has already been discussed as a way
>>>> to
>>>> interface the engine with libvdsm.so:
>>>>
>>>> http://lists.ovirt.org/pipermail/arch/2012-August/000760.html
>>>>
>>>> If I remember correctly the outcome of that discussion was that it
>>>> shouldn't be used because it is considered dangerous to use native
>>>> libraries from Java and because JGIR itself is very outdated. I
>>>> don't
>>>> 100% agree with the first part, but the second part is completely
>>>> true.
>>>>
>>>> Anyhow, as it seems to be the right way to interface with
>>>> libosinfo
>>>> and
>>>> future libvirt capabilities, we may want to rethink and put some
>>>> effort
>>>> to bring it up to date.
>>> I don't think Java should go native to obtain data, using JNI or
>>> whatever native interface that is offered. It can totally break
>>> the JVM and cause leaks and stability issues JRE would like to
>>> avoid.
>> This is were I disagree, only a bit. It is true that using JNI
>> *without
>> care* can totally break the JVM, generate leaks, and bring down all
>> the
>> application. But if the JNI layer is developed carefully this should
>> not
>> be more risky than any of the other multiple native libraries that
>> you
>> use every day from Java without noticing.
>> In addition JGIR is based on JNA (which is well maintained and
>> stable)
>> and doesn't include any "dangerous" native C code.
>>
> Well, you can call me old fashioned...
>
> But I don't think that any none critical task should be run out of process of the production component.
>
> It is not GObject layer I am afraid of, it is what it eventually runs.
>
> I believe that the data provided by libosinfo is not vital for our runtime, no problem in fetching it before start.
>
> And again, I don't think data provider should be language binding... this concept alone is making me scared, as instead of using well defined data container people will start to use mesh of components... I don't see how you can stabilize an application this way.
>
> For the native library that you use "every day", I don't know what you referring to, I've done lsof of engine and saw no exception... which other native library we use these days?
>   
>>> Having language binding for data access logic seems a bit strange
>>> for me, as it should be easy to stabilize a schema either using
>>> XML or a set of CSVs, for all to read and use.
>>>
>>> I do understand why JNI is to be used when OS specific features are
>>> to be used, for example unix domain sockets, shared memory,
>>> security, but not for data access.
>>>
>>> The alternatives would be:
>>>
>>> 1. Run a python program at ovirt-engine start-up, use the python
>>> API to fetch the data we need, and generate our own tiny XML of
>>> the subset or even load it to the database. We can use the output
>>> safely in Java. Disadvantage is that we need to stop/start service
>>> or have a "refresh" verb for reload after libosinfo update.
>>>
>>> 2. When data is requested, execute a python process with cmdline
>>> arguments and get result from stdout of the process. The
>>> disadvantage is the overhead of clone().
>>>
>>> 3. Read the XML directly on runtime, disadvantage is if XML is
>>> changed.
>>>
>>> 4. Use the libosinfo python API during build and distribute an
>>> update to the information in versions of ovirt, disadvantage is
>>> slow refresh of data, not sure how often this data is updated.
>>>
>>> 5. Provide a web service that has specific XML-RPC / RESTful API,
>>> that can provide data out of the libosinfo, and publish it at
>>> internet. Configure ovirt to use it, disadvantage is the need for
>>> internet connection.
>>>
>>> 6. Configure cgi-bin locally at apache where ovirt is running,
>>> which provide XML-RPC / RESTful API, and access it locally.
>>>
>>> In any case I would avoid exiting the JVM to interact with data
>>> provider.
>>>
>>> Regards,
>>> Alon.
>>>
>>
>> --
>> Dirección Comercial: C/Jose Bardasano Baos, 9, Edif. Gorbea 3, planta
>> 3ºD, 28016 Madrid, Spain
>> Inscrita en el Reg. Mercantil de Madrid – C.I.F. B82657941 - Red Hat
>> S.L.
>>
> _______________________________________________
> Engine-devel mailing list
> Engine-devel at ovirt.org
> http://lists.ovirt.org/mailman/listinfo/engine-devel