[Engine-devel] libosinfo integration
Here's a draft wiki of the libosinfo integration with the engine http://wiki.ovirt.org/wiki/Libosinfo please share your comments on this thread and I'll update the wiki accordingly. Thanks, Roy
Roy Golan píše v Po 24. 09. 2012 v 16:51 +0200:
Here's a draft wiki of the libosinfo integration with the engine http://wiki.ovirt.org/wiki/Libosinfo
please share your comments on this thread and I'll update the wiki accordingly.
Thanks, Roy
If you use java bindings created by GObject Introspection, will the program leave JVM when invoking them or not? If not, that would simplify your choice by great deal. David -- David Jaša, RHCE SPICE QE based in Brno GPG Key: 22C33E24 Fingerprint: 513A 060B D1B4 2A72 7F0D 0278 B125 CD00 22C3 3E24
On Mon, Sep 24, 2012 at 04:51:37PM +0200, Roy Golan wrote:
Here's a draft wiki of the libosinfo integration with the engine http://wiki.ovirt.org/wiki/Libosinfo
please share your comments on this thread and I'll update the wiki accordingly.
As a libosinfo maintainer I strongly discourage you from writing a custom API directly against the XML. While the format is stable from the POV that vendors/users can create new XML docs for new/customized OS variants they have, we do not consider it something for applications to be using directly. The libosinfo library API includes alot of logic beyond simply being an convenient way to process the XML. In particular dealing with the inheritance of data across OS, searching for data across a variety of data sources, relying on GIO for URI resolving, live update of the database, and more. We're also not really interested in maintaining manually written bindings for any languages. Rather than putting effort into such bindings for Java, time is better spent on finishing the GOBject introspection mapping layer for Java. There was a proof of concept, which was never finished / updated to the final introspection spec, which can be used as a basis for ongoing development. https://live.gnome.org/JGIR This work will be applicable way beyond just libosinfo, to a wide variety of useful APIs. Of possible interest to VDSM in the future will be things like the new libvirt-designer & libvirt-builder APIs which are going to provide a bridge between libvirt + libosinfo to simply life for application developers constructing XML for provisioning guests. Daniel -- |: http://berrange.com -o- http://www.flickr.com/photos/dberrange/ :| |: http://libvirt.org -o- http://virt-manager.org :| |: http://autobuild.org -o- http://search.cpan.org/~danberr/ :| |: http://entangle-photo.org -o- http://live.gnome.org/gtk-vnc :|
On 09/24/2012 05:24 PM, Daniel P. Berrange wrote:
On Mon, Sep 24, 2012 at 04:51:37PM +0200, Roy Golan wrote:
Here's a draft wiki of the libosinfo integration with the engine http://wiki.ovirt.org/wiki/Libosinfo
please share your comments on this thread and I'll update the wiki accordingly.
As a libosinfo maintainer I strongly discourage you from writing a custom API directly against the XML. While the format is stable from the POV that vendors/users can create new XML docs for new/customized OS variants they have, we do not consider it something for applications to be using directly. The libosinfo library API includes alot of logic beyond simply being an convenient way to process the XML. In particular dealing with the inheritance of data across OS, searching for data across a variety of data sources, relying on GIO for URI resolving, live update of the database, and more.
We're also not really interested in maintaining manually written bindings for any languages. Rather than putting effort into such bindings for Java, time is better spent on finishing the GOBject introspection mapping layer for Java. There was a proof of concept, which was never finished / updated to the final introspection spec, which can be used as a basis for ongoing development.
This work will be applicable way beyond just libosinfo, to a wide variety of useful APIs. Of possible interest to VDSM in the future will be things like the new libvirt-designer & libvirt-builder APIs which are going to provide a bridge between libvirt + libosinfo to simply life for application developers constructing XML for provisioning guests.
I think that the use of JGIR has already been discussed as a way to interface the engine with libvdsm.so: http://lists.ovirt.org/pipermail/arch/2012-August/000760.html If I remember correctly the outcome of that discussion was that it shouldn't be used because it is considered dangerous to use native libraries from Java and because JGIR itself is very outdated. I don't 100% agree with the first part, but the second part is completely true. Anyhow, as it seems to be the right way to interface with libosinfo and future libvirt capabilities, we may want to rethink and put some effort to bring it up to date. -- Dirección Comercial: C/Jose Bardasano Baos, 9, Edif. Gorbea 3, planta 3ºD, 28016 Madrid, Spain Inscrita en el Reg. Mercantil de Madrid – C.I.F. B82657941 - Red Hat S.L.
----- Original Message -----
From: "Juan Hernandez" <jhernand@redhat.com> To: "Daniel P. Berrange" <berrange@redhat.com> Cc: "arch" <arch@ovirt.org>, engine-devel@ovirt.org Sent: Monday, September 24, 2012 6:30:43 PM Subject: Re: [Engine-devel] libosinfo integration
On 09/24/2012 05:24 PM, Daniel P. Berrange wrote:
On Mon, Sep 24, 2012 at 04:51:37PM +0200, Roy Golan wrote:
Here's a draft wiki of the libosinfo integration with the engine http://wiki.ovirt.org/wiki/Libosinfo
please share your comments on this thread and I'll update the wiki accordingly.
As a libosinfo maintainer I strongly discourage you from writing a custom API directly against the XML. While the format is stable from the POV that vendors/users can create new XML docs for new/customized OS variants they have, we do not consider it something for applications to be using directly. The libosinfo library API includes alot of logic beyond simply being an convenient way to process the XML. In particular dealing with the inheritance of data across OS, searching for data across a variety of data sources, relying on GIO for URI resolving, live update of the database, and more.
We're also not really interested in maintaining manually written bindings for any languages. Rather than putting effort into such bindings for Java, time is better spent on finishing the GOBject introspection mapping layer for Java. There was a proof of concept, which was never finished / updated to the final introspection spec, which can be used as a basis for ongoing development.
This work will be applicable way beyond just libosinfo, to a wide variety of useful APIs. Of possible interest to VDSM in the future will be things like the new libvirt-designer & libvirt-builder APIs which are going to provide a bridge between libvirt + libosinfo to simply life for application developers constructing XML for provisioning guests.
I think that the use of JGIR has already been discussed as a way to interface the engine with libvdsm.so:
http://lists.ovirt.org/pipermail/arch/2012-August/000760.html
If I remember correctly the outcome of that discussion was that it shouldn't be used because it is considered dangerous to use native libraries from Java and because JGIR itself is very outdated. I don't 100% agree with the first part, but the second part is completely true.
Anyhow, as it seems to be the right way to interface with libosinfo and future libvirt capabilities, we may want to rethink and put some effort to bring it up to date.
I don't think Java should go native to obtain data, using JNI or whatever native interface that is offered. It can totally break the JVM and cause leaks and stability issues JRE would like to avoid. Having language binding for data access logic seems a bit strange for me, as it should be easy to stabilize a schema either using XML or a set of CSVs, for all to read and use. I do understand why JNI is to be used when OS specific features are to be used, for example unix domain sockets, shared memory, security, but not for data access. The alternatives would be: 1. Run a python program at ovirt-engine start-up, use the python API to fetch the data we need, and generate our own tiny XML of the subset or even load it to the database. We can use the output safely in Java. Disadvantage is that we need to stop/start service or have a "refresh" verb for reload after libosinfo update. 2. When data is requested, execute a python process with cmdline arguments and get result from stdout of the process. The disadvantage is the overhead of clone(). 3. Read the XML directly on runtime, disadvantage is if XML is changed. 4. Use the libosinfo python API during build and distribute an update to the information in versions of ovirt, disadvantage is slow refresh of data, not sure how often this data is updated. 5. Provide a web service that has specific XML-RPC / RESTful API, that can provide data out of the libosinfo, and publish it at internet. Configure ovirt to use it, disadvantage is the need for internet connection. 6. Configure cgi-bin locally at apache where ovirt is running, which provide XML-RPC / RESTful API, and access it locally. In any case I would avoid exiting the JVM to interact with data provider. Regards, Alon.
On 09/24/2012 08:59 PM, Alon Bar-Lev wrote:
----- Original Message -----
From: "Juan Hernandez" <jhernand@redhat.com> To: "Daniel P. Berrange" <berrange@redhat.com> Cc: "arch" <arch@ovirt.org>, engine-devel@ovirt.org Sent: Monday, September 24, 2012 6:30:43 PM Subject: Re: [Engine-devel] libosinfo integration
On 09/24/2012 05:24 PM, Daniel P. Berrange wrote:
On Mon, Sep 24, 2012 at 04:51:37PM +0200, Roy Golan wrote:
Here's a draft wiki of the libosinfo integration with the engine http://wiki.ovirt.org/wiki/Libosinfo
please share your comments on this thread and I'll update the wiki accordingly.
As a libosinfo maintainer I strongly discourage you from writing a custom API directly against the XML. While the format is stable from the POV that vendors/users can create new XML docs for new/customized OS variants they have, we do not consider it something for applications to be using directly. The libosinfo library API includes alot of logic beyond simply being an convenient way to process the XML. In particular dealing with the inheritance of data across OS, searching for data across a variety of data sources, relying on GIO for URI resolving, live update of the database, and more.
We're also not really interested in maintaining manually written bindings for any languages. Rather than putting effort into such bindings for Java, time is better spent on finishing the GOBject introspection mapping layer for Java. There was a proof of concept, which was never finished / updated to the final introspection spec, which can be used as a basis for ongoing development.
This work will be applicable way beyond just libosinfo, to a wide variety of useful APIs. Of possible interest to VDSM in the future will be things like the new libvirt-designer & libvirt-builder APIs which are going to provide a bridge between libvirt + libosinfo to simply life for application developers constructing XML for provisioning guests.
I think that the use of JGIR has already been discussed as a way to interface the engine with libvdsm.so:
http://lists.ovirt.org/pipermail/arch/2012-August/000760.html
If I remember correctly the outcome of that discussion was that it shouldn't be used because it is considered dangerous to use native libraries from Java and because JGIR itself is very outdated. I don't 100% agree with the first part, but the second part is completely true.
Anyhow, as it seems to be the right way to interface with libosinfo and future libvirt capabilities, we may want to rethink and put some effort to bring it up to date.
I don't think Java should go native to obtain data, using JNI or whatever native interface that is offered. It can totally break the JVM and cause leaks and stability issues JRE would like to avoid.
This is were I disagree, only a bit. It is true that using JNI *without care* can totally break the JVM, generate leaks, and bring down all the application. But if the JNI layer is developed carefully this should not be more risky than any of the other multiple native libraries that you use every day from Java without noticing. In addition JGIR is based on JNA (which is well maintained and stable) and doesn't include any "dangerous" native C code.
Having language binding for data access logic seems a bit strange for me, as it should be easy to stabilize a schema either using XML or a set of CSVs, for all to read and use.
I do understand why JNI is to be used when OS specific features are to be used, for example unix domain sockets, shared memory, security, but not for data access.
The alternatives would be:
1. Run a python program at ovirt-engine start-up, use the python API to fetch the data we need, and generate our own tiny XML of the subset or even load it to the database. We can use the output safely in Java. Disadvantage is that we need to stop/start service or have a "refresh" verb for reload after libosinfo update.
2. When data is requested, execute a python process with cmdline arguments and get result from stdout of the process. The disadvantage is the overhead of clone().
3. Read the XML directly on runtime, disadvantage is if XML is changed.
4. Use the libosinfo python API during build and distribute an update to the information in versions of ovirt, disadvantage is slow refresh of data, not sure how often this data is updated.
5. Provide a web service that has specific XML-RPC / RESTful API, that can provide data out of the libosinfo, and publish it at internet. Configure ovirt to use it, disadvantage is the need for internet connection.
6. Configure cgi-bin locally at apache where ovirt is running, which provide XML-RPC / RESTful API, and access it locally.
In any case I would avoid exiting the JVM to interact with data provider.
Regards, Alon.
-- Dirección Comercial: C/Jose Bardasano Baos, 9, Edif. Gorbea 3, planta 3ºD, 28016 Madrid, Spain Inscrita en el Reg. Mercantil de Madrid – C.I.F. B82657941 - Red Hat S.L.
----- Original Message -----
From: "Juan Hernandez" <jhernand@redhat.com> To: "Alon Bar-Lev" <alonbl@redhat.com> Cc: "arch" <arch@ovirt.org>, engine-devel@ovirt.org, "Daniel P. Berrange" <berrange@redhat.com> Sent: Monday, September 24, 2012 9:33:38 PM Subject: Re: [Engine-devel] libosinfo integration
On 09/24/2012 08:59 PM, Alon Bar-Lev wrote:
----- Original Message -----
From: "Juan Hernandez" <jhernand@redhat.com> To: "Daniel P. Berrange" <berrange@redhat.com> Cc: "arch" <arch@ovirt.org>, engine-devel@ovirt.org Sent: Monday, September 24, 2012 6:30:43 PM Subject: Re: [Engine-devel] libosinfo integration
On 09/24/2012 05:24 PM, Daniel P. Berrange wrote:
On Mon, Sep 24, 2012 at 04:51:37PM +0200, Roy Golan wrote:
Here's a draft wiki of the libosinfo integration with the engine http://wiki.ovirt.org/wiki/Libosinfo
please share your comments on this thread and I'll update the wiki accordingly.
As a libosinfo maintainer I strongly discourage you from writing a custom API directly against the XML. While the format is stable from the POV that vendors/users can create new XML docs for new/customized OS variants they have, we do not consider it something for applications to be using directly. The libosinfo library API includes alot of logic beyond simply being an convenient way to process the XML. In particular dealing with the inheritance of data across OS, searching for data across a variety of data sources, relying on GIO for URI resolving, live update of the database, and more.
We're also not really interested in maintaining manually written bindings for any languages. Rather than putting effort into such bindings for Java, time is better spent on finishing the GOBject introspection mapping layer for Java. There was a proof of concept, which was never finished / updated to the final introspection spec, which can be used as a basis for ongoing development.
This work will be applicable way beyond just libosinfo, to a wide variety of useful APIs. Of possible interest to VDSM in the future will be things like the new libvirt-designer & libvirt-builder APIs which are going to provide a bridge between libvirt + libosinfo to simply life for application developers constructing XML for provisioning guests.
I think that the use of JGIR has already been discussed as a way to interface the engine with libvdsm.so:
http://lists.ovirt.org/pipermail/arch/2012-August/000760.html
If I remember correctly the outcome of that discussion was that it shouldn't be used because it is considered dangerous to use native libraries from Java and because JGIR itself is very outdated. I don't 100% agree with the first part, but the second part is completely true.
Anyhow, as it seems to be the right way to interface with libosinfo and future libvirt capabilities, we may want to rethink and put some effort to bring it up to date.
I don't think Java should go native to obtain data, using JNI or whatever native interface that is offered. It can totally break the JVM and cause leaks and stability issues JRE would like to avoid.
This is were I disagree, only a bit. It is true that using JNI *without care* can totally break the JVM, generate leaks, and bring down all the application. But if the JNI layer is developed carefully this should not be more risky than any of the other multiple native libraries that you use every day from Java without noticing. In addition JGIR is based on JNA (which is well maintained and stable) and doesn't include any "dangerous" native C code.
Well, you can call me old fashioned... But I don't think that any none critical task should be run out of process of the production component. It is not GObject layer I am afraid of, it is what it eventually runs. I believe that the data provided by libosinfo is not vital for our runtime, no problem in fetching it before start. And again, I don't think data provider should be language binding... this concept alone is making me scared, as instead of using well defined data container people will start to use mesh of components... I don't see how you can stabilize an application this way. For the native library that you use "every day", I don't know what you referring to, I've done lsof of engine and saw no exception... which other native library we use these days?
Having language binding for data access logic seems a bit strange for me, as it should be easy to stabilize a schema either using XML or a set of CSVs, for all to read and use.
I do understand why JNI is to be used when OS specific features are to be used, for example unix domain sockets, shared memory, security, but not for data access.
The alternatives would be:
1. Run a python program at ovirt-engine start-up, use the python API to fetch the data we need, and generate our own tiny XML of the subset or even load it to the database. We can use the output safely in Java. Disadvantage is that we need to stop/start service or have a "refresh" verb for reload after libosinfo update.
2. When data is requested, execute a python process with cmdline arguments and get result from stdout of the process. The disadvantage is the overhead of clone().
3. Read the XML directly on runtime, disadvantage is if XML is changed.
4. Use the libosinfo python API during build and distribute an update to the information in versions of ovirt, disadvantage is slow refresh of data, not sure how often this data is updated.
5. Provide a web service that has specific XML-RPC / RESTful API, that can provide data out of the libosinfo, and publish it at internet. Configure ovirt to use it, disadvantage is the need for internet connection.
6. Configure cgi-bin locally at apache where ovirt is running, which provide XML-RPC / RESTful API, and access it locally.
In any case I would avoid exiting the JVM to interact with data provider.
Regards, Alon.
-- Dirección Comercial: C/Jose Bardasano Baos, 9, Edif. Gorbea 3, planta 3ºD, 28016 Madrid, Spain Inscrita en el Reg. Mercantil de Madrid – C.I.F. B82657941 - Red Hat S.L.
After considering the pros and cons and making poc's I've decided to take the direct XML and java pure implementation. http://gerrit.ovirt.org/#/c/8472/ http://gerrit.ovirt.org/#/c/8473/ the main motivation was is that the XML loading was fast and standard. it is very easy to expose a subset of the API while leaving behind the extra library magic which we don't currently need. Maybe at some point the inheritance feature will come in handy and the jgir implementation will be done so we could switch to it (I'm really not sure whats it there which is not complete) On 09/24/2012 10:01 PM, Alon Bar-Lev wrote:
----- Original Message -----
From: "Juan Hernandez" <jhernand@redhat.com> To: "Alon Bar-Lev" <alonbl@redhat.com> Cc: "arch" <arch@ovirt.org>, engine-devel@ovirt.org, "Daniel P. Berrange" <berrange@redhat.com> Sent: Monday, September 24, 2012 9:33:38 PM Subject: Re: [Engine-devel] libosinfo integration
On 09/24/2012 08:59 PM, Alon Bar-Lev wrote:
From: "Juan Hernandez" <jhernand@redhat.com> To: "Daniel P. Berrange" <berrange@redhat.com> Cc: "arch" <arch@ovirt.org>, engine-devel@ovirt.org Sent: Monday, September 24, 2012 6:30:43 PM Subject: Re: [Engine-devel] libosinfo integration
On 09/24/2012 05:24 PM, Daniel P. Berrange wrote:
Here's a draft wiki of the libosinfo integration with the engine http://wiki.ovirt.org/wiki/Libosinfo
please share your comments on this thread and I'll update the wiki accordingly. As a libosinfo maintainer I strongly discourage you from writing a custom API directly against the XML. While the format is stable from the POV
On Mon, Sep 24, 2012 at 04:51:37PM +0200, Roy Golan wrote: that vendors/users can create new XML docs for new/customized OS variants they have, we do not consider it something for applications to be using directly. The libosinfo library API includes alot of logic beyond simply being an convenient way to process the XML. In particular dealing with the inheritance of data across OS, searching for data across a variety of data sources, relying on GIO for URI resolving, live update of the database, and more.
We're also not really interested in maintaining manually written bindings for any languages. Rather than putting effort into such bindings for Java, time is better spent on finishing the GOBject introspection mapping layer for Java. There was a proof of concept, which was never finished / updated to the final introspection spec, which can be used as a basis for ongoing development.
This work will be applicable way beyond just libosinfo, to a wide variety of useful APIs. Of possible interest to VDSM in the future will be things like the new libvirt-designer & libvirt-builder APIs which are going to provide a bridge between libvirt + libosinfo to simply life for application developers constructing XML for provisioning guests. I think that the use of JGIR has already been discussed as a way to interface the engine with libvdsm.so:
http://lists.ovirt.org/pipermail/arch/2012-August/000760.html
If I remember correctly the outcome of that discussion was that it shouldn't be used because it is considered dangerous to use native libraries from Java and because JGIR itself is very outdated. I don't 100% agree with the first part, but the second part is completely true.
Anyhow, as it seems to be the right way to interface with libosinfo and future libvirt capabilities, we may want to rethink and put some effort to bring it up to date. I don't think Java should go native to obtain data, using JNI or whatever native interface that is offered. It can totally break
----- Original Message ----- the JVM and cause leaks and stability issues JRE would like to avoid.
This is were I disagree, only a bit. It is true that using JNI *without care* can totally break the JVM, generate leaks, and bring down all the application. But if the JNI layer is developed carefully this should not be more risky than any of the other multiple native libraries that you use every day from Java without noticing. In addition JGIR is based on JNA (which is well maintained and stable) and doesn't include any "dangerous" native C code.
Well, you can call me old fashioned...
But I don't think that any none critical task should be run out of process of the production component.
It is not GObject layer I am afraid of, it is what it eventually runs.
I believe that the data provided by libosinfo is not vital for our runtime, no problem in fetching it before start.
And again, I don't think data provider should be language binding... this concept alone is making me scared, as instead of using well defined data container people will start to use mesh of components... I don't see how you can stabilize an application this way.
For the native library that you use "every day", I don't know what you referring to, I've done lsof of engine and saw no exception... which other native library we use these days?
Having language binding for data access logic seems a bit strange for me, as it should be easy to stabilize a schema either using XML or a set of CSVs, for all to read and use.
I do understand why JNI is to be used when OS specific features are to be used, for example unix domain sockets, shared memory, security, but not for data access.
The alternatives would be:
1. Run a python program at ovirt-engine start-up, use the python API to fetch the data we need, and generate our own tiny XML of the subset or even load it to the database. We can use the output safely in Java. Disadvantage is that we need to stop/start service or have a "refresh" verb for reload after libosinfo update.
2. When data is requested, execute a python process with cmdline arguments and get result from stdout of the process. The disadvantage is the overhead of clone().
3. Read the XML directly on runtime, disadvantage is if XML is changed.
4. Use the libosinfo python API during build and distribute an update to the information in versions of ovirt, disadvantage is slow refresh of data, not sure how often this data is updated.
5. Provide a web service that has specific XML-RPC / RESTful API, that can provide data out of the libosinfo, and publish it at internet. Configure ovirt to use it, disadvantage is the need for internet connection.
6. Configure cgi-bin locally at apache where ovirt is running, which provide XML-RPC / RESTful API, and access it locally.
In any case I would avoid exiting the JVM to interact with data provider.
Regards, Alon.
-- Dirección Comercial: C/Jose Bardasano Baos, 9, Edif. Gorbea 3, planta 3ºD, 28016 Madrid, Spain Inscrita en el Reg. Mercantil de Madrid – C.I.F. B82657941 - Red Hat S.L.
_______________________________________________ Engine-devel mailing list Engine-devel@ovirt.org http://lists.ovirt.org/mailman/listinfo/engine-devel
----- Original Message -----
From: "Roy Golan" <rgolan@redhat.com> To: engine-devel@ovirt.org Sent: Thursday, October 11, 2012 12:22:51 PM Subject: Re: [Engine-devel] libosinfo integration
After considering the pros and cons and making poc's I've decided to take the direct XML and java pure implementation. http://gerrit.ovirt.org/#/c/8472/ http://gerrit.ovirt.org/#/c/8473/
the main motivation was is that the XML loading was fast and standard. it is very easy to expose a subset of the API while leaving behind the extra library magic which we don't currently need. Maybe at some point the inheritance feature will come in handy and the jgir implementation will be done so we could switch to it (I'm really not sure whats it there which is not complete)
Great! This is the simplest solution indeed. Me happy :)))
On 09/24/2012 10:01 PM, Alon Bar-Lev wrote:
----- Original Message -----
From: "Juan Hernandez" <jhernand@redhat.com> To: "Alon Bar-Lev" <alonbl@redhat.com> Cc: "arch" <arch@ovirt.org>, engine-devel@ovirt.org, "Daniel P. Berrange" <berrange@redhat.com> Sent: Monday, September 24, 2012 9:33:38 PM Subject: Re: [Engine-devel] libosinfo integration
On 09/24/2012 08:59 PM, Alon Bar-Lev wrote:
From: "Juan Hernandez" <jhernand@redhat.com> To: "Daniel P. Berrange" <berrange@redhat.com> Cc: "arch" <arch@ovirt.org>, engine-devel@ovirt.org Sent: Monday, September 24, 2012 6:30:43 PM Subject: Re: [Engine-devel] libosinfo integration
On 09/24/2012 05:24 PM, Daniel P. Berrange wrote:
On Mon, Sep 24, 2012 at 04:51:37PM +0200, Roy Golan wrote: > Here's a draft wiki of the libosinfo integration with the > engine > http://wiki.ovirt.org/wiki/Libosinfo > > please share your comments on this thread and I'll update the > wiki > accordingly. As a libosinfo maintainer I strongly discourage you from writing a custom API directly against the XML. While the format is stable from the POV that vendors/users can create new XML docs for new/customized OS variants they have, we do not consider it something for applications to be using directly. The libosinfo library API includes alot of logic beyond simply being an convenient way to process the XML. In particular dealing with the inheritance of data across OS, searching for data across a variety of data sources, relying on GIO for URI resolving, live update of the database, and more.
We're also not really interested in maintaining manually written bindings for any languages. Rather than putting effort into such bindings for Java, time is better spent on finishing the GOBject introspection mapping layer for Java. There was a proof of concept, which was never finished / updated to the final introspection spec, which can be used as a basis for ongoing development.
This work will be applicable way beyond just libosinfo, to a wide variety of useful APIs. Of possible interest to VDSM in the future will be things like the new libvirt-designer & libvirt-builder APIs which are going to provide a bridge between libvirt + libosinfo to simply life for application developers constructing XML for provisioning guests. I think that the use of JGIR has already been discussed as a way to interface the engine with libvdsm.so:
http://lists.ovirt.org/pipermail/arch/2012-August/000760.html
If I remember correctly the outcome of that discussion was that it shouldn't be used because it is considered dangerous to use native libraries from Java and because JGIR itself is very outdated. I don't 100% agree with the first part, but the second part is completely true.
Anyhow, as it seems to be the right way to interface with libosinfo and future libvirt capabilities, we may want to rethink and put some effort to bring it up to date. I don't think Java should go native to obtain data, using JNI or whatever native interface that is offered. It can totally break
----- Original Message ----- the JVM and cause leaks and stability issues JRE would like to avoid.
This is were I disagree, only a bit. It is true that using JNI *without care* can totally break the JVM, generate leaks, and bring down all the application. But if the JNI layer is developed carefully this should not be more risky than any of the other multiple native libraries that you use every day from Java without noticing. In addition JGIR is based on JNA (which is well maintained and stable) and doesn't include any "dangerous" native C code.
Well, you can call me old fashioned...
But I don't think that any none critical task should be run out of process of the production component.
It is not GObject layer I am afraid of, it is what it eventually runs.
I believe that the data provided by libosinfo is not vital for our runtime, no problem in fetching it before start.
And again, I don't think data provider should be language binding... this concept alone is making me scared, as instead of using well defined data container people will start to use mesh of components... I don't see how you can stabilize an application this way.
For the native library that you use "every day", I don't know what you referring to, I've done lsof of engine and saw no exception... which other native library we use these days?
Having language binding for data access logic seems a bit strange for me, as it should be easy to stabilize a schema either using XML or a set of CSVs, for all to read and use.
I do understand why JNI is to be used when OS specific features are to be used, for example unix domain sockets, shared memory, security, but not for data access.
The alternatives would be:
1. Run a python program at ovirt-engine start-up, use the python API to fetch the data we need, and generate our own tiny XML of the subset or even load it to the database. We can use the output safely in Java. Disadvantage is that we need to stop/start service or have a "refresh" verb for reload after libosinfo update.
2. When data is requested, execute a python process with cmdline arguments and get result from stdout of the process. The disadvantage is the overhead of clone().
3. Read the XML directly on runtime, disadvantage is if XML is changed.
4. Use the libosinfo python API during build and distribute an update to the information in versions of ovirt, disadvantage is slow refresh of data, not sure how often this data is updated.
5. Provide a web service that has specific XML-RPC / RESTful API, that can provide data out of the libosinfo, and publish it at internet. Configure ovirt to use it, disadvantage is the need for internet connection.
6. Configure cgi-bin locally at apache where ovirt is running, which provide XML-RPC / RESTful API, and access it locally.
In any case I would avoid exiting the JVM to interact with data provider.
Regards, Alon.
-- Dirección Comercial: C/Jose Bardasano Baos, 9, Edif. Gorbea 3, planta 3ºD, 28016 Madrid, Spain Inscrita en el Reg. Mercantil de Madrid – C.I.F. B82657941 - Red Hat S.L.
_______________________________________________ Engine-devel mailing list Engine-devel@ovirt.org http://lists.ovirt.org/mailman/listinfo/engine-devel
_______________________________________________ Engine-devel mailing list Engine-devel@ovirt.org http://lists.ovirt.org/mailman/listinfo/engine-devel
On Thu, Oct 11, 2012 at 12:22:51PM +0200, Roy Golan wrote:
After considering the pros and cons and making poc's I've decided to take the direct XML and java pure implementation. http://gerrit.ovirt.org/#/c/8472/ http://gerrit.ovirt.org/#/c/8473/
Any particular reason I have no permission to view these?
----- Original Message -----
From: "Ewoud Kohl van Wijngaarden" <ewoud+ovirt@kohlvanwijngaarden.nl> To: engine-devel@ovirt.org Sent: Thursday, October 11, 2012 12:38:12 PM Subject: Re: [Engine-devel] libosinfo integration
On Thu, Oct 11, 2012 at 12:22:51PM +0200, Roy Golan wrote:
After considering the pros and cons and making poc's I've decided to take the direct XML and java pure implementation. http://gerrit.ovirt.org/#/c/8472/ http://gerrit.ovirt.org/#/c/8473/
Any particular reason I have no permission to view these?
I cannot either... it must be a draft. We should solve the drafts permission issues and make them public. Alon.
On 10/11/2012 12:39 PM, Alon Bar-Lev wrote:
----- Original Message -----
From: "Ewoud Kohl van Wijngaarden" <ewoud+ovirt@kohlvanwijngaarden.nl> To: engine-devel@ovirt.org Sent: Thursday, October 11, 2012 12:38:12 PM Subject: Re: [Engine-devel] libosinfo integration
On Thu, Oct 11, 2012 at 12:22:51PM +0200, Roy Golan wrote:
After considering the pros and cons and making poc's I've decided to take the direct XML and java pure implementation. http://gerrit.ovirt.org/#/c/8472/ http://gerrit.ovirt.org/#/c/8473/ Any particular reason I have no permission to view these? I cannot either... it must be a draft. We should solve the drafts permission issues and make them public.
Eedri do you know how to solve this? I'll just add you manually meanwhile.
Alon. _______________________________________________ Engine-devel mailing list Engine-devel@ovirt.org http://lists.ovirt.org/mailman/listinfo/engine-devel
----- Original Message -----
From: "Roy Golan" <rgolan@redhat.com> To: engine-devel@ovirt.org Sent: Thursday, October 11, 2012 12:45:58 PM Subject: Re: [Engine-devel] libosinfo integration
On 10/11/2012 12:39 PM, Alon Bar-Lev wrote:
----- Original Message -----
From: "Ewoud Kohl van Wijngaarden" <ewoud+ovirt@kohlvanwijngaarden.nl> To: engine-devel@ovirt.org Sent: Thursday, October 11, 2012 12:38:12 PM Subject: Re: [Engine-devel] libosinfo integration
On Thu, Oct 11, 2012 at 12:22:51PM +0200, Roy Golan wrote:
After considering the pros and cons and making poc's I've decided to take the direct XML and java pure implementation. http://gerrit.ovirt.org/#/c/8472/ http://gerrit.ovirt.org/#/c/8473/ Any particular reason I have no permission to view these? I cannot either... it must be a draft. We should solve the drafts permission issues and make them public.
Eedri do you know how to solve this?
don't have permissions to gerrit.ovirt.org. you should ask on infra@ovirt.org
I'll just add you manually meanwhile.
Alon. _______________________________________________ Engine-devel mailing list Engine-devel@ovirt.org http://lists.ovirt.org/mailman/listinfo/engine-devel
_______________________________________________ Engine-devel mailing list Engine-devel@ovirt.org http://lists.ovirt.org/mailman/listinfo/engine-devel
----- Original Message -----
From: "Eyal Edri" <eedri@redhat.com> To: "Roy Golan" <rgolan@redhat.com> Cc: engine-devel@ovirt.org Sent: Thursday, October 11, 2012 2:01:30 PM Subject: Re: [Engine-devel] libosinfo integration
----- Original Message -----
From: "Roy Golan" <rgolan@redhat.com> To: engine-devel@ovirt.org Sent: Thursday, October 11, 2012 12:45:58 PM Subject: Re: [Engine-devel] libosinfo integration
On 10/11/2012 12:39 PM, Alon Bar-Lev wrote:
From: "Ewoud Kohl van Wijngaarden" <ewoud+ovirt@kohlvanwijngaarden.nl> To: engine-devel@ovirt.org Sent: Thursday, October 11, 2012 12:38:12 PM Subject: Re: [Engine-devel] libosinfo integration
On Thu, Oct 11, 2012 at 12:22:51PM +0200, Roy Golan wrote:
After considering the pros and cons and making poc's I've decided to take the direct XML and java pure implementation. http://gerrit.ovirt.org/#/c/8472/ http://gerrit.ovirt.org/#/c/8473/ Any particular reason I have no permission to view these? I cannot either... it must be a draft. We should solve the drafts permission issues and make them
----- Original Message ----- public.
Eedri do you know how to solve this?
don't have permissions to gerrit.ovirt.org. you should ask on infra@ovirt.org
Eyal, we talked about this. We should make the access to drafts as public not only to people that are CCed.
I'll just add you manually meanwhile.
Alon. _______________________________________________ Engine-devel mailing list Engine-devel@ovirt.org http://lists.ovirt.org/mailman/listinfo/engine-devel
_______________________________________________ Engine-devel mailing list Engine-devel@ovirt.org http://lists.ovirt.org/mailman/listinfo/engine-devel
_______________________________________________ Engine-devel mailing list Engine-devel@ovirt.org http://lists.ovirt.org/mailman/listinfo/engine-devel
participants (7)
-
Alon Bar-Lev -
Daniel P. Berrange -
David Jaša -
Ewoud Kohl van Wijngaarden -
Eyal Edri -
Juan Hernandez -
Roy Golan