This message is in response to an earlier thread regarding a UI plugin to upload ISO files. Like the original poster, Lucas, I began work on a UI plugin to allow uploading ISO files through a UI plugin. After reading the previous thread I'm re-thinking the architecture. It was suggested that the recommended approach to upload files to a storage domain is through the VDSM API [1]. I'm pretty familiar with the oVirt REST API but have been unable to find documentation regarding accessing the VDSM API. Should the VDSM API be accessible by a UI plugin? If so, is there documentation available to do so? [1] http://lists.ovirt.org/pipermail/devel/2014-December/009497.html -Tony
----- Original Message -----
From: "Tony James" <tony@anthonyjames.org> To: devel@ovirt.org Sent: Monday, December 29, 2014 3:30:49 AM Subject: [ovirt-devel] UI Plugin to Upload ISO Files
This message is in response to an earlier thread regarding a UI plugin to upload ISO files. Like the original poster, Lucas, I began work on a UI plugin to allow uploading ISO files through a UI plugin. After reading the previous thread I'm re-thinking the architecture.
It was suggested that the recommended approach to upload files to a storage domain is through the VDSM API [1]. I'm pretty familiar with the oVirt REST API but have been unable to find documentation regarding accessing the VDSM API. Should the VDSM API be accessible by a UI plugin? If so, is there documentation available to do so?
[1] http://lists.ovirt.org/pipermail/devel/2014-December/009497.html
Basically you have to: 1. Use the vdsm xmlrpc/jsonrpc to create an image 2. Use the vdsm http api to upload the data to the image. This will create a task and return a task id. 3. Use the vdsm xmlrpc/jsonrpc api to check the task status, and clear the task when done The xmlrpc/jsonrpc api is documented here: http://gerrit.ovirt.org/gitweb?p=vdsm.git;a=blob;f=vdsm/rpc/vdsmapi-schema.j... You can check the code for upload here: http://gerrit.ovirt.org/gitweb?p=vdsm.git;a=blob;f=vdsm/rpc/BindingXMLRPC.py... Nir
On 12/29/2014 09:25 AM, Nir Soffer wrote:
----- Original Message -----
From: "Tony James" <tony@anthonyjames.org> To: devel@ovirt.org Sent: Monday, December 29, 2014 3:30:49 AM Subject: [ovirt-devel] UI Plugin to Upload ISO Files
This message is in response to an earlier thread regarding a UI plugin to upload ISO files. Like the original poster, Lucas, I began work on a UI plugin to allow uploading ISO files through a UI plugin. After reading the previous thread I'm re-thinking the architecture.
It was suggested that the recommended approach to upload files to a storage domain is through the VDSM API [1]. I'm pretty familiar with the oVirt REST API but have been unable to find documentation regarding accessing the VDSM API. Should the VDSM API be accessible by a UI plugin? If so, is there documentation available to do so?
[1] http://lists.ovirt.org/pipermail/devel/2014-December/009497.html
Basically you have to: 1. Use the vdsm xmlrpc/jsonrpc to create an image 2. Use the vdsm http api to upload the data to the image. This will create a task and return a task id. 3. Use the vdsm xmlrpc/jsonrpc api to check the task status, and clear the task when done
The xmlrpc/jsonrpc api is documented here: http://gerrit.ovirt.org/gitweb?p=vdsm.git;a=blob;f=vdsm/rpc/vdsmapi-schema.j...
You can check the code for upload here: http://gerrit.ovirt.org/gitweb?p=vdsm.git;a=blob;f=vdsm/rpc/BindingXMLRPC.py...
I assume the upload will be done via a servlet on the engine, not directly by the ui plugin accessing vdsm. worth discussing your plans here, to make sure architecture/security are correct.
On Mon, Dec 29, 2014 at 5:26 AM, Itamar Heim <iheim@redhat.com> wrote:
On 12/29/2014 09:25 AM, Nir Soffer wrote:
----- Original Message -----
From: "Tony James" <tony@anthonyjames.org> To: devel@ovirt.org Sent: Monday, December 29, 2014 3:30:49 AM Subject: [ovirt-devel] UI Plugin to Upload ISO Files
This message is in response to an earlier thread regarding a UI plugin to upload ISO files. Like the original poster, Lucas, I began work on a UI plugin to allow uploading ISO files through a UI plugin. After reading the previous thread I'm re-thinking the architecture.
It was suggested that the recommended approach to upload files to a storage domain is through the VDSM API [1]. I'm pretty familiar with the oVirt REST API but have been unable to find documentation regarding accessing the VDSM API. Should the VDSM API be accessible by a UI plugin? If so, is there documentation available to do so?
[1] http://lists.ovirt.org/pipermail/devel/2014-December/009497.html
Basically you have to: 1. Use the vdsm xmlrpc/jsonrpc to create an image 2. Use the vdsm http api to upload the data to the image. This will create a task and return a task id. 3. Use the vdsm xmlrpc/jsonrpc api to check the task status, and clear the task when done
The xmlrpc/jsonrpc api is documented here:
http://gerrit.ovirt.org/gitweb?p=vdsm.git;a=blob;f=vdsm/rpc/vdsmapi-schema.j...
You can check the code for upload here:
http://gerrit.ovirt.org/gitweb?p=vdsm.git;a=blob;f=vdsm/rpc/BindingXMLRPC.py...
I assume the upload will be done via a servlet on the engine, not directly by the ui plugin accessing vdsm. worth discussing your plans here, to make sure architecture/security are correct.
I was planning on using a python CGI script which would accept the upload via POST from the UI plugin. The file would be stored in /tmp on the engine host. After the file was successfully uploaded, the CGI script would send a POST to a python HTTP server (BaseHTTPServer, also running on engine host) with the filename and storage domain information. This python script would then take care of mounting the storage domain and copying the file to the appropriate location. This was my initial approach, I plan on checking out the VDSM API as well.
On Dec 29, 2014, at 11:13 AM, Tony James <tony@anthonyjames.org> wrote:
On Mon, Dec 29, 2014 at 5:26 AM, Itamar Heim <iheim@redhat.com> wrote:
On 12/29/2014 09:25 AM, Nir Soffer wrote:
----- Original Message -----
From: "Tony James" <tony@anthonyjames.org> To: devel@ovirt.org Sent: Monday, December 29, 2014 3:30:49 AM Subject: [ovirt-devel] UI Plugin to Upload ISO Files
This message is in response to an earlier thread regarding a UI plugin to upload ISO files. Like the original poster, Lucas, I began work on a UI plugin to allow uploading ISO files through a UI plugin. After reading the previous thread I'm re-thinking the architecture.
It was suggested that the recommended approach to upload files to a storage domain is through the VDSM API [1]. I'm pretty familiar with the oVirt REST API but have been unable to find documentation regarding accessing the VDSM API. Should the VDSM API be accessible by a UI plugin? If so, is there documentation available to do so?
[1] http://lists.ovirt.org/pipermail/devel/2014-December/009497.html
Basically you have to: 1. Use the vdsm xmlrpc/jsonrpc to create an image 2. Use the vdsm http api to upload the data to the image. This will create a task and return a task id. 3. Use the vdsm xmlrpc/jsonrpc api to check the task status, and clear the task when done
The xmlrpc/jsonrpc api is documented here:
http://gerrit.ovirt.org/gitweb?p=vdsm.git;a=blob;f=vdsm/rpc/vdsmapi-schema.j...
You can check the code for upload here:
http://gerrit.ovirt.org/gitweb?p=vdsm.git;a=blob;f=vdsm/rpc/BindingXMLRPC.py...
I assume the upload will be done via a servlet on the engine, not directly by the ui plugin accessing vdsm. worth discussing your plans here, to make sure architecture/security are correct.
I was planning on using a python CGI script which would accept the upload via POST from the UI plugin. The file would be stored in /tmp on the engine host.
After the file was successfully uploaded, the CGI script would send a POST to a python HTTP server (BaseHTTPServer, also running on engine host) with the filename and storage domain information. This python script would then take care of mounting the storage domain and copying the file to the appropriate location.
This was my initial approach, I plan on checking out the VDSM API as well.
I’ve made some progress interacting with the VDSM API and wanted to get feedback on the connection method before proceeding. Below is the Python code I’m using to connect to the API: import xmlrpclib host = ’node.example.com' port = '54321' key_file = '/etc/pki/ovirt-engine/keys/engine_id_rsa' cert_file = '/etc/pki/ovirt-engine/certs/engine.cer' class MySafeTransport(xmlrpclib.SafeTransport): host_tuple = ('%s:%s' % (host, port), {'key_file': key_file, 'cert_file': cert_file}) def make_connection(self, host): return xmlrpclib.SafeTransport.make_connection(self, self.host_tuple) transport = MySafeTransport() server = xmlrpclib.ServerProxy('https://%s:%s' % (host, port), transport) This will be run from the engine host. If all is well I assume the ‘uploadImage’ method is the one to use for actually uploading an ISO file? -Tony
On 12/29/2014 06:13 PM, Tony James wrote:
On Mon, Dec 29, 2014 at 5:26 AM, Itamar Heim <iheim@redhat.com> wrote:
On 12/29/2014 09:25 AM, Nir Soffer wrote:
----- Original Message -----
From: "Tony James" <tony@anthonyjames.org> To: devel@ovirt.org Sent: Monday, December 29, 2014 3:30:49 AM Subject: [ovirt-devel] UI Plugin to Upload ISO Files
This message is in response to an earlier thread regarding a UI plugin to upload ISO files. Like the original poster, Lucas, I began work on a UI plugin to allow uploading ISO files through a UI plugin. After reading the previous thread I'm re-thinking the architecture.
It was suggested that the recommended approach to upload files to a storage domain is through the VDSM API [1]. I'm pretty familiar with the oVirt REST API but have been unable to find documentation regarding accessing the VDSM API. Should the VDSM API be accessible by a UI plugin? If so, is there documentation available to do so?
[1] http://lists.ovirt.org/pipermail/devel/2014-December/009497.html
Basically you have to: 1. Use the vdsm xmlrpc/jsonrpc to create an image 2. Use the vdsm http api to upload the data to the image. This will create a task and return a task id. 3. Use the vdsm xmlrpc/jsonrpc api to check the task status, and clear the task when done
The xmlrpc/jsonrpc api is documented here:
http://gerrit.ovirt.org/gitweb?p=vdsm.git;a=blob;f=vdsm/rpc/vdsmapi-schema.j...
You can check the code for upload here:
http://gerrit.ovirt.org/gitweb?p=vdsm.git;a=blob;f=vdsm/rpc/BindingXMLRPC.py...
I assume the upload will be done via a servlet on the engine, not directly by the ui plugin accessing vdsm. worth discussing your plans here, to make sure architecture/security are correct.
I was planning on using a python CGI script which would accept the upload via POST from the UI plugin. The file would be stored in /tmp on the engine host.
After the file was successfully uploaded, the CGI script would send a POST to a python HTTP server (BaseHTTPServer, also running on engine host) with the filename and storage domain information. This python script would then take care of mounting the storage domain and copying the file to the appropriate location.
This was my initial approach, I plan on checking out the VDSM API as well.
my preference would be to stream via a servlet to the vdsm api, rather than "store and forward" to avoid potentially exhausting space on engine or having to deal with two phased task tracking. the tricky part which requires a review is validating authentication and authorization by the servlet - to make sure one has the permission to write to a certain disk (for data domains) / iso domain. this should be similar to the websocket novnc approach of validating user has access to relevant VM (but Alon may correct me if its different) notice there is one caveat for iso domains to having vdsm do the upload vs. the iso-uploader utility - it would require vdsm to have write permissions to the iso nfs path. but it allows uploading disks/vm's as well to data stores, which i think is worth having the same pattern for both.
----- Original Message -----
From: "Itamar Heim" <iheim@redhat.com> To: "Tony James" <tony@anthonyjames.org> Cc: "Nir Soffer" <nsoffer@redhat.com>, "Liron Aravot" <laravot@redhat.com>, devel@ovirt.org, "Alon Bar-Lev" <alonbl@redhat.com> Sent: Friday, January 2, 2015 9:44:45 AM Subject: Re: [ovirt-devel] UI Plugin to Upload ISO Files
On 12/29/2014 06:13 PM, Tony James wrote:
On Mon, Dec 29, 2014 at 5:26 AM, Itamar Heim <iheim@redhat.com> wrote:
On 12/29/2014 09:25 AM, Nir Soffer wrote:
----- Original Message -----
From: "Tony James" <tony@anthonyjames.org> To: devel@ovirt.org Sent: Monday, December 29, 2014 3:30:49 AM Subject: [ovirt-devel] UI Plugin to Upload ISO Files
This message is in response to an earlier thread regarding a UI plugin to upload ISO files. Like the original poster, Lucas, I began work on a UI plugin to allow uploading ISO files through a UI plugin. After reading the previous thread I'm re-thinking the architecture.
It was suggested that the recommended approach to upload files to a storage domain is through the VDSM API [1]. I'm pretty familiar with the oVirt REST API but have been unable to find documentation regarding accessing the VDSM API. Should the VDSM API be accessible by a UI plugin? If so, is there documentation available to do so?
[1] http://lists.ovirt.org/pipermail/devel/2014-December/009497.html
Basically you have to: 1. Use the vdsm xmlrpc/jsonrpc to create an image 2. Use the vdsm http api to upload the data to the image. This will create a task and return a task id. 3. Use the vdsm xmlrpc/jsonrpc api to check the task status, and clear the task when done
The xmlrpc/jsonrpc api is documented here:
http://gerrit.ovirt.org/gitweb?p=vdsm.git;a=blob;f=vdsm/rpc/vdsmapi-schema.j...
You can check the code for upload here:
http://gerrit.ovirt.org/gitweb?p=vdsm.git;a=blob;f=vdsm/rpc/BindingXMLRPC.py...
I assume the upload will be done via a servlet on the engine, not directly by the ui plugin accessing vdsm. worth discussing your plans here, to make sure architecture/security are correct.
I was planning on using a python CGI script which would accept the upload via POST from the UI plugin. The file would be stored in /tmp on the engine host.
After the file was successfully uploaded, the CGI script would send a POST to a python HTTP server (BaseHTTPServer, also running on engine host) with the filename and storage domain information. This python script would then take care of mounting the storage domain and copying the file to the appropriate location.
This was my initial approach, I plan on checking out the VDSM API as well.
my preference would be to stream via a servlet to the vdsm api, rather than "store and forward" to avoid potentially exhausting space on engine or having to deal with two phased task tracking.
the tricky part which requires a review is validating authentication and authorization by the servlet - to make sure one has the permission to write to a certain disk (for data domains) / iso domain. this should be similar to the websocket novnc approach of validating user has access to relevant VM (but Alon may correct me if its different)
these requirements are for core product feature, while I guess Tony is after a solution to bridge the gap of the missing basic product feature within his environment. his solution is the most simple to solve his local problem, please assign core developer(s) to resolve the generic problem. Expecting a contributer to apply something the product was not designed for or to fix the entire product is incorrect. we have long way to enable pluggable server side logic; we have no stable interface for authentication check (although we plan to have one for 3.6), we have no stable xmlrpc wrapper, not to mention that the xmlrpc should be obsoleted once the json rpc is out, we do not have access to dao to obtain information about entities, we do not have stable interface to engine key and track its usage, we do not have restapi extensions to extend server side functionality, I guess we do not have many more, as engine was never designed as modular extensible component. we should also consider dropping the image upload to vdsm in favor of glance, so that core product will support glance directly either by embedding it or by delegating into it. this will enable us to retire the xmlrpc entirely as far as I understand, it will also enable to implement similar functionality of what initially requested within this thread. Regards, Alon
On Fri, Jan 2, 2015 at 2:44 AM, Itamar Heim <iheim@redhat.com> wrote:
On 12/29/2014 06:13 PM, Tony James wrote:
On Mon, Dec 29, 2014 at 5:26 AM, Itamar Heim <iheim@redhat.com> wrote:
On 12/29/2014 09:25 AM, Nir Soffer wrote:
----- Original Message -----
From: "Tony James" <tony@anthonyjames.org> To: devel@ovirt.org Sent: Monday, December 29, 2014 3:30:49 AM Subject: [ovirt-devel] UI Plugin to Upload ISO Files
This message is in response to an earlier thread regarding a UI plugin to upload ISO files. Like the original poster, Lucas, I began work on a UI plugin to allow uploading ISO files through a UI plugin. After reading the previous thread I'm re-thinking the architecture.
It was suggested that the recommended approach to upload files to a storage domain is through the VDSM API [1]. I'm pretty familiar with the oVirt REST API but have been unable to find documentation regarding accessing the VDSM API. Should the VDSM API be accessible by a UI plugin? If so, is there documentation available to do so?
[1] http://lists.ovirt.org/pipermail/devel/2014-December/009497.html
Basically you have to: 1. Use the vdsm xmlrpc/jsonrpc to create an image 2. Use the vdsm http api to upload the data to the image. This will create a task and return a task id. 3. Use the vdsm xmlrpc/jsonrpc api to check the task status, and clear the task when done
The xmlrpc/jsonrpc api is documented here:
http://gerrit.ovirt.org/gitweb?p=vdsm.git;a=blob;f=vdsm/rpc/vdsmapi-schema.j...
You can check the code for upload here:
http://gerrit.ovirt.org/gitweb?p=vdsm.git;a=blob;f=vdsm/rpc/BindingXMLRPC.py...
I assume the upload will be done via a servlet on the engine, not directly by the ui plugin accessing vdsm. worth discussing your plans here, to make sure architecture/security are correct.
I was planning on using a python CGI script which would accept the upload via POST from the UI plugin. The file would be stored in /tmp on the engine host.
After the file was successfully uploaded, the CGI script would send a POST to a python HTTP server (BaseHTTPServer, also running on engine host) with the filename and storage domain information. This python script would then take care of mounting the storage domain and copying the file to the appropriate location.
This was my initial approach, I plan on checking out the VDSM API as well.
my preference would be to stream via a servlet to the vdsm api, rather than "store and forward" to avoid potentially exhausting space on engine or having to deal with two phased task tracking.
the tricky part which requires a review is validating authentication and authorization by the servlet - to make sure one has the permission to write to a certain disk (for data domains) / iso domain. this should be similar to the websocket novnc approach of validating user has access to relevant VM (but Alon may correct me if its different)
notice there is one caveat for iso domains to having vdsm do the upload vs. the iso-uploader utility - it would require vdsm to have write permissions to the iso nfs path. but it allows uploading disks/vm's as well to data stores, which i think is worth having the same pattern for both.
Would it be sufficient to verify that the user has been given the StorageAdmin role before allowing an upload?
On 01/02/2015 05:16 PM, Tony James wrote:
On Fri, Jan 2, 2015 at 2:44 AM, Itamar Heim <iheim@redhat.com> wrote:
On 12/29/2014 06:13 PM, Tony James wrote:
On Mon, Dec 29, 2014 at 5:26 AM, Itamar Heim <iheim@redhat.com> wrote:
On 12/29/2014 09:25 AM, Nir Soffer wrote:
----- Original Message -----
From: "Tony James" <tony@anthonyjames.org> To: devel@ovirt.org Sent: Monday, December 29, 2014 3:30:49 AM Subject: [ovirt-devel] UI Plugin to Upload ISO Files
This message is in response to an earlier thread regarding a UI plugin to upload ISO files. Like the original poster, Lucas, I began work on a UI plugin to allow uploading ISO files through a UI plugin. After reading the previous thread I'm re-thinking the architecture.
It was suggested that the recommended approach to upload files to a storage domain is through the VDSM API [1]. I'm pretty familiar with the oVirt REST API but have been unable to find documentation regarding accessing the VDSM API. Should the VDSM API be accessible by a UI plugin? If so, is there documentation available to do so?
[1] http://lists.ovirt.org/pipermail/devel/2014-December/009497.html
Basically you have to: 1. Use the vdsm xmlrpc/jsonrpc to create an image 2. Use the vdsm http api to upload the data to the image. This will create a task and return a task id. 3. Use the vdsm xmlrpc/jsonrpc api to check the task status, and clear the task when done
The xmlrpc/jsonrpc api is documented here:
http://gerrit.ovirt.org/gitweb?p=vdsm.git;a=blob;f=vdsm/rpc/vdsmapi-schema.j...
You can check the code for upload here:
http://gerrit.ovirt.org/gitweb?p=vdsm.git;a=blob;f=vdsm/rpc/BindingXMLRPC.py...
I assume the upload will be done via a servlet on the engine, not directly by the ui plugin accessing vdsm. worth discussing your plans here, to make sure architecture/security are correct.
I was planning on using a python CGI script which would accept the upload via POST from the UI plugin. The file would be stored in /tmp on the engine host.
After the file was successfully uploaded, the CGI script would send a POST to a python HTTP server (BaseHTTPServer, also running on engine host) with the filename and storage domain information. This python script would then take care of mounting the storage domain and copying the file to the appropriate location.
This was my initial approach, I plan on checking out the VDSM API as well.
my preference would be to stream via a servlet to the vdsm api, rather than "store and forward" to avoid potentially exhausting space on engine or having to deal with two phased task tracking.
the tricky part which requires a review is validating authentication and authorization by the servlet - to make sure one has the permission to write to a certain disk (for data domains) / iso domain. this should be similar to the websocket novnc approach of validating user has access to relevant VM (but Alon may correct me if its different)
notice there is one caveat for iso domains to having vdsm do the upload vs. the iso-uploader utility - it would require vdsm to have write permissions to the iso nfs path. but it allows uploading disks/vm's as well to data stores, which i think is worth having the same pattern for both.
Would it be sufficient to verify that the user has been given the StorageAdmin role before allowing an upload?
1. lets see if we can design this as a full blown feature wrt "how it should look like". I still believe it should have a servlet for the actual communication streaming to vdsm. of course, if you want to do something until this is provided "properly", you can do that. 2. allon - thoughts wrt permissions? would be relevant to the general use case. question is if this should be limited to admins, then probably storage domain level permission is enough, or we'd like to check disk level permissions (for data disks) to allow users to upload vm's as well. my personal take is we can probably start with "admins first" which means storage domain level permission check should be enough to start with.
participants (4)
-
Alon Bar-Lev -
Itamar Heim -
Nir Soffer -
Tony James