[RFC] ISO pool: Support to download ISO from URL
The following API changes should be made to implement the remote download ISO feature: 1) POST /storagepools/ISO/storagevolumes/download {'url': <http|https|ftp|ftps>, 'name': <volume name>} Description: Starts downloading a remote ISO image to Kimchi's storage pool. Parameters: 'url': specifies where the image is hosted. Required. 'name': specifies the name of the local file when the download finishes. If this parameter is omitted, Kimchi will try to use the same file name as the remote file. Optional. Return codes: 200: the remote URL is valid and the download started successfully. 400: the remote URL is not valid; the provided 'name' is already in use (this error shouldn't happen if 'name' is omitted). the storage pool doesn't have enough free space to hold the remote file. Return data: {'name': <vol-name>} 2) POST /storagepools/ISO/storagevolumes/download/pause {'name': <volume name>} Description: Pauses a remote ISO download. Parameters: 'name': the remote volume name to be paused. Required. Return codes: 200: 'name' is an ongoing download file and the operation was paused successfully. 400: 'name' is an invalid volume name (doesn't exist / isn't an ongoing download); download referred by 'name' cannot be paused. Return data: {} 3) POST /storagepools/ISO/storagevolumes/download/resume {'name': <volume name>} Description: Resumes a paused remote ISO download. Parameters: 'name': the remote volume name to be resumed. Required. Return codes: 200: 'name' is a paused download file and the operation was resumed successfully. 400: 'name' is an invalid volume name (doesn't exist / isn't a paused download). Return data: {} 4) POST /storagepools/ISO/storagevolumes/download/cancel {'name': <volume name>} Description: Cancels a remote ISO download. Parameters: 'name': the remote volume name to be canceled. Required. Return codes: 200: 'name' is an ongoing download file and the operation was canceled successfully. Whatever data has been downloaded so far should be deleted. 400: 'name' is an invalid volume name (doesn't exist / isn't an ongoing download). Return data: {} 5) POST /storagepools/ISO/storagevolumes/download/status {'name': <volume name>} Description: Provides status of a remote ISO download. Parameters: 'name': the remote volume name. Required. Return codes: 200: 'name' is a download file and its status information was returned successfully. 400: 'name' is an invalid volume name (doesn't exist / isn't a remote download). Return data: {'name': <volume name>, 'status': [downloading|paused], 'total_size': <size in bytes>, 'downloaded_size': <size in bytes>} I tried to put together the feedback given in another e-mail thread, especially by Aline, Royce and Yu Xin Huo, along with some ideas of my own. This RFC doesn't support choosing a different storage pool other than "ISO" to host the downloaded files. If we're doing this, wel need to update this new API accordingly. The implementation details will be discussed later. Please share your feedback with us.
On 2014年08月19日 08:42, Crístian Viana wrote:
The following API changes should be made to implement the remote download ISO feature:
1) POST /storagepools/ISO/storagevolumes/download {'url': <http|https|ftp|ftps>, 'name': <volume name>} Description: Starts downloading a remote ISO image to Kimchi's storage pool. Parameters: 'url': specifies where the image is hosted. Required. 'name': specifies the name of the local file when the download finishes. If this parameter is omitted, Kimchi will try to use the same file name as the remote file. Optional. Return codes: 200: the remote URL is valid and the download started successfully. 400: the remote URL is not valid; Cristian,
Thanks for this comprehensive and detailed RFC which we can take as a good example! Some suggestion according to me below: 1. As for me, if download/upload is a time consuming task, we may want to use AsyncTask to track it. So for POST /storagepools/ISO/storagevolumes/download we may want to return 202 and a 'task_id' in the response data. 2.To query status, pause and cancel the download, the target will be the task: POST /tasks/download-task-id/cancel This may require enhancement of tasks API.
the provided 'name' is already in use (this error shouldn't happen if 'name' is omitted). the storage pool doesn't have enough free space to hold the remote file. Return data: {'name': <vol-name>}
2) POST /storagepools/ISO/storagevolumes/download/pause {'name': <volume name>} Description: Pauses a remote ISO download. Parameters: 'name': the remote volume name to be paused. Required. Return codes: 200: 'name' is an ongoing download file and the operation was paused successfully. 400: 'name' is an invalid volume name (doesn't exist / isn't an ongoing download); download referred by 'name' cannot be paused. Return data: {}
3) POST /storagepools/ISO/storagevolumes/download/resume {'name': <volume name>} Description: Resumes a paused remote ISO download. Parameters: 'name': the remote volume name to be resumed. Required. Return codes: 200: 'name' is a paused download file and the operation was resumed successfully. 400: 'name' is an invalid volume name (doesn't exist / isn't a paused download). Return data: {}
4) POST /storagepools/ISO/storagevolumes/download/cancel {'name': <volume name>} Description: Cancels a remote ISO download. Parameters: 'name': the remote volume name to be canceled. Required. Return codes: 200: 'name' is an ongoing download file and the operation was canceled successfully. Whatever data has been downloaded so far should be deleted. 400: 'name' is an invalid volume name (doesn't exist / isn't an ongoing download). Return data: {}
5) POST /storagepools/ISO/storagevolumes/download/status {'name': <volume name>} Description: Provides status of a remote ISO download. Parameters: 'name': the remote volume name. Required. Return codes: 200: 'name' is a download file and its status information was returned successfully. 400: 'name' is an invalid volume name (doesn't exist / isn't a remote download). Return data: {'name': <volume name>, 'status': [downloading|paused], 'total_size': <size in bytes>, 'downloaded_size': <size in bytes>}
I tried to put together the feedback given in another e-mail thread, especially by Aline, Royce and Yu Xin Huo, along with some ideas of my own. This RFC doesn't support choosing a different storage pool other than "ISO" to host the downloaded files. If we're doing this, wel need to update this new API accordingly.
The implementation details will be discussed later.
Please share your feedback with us.
_______________________________________________ Kimchi-devel mailing list Kimchi-devel@ovirt.org http://lists.ovirt.org/mailman/listinfo/kimchi-devel
On 19-08-2014 00:30, Royce Lv wrote:
Cristian,
Thanks for this comprehensive and detailed RFC which we can take as a good example! Some suggestion according to me below: 1. As for me, if download/upload is a time consuming task, we may want to use AsyncTask to track it. So for POST /storagepools/ISO/storagevolumes/download we may want to return 202 and a 'task_id' in the response data.
2.To query status, pause and cancel the download, the target will be the task: POST /tasks/download-task-id/cancel This may require enhancement of tasks API.
Thanks for the suggestions! I proposed returning the "name" when a download starts exactly so it could be tracked later (paused, canceled, status), but an ID sounds better for that. Especially because there's already a Tasks API we can use here.
On 08/18/2014 09:42 PM, Crístian Viana wrote:
The following API changes should be made to implement the remote download ISO feature:
1) POST /storagepools/ISO/storagevolumes/download {'url': <http|https|ftp|ftps>, 'name': <volume name>} Description: Starts downloading a remote ISO image to Kimchi's storage pool. Parameters: 'url': specifies where the image is hosted. Required. 'name': specifies the name of the local file when the download finishes. If this parameter is omitted, Kimchi will try to use the same file name as the remote file. Optional. Return codes: 200: the remote URL is valid and the download started successfully. 400: the remote URL is not valid; the provided 'name' is already in use (this error shouldn't happen if 'name' is omitted). the storage pool doesn't have enough free space to hold the remote file. Return data: {'name': <vol-name>}
2) POST /storagepools/ISO/storagevolumes/download/pause {'name': <volume name>} Description: Pauses a remote ISO download. Parameters: 'name': the remote volume name to be paused. Required. Return codes: 200: 'name' is an ongoing download file and the operation was paused successfully. 400: 'name' is an invalid volume name (doesn't exist / isn't an ongoing download); download referred by 'name' cannot be paused. Return data: {}
3) POST /storagepools/ISO/storagevolumes/download/resume {'name': <volume name>} Description: Resumes a paused remote ISO download. Parameters: 'name': the remote volume name to be resumed. Required. Return codes: 200: 'name' is a paused download file and the operation was resumed successfully. 400: 'name' is an invalid volume name (doesn't exist / isn't a paused download). Return data: {}
4) POST /storagepools/ISO/storagevolumes/download/cancel {'name': <volume name>} Description: Cancels a remote ISO download. Parameters: 'name': the remote volume name to be canceled. Required. Return codes: 200: 'name' is an ongoing download file and the operation was canceled successfully. Whatever data has been downloaded so far should be deleted. 400: 'name' is an invalid volume name (doesn't exist / isn't an ongoing download). Return data: {}
5) POST /storagepools/ISO/storagevolumes/download/status {'name': <volume name>} Description: Provides status of a remote ISO download. Parameters: 'name': the remote volume name. Required. Return codes: 200: 'name' is a download file and its status information was returned successfully. 400: 'name' is an invalid volume name (doesn't exist / isn't a remote download). Return data: {'name': <volume name>, 'status': [downloading|paused], 'total_size': <size in bytes>, 'downloaded_size': <size in bytes>}
I tried to put together the feedback given in another e-mail thread, especially by Aline, Royce and Yu Xin Huo, along with some ideas of my own. This RFC doesn't support choosing a different storage pool other than "ISO" to host the downloaded files. If we're doing this, wel need to update this new API accordingly.
Just one more comment along the others made by Royce. Yes - download and upload functionality will be available to all pools.
The implementation details will be discussed later.
Please share your feedback with us.
_______________________________________________ Kimchi-devel mailing list Kimchi-devel@ovirt.org http://lists.ovirt.org/mailman/listinfo/kimchi-devel
On 19-08-2014 10:49, Aline Manera wrote:
Just one more comment along the others made by Royce.
Yes - download and upload functionality will be available to all pools.
Ok! So I'll update the RFC to support multiple storage pools and to use the Tasks API to handle the pause/resume/cancel operations.
On 19-08-2014 13:53, Crístian Viana wrote:
On 19-08-2014 10:49, Aline Manera wrote:
Just one more comment along the others made by Royce.
Yes - download and upload functionality will be available to all pools.
Ok! So I'll update the RFC to support multiple storage pools and to use the Tasks API to handle the pause/resume/cancel operations.
Well, Aline and I found a problem with this proposal. Currently, the REST API always redirects "POST" requests in a Collection to the function "<colletion-name>_create". In our case, if we send "POST /storagepools/<pool-name>/storagevolumes/download", that would be redirected to the backend function "storagevolumes_create". There is no way to redirect those requests to different functions (like, ideally, "storagevolumes_download" and "storagevolumes_upload"). So I see two options from here: 1) Use the first API proposed by Aline: "POST /storagepools/<pool-name>/storagevolumes". By looking at the parameters (i.e. whether 'url' starts with "file://" or "[http|ftp]://") we can tell if the operation is download or upload. 2) Remodel the control code to support multiple actions when POST'ing to a Collection.
On 08/19/2014 04:03 PM, Crístian Viana wrote:
On 19-08-2014 13:53, Crístian Viana wrote:
On 19-08-2014 10:49, Aline Manera wrote:
Just one more comment along the others made by Royce.
Yes - download and upload functionality will be available to all pools.
Ok! So I'll update the RFC to support multiple storage pools and to use the Tasks API to handle the pause/resume/cancel operations.
Well, Aline and I found a problem with this proposal. Currently, the REST API always redirects "POST" requests in a Collection to the function "<colletion-name>_create". In our case, if we send "POST /storagepools/<pool-name>/storagevolumes/download", that would be redirected to the backend function "storagevolumes_create". There is no way to redirect those requests to different functions (like, ideally, "storagevolumes_download" and "storagevolumes_upload").
So I see two options from here:
1) Use the first API proposed by Aline: "POST /storagepools/<pool-name>/storagevolumes". By looking at the parameters (i.e. whether 'url' starts with "file://" or "[http|ftp]://") we can tell if the operation is download or upload.
We can use "url" for download and "file" for upload. if "url" in params.keys(): self.download() elif "file" in params.keys(): self.upload()
2) Remodel the control code to support multiple actions when POST'ing to a Collection.
It will not be possible as we expect to have an resource id instead of an action name. To be able to do it we should treat the action name as a reserved name to avoid having a id and action with the same name # This is a resource named "upload" /storagepools/upload # This is an action upload on storagepools collection /storagepools/upload I strongly suggest to adopt option 1. The if/else will not be a big deal as we can do the functions self contained for easy maintenance.
On 19-08-2014 16:17, Aline Manera wrote:
We can use "url" for download and "file" for upload.
What if both parameters are provided?
if "url" in params.keys(): self.download() elif "file" in params.keys(): self.upload()
According to this code, the parameter 'url' overrides 'file'. Should we consider that behavior?
On 08/19/2014 05:20 PM, Crístian Viana wrote:
On 19-08-2014 16:17, Aline Manera wrote:
We can use "url" for download and "file" for upload.
What if both parameters are provided?
Raise an error. keys = params.keys() if "url" in keys and "url" in keys: raise InvalidParameter() if url in keys: return self.download if file in keys: return self.upload()
if "url" in params.keys(): self.download() elif "file" in params.keys(): self.upload()
According to this code, the parameter 'url' overrides 'file'. Should we consider that behavior?
On 2014年08月20日 03:17, Aline Manera wrote:
On 08/19/2014 04:03 PM, Crístian Viana wrote:
On 19-08-2014 13:53, Crístian Viana wrote:
On 19-08-2014 10:49, Aline Manera wrote:
Just one more comment along the others made by Royce.
Yes - download and upload functionality will be available to all pools.
Ok! So I'll update the RFC to support multiple storage pools and to use the Tasks API to handle the pause/resume/cancel operations.
Well, Aline and I found a problem with this proposal. Currently, the REST API always redirects "POST" requests in a Collection to the function "<colletion-name>_create". In our case, if we send "POST /storagepools/<pool-name>/storagevolumes/download", that would be redirected to the backend function "storagevolumes_create". There is no way to redirect those requests to different functions (like, ideally, "storagevolumes_download" and "storagevolumes_upload").
So I see two options from here:
1) Use the first API proposed by Aline: "POST /storagepools/<pool-name>/storagevolumes". By looking at the parameters (i.e. whether 'url' starts with "file://" or "[http|ftp]://") we can tell if the operation is download or upload.
We can use "url" for download and "file" for upload.
if "url" in params.keys(): self.download() elif "file" in params.keys(): self.upload()
2) Remodel the control code to support multiple actions when POST'ing to a Collection.
It will not be possible as we expect to have an resource id instead of an action name. To be able to do it we should treat the action name as a reserved name to avoid having a id and action with the same name
# This is a resource named "upload" /storagepools/upload
# This is an action upload on storagepools collection /storagepools/upload
I strongly suggest to adopt option 1. The if/else will not be a big deal as we can do the functions self contained for easy maintenance.
Yeah, I realize this is an error which actions are always on resource, not on collection, so if we want to adopt this, the API will be: POST /storagepools/pool-name/upload {parameter} I think this would be proper. What do you think?
On 2014年08月20日 14:06, Royce Lv wrote:
On 2014年08月20日 03:17, Aline Manera wrote:
On 08/19/2014 04:03 PM, Crístian Viana wrote:
On 19-08-2014 13:53, Crístian Viana wrote:
On 19-08-2014 10:49, Aline Manera wrote:
Just one more comment along the others made by Royce.
Yes - download and upload functionality will be available to all pools.
Ok! So I'll update the RFC to support multiple storage pools and to use the Tasks API to handle the pause/resume/cancel operations.
Well, Aline and I found a problem with this proposal. Currently, the REST API always redirects "POST" requests in a Collection to the function "<colletion-name>_create". In our case, if we send "POST /storagepools/<pool-name>/storagevolumes/download", that would be redirected to the backend function "storagevolumes_create". There is no way to redirect those requests to different functions (like, ideally, "storagevolumes_download" and "storagevolumes_upload").
So I see two options from here:
1) Use the first API proposed by Aline: "POST /storagepools/<pool-name>/storagevolumes". By looking at the parameters (i.e. whether 'url' starts with "file://" or "[http|ftp]://") we can tell if the operation is download or upload.
We can use "url" for download and "file" for upload.
if "url" in params.keys(): self.download() elif "file" in params.keys(): self.upload()
2) Remodel the control code to support multiple actions when POST'ing to a Collection.
It will not be possible as we expect to have an resource id instead of an action name. To be able to do it we should treat the action name as a reserved name to avoid having a id and action with the same name
# This is a resource named "upload" /storagepools/upload
# This is an action upload on storagepools collection /storagepools/upload
I strongly suggest to adopt option 1. The if/else will not be a big deal as we can do the functions self contained for easy maintenance.
Yeah, I realize this is an error which actions are always on resource, not on collection, so if we want to adopt this, the API will be: POST /storagepools/pool-name/upload {parameter}
I think this would be proper. What do you think? After a second thought, I think POST and return resource may be the only reasonable option. It returns the volume information (may be future task id). We need to consider how to make the model part more clean in implementation.
_______________________________________________ Kimchi-devel mailing list Kimchi-devel@ovirt.org http://lists.ovirt.org/mailman/listinfo/kimchi-devel
participants (3)
-
Aline Manera -
Crístian Viana -
Royce Lv