[Engine-devel] Proposal for support ISO domain on other types of file based storage.
This is a multi-part message in MIME format. --------------070305010507040809030004 Content-Type: text/plain; charset=ISO-8859-1; format=flowed Content-Transfer-Encoding: 7bit Hi guys, Currently, ISO domain is only supported on NFS storage. It could improve the ease of use if it allows other types of file based storage to store ISO images. After an investigation, I found there's not any restriction on this idea. So the whole work is removing the limitation on engine side. That means engine should allow ISO domain could have different storage type from the data center it's attached, like what we do with nfs ISO domain in SAN DC. I start this idea with localfs. I know local storage can't be seen in cluster level. But it also provides a choice if no NFS available. VMs can be created on the host which has the ISO repo, and then be migrated to any other host in the cluster. I have done the initial patches: allow creation ISO domain on localfs [1] and support import ISO domain on localfs [2] I don't have much experience in java/j2ee/web development and engine architecture. The patches just work for me. I am not sure if it will bring some potential problems. So any feedback on the patch or the idea will be appreciated very much. Mark. [1] http://gerrit.ovirt.org/#/c/12687/ [2] http://gerrit.ovirt.org/#/c/12916/ --------------070305010507040809030004 Content-Type: text/html; charset=ISO-8859-1 Content-Transfer-Encoding: 7bit <html> <head> <meta http-equiv="content-type" content="text/html; charset=ISO-8859-1"> </head> <body bgcolor="#FFFFFF" text="#000000"> Hi guys, <br> <br> Currently, ISO domain is only supported on NFS storage. It could improve the ease of use if it allows other types<br> of file based storage to store ISO images. After an investigation, I found there's not any restriction on this idea.<br> So the whole work is removing the limitation on engine side. That means engine should allow ISO domain could <br> have different storage type from the data center it's attached, like what we do with nfs ISO domain in SAN DC.<br> <br> I start this idea with localfs. I know local storage can't be seen in cluster level. But it also provides a choice if no<br> NFS available. VMs can be created on the host which has the ISO repo, and then be migrated to any other host in the cluster.<br> I have done the initial patches: allow creation ISO domain on localfs [1] and support import ISO domain on localfs [2]<br> I don't have much experience in java/j2ee/web development and engine architecture. The patches just work for me.<br> I am not sure if it will bring some potential problems. So any feedback on the patch or the idea will be appreciated very much.<br> <br> <br> Mark.<br> <meta http-equiv="content-type" content="text/html; charset=ISO-8859-1"> <br> [1] <a href="http://gerrit.ovirt.org/#/c/12687/">http://gerrit.ovirt.org/#/c/12687/</a><br> [2] <meta http-equiv="content-type" content="text/html; charset=ISO-8859-1"> <a href="http://gerrit.ovirt.org/#/c/12916/">http://gerrit.ovirt.org/#/c/12916/<br> </a> </body> </html> --------------070305010507040809030004--
----- Original Message -----
Hi guys,
Currently, ISO domain is only supported on NFS storage. It could improve the ease of use if it allows other types of file based storage to store ISO images. After an investigation, I found there's not any restriction on this idea. So the whole work is removing the limitation on engine side. That means engine should allow ISO domain could have different storage type from the data center it's attached, like what we do with nfs ISO domain in SAN DC.
I start this idea with localfs. I know local storage can't be seen in cluster level. But it also provides a choice if no NFS available. VMs can be created on the host which has the ISO repo, and then be migrated to any other host in the cluster. I have done the initial patches: allow creation ISO domain on localfs [1] and support import ISO domain on localfs [2] I don't have much experience in java/j2ee/web development and engine architecture. The patches just work for me. I am not sure if it will bring some potential problems. So any feedback on the patch or the idea will be appreciated very much.
Haven't looked at the patches yet, but wrt the idea, I agree on the need (being able to attach ISOs from anywhere and not just nfs) but I think the way to do this should be by getting rid of the ISO domain type altogether. Basically what we need is: 1. a way to connect to file based storage (let's leave block aside for now) - this already exists via the connectStorageServer verb 2. a way to list and present a file system tree in gui (give an arbitrary path to vdsm and list content) and possibly filter results by type (vfd, iso) - does not exist today. Possibly some security aspects here that need hashing out. 3. a way to specify a path to a file when attaching an iso/vfd to a VM - this is the way it works today This would devoid the need for isoUploader and allow users to simply manage an nfs export with files. Next step would be to make connectStorageServer support httpfs [1] and then we'd be able to mount ISOs directly over http (hopefully this would be sufficient to support ISOs stored on S3, swift, glance, etc). [1] http://httpfs.sourceforge.net/
Mark.
[1] http://gerrit.ovirt.org/#/c/12687/ [2] http://gerrit.ovirt.org/#/c/12916/
_______________________________________________ Engine-devel mailing list Engine-devel@ovirt.org http://lists.ovirt.org/mailman/listinfo/engine-devel
On Sun 17 Mar 2013 10:12:55 PM CST, Ayal Baron wrote:
----- Original Message -----
Hi guys,
Currently, ISO domain is only supported on NFS storage. It could improve the ease of use if it allows other types of file based storage to store ISO images. After an investigation, I found there's not any restriction on this idea. So the whole work is removing the limitation on engine side. That means engine should allow ISO domain could have different storage type from the data center it's attached, like what we do with nfs ISO domain in SAN DC.
I start this idea with localfs. I know local storage can't be seen in cluster level. But it also provides a choice if no NFS available. VMs can be created on the host which has the ISO repo, and then be migrated to any other host in the cluster. I have done the initial patches: allow creation ISO domain on localfs [1] and support import ISO domain on localfs [2] I don't have much experience in java/j2ee/web development and engine architecture. The patches just work for me. I am not sure if it will bring some potential problems. So any feedback on the patch or the idea will be appreciated very much.
Haven't looked at the patches yet, but wrt the idea, I agree on the need (being able to attach ISOs from anywhere and not just nfs) but I think the way to do this should be by getting rid of the ISO domain type altogether.
I think ISO domain on localfs is useful for a simple setup or demo, such as oVirt all-in-one.
Basically what we need is: 1. a way to connect to file based storage (let's leave block aside for now) - this already exists via the connectStorageServer verb 2. a way to list and present a file system tree in gui (give an arbitrary path to vdsm and list content) and possibly filter results by type (vfd, iso) - does not exist today. Possibly some security aspects here that need hashing out. 3. a way to specify a path to a file when attaching an iso/vfd to a VM - this is the way it works today
This would devoid the need for isoUploader and allow users to simply manage an nfs export with files. Next step would be to make connectStorageServer support httpfs [1] and then we'd be able to mount ISOs directly over http (hopefully this would be sufficient to support ISOs stored on S3, swift, glance, etc).
Actually, we could use the qemu curl backend image support directly. That means we don't need mount the place storing ISO images. We can just maintain a list of ISO image with its link, which could be http, ftp and ssh.
[1] http://httpfs.sourceforge.net/
Mark.
[1] http://gerrit.ovirt.org/#/c/12687/ [2] http://gerrit.ovirt.org/#/c/12916/
_______________________________________________ Engine-devel mailing list Engine-devel@ovirt.org http://lists.ovirt.org/mailman/listinfo/engine-devel
----- Original Message -----
On Sun 17 Mar 2013 10:12:55 PM CST, Ayal Baron wrote:
----- Original Message -----
Hi guys,
Currently, ISO domain is only supported on NFS storage. It could improve the ease of use if it allows other types of file based storage to store ISO images. After an investigation, I found there's not any restriction on this idea. So the whole work is removing the limitation on engine side. That means engine should allow ISO domain could have different storage type from the data center it's attached, like what we do with nfs ISO domain in SAN DC.
I start this idea with localfs. I know local storage can't be seen in cluster level. But it also provides a choice if no NFS available. VMs can be created on the host which has the ISO repo, and then be migrated to any other host in the cluster. I have done the initial patches: allow creation ISO domain on localfs [1] and support import ISO domain on localfs [2] I don't have much experience in java/j2ee/web development and engine architecture. The patches just work for me. I am not sure if it will bring some potential problems. So any feedback on the patch or the idea will be appreciated very much.
Haven't looked at the patches yet, but wrt the idea, I agree on the need (being able to attach ISOs from anywhere and not just nfs) but I think the way to do this should be by getting rid of the ISO domain type altogether.
I think ISO domain on localfs is useful for a simple setup or demo, such as oVirt all-in-one.
As I said above, I totally agree that we need to be able to attach ISOs on local host, what I'm saying is that we don't need a local ISO *domain*. All we need is the ability to list the content of a directory on the local host (to be able to choose the ISO) and attach it to the VM. Since the attach part already exists then the gap is just the listing issue.
Basically what we need is: 1. a way to connect to file based storage (let's leave block aside for now) - this already exists via the connectStorageServer verb 2. a way to list and present a file system tree in gui (give an arbitrary path to vdsm and list content) and possibly filter results by type (vfd, iso) - does not exist today. Possibly some security aspects here that need hashing out. 3. a way to specify a path to a file when attaching an iso/vfd to a VM - this is the way it works today
This would devoid the need for isoUploader and allow users to simply manage an nfs export with files. Next step would be to make connectStorageServer support httpfs [1] and then we'd be able to mount ISOs directly over http (hopefully this would be sufficient to support ISOs stored on S3, swift, glance, etc).
Actually, we could use the qemu curl backend image support directly. That means we don't need mount the place storing ISO images. We can just maintain a list of ISO image with its link, which could be http, ftp and ssh.
even better!
[1] http://httpfs.sourceforge.net/
Mark.
[1] http://gerrit.ovirt.org/#/c/12687/ [2] http://gerrit.ovirt.org/#/c/12916/
_______________________________________________ Engine-devel mailing list Engine-devel@ovirt.org http://lists.ovirt.org/mailman/listinfo/engine-devel
Mark Wu :
On Sun 17 Mar 2013 10:12:55 PM CST, Ayal Baron wrote:
----- Original Message -----
Hi guys,
Currently, ISO domain is only supported on NFS storage. It could improve the ease of use if it allows other types of file based storage to store ISO images. After an investigation, I found there's not any restriction on this idea. So the whole work is removing the limitation on engine side. That means engine should allow ISO domain could have different storage type from the data center it's attached, like what we do with nfs ISO domain in SAN DC.
I start this idea with localfs. I know local storage can't be seen in cluster level. But it also provides a choice if no NFS available. VMs can be created on the host which has the ISO repo, and then be migrated to any other host in the cluster. I have done the initial patches: allow creation ISO domain on localfs [1] and support import ISO domain on localfs [2] I don't have much experience in java/j2ee/web development and engine architecture. The patches just work for me. I am not sure if it will bring some potential problems. So any feedback on the patch or the idea will be appreciated very much.
Haven't looked at the patches yet, but wrt the idea, I agree on the need (being able to attach ISOs from anywhere and not just nfs) but I think the way to do this should be by getting rid of the ISO domain type altogether.
I think ISO domain on localfs is useful for a simple setup or demo, such as oVirt all-in-one.
Basically what we need is: 1. a way to connect to file based storage (let's leave block aside for now) - this already exists via the connectStorageServer verb 2. a way to list and present a file system tree in gui (give an arbitrary path to vdsm and list content) and possibly filter results by type (vfd, iso) - does not exist today. Possibly some security aspects here that need hashing out. 3. a way to specify a path to a file when attaching an iso/vfd to a VM - this is the way it works today
This would devoid the need for isoUploader and allow users to simply manage an nfs export with files. Next step would be to make connectStorageServer support httpfs [1] and then we'd be able to mount ISOs directly over http (hopefully this would be sufficient to support ISOs stored on S3, swift, glance, etc).
Actually, we could use the qemu curl backend image support directly. That means we don't need mount the place storing ISO images. We can just maintain a list of ISO image with its link, which could be http, ftp and ssh.
That will be fine to start a VM on a existing extern ISO image. I also would like to maintain a ISO image cache on the local host to avoid to re-streaming the ISO image from the ISO image repositories every time. That will be helpful for people who is suffered from the network bottleneck.
[1] http://httpfs.sourceforge.net/
Mark.
[1] http://gerrit.ovirt.org/#/c/12687/ [2] http://gerrit.ovirt.org/#/c/12916/
_______________________________________________ Engine-devel mailing list Engine-devel@ovirt.org http://lists.ovirt.org/mailman/listinfo/engine-devel
_______________________________________________ Engine-devel mailing list Engine-devel@ovirt.org http://lists.ovirt.org/mailman/listinfo/engine-devel
-- --- 舒明 Shu Ming Open Virtualization Engineerning; CSTL, IBM Corp. Tel: 86-10-82451626 Tieline: 9051626 E-mail: shuming@cn.ibm.com or shuming@linux.vnet.ibm.com Address: 3/F Ring Building, ZhongGuanCun Software Park, Haidian District, Beijing 100193, PRC
On 03/18/2013 02:51 PM, Ayal Baron wrote:
----- Original Message -----
On Sun 17 Mar 2013 10:12:55 PM CST, Ayal Baron wrote:
----- Original Message -----
Hi guys,
Currently, ISO domain is only supported on NFS storage. It could improve the ease of use if it allows other types of file based storage to store ISO images. After an investigation, I found there's not any restriction on this idea. So the whole work is removing the limitation on engine side. That means engine should allow ISO domain could have different storage type from the data center it's attached, like what we do with nfs ISO domain in SAN DC.
I start this idea with localfs. I know local storage can't be seen in cluster level. But it also provides a choice if no NFS available. VMs can be created on the host which has the ISO repo, and then be migrated to any other host in the cluster. I have done the initial patches: allow creation ISO domain on localfs [1] and support import ISO domain on localfs [2] I don't have much experience in java/j2ee/web development and engine architecture. The patches just work for me. I am not sure if it will bring some potential problems. So any feedback on the patch or the idea will be appreciated very much. Haven't looked at the patches yet, but wrt the idea, I agree on the need (being able to attach ISOs from anywhere and not just nfs) but I think the way to do this should be by getting rid of the ISO domain type altogether.
I think ISO domain on localfs is useful for a simple setup or demo, such as oVirt all-in-one. As I said above, I totally agree that we need to be able to attach ISOs on local host, what I'm saying is that we don't need a local ISO *domain*. All we need is the ability to list the content of a directory on the local host (to be able to choose the ISO) and attach it to the VM. Since the attach part already exists then the gap is just the listing issue.
Got it. Thanks for your explanation!
Basically what we need is: 1. a way to connect to file based storage (let's leave block aside for now) - this already exists via the connectStorageServer verb 2. a way to list and present a file system tree in gui (give an arbitrary path to vdsm and list content) and possibly filter results by type (vfd, iso) - does not exist today. Possibly some security aspects here that need hashing out. 3. a way to specify a path to a file when attaching an iso/vfd to a VM - this is the way it works today
This would devoid the need for isoUploader and allow users to simply manage an nfs export with files. Next step would be to make connectStorageServer support httpfs [1] and then we'd be able to mount ISOs directly over http (hopefully this would be sufficient to support ISOs stored on S3, swift, glance, etc). Actually, we could use the qemu curl backend image support directly. That means we don't need mount the place storing ISO images. We can just maintain a list of ISO image with its link, which could be http, ftp and ssh. even better!
[1] http://httpfs.sourceforge.net/
Mark.
[1] http://gerrit.ovirt.org/#/c/12687/ [2] http://gerrit.ovirt.org/#/c/12916/
_______________________________________________ Engine-devel mailing list Engine-devel@ovirt.org http://lists.ovirt.org/mailman/listinfo/engine-devel
----- Original Message -----
Mark Wu :
On Sun 17 Mar 2013 10:12:55 PM CST, Ayal Baron wrote:
----- Original Message -----
Hi guys,
Currently, ISO domain is only supported on NFS storage. It could improve the ease of use if it allows other types of file based storage to store ISO images. After an investigation, I found there's not any restriction on this idea. So the whole work is removing the limitation on engine side. That means engine should allow ISO domain could have different storage type from the data center it's attached, like what we do with nfs ISO domain in SAN DC.
I start this idea with localfs. I know local storage can't be seen in cluster level. But it also provides a choice if no NFS available. VMs can be created on the host which has the ISO repo, and then be migrated to any other host in the cluster. I have done the initial patches: allow creation ISO domain on localfs [1] and support import ISO domain on localfs [2] I don't have much experience in java/j2ee/web development and engine architecture. The patches just work for me. I am not sure if it will bring some potential problems. So any feedback on the patch or the idea will be appreciated very much.
Haven't looked at the patches yet, but wrt the idea, I agree on the need (being able to attach ISOs from anywhere and not just nfs) but I think the way to do this should be by getting rid of the ISO domain type altogether.
I think ISO domain on localfs is useful for a simple setup or demo, such as oVirt all-in-one.
Basically what we need is: 1. a way to connect to file based storage (let's leave block aside for now) - this already exists via the connectStorageServer verb 2. a way to list and present a file system tree in gui (give an arbitrary path to vdsm and list content) and possibly filter results by type (vfd, iso) - does not exist today. Possibly some security aspects here that need hashing out. 3. a way to specify a path to a file when attaching an iso/vfd to a VM - this is the way it works today
This would devoid the need for isoUploader and allow users to simply manage an nfs export with files. Next step would be to make connectStorageServer support httpfs [1] and then we'd be able to mount ISOs directly over http (hopefully this would be sufficient to support ISOs stored on S3, swift, glance, etc).
Actually, we could use the qemu curl backend image support directly. That means we don't need mount the place storing ISO images. We can just maintain a list of ISO image with its link, which could be http, ftp and ssh.
That will be fine to start a VM on a existing extern ISO image. I also would like to maintain a ISO image cache on the local host to avoid to re-streaming the ISO image from the ISO image repositories every time. That will be helpful for people who is suffered from the network bottleneck.
We have a similar requirement for glance support (not limited to ISOs, rather to all read-only images) so that should be tackled with a broader solution.
[1] http://httpfs.sourceforge.net/
Mark.
[1] http://gerrit.ovirt.org/#/c/12687/ [2] http://gerrit.ovirt.org/#/c/12916/
_______________________________________________ Engine-devel mailing list Engine-devel@ovirt.org http://lists.ovirt.org/mailman/listinfo/engine-devel
_______________________________________________ Engine-devel mailing list Engine-devel@ovirt.org http://lists.ovirt.org/mailman/listinfo/engine-devel
-- --- 舒明 Shu Ming Open Virtualization Engineerning; CSTL, IBM Corp. Tel: 86-10-82451626 Tieline: 9051626 E-mail: shuming@cn.ibm.com or shuming@linux.vnet.ibm.com Address: 3/F Ring Building, ZhongGuanCun Software Park, Haidian District, Beijing 100193, PRC
On 03/18/2013 03:11 PM, Shu Ming wrote:
Mark Wu :
On Sun 17 Mar 2013 10:12:55 PM CST, Ayal Baron wrote:
----- Original Message -----
Hi guys,
Currently, ISO domain is only supported on NFS storage. It could improve the ease of use if it allows other types of file based storage to store ISO images. After an investigation, I found there's not any restriction on this idea. So the whole work is removing the limitation on engine side. That means engine should allow ISO domain could have different storage type from the data center it's attached, like what we do with nfs ISO domain in SAN DC.
I start this idea with localfs. I know local storage can't be seen in cluster level. But it also provides a choice if no NFS available. VMs can be created on the host which has the ISO repo, and then be migrated to any other host in the cluster. I have done the initial patches: allow creation ISO domain on localfs [1] and support import ISO domain on localfs [2] I don't have much experience in java/j2ee/web development and engine architecture. The patches just work for me. I am not sure if it will bring some potential problems. So any feedback on the patch or the idea will be appreciated very much.
Haven't looked at the patches yet, but wrt the idea, I agree on the need (being able to attach ISOs from anywhere and not just nfs) but I think the way to do this should be by getting rid of the ISO domain type altogether.
I think ISO domain on localfs is useful for a simple setup or demo, such as oVirt all-in-one.
Basically what we need is: 1. a way to connect to file based storage (let's leave block aside for now) - this already exists via the connectStorageServer verb 2. a way to list and present a file system tree in gui (give an arbitrary path to vdsm and list content) and possibly filter results by type (vfd, iso) - does not exist today. Possibly some security aspects here that need hashing out. 3. a way to specify a path to a file when attaching an iso/vfd to a VM - this is the way it works today
This would devoid the need for isoUploader and allow users to simply manage an nfs export with files. Next step would be to make connectStorageServer support httpfs [1] and then we'd be able to mount ISOs directly over http (hopefully this would be sufficient to support ISOs stored on S3, swift, glance, etc).
Actually, we could use the qemu curl backend image support directly. That means we don't need mount the place storing ISO images. We can just maintain a list of ISO image with its link, which could be http, ftp and ssh.
That will be fine to start a VM on a existing extern ISO image. I also would like to maintain a ISO image cache on the local host to avoid to re-streaming the ISO image from the ISO image repositories every time. That will be helpful for people who is suffered from the network bottleneck. I think this problem could be avoided by using VM template. It will significantly reduce the number of access on remote ISO images.
[1] http://httpfs.sourceforge.net/
Mark.
[1] http://gerrit.ovirt.org/#/c/12687/ [2] http://gerrit.ovirt.org/#/c/12916/
_______________________________________________ Engine-devel mailing list Engine-devel@ovirt.org http://lists.ovirt.org/mailman/listinfo/engine-devel
_______________________________________________ Engine-devel mailing list Engine-devel@ovirt.org http://lists.ovirt.org/mailman/listinfo/engine-devel
This is a multi-part message in MIME format. --------------090705080902000503070107 Content-Type: text/plain; charset=UTF-8; format=flowed Content-Transfer-Encoding: 8bit On 03/18/2013 04:43 PM, Ayal Baron wrote:
----- Original Message -----
Mark Wu :
On Sun 17 Mar 2013 10:12:55 PM CST, Ayal Baron wrote:
Hi guys,
Currently, ISO domain is only supported on NFS storage. It could improve the ease of use if it allows other types of file based storage to store ISO images. After an investigation, I found there's not any restriction on this idea. So the whole work is removing the limitation on engine side. That means engine should allow ISO domain could have different storage type from the data center it's attached, like what we do with nfs ISO domain in SAN DC.
I start this idea with localfs. I know local storage can't be seen in cluster level. But it also provides a choice if no NFS available. VMs can be created on the host which has the ISO repo, and then be migrated to any other host in the cluster. I have done the initial patches: allow creation ISO domain on localfs [1] and support import ISO domain on localfs [2] I don't have much experience in java/j2ee/web development and engine architecture. The patches just work for me. I am not sure if it will bring some potential problems. So any feedback on the patch or the idea will be appreciated very much. Haven't looked at the patches yet, but wrt the idea, I agree on
----- Original Message ----- the need (being able to attach ISOs from anywhere and not just nfs) but I think the way to do this should be by getting rid of the ISO domain type altogether.
I think ISO domain on localfs is useful for a simple setup or demo, such as oVirt all-in-one.
Basically what we need is: 1. a way to connect to file based storage (let's leave block aside for now) - this already exists via the connectStorageServer verb 2. a way to list and present a file system tree in gui (give an arbitrary path to vdsm and list content) and possibly filter results by type (vfd, iso) - does not exist today. Possibly some security aspects here that need hashing out. 3. a way to specify a path to a file when attaching an iso/vfd to a VM - this is the way it works today
This would devoid the need for isoUploader and allow users to simply manage an nfs export with files. Next step would be to make connectStorageServer support httpfs [1] and then we'd be able to mount ISOs directly over http (hopefully this would be sufficient to support ISOs stored on S3, swift, glance, etc). Actually, we could use the qemu curl backend image support directly. That means we don't need mount the place storing ISO images. We can just maintain a list of ISO image with its link, which could be http, ftp and ssh. That will be fine to start a VM on a existing extern ISO image. I also would like to maintain a ISO image cache on the local host to avoid to re-streaming the ISO image from the ISO image repositories every time. That will be helpful for people who is suffered from the network bottleneck. We have a similar requirement for glance support (not limited to ISOs, rather to all read-only images) so that should be tackled with a broader solution.
In that case, I suppose qemu's copy-on-read + httpfs (you mentioned above) could help. It can avoid accessing the same backing file sectors repeatedly. But the copied content can't be shared by multiple VM using the same backend image.
[1] http://httpfs.sourceforge.net/
Mark.
[1] http://gerrit.ovirt.org/#/c/12687/ [2] http://gerrit.ovirt.org/#/c/12916/
_______________________________________________ Engine-devel mailing list Engine-devel@ovirt.org http://lists.ovirt.org/mailman/listinfo/engine-devel
_______________________________________________ Engine-devel mailing list Engine-devel@ovirt.org http://lists.ovirt.org/mailman/listinfo/engine-devel
-- --- 舒明 Shu Ming Open Virtualization Engineerning; CSTL, IBM Corp. Tel: 86-10-82451626 Tieline: 9051626 E-mail: shuming@cn.ibm.com or shuming@linux.vnet.ibm.com Address: 3/F Ring Building, ZhongGuanCun Software Park, Haidian District, Beijing 100193, PRC
--------------090705080902000503070107 Content-Type: text/html; charset=UTF-8 Content-Transfer-Encoding: 8bit <html> <head> <meta content="text/html; charset=UTF-8" http-equiv="Content-Type"> </head> <body bgcolor="#FFFFFF" text="#000000"> <div class="moz-cite-prefix">On 03/18/2013 04:43 PM, Ayal Baron wrote:<br> </div> <blockquote cite="mid:1599625604.9482831.1363596188030.JavaMail.root@redhat.com" type="cite"> <pre wrap=""> ----- Original Message ----- </pre> <blockquote type="cite"> <pre wrap="">Mark Wu : </pre> <blockquote type="cite"> <pre wrap="">On Sun 17 Mar 2013 10:12:55 PM CST, Ayal Baron wrote: </pre> <blockquote type="cite"> <pre wrap=""> ----- Original Message ----- </pre> <blockquote type="cite"> <pre wrap=""> Hi guys, Currently, ISO domain is only supported on NFS storage. It could improve the ease of use if it allows other types of file based storage to store ISO images. After an investigation, I found there's not any restriction on this idea. So the whole work is removing the limitation on engine side. That means engine should allow ISO domain could have different storage type from the data center it's attached, like what we do with nfs ISO domain in SAN DC. I start this idea with localfs. I know local storage can't be seen in cluster level. But it also provides a choice if no NFS available. VMs can be created on the host which has the ISO repo, and then be migrated to any other host in the cluster. I have done the initial patches: allow creation ISO domain on localfs [1] and support import ISO domain on localfs [2] I don't have much experience in java/j2ee/web development and engine architecture. The patches just work for me. I am not sure if it will bring some potential problems. So any feedback on the patch or the idea will be appreciated very much. </pre> </blockquote> <pre wrap=""> Haven't looked at the patches yet, but wrt the idea, I agree on the need (being able to attach ISOs from anywhere and not just nfs) but I think the way to do this should be by getting rid of the ISO domain type altogether. </pre> </blockquote> <pre wrap=""> I think ISO domain on localfs is useful for a simple setup or demo, such as oVirt all-in-one. </pre> <blockquote type="cite"> <pre wrap="">Basically what we need is: 1. a way to connect to file based storage (let's leave block aside for now) - this already exists via the connectStorageServer verb 2. a way to list and present a file system tree in gui (give an arbitrary path to vdsm and list content) and possibly filter results by type (vfd, iso) - does not exist today. Possibly some security aspects here that need hashing out. 3. a way to specify a path to a file when attaching an iso/vfd to a VM - this is the way it works today This would devoid the need for isoUploader and allow users to simply manage an nfs export with files. Next step would be to make connectStorageServer support httpfs [1] and then we'd be able to mount ISOs directly over http (hopefully this would be sufficient to support ISOs stored on S3, swift, glance, etc). </pre> </blockquote> <pre wrap=""> Actually, we could use the qemu curl backend image support directly. That means we don't need mount the place storing ISO images. We can just maintain a list of ISO image with its link, which could be http, ftp and ssh. </pre> </blockquote> <pre wrap=""> That will be fine to start a VM on a existing extern ISO image. I also would like to maintain a ISO image cache on the local host to avoid to re-streaming the ISO image from the ISO image repositories every time. That will be helpful for people who is suffered from the network bottleneck. </pre> </blockquote> <pre wrap=""> We have a similar requirement for glance support (not limited to ISOs, rather to all read-only images) so that should be tackled with a broader solution.</pre> </blockquote> In that case, I suppose qemu's copy-on-read + httpfs (you mentioned above) could help. It <meta http-equiv="content-type" content="text/html; charset=UTF-8"> can avoid accessing the same backing file sectors repeatedly. <br> But the copied content can't be shared by multiple VM using the same backend image.<br> <br> <blockquote cite="mid:1599625604.9482831.1363596188030.JavaMail.root@redhat.com" type="cite"> <pre wrap=""> </pre> <blockquote type="cite"> <pre wrap=""> </pre> <blockquote type="cite"> <pre wrap=""> </pre> <blockquote type="cite"> <pre wrap=""> [1] <a class="moz-txt-link-freetext" href="http://httpfs.sourceforge.net/">http://httpfs.sourceforge.net/</a> </pre> <blockquote type="cite"> <pre wrap=""> Mark. [1] <a class="moz-txt-link-freetext" href="http://gerrit.ovirt.org/#/c/12687/">http://gerrit.ovirt.org/#/c/12687/</a> [2] <a class="moz-txt-link-freetext" href="http://gerrit.ovirt.org/#/c/12916/">http://gerrit.ovirt.org/#/c/12916/</a> _______________________________________________ Engine-devel mailing list <a class="moz-txt-link-abbreviated" href="mailto:Engine-devel@ovirt.org">Engine-devel@ovirt.org</a> <a class="moz-txt-link-freetext" href="http://lists.ovirt.org/mailman/listinfo/engine-devel">http://lists.ovirt.org/mailman/listinfo/engine-devel</a> </pre> </blockquote> <pre wrap=""> </pre> </blockquote> <pre wrap=""> _______________________________________________ Engine-devel mailing list <a class="moz-txt-link-abbreviated" href="mailto:Engine-devel@ovirt.org">Engine-devel@ovirt.org</a> <a class="moz-txt-link-freetext" href="http://lists.ovirt.org/mailman/listinfo/engine-devel">http://lists.ovirt.org/mailman/listinfo/engine-devel</a> </pre> </blockquote> <pre wrap=""> -- --- 舒明 Shu Ming Open Virtualization Engineerning; CSTL, IBM Corp. Tel: 86-10-82451626 Tieline: 9051626 E-mail: <a class="moz-txt-link-abbreviated" href="mailto:shuming@cn.ibm.com">shuming@cn.ibm.com</a> or <a class="moz-txt-link-abbreviated" href="mailto:shuming@linux.vnet.ibm.com">shuming@linux.vnet.ibm.com</a> Address: 3/F Ring Building, ZhongGuanCun Software Park, Haidian District, Beijing 100193, PRC </pre> </blockquote> <pre wrap=""> </pre> </blockquote> <br> </body> </html> --------------090705080902000503070107--
----- Original Message -----
On 03/18/2013 04:43 PM, Ayal Baron wrote:
----- Original Message -----
Mark Wu :
On Sun 17 Mar 2013 10:12:55 PM CST, Ayal Baron wrote:
----- Original Message -----
Hi guys,
Currently, ISO domain is only supported on NFS storage. It could improve the ease of use if it allows other types of file based storage to store ISO images. After an investigation, I found there's not any restriction on this idea. So the whole work is removing the limitation on engine side. That means engine should allow ISO domain could have different storage type from the data center it's attached, like what we do with nfs ISO domain in SAN DC.
I start this idea with localfs. I know local storage can't be seen in cluster level. But it also provides a choice if no NFS available. VMs can be created on the host which has the ISO repo, and then be migrated to any other host in the cluster. I have done the initial patches: allow creation ISO domain on localfs [1] and support import ISO domain on localfs [2] I don't have much experience in java/j2ee/web development and engine architecture. The patches just work for me. I am not sure if it will bring some potential problems. So any feedback on the patch or the idea will be appreciated very much. Haven't looked at the patches yet, but wrt the idea, I agree on the need (being able to attach ISOs from anywhere and not just nfs) but I think the way to do this should be by getting rid of the ISO domain type altogether. I think ISO domain on localfs is useful for a simple setup or demo, such as oVirt all-in-one.
Basically what we need is: 1. a way to connect to file based storage (let's leave block aside for now) - this already exists via the connectStorageServer verb 2. a way to list and present a file system tree in gui (give an arbitrary path to vdsm and list content) and possibly filter results by type (vfd, iso) - does not exist today. Possibly some security aspects here that need hashing out. 3. a way to specify a path to a file when attaching an iso/vfd to a VM - this is the way it works today
This would devoid the need for isoUploader and allow users to simply manage an nfs export with files. Next step would be to make connectStorageServer support httpfs [1] and then we'd be able to mount ISOs directly over http (hopefully this would be sufficient to support ISOs stored on S3, swift, glance, etc). Actually, we could use the qemu curl backend image support directly. That means we don't need mount the place storing ISO images. We can just maintain a list of ISO image with its link, which could be http, ftp and ssh. That will be fine to start a VM on a existing extern ISO image. I also would like to maintain a ISO image cache on the local host to avoid to re-streaming the ISO image from the ISO image repositories every time. That will be helpful for people who is suffered from the network bottleneck. We have a similar requirement for glance support (not limited to ISOs, rather to all read-only images) so that should be tackled with a broader solution. In that case, I suppose qemu's copy-on-read + httpfs (you mentioned above) could help. It can avoid accessing the same backing file sectors repeatedly. But the copied content can't be shared by multiple VM using the same backend image.
Sounds like an rfe for qemu. Basically we need to try and split the storage part in qemu to a separate process to enable such capabilities.
[1] http://httpfs.sourceforge.net/
Mark.
[1] http://gerrit.ovirt.org/#/c/12687/ [2] http://gerrit.ovirt.org/#/c/12916/ _______________________________________________ Engine-devel mailing list Engine-devel@ovirt.org http://lists.ovirt.org/mailman/listinfo/engine-devel _______________________________________________ Engine-devel mailing list Engine-devel@ovirt.org http://lists.ovirt.org/mailman/listinfo/engine-devel -- --- 舒明 Shu Ming Open Virtualization Engineerning; CSTL, IBM Corp. Tel: 86-10-82451626 Tieline: 9051626 E-mail: shuming@cn.ibm.com or shuming@linux.vnet.ibm.com Address: 3/F Ring Building, ZhongGuanCun Software Park, Haidian District, Beijing 100193, PRC
participants (3)
-
Ayal Baron -
Mark Wu -
Shu Ming