Feature: enhanced OVA support
Hi everyone, We would like to share our plan for extending the currently provided support for OVA files with: 1. Support for uploading OVA. 2. Support for exporting a VM/template as OVA. 3. Support for importing OVA that was generated by oVirt (today, we only support those that are VMware-compatible). 4. Support for downloading OVA. This can be found on the feature page <http://www.ovirt.org/develop/release-management/features/virt/enhance-import-export-with-ova/> . Your feedback and cooperation will be highly appreciated. Thanks, Arik
Hi, Arik Hadas <ahadas@redhat.com> writes:
Hi everyone,
We would like to share our plan for extending the currently provided support for OVA files with: 1. Support for uploading OVA. 2. Support for exporting a VM/template as OVA. 3. Support for importing OVA that was generated by oVirt (today, we only support those that are VMware-compatible). 4. Support for downloading OVA.
This can be found on the feature page.
Your feedback and cooperation will be highly appreciated.
Sounds like a great plan. What's the chance that the export would enable an OVA that could be loaded back into VMware?
Thanks, Arik
-derek -- Derek Atkins 617-623-3745 derek@ihtfp.com www.ihtfp.com Computer and Internet Security Consultant
On Mon, May 15, 2017 at 6:11 PM, Derek Atkins <derek@ihtfp.com> wrote:
Hi,
Arik Hadas <ahadas@redhat.com> writes:
Hi everyone,
We would like to share our plan for extending the currently provided support for OVA files with: 1. Support for uploading OVA. 2. Support for exporting a VM/template as OVA. 3. Support for importing OVA that was generated by oVirt (today, we only support those that are VMware-compatible). 4. Support for downloading OVA.
This can be found on the feature page.
Your feedback and cooperation will be highly appreciated.
Sounds like a great plan.
What's the chance that the export would enable an OVA that could be loaded back into VMware?
I'm afraid that would be too costly. As far as I know, there are pretty good conversion tools for converting VMs from KVM to VMware today [1] (which, btw, I consider to be a strength as Eduardo mentioned before. we certainly not in favor of vendor lock-in). It would only make sense for them to support converting the OVA we produce, much like we extended virt-v2v for converting VMware's OVA not long ago - and I'm pretty sure they won't object to do this, they have all the building blocks - getting a VM as a single archive that complies with the OVA standard should make it easy for them). [1] https://www.youtube.com/watch?v=DJedam7TJWo (the relevant part starts at 14:40)
Thanks, Arik
-derek
-- Derek Atkins 617-623-3745 derek@ihtfp.com www.ihtfp.com Computer and Internet Security Consultant
On Sun, May 14, 2017 at 04:56:56PM +0300, Arik Hadas wrote:
Hi everyone,
We would like to share our plan for extending the currently provided support for OVA files with: 1. Support for uploading OVA. 2. Support for exporting a VM/template as OVA. 3. Support for importing OVA that was generated by oVirt (today, we only support those that are VMware-compatible). 4. Support for downloading OVA.
This can be found on the feature page <http://www.ovirt.org/develop/release-management/features/virt/enhance-import-export-with-ova/> .
Your feedback and cooperation will be highly appreciated.
The plan as stated seems fine, but I have some reservations which I don't think are answered by the page: (1) How will oVirt know the difference between an OVA generated by oVirt and one generated by VMware (or indeed other sources)? A VMware OVF has an XML comment: <!--Generated by VMware ESX Server, [...] --> but not any official metadata that I could see. (By the way, I don't think importing via virt-v2v vs directly will be any quicker. The v2v conversion / device installation takes only a fraction of the time. Most of the time is consumed doing the format conversion from VMDK to qcow2. However you are correct that when you know that the source is oVirt/KVM, you should not run virt-v2v.) (2) I think you're going to have a lot of fun generating OVAs which work on VMware. As Yaniv says, the devices aren't the same so you'd be having to do some virt-v2v -like driver installation / registry modification. Plus the OVF file is essentially a VMware data dump encoded as XML. OVF isn't a real standard. I bet there are a million strange corner cases. Even writing VMDK files is full of pitfalls. VMware has a reasonable V2V import tool (actually their P2V tooling is very decent). Of course it's proprietary, but then so is their hypervisor. Maybe oVirt can drive their tools? Rich. -- Richard Jones, Virtualization Group, Red Hat http://people.redhat.com/~rjones Read my programming and virtualization blog: http://rwmj.wordpress.com Fedora Windows cross-compiler. Compile Windows programs, test, and build Windows installers. Over 100 libraries supported. http://fedoraproject.org/wiki/MinGW
On Mon, May 15, 2017 at 8:05 PM, Richard W.M. Jones <rjones@redhat.com> wrote:
On Sun, May 14, 2017 at 04:56:56PM +0300, Arik Hadas wrote:
Hi everyone,
We would like to share our plan for extending the currently provided support for OVA files with: 1. Support for uploading OVA. 2. Support for exporting a VM/template as OVA. 3. Support for importing OVA that was generated by oVirt (today, we only support those that are VMware-compatible). 4. Support for downloading OVA.
This can be found on the feature page <http://www.ovirt.org/develop/release-management/features/ virt/enhance-import-export-with-ova/> .
Your feedback and cooperation will be highly appreciated.
The plan as stated seems fine, but I have some reservations which I don't think are answered by the page:
(1) How will oVirt know the difference between an OVA generated by oVirt and one generated by VMware (or indeed other sources)? A VMware OVF has an XML comment:
<!--Generated by VMware ESX Server, [...] -->
but not any official metadata that I could see.
So that's something that we have not decided on yet. Indeed, we need some indication of the system that generated the OVA and it makes sense to have it inside the OVF. I thought about a field that is part of the VM configuration, like the "origin" field of VMs in ovirt-engine. Having a comment like you mentioned is also an option.
(By the way, I don't think importing via virt-v2v vs directly will be any quicker. The v2v conversion / device installation takes only a fraction of the time. Most of the time is consumed doing the format conversion from VMDK to qcow2. However you are correct that when you know that the source is oVirt/KVM, you should not run virt-v2v.)
Note that the disks within the OVA will be of type qcow2. So not only that no v2v conversion / device installation is needed, but also no format conversion will be needed on the import and upload flows.
(2) I think you're going to have a lot of fun generating OVAs which work on VMware. As Yaniv says, the devices aren't the same so you'd be having to do some virt-v2v -like driver installation / registry modification. Plus the OVF file is essentially a VMware data dump encoded as XML. OVF isn't a real standard. I bet there are a million strange corner cases. Even writing VMDK files is full of pitfalls.
VMware has a reasonable V2V import tool (actually their P2V tooling is very decent). Of course it's proprietary, but then so is their hypervisor. Maybe oVirt can drive their tools?
No no, that fun is not part of the plan :) The OVAs we'll generate are supposed to contain: 1. OVF - it should be similar to the one virt-v2v generates for oVirt (that is similar to the one we use internally in oVirt for snapshots and for backup within storage domains, i.e., OVF-stores). We will definitely need some extensions, like an indication that the OVA was generated by oVirt. We may make some tweaks here and there, like removing network interfaces from the list of resources. But we already are generally aligned with what the specification says about OVFs. 2. qcow2 disks - thus, no conversion (device installation) and no format conversion will be needed (we may consider to convert them to raw later on, since they are expected to be the base volumes of the disks, not sure it worth the effort though). I will add this to the feature page. We are aware to the fact that with this design, the OVAs could not be directly consumed by others, like VMware. But this could make it easier for them to make the needed conversion - they won't need to query the VM configuration from oVirt and won't need to lookup the disks inside oVirt's storage domains. Anyway, we assume that this conversion is done by other tools.
Rich.
-- Richard Jones, Virtualization Group, Red Hat http://people.redhat.com/~ rjones Read my programming and virtualization blog: http://rwmj.wordpress.com Fedora Windows cross-compiler. Compile Windows programs, test, and build Windows installers. Over 100 libraries supported. http://fedoraproject.org/wiki/MinGW
On Mon, May 15, 2017 at 10:52 PM, Arik Hadas <ahadas@redhat.com> wrote:
On Mon, May 15, 2017 at 8:05 PM, Richard W.M. Jones <rjones@redhat.com> wrote:
On Sun, May 14, 2017 at 04:56:56PM +0300, Arik Hadas wrote:
Hi everyone,
We would like to share our plan for extending the currently provided support for OVA files with: 1. Support for uploading OVA. 2. Support for exporting a VM/template as OVA. 3. Support for importing OVA that was generated by oVirt (today, we only support those that are VMware-compatible). 4. Support for downloading OVA.
This can be found on the feature page <http://www.ovirt.org/develop/release-management/features/vi rt/enhance-import-export-with-ova/> .
Your feedback and cooperation will be highly appreciated.
The plan as stated seems fine, but I have some reservations which I don't think are answered by the page:
(1) How will oVirt know the difference between an OVA generated by oVirt and one generated by VMware (or indeed other sources)? A VMware OVF has an XML comment:
<!--Generated by VMware ESX Server, [...] -->
but not any official metadata that I could see.
So that's something that we have not decided on yet. Indeed, we need some indication of the system that generated the OVA and it makes sense to have it inside the OVF. I thought about a field that is part of the VM configuration, like the "origin" field of VMs in ovirt-engine. Having a comment like you mentioned is also an option.
(By the way, I don't think importing via virt-v2v vs directly will be any quicker. The v2v conversion / device installation takes only a fraction of the time. Most of the time is consumed doing the format conversion from VMDK to qcow2. However you are correct that when you know that the source is oVirt/KVM, you should not run virt-v2v.)
Note that the disks within the OVA will be of type qcow2. So not only that no v2v conversion / device installation is needed, but also no format conversion will be needed on the import and upload flows.
(2) I think you're going to have a lot of fun generating OVAs which work on VMware. As Yaniv says, the devices aren't the same so you'd be having to do some virt-v2v -like driver installation / registry modification. Plus the OVF file is essentially a VMware data dump encoded as XML. OVF isn't a real standard. I bet there are a million strange corner cases. Even writing VMDK files is full of pitfalls.
VMware has a reasonable V2V import tool (actually their P2V tooling is very decent). Of course it's proprietary, but then so is their hypervisor. Maybe oVirt can drive their tools?
No no, that fun is not part of the plan :) The OVAs we'll generate are supposed to contain: 1. OVF - it should be similar to the one virt-v2v generates for oVirt (that is similar to the one we use internally in oVirt for snapshots and for backup within storage domains, i.e., OVF-stores). We will definitely need some extensions, like an indication that the OVA was generated by oVirt. We may make some tweaks here and there, like removing network interfaces from the list of resources. But we already are generally aligned with what the specification says about OVFs.
2. qcow2 disks - thus, no conversion (device installation) and no format conversion will be needed (we may consider to convert them to raw later on, since they are expected to be the base volumes of the disks, not sure it worth the effort though).
2. Run virt-sparsify on the disks.
I wonder if it's worth looking into several options when exporting: 1. Run virt-sysprep with 'harmless' operations such as 'abrt-data,backup-files,crash-data,tmp-files' 3. Convert to compressed qcow2. In most cases it certainly can have a significant saving (1:2 should be reasonable, but some can get 1:6 even) of disk space (and later, file transfer times, etc). If we are already converting, we can also convert from qcow2 to qcow2v3 (compat 0.1 to compat 1.1). Y.
I will add this to the feature page.
We are aware to the fact that with this design, the OVAs could not be directly consumed by others, like VMware. But this could make it easier for them to make the needed conversion - they won't need to query the VM configuration from oVirt and won't need to lookup the disks inside oVirt's storage domains. Anyway, we assume that this conversion is done by other tools.
Rich.
-- Richard Jones, Virtualization Group, Red Hat http://people.redhat.com/~rjones Read my programming and virtualization blog: http://rwmj.wordpress.com Fedora Windows cross-compiler. Compile Windows programs, test, and build Windows installers. Over 100 libraries supported. http://fedoraproject.org/wiki/MinGW
_______________________________________________ Devel mailing list Devel@ovirt.org http://lists.ovirt.org/mailman/listinfo/devel
Great feature! I am glad to see you plan to use the existing imageio framework for transferring data. Will you allow export of VMs from a particular snapshot? I guess that's how you'll have to do it if you want to support export of running VMs. I think you should definitely have a comment in the ovf to indicate that an OVA was generated by a oVirt. People will try to use this new feature to import random OVAs from who knows where. I'd also recommend adding a version to this comment: <!-- Generated by oVirt version 4.1.2 --> or perhaps even a schema version in case you need to deal with compatibility issues in the future. I agree with Yaniv Kaul that we should offer to sparsify the VM to optimize it for export. We should also return compressed data. When exporting, does it make sense to cache the stored OVA file in some sort of ephemeral storage (host local is fine, storage domain may be better) in order to allow the client to resume or restart an interrupted download without having to start from scratch? On Sun, May 14, 2017 at 9:56 AM, Arik Hadas <ahadas@redhat.com> wrote:
Hi everyone,
We would like to share our plan for extending the currently provided support for OVA files with: 1. Support for uploading OVA. 2. Support for exporting a VM/template as OVA. 3. Support for importing OVA that was generated by oVirt (today, we only support those that are VMware-compatible). 4. Support for downloading OVA.
This can be found on the feature page <http://www.ovirt.org/develop/release-management/features/virt/enhance-import-export-with-ova/> .
Your feedback and cooperation will be highly appreciated.
Thanks, Arik
_______________________________________________ Users mailing list Users@ovirt.org http://lists.ovirt.org/mailman/listinfo/users
-- Adam Litke
On Tue, May 16, 2017 at 5:56 PM, Adam Litke <alitke@redhat.com> wrote:
Great feature! I am glad to see you plan to use the existing imageio framework for transferring data.
Yep, and that's the part we need the feedback on the most.
Will you allow export of VMs from a particular snapshot? I guess that's how you'll have to do it if you want to support export of running VMs.
Exactly, that's how we intend to support exporting VMs with no downtime.
I think you should definitely have a comment in the ovf to indicate that an OVA was generated by a oVirt. People will try to use this new feature to import random OVAs from who knows where. I'd also recommend adding a version to this comment: <!-- Generated by oVirt version 4.1.2 --> or perhaps even a schema version in case you need to deal with compatibility issues in the future.
So yeah, there will be some sort of marking - either by a field or a comment. We'll see about that. As for the schema version, it already exists in OVFs generated by oVirt today exactly for that purpose.
I agree with Yaniv Kaul that we should offer to sparsify the VM to optimize it for export. We should also return compressed data. When exporting, does it make sense to cache the stored OVA file in some sort of ephemeral storage (host local is fine, storage domain may be better) in order to allow the client to resume or restart an interrupted download without having to start from scratch?
Well, on the one hand it makes sense and I would expect such a mechanism to be used for image-download as well. On the other hand, we don't support this concept of pausing ongoing operations or resuming interrupted opeations on the engine side, so in the context of ova-download (where the streaming process is comprised of several steps orchestrated by the engine) supporting this may require too much effort.
On Sun, May 14, 2017 at 9:56 AM, Arik Hadas <ahadas@redhat.com> wrote:
Hi everyone,
We would like to share our plan for extending the currently provided support for OVA files with: 1. Support for uploading OVA. 2. Support for exporting a VM/template as OVA. 3. Support for importing OVA that was generated by oVirt (today, we only support those that are VMware-compatible). 4. Support for downloading OVA.
This can be found on the feature page <http://www.ovirt.org/develop/release-management/features/virt/enhance-import-export-with-ova/> .
Your feedback and cooperation will be highly appreciated.
Thanks, Arik
_______________________________________________ Users mailing list Users@ovirt.org http://lists.ovirt.org/mailman/listinfo/users
-- Adam Litke
participants (5)
-
Adam Litke -
Arik Hadas -
Derek Atkins -
Richard W.M. Jones -
Yaniv Kaul