Better late than never - thank you all for the input, it was very useful!
With the ability to measure the collapsed form of a volume chain in the qcow2 format, we managed to simplify and improve the process of creating an OVA significantly. What we do now is:
1. Measure the collapsed qcow2 volumes that would be written to the OVA
2. Create the OVA file accordingly
3. Write the "metadata" of the OVA - headers and OVF, while skipping the reserved places for disks
4. Mount each reserved place for a disk as a loopback device and convert the volume-chain directly to it [1]
This saves us from allocating temporary disks and extra copies.
That change would hopefully land in one of the next updates of oVirt 4.2.
Am 30.05.2018 um 18:14 hat Arik Hadas geschrieben:
> On Wed, May 30, 2018 at 6:33 PM, Kevin Wolf <kwolf@redhat.com> wrote:
> > I think the problem is that we're talking about two different things in
> > one thread. If I understand correctly, what oVirt does today is:
> >
> > 1. qemu-img convert to create a temporary qcow2 image that merges the
> > whole backing chain in a single file
> >
> > 2. tar to create an temporary OVA archive that contains, amongst others,
> > the temporary qcow2 image. This is a second temporary file.
> >
> > 3. Stream this temporary OVA archive over HTTP
> >
>
> Well, today we suggest users to mount a shared storage to multiple hosts
> that reside in different oVirt/RHV deployments so they could export
> VMs/templates as OVAs to that shared storage and import these OVAs from the
> shared storage to a destination deployment. This process involves only #1
> and #2.
>
> The technique you proposed earlier for writing disks directly into an OVA,
> assuming that the target size can be retrieved with 'qemu-img measure',
> sounds like a nice approach to accelerate this process. I think we should
> really consider doing that if that's as easy as it sounds.
Writing the image to a given offset in a file is the example that I gave
further down in the mail:
> > You added another host into the mix, which just receives the image
> > content via NBD and then re-exports it as HTTP. Does this host actually
> > exist or is it the same host where the original images are located?
> >
> > Because if you stay local for this step, there is no need to use NBD at
> > all:
> >
> > $ ./qemu-img measure -O qcow2 ~/images/hd.img
> > required size: 67436544
> > fully allocated size: 67436544
> > $ ./qemu-img create -f file /tmp/test.qcow2 67436544
> > Formatting '/tmp/test.qcow2', fmt=file size=67436544
> > $ ./qemu-img convert -n --target-image-opts ~/images/hd.img
> > driver=raw,file.driver=file,file.filename=/tmp/test.qcow2,offset=65536
> >
> > hexdump verifies that this does the expected thing.
> But #3 is definitely something we are interested in because we expect the
> next step to be exporting the OVAs to a remote instance of Glance that
> serves as a shared repository for the different deployments. Being able to
> stream the collapsed form of a volume chain without writing anything to the
> storage device would be fantastic. I think that even at the expense of
> iterating the chain twice - once to map the structure of the jump tables
> (right?) and once to stream the whole data.
If the target is not a stupid web browser, but something actually
virt-related like Glance, I'm sure it can offer a more suitable protocol
than HTTP? If you could talk NBD to Glance, you'd get rid of the
streaming requirement. I think it would make more sense to invest the
effort there.
Kevin