[ovirt-users] import thin provisioned disks with image upload
Nir Soffer
nsoffer at redhat.com
Thu Dec 7 20:33:29 UTC 2017
On Thu, Dec 7, 2017 at 10:15 AM Gianluca Cecchi <gianluca.cecchi at gmail.com>
wrote:
> On Thu, Dec 7, 2017 at 1:28 AM, Nir Soffer <nsoffer at redhat.com> wrote:
>
>>
>>
>> On Thu, Dec 7, 2017 at 1:22 AM Gianluca Cecchi <gianluca.cecchi at gmail.com>
>> wrote:
>>
>>> On Wed, Dec 6, 2017 at 11:42 PM, Nir Soffer <nsoffer at redhat.com> wrote:
>>>
>>>>
>>>>
>>>>>
>>>>> BTW: I notice that the disk seems preallocated even if original qcow2
>>>>> is thin... is this expected?
>>>>> This obviously also impacts the time to upload (20Gb virtual disk with
>>>>> actual 1.2Gb occupied needs the time equivalent for full 20Gb...)
>>>>>
>>>>
>>>> We upload exactly the file you provided, there is no way we can upload
>>>> 20G from 1.2G file :-)
>>>>
>>>
>>> But the upload process at a medium rate of 40-50MB/s has last about 9
>>> minutes that confirms the 20Gb size
>>> The disk at source has been created as virtio type and format qcow2 from
>>> virt-manager and then only installed a CentOS 7.2 OS with infrastructure
>>> server configuration.
>>> Apart from qemu-img also ls:
>>> # ls -lhs c7lab1.qcow2
>>> 1.3G -rw------- 1 root root 21G Dec 6 23:05 c7lab1.qcow2
>>>
>>
>> The fiile size is 21G - matching what you see. This is the size we upload.
>>
>
> I changed the subject to better address and track the thread..
> You are not convincing me...
>
Trying harder...
> The term "file size" is ambiguous in this context...
>
It is not. file size is what you get from stat:
$ truncate -s 1g empty
$ stat empty
File: 'empty'
Size: 1073741824 Blocks: 0 IO Block: 4096 regular file
...
$ qemu-img info empty
image: empty
file format: raw
virtual size: 1.0G (1073741824 bytes)
disk size: 0
The value "disk size" used by qemu-img is confusing and not useful
when you want to transfer the file to another host.
I don't know why qemu-img display this value instead of the actual
file size, adding qemu-block mailing list in case someone can explain
this.
When you upload or download this file, you will transfer 1g of zeros.
>
> If I take a disk that was born on this oVirt environment itself I have
> this:
>
> https://drive.google.com/file/d/1Dtb69EKa8adNYiwUDs2d-plSMZIM0Bzr/view?usp=sharing
>
> that shows that is thin provisioned
> Its id shown here:
>
> https://drive.google.com/file/d/1PvLjJVQ3JR4_6v5Da7ATac5ReEObxH-A/view?usp=sharing
>
> If I go and check on my exported file system I get this
>
> [root at ovirt01 NFS_DOMAIN]# find . -name
> "3c68d43f-0f28-4564-b557-d390a125daa6"
>
> ./572eabe7-15d0-42c2-8fa9-0bd773e22e2e/images/3c68d43f-0f28-4564-b557-d390a125daa6
> [root at ovirt01 NFS_DOMAIN]# ls -lsh
> ./572eabe7-15d0-42c2-8fa9-0bd773e22e2e/images/3c68d43f-0f28-4564-b557-d390a125daa6
> total 8.6G
> 8.6G -rw-rw----. 1 vdsm kvm 10G Dec 7 08:42
> 09ad8e53-0b22-4fe3-b718-d14352b8290a
> 1.0M -rw-rw----. 1 vdsm kvm 1.0M May 1 2016
> 09ad8e53-0b22-4fe3-b718-d14352b8290a.lease
> 4.0K -rw-r--r--. 1 vdsm kvm 317 Jun 21 10:41
> 09ad8e53-0b22-4fe3-b718-d14352b8290a.meta
>
> [root at ovirt01 NFS_DOMAIN]# qemu-img info
> ./572eabe7-15d0-42c2-8fa9-0bd773e22e2e/images/3c68d43f-0f28-4564-b557-d390a125daa6/09ad8e53-0b22-4fe3-b718-d14352b8290a
> image:
> ./572eabe7-15d0-42c2-8fa9-0bd773e22e2e/images/3c68d43f-0f28-4564-b557-d390a125daa6/09ad8e53-0b22-4fe3-b718-d14352b8290a
> file format: raw
> virtual size: 10G (10737418240 bytes)
> disk size: 8.5G
> [root at ovirt01 NFS_DOMAIN]#
>
> So it seems it is raw/sparse
> https://www.ovirt.org/documentation/admin-guide/chap-Virtual_Machine_Disks/
>
Yes this is raw sparse file.
> 1.3G is the used size on the file system, we cannot upload only used
>> blocks.
>> qemu-img info "Disk size" is not the file size the the used size, not
>> useful
>> for upload.
>>
>
> Why not? You can detect the format of source and use a similar strategy of
> the bugzilla I'm referring (see below)
> Or couldn't you use something like
>
That bug is not related. It was about converting raw sparse format
to qcow2.
In 4.2 we use qemu-img map and the same algorithm used in qemu to
estimate the required size needed to convert raw volume to qcow2 volume.
For transferring images over http, there is no way to avoid sending
unused blocks, except compression.
>
> rsync -av --sparse source target
>
You can use rsync to transfer images if you like, but we support
only http.
>
> if the target is NFS?
>
>
>> Maybe this file was crated with preallocation=full?
>>
>
> In virt-manager there is not this logic by default.
> The default format is qcow2/sparse
> When you create/add a disk you can choose "Select or create custom
> storage":
>
> https://drive.google.com/file/d/1gAwjAG5aRFC5fFgkoZXT5wINvQnov88p/view?usp=sharing
>
> And then choose a format between:
> raw, qcow, qcow2, qed, vmdk, vpc, vdi
>
> In my case it was the default one, so qcow2, as qemu-img correctly detects.
>
I tested creating qcow2 files with virt manager, and it seems to use
the preallocation=metadata by default (behind your back), I guess
for improving performance in some cases. oVirt doe not use this
option so we don't have this issue when downloading files.
Here example of image created by virt-manager:
# ls -lsh fedora26-2.img
3.4M -rw-------. 1 root root 21G Dec 7 21:36 fedora26-2.img
# qemu-img info fedora26-2.img
image: fedora26-2.img
file format: qcow2
virtual size: 20G (21474836480 bytes)
disk size: 3.3M
cluster_size: 65536
Format specific information:
compat: 1.1
lazy refcounts: true
refcount bits: 16
corrupt: false
And here is same image as ovirt create images:
# qemu-img create -f qcow2 test.qcow2 20g
Formatting 'test.qcow2', fmt=qcow2 size=21474836480 encryption=off
cluster_size=65536 lazy_refcounts=off refcount_bits=16
# qemu-img info test.qcow2
image: test.qcow2
file format: qcow2
virtual size: 20G (21474836480 bytes)
disk size: 196K
cluster_size: 65536
Format specific information:
compat: 1.1
lazy refcounts: false
refcount bits: 16
corrupt: false
# ls -lhs test.qcow2
196K -rw-r--r--. 1 root root 193K Dec 7 21:40 test.qcow2
So to download this image you need to transfer only 196k of data, not 20g.
We can get the same results as virt-manager if we use
preallocation=metadata:
# qemu-img create -f qcow2 -o preallocation=metadata
test-preallocation-metadata.qcow2 20g
Formatting 'test-preallocation-metadata.qcow2', fmt=qcow2 size=21474836480
encryption=off cluster_size=65536 preallocation=metadata lazy_refcounts=off
refcount_bits=16
# qemu-img info test-preallocation-metadata.qcow2
image: test-preallocation-metadata.qcow2
file format: qcow2
virtual size: 20G (21474836480 bytes)
disk size: 3.3M
cluster_size: 65536
Format specific information:
compat: 1.1
lazy refcounts: false
refcount bits: 16
corrupt: false
# ls -lhs test-preallocation-metadata.qcow2
3.4M -rw-r--r--. 1 root root 21G Dec 7 21:42
test-preallocation-metadata.qcow2
First message at beginning of upload:
>>>
>>> 2017-12-06 23:09:50,183+0100 INFO (jsonrpc/4) [vdsm.api] START
>>> createVolume(sdUUID=u'572eabe7-15d0-42c2-8fa9-0bd773e22e2e',
>>> spUUID=u'00000001-0001-0001-0001-000000000343',
>>> imgUUID=u'251063f6-5570-4bdc-b28f-21e82aa5e185', size=u'22548578304',
>>> volFormat=4, preallocate=2, diskType=2,
>>> volUUID=u'77aacfb3-9e67-4ad2-96f6-242b5ba4d9e0',
>>> desc=u'{"DiskAlias":"c7lab1","DiskDescription":""}',
>>> srcImgUUID=u'00000000-0000-0000-0000-000000000000',
>>> srcVolUUID=u'00000000-0000-0000-0000-000000000000', initialSize=None)
>>> from=192.168.1.212,56846, flow_id=18c6bd3b-76ab-45f9-b8c7-09c727f44c91,
>>> task_id=e7cc67e6-4b61-4bb3-81b1-6bc687ea5ee9 (api:46)
>>>
>>>
> In this bugzilla, that is related to export from NFS to block:
> https://bugzilla.redhat.com/show_bug.cgi?id=1358717
>
> it seems to understand, from a comment of Maor, that
> preallocate=2 --> sparse
>
> what are instead the possible values of volFormat and their decodification?
> In my case it detects
> volFormat=4,
>
> Could it be something similar to the bugzilla above? Can it change
> anything if exporting with NFS 4.2 or using oVirt 4.2?
>
> I think that if one has big qcow2 disk files that he/she desire to migrate
> into oVirt, the process could be optimized
>
To optimize the import of images with unneeded preallocation, you can
convert them to unallocated and compressed images as I showed
in the previous mail.
Here is an example with the preallocated images, like the one from
virt-manager.
I'll create file with some data - I'm using data that will compress well
to make the example more convincing :-)
# dd if=/dev/zero bs=8M count=128 | tr "\0" "\1" > image.raw
128+0 records in
128+0 records out
1073741824 bytes (1.1 GB, 1.0 GiB) copied, 1.88171 s, 571 MB/s
Add some empty space:
# truncate -s 10g image.raw
# ls -lhs image.raw
1.1G -rw-r--r--. 1 root root 10G Dec 7 22:01 image.raw
Now lest create a qcow2 preallocated image like virt-manger does:
# qemu-img convert -f raw -O qcow2 -o preallocation=metadata image.raw
image-preallocated-metadata.qcow2
# qemu-img info image-preallocated-metadata.qcow2
image: image-preallocated-metadata.qcow2
file format: qcow2
virtual size: 10G (10737418240 bytes)
disk size: 1.0G
cluster_size: 65536
Format specific information:
compat: 1.1
lazy refcounts: false
refcount bits: 16
corrupt: false
# ls -lhs image*
1.1G -rw-r--r--. 1 root root 11G Dec 7 22:02
image-preallocated-metadata.qcow2
1.1G -rw-r--r--. 1 root root 10G Dec 7 22:01 image.raw
Finally, create a compressed qcow2, optimized for upload:
# qemu-img convert -c -f qcow2 -O qcow2 image-preallocated-metadata.qcow2
image-preallocated-metadata-optimized.qcow2
# qemu-img info image-preallocated-metadata-optimized.qcow2
image: image-preallocated-metadata-optimized.qcow2
file format: qcow2
virtual size: 10G (10737418240 bytes)
disk size: 1.6M
cluster_size: 65536
Format specific information:
compat: 1.1
lazy refcounts: false
refcount bits: 16
corrupt: false
# ls -lhs image*
1.6M -rw-r--r--. 1 root root 1.7M Dec 7 22:04
image-preallocated-metadata-optimized.qcow2
1.1G -rw-r--r--. 1 root root 11G Dec 7 22:02
image-preallocated-metadata.qcow2
1.1G -rw-r--r--. 1 root root 10G Dec 7 22:01 image.raw
All the images contain exactly the same data.
Now, how can we optimize your image on upload? we can't.
When uploading from the UI, our code run in the browser
and we don't have access to your file system.
When using the oVirt SDK, you write the code for uploading so you can
do the conversion if want to save upload time and bandwidth and can
spend the time to do the conversion.
We can provide an upload command line tool that will convert
and compress images before upload, and will use the oVirt SDK
to upload the images.
But I think it will be simpler and easier to maintain to support
compression when uploading or downloading data. This is
already supported by browsers and common tools like curl,
and it works with any file format without conversion.
I hope I convinced you this time ;-)
Nir
>
> Thanks,
> Gianluca
>
>
>
-------------- next part --------------
An HTML attachment was scrubbed...
URL: <http://lists.ovirt.org/pipermail/users/attachments/20171207/ece3fa44/attachment.html>
More information about the Users
mailing list