[ovirt-users] import thin provisioned disks with image upload

Thu Dec 7 08:15:22 UTC 2017

On Thu, Dec 7, 2017 at 1:28 AM, Nir Soffer <nsoffer at redhat.com> wrote:

>
>
> On Thu, Dec 7, 2017 at 1:22 AM Gianluca Cecchi <gianluca.cecchi at gmail.com>
> wrote:
>
>> On Wed, Dec 6, 2017 at 11:42 PM, Nir Soffer <nsoffer at redhat.com> wrote:
>>
>>>
>>>
>>>>
>>>> BTW: I notice that the disk seems preallocated even if original qcow2
>>>> is thin... is this expected?
>>>> This obviously also impacts the time to upload (20Gb virtual disk with
>>>> actual 1.2Gb occupied needs the time equivalent for full 20Gb...)
>>>>
>>>
>>> We upload exactly the file you provided, there is no way we can upload
>>> 20G from 1.2G file :-)
>>>
>>
>> But the upload process at a medium rate of 40-50MB/s has last about 9
>> minutes that confirms the 20Gb size
>> The disk at source has been created as virtio type and format qcow2 from
>> virt-manager and then only installed a CentOS 7.2 OS with infrastructure
>> server configuration.
>> Apart from qemu-img also ls:
>> # ls -lhs c7lab1.qcow2
>> 1.3G -rw------- 1 root root 21G Dec  6 23:05 c7lab1.qcow2
>>
>
> The fiile size is 21G - matching what you see. This is the size we upload.
>

I changed the subject to better address and track the thread..
You are not convincing me...
The term "file size" is ambiguous in this context...

If I take a disk that was born on this oVirt environment itself I have this:
https://drive.google.com/file/d/1Dtb69EKa8adNYiwUDs2d-plSMZIM0Bzr/view?usp=sharing

that shows that is thin provisioned
Its id shown here:
https://drive.google.com/file/d/1PvLjJVQ3JR4_6v5Da7ATac5ReEObxH-A/view?usp=sharing

If I go and check on my exported file system I get this

[root at ovirt01 NFS_DOMAIN]# find . -name
"3c68d43f-0f28-4564-b557-d390a125daa6"
./572eabe7-15d0-42c2-8fa9-0bd773e22e2e/images/3c68d43f-0f28-4564-b557-d390a125daa6
[root at ovirt01 NFS_DOMAIN]# ls -lsh
./572eabe7-15d0-42c2-8fa9-0bd773e22e2e/images/3c68d43f-0f28-4564-b557-d390a125daa6
total 8.6G
8.6G -rw-rw----. 1 vdsm kvm  10G Dec  7 08:42
09ad8e53-0b22-4fe3-b718-d14352b8290a
1.0M -rw-rw----. 1 vdsm kvm 1.0M May  1  2016
09ad8e53-0b22-4fe3-b718-d14352b8290a.lease
4.0K -rw-r--r--. 1 vdsm kvm  317 Jun 21 10:41
09ad8e53-0b22-4fe3-b718-d14352b8290a.meta

[root at ovirt01 NFS_DOMAIN]# qemu-img info
./572eabe7-15d0-42c2-8fa9-0bd773e22e2e/images/3c68d43f-0f28-4564-b557-d390a125daa6/09ad8e53-0b22-4fe3-b718-d14352b8290a
image:
./572eabe7-15d0-42c2-8fa9-0bd773e22e2e/images/3c68d43f-0f28-4564-b557-d390a125daa6/09ad8e53-0b22-4fe3-b718-d14352b8290a
file format: raw
virtual size: 10G (10737418240 bytes)
disk size: 8.5G
[root at ovirt01 NFS_DOMAIN]#

So it seems it is raw/sparse
https://www.ovirt.org/documentation/admin-guide/chap-Virtual_Machine_Disks/

>
> 1.3G is the used size on the file system, we cannot upload only used
> blocks.
> qemu-img info "Disk size" is not the file size the the used size, not
> useful
> for upload.
>

Why not? You can detect the format of source and use a similar strategy of
the bugzilla I'm referring (see below)
Or couldn't you use something like

 rsync -av --sparse source target

if the target is NFS?

> Maybe this file was crated with preallocation=full?
>

In virt-manager there is not this logic by default.
The default format is qcow2/sparse
When you create/add a disk you can choose "Select or create custom storage":
https://drive.google.com/file/d/1gAwjAG5aRFC5fFgkoZXT5wINvQnov88p/view?usp=sharing

And then choose a format between:
raw, qcow, qcow2, qed, vmdk, vpc, vdi

In my case it was the default one, so qcow2, as qemu-img correctly detects.

>
>>>>
>> First message at beginning of upload:
>>
>> 2017-12-06 23:09:50,183+0100 INFO  (jsonrpc/4) [vdsm.api] START
>> createVolume(sdUUID=u'572eabe7-15d0-42c2-8fa9-0bd773e22e2e',
>> spUUID=u'00000001-0001-0001-0001-000000000343',
>> imgUUID=u'251063f6-5570-4bdc-b28f-21e82aa5e185', size=u'22548578304',
>> volFormat=4, preallocate=2, diskType=2, volUUID=u'77aacfb3-9e67-4ad2-96f6-242b5ba4d9e0',
>> desc=u'{"DiskAlias":"c7lab1","DiskDescription":""}',
>> srcImgUUID=u'00000000-0000-0000-0000-000000000000',
>> srcVolUUID=u'00000000-0000-0000-0000-000000000000', initialSize=None)
>> from=192.168.1.212,56846, flow_id=18c6bd3b-76ab-45f9-b8c7-09c727f44c91,
>> task_id=e7cc67e6-4b61-4bb3-81b1-6bc687ea5ee9 (api:46)
>>
>>
In this bugzilla, that is related to export from NFS to block:
https://bugzilla.redhat.com/show_bug.cgi?id=1358717

it seems to understand, from a comment of Maor, that
preallocate=2 --> sparse

what are instead the possible values of volFormat and their decodification?
In my case it detects
volFormat=4,

Could it be something similar to the bugzilla above? Can it change anything
if exporting with NFS 4.2 or using oVirt 4.2?

I think that if one has big qcow2 disk files that he/she desire to migrate
into oVirt, the process could be optimized

Thanks,
Gianluca
-------------- next part --------------
An HTML attachment was scrubbed...
URL: <http://lists.ovirt.org/pipermail/users/attachments/20171207/1a1dab02/attachment.html>