On Thu, Dec 7, 2017 at 1:28 AM, Nir Soffer <nsoffer@redhat.com> wrote:

On Thu, Dec 7, 2017 at 1:22 AM Gianluca Cecchi <gianluca.cecchi@gmail.com> wrote:
On Wed, Dec 6, 2017 at 11:42 PM, Nir Soffer <nsoffer@redhat.com> wrote:

BTW: I notice that the disk seems preallocated even if original qcow2 is thin... is this expected?
This obviously also impacts the time to upload (20Gb virtual disk with actual 1.2Gb occupied needs the time equivalent for full 20Gb...)

We upload exactly the file you provided, there is no way we can upload 20G from 1.2G file :-)

But the upload process at a medium rate of 40-50MB/s has last about 9 minutes that confirms the 20Gb size
The disk at source has been created as virtio type and format qcow2 from virt-manager and then only installed a CentOS 7.2 OS with infrastructure server configuration.
Apart from qemu-img also ls:
# ls -lhs c7lab1.qcow2
1.3G -rw------- 1 root root 21G Dec 6 23:05 c7lab1.qcow2

The fiile size is 21G - matching what you see. This is the size we upload.

I changed the subject to better address and track the thread..

You are not convincing me...

The term "file size" is ambiguous in this context...

If I take a disk that was born on this oVirt environment itself I have this:

https://drive.google.com/file/d/1Dtb69EKa8adNYiwUDs2d-plSMZIM0Bzr/view?usp=sharing

that shows that is thin provisioned

Its id shown here:

https://drive.google.com/file/d/1PvLjJVQ3JR4_6v5Da7ATac5ReEObxH-A/view?usp=sharing

If I go and check on my exported file system I get this

[root@ovirt01 NFS_DOMAIN]# find . -name "3c68d43f-0f28-4564-b557-d390a125daa6"

./572eabe7-15d0-42c2-8fa9-0bd773e22e2e/images/3c68d43f-0f28-4564-b557-d390a125daa6

[root@ovirt01 NFS_DOMAIN]# ls -lsh ./572eabe7-15d0-42c2-8fa9-0bd773e22e2e/images/3c68d43f-0f28-4564-b557-d390a125daa6

total 8.6G

8.6G -rw-rw----. 1 vdsm kvm 10G Dec 7 08:42 09ad8e53-0b22-4fe3-b718-d14352b8290a

1.0M -rw-rw----. 1 vdsm kvm 1.0M May 1 2016 09ad8e53-0b22-4fe3-b718-d14352b8290a.lease

4.0K -rw-r--r--. 1 vdsm kvm 317 Jun 21 10:41 09ad8e53-0b22-4fe3-b718-d14352b8290a.meta

[root@ovirt01 NFS_DOMAIN]# qemu-img info ./572eabe7-15d0-42c2-8fa9-0bd773e22e2e/images/3c68d43f-0f28-4564-b557-d390a125daa6/09ad8e53-0b22-4fe3-b718-d14352b8290a

image: ./572eabe7-15d0-42c2-8fa9-0bd773e22e2e/images/3c68d43f-0f28-4564-b557-d390a125daa6/09ad8e53-0b22-4fe3-b718-d14352b8290a

file format: raw

virtual size: 10G (10737418240 bytes)

disk size: 8.5G

[root@ovirt01 NFS_DOMAIN]#

So it seems it is raw/sparse

https://www.ovirt.org/documentation/admin-guide/chap-Virtual_Machine_Disks/

1.3G is the used size on the file system, we cannot upload only used blocks.
qemu-img info "Disk size" is not the file size the the used size, not useful
for upload.

Why not? You can detect the format of source and use a similar strategy of the bugzilla I'm referring (see below)

Or couldn't you use something like

rsync -av --sparse source target

if the target is NFS?

Maybe this file was crated with preallocation=full?

In virt-manager there is not this logic by default.

The default format is qcow2/sparse

When you create/add a disk you can choose "Select or create custom storage":

https://drive.google.com/file/d/1gAwjAG5aRFC5fFgkoZXT5wINvQnov88p/view?usp=sharing

And then choose a format between:

raw, qcow, qcow2, qed, vmdk, vpc, vdi

In my case it was the default one, so qcow2, as qemu-img correctly detects.

First message at beginning of upload:

2017-12-06 23:09:50,183+0100 INFO (jsonrpc/4) [vdsm.api] START createVolume(sdUUID=u'572eabe7-15d0-42c2-8fa9-0bd773e22e2e', spUUID=u'00000001-0001-0001-0001-000000000343', imgUUID=u'251063f6-5570-4bdc-b28f-21e82aa5e185', size=u'22548578304', volFormat=4, preallocate=2, diskType=2, volUUID=u'77aacfb3-9e67-4ad2-96f6-242b5ba4d9e0', desc=u'{"DiskAlias":"c7lab1","DiskDescription":""}', srcImgUUID=u'00000000-0000-0000-0000-000000000000', srcVolUUID=u'00000000-0000-0000-0000-000000000000', initialSize=None) from=192.168.1.212,56846, flow_id=18c6bd3b-76ab-45f9-b8c7-09c727f44c91, task_id=e7cc67e6-4b61-4bb3-81b1-6bc687ea5ee9 (api:46)

In this bugzilla, that is related to export from NFS to block:

https://bugzilla.redhat.com/show_bug.cgi?id=1358717

it seems to understand, from a comment of Maor, that

preallocate=2 --> sparse

what are instead the possible values of volFormat and their decodification?

In my case it detects

volFormat=4,

Could it be something similar to the bugzilla above? Can it change anything if exporting with NFS 4.2 or using oVirt 4.2?

I think that if one has big qcow2 disk files that he/she desire to migrate into oVirt, the process could be optimized

Thanks,

Gianluca