On 06/11/2018 06:07 PM, Nir Soffer wrote:
Correct image files should be aligned to 512 byte sectors, so something is wrong with your image to start with (hard disks don't have half sectors).
Ideally, a guest will never see an unaligned image size: qemu should round UP when creating an image, and then truncate DOWN when opening an unaligned image that it cannot resize. qemu-img DOES round the guest virtual size up on creation; and thus, for the raw format, a file created by qemu-img will be aligned, and you can only get an unaligned image by manual efforts. But for qcow2 files, the qcow2 spec is clear that an incomplete final cluster (whether or not that last cluster end is also sector-aligned) is well-formed with the tail reading as zeroes; if the last cluster is guest-visible, qemu-img will have rounded it up to sector (but not necessarily cluster) alignment. But if the last cluster is something else, like the refcount, L1, or L2 table, then yes, it is very common that the consumed disk size is not sector aligned. $ qemu-img create -f qcow2 test3 123 Formatting 'test3', fmt=qcow2 size=123 cluster_size=65536 lazy_refcounts=off refcount_bits=16 $ qemu-img info test3 image: test3 file format: qcow2 virtual size: 512 (512 bytes) disk size: 196K cluster_size: 65536 Format specific information: compat: 1.1 lazy refcounts: false refcount bits: 16 corrupt: false $ ls -l test3 -rw-r--r--. 1 eblake eblake 196616 Jun 11 20:32 test3 $ echo $((196616/512*512)) 196608 So, you can see that the guest-visible size was rounded up (123 became 512), which is now sector-aligned but not cluster-aligned; and that the final cluster occupies only 8 bytes at the moment (the difference between 196616 and 196608). Maybe we should improve qemu to always create cluster-aligned qcow2 images, but that's a bigger audit.
Right. We indeed forbid uploads on unaligned images in the UI.
But we found that qemu-img creates unaligned images:
$ qemu-img create -f qcow2 empty.raw 10g Formatting 'empty.raw', fmt=qcow2 size=10737418240 cluster_size=65536 lazy_refcounts=off refcount_bits=16
$ ls -l empty.raw -rw-r--r--. 1 nsoffer nsoffer 196768 Jun 12 01:59 empty.raw
$ python -c "print 196768 % 512" 160
$ qemu-img map -f raw --output json test.raw
Wait; where did test.raw come from; given that your earlier commands were dealing with empty.raw? And mapping a qcow2 file as though it were raw is indeed confusing (you don't want to do that for a guest using the image).
The image becomes aligned once you write anything into it, but I still find it strange that qemu-img create such files. Shouldn't it create always 3 complete clusters?
Because historical versions of qemu do not, we already have to cope with unaligned images (when opening them as qcow2; but not when transliterating and opening them as raw, since it took this long to find the regression); we could make future versions of qemu always create cluster-aligned images, but you'd still have to cope with existing images that aren't. -- Eric Blake, Principal Software Engineer Red Hat, Inc. +1-919-301-3266 Virtualization: qemu.org | libvirt.org