4:42 a.m.
On 06/11/2018 06:07 PM, Nir Soffer wrote:
> Correct image files should be
> aligned to 512 byte sectors, so something is wrong with your image to
> start with (hard disks don't have half sectors).
Ideally, a guest will never see an unaligned image size: qemu should
round UP when creating an image, and then truncate DOWN when opening an
unaligned image that it cannot resize. qemu-img DOES round the guest
virtual size up on creation; and thus, for the raw format, a file
created by qemu-img will be aligned, and you can only get an unaligned
image by manual efforts.
But for qcow2 files, the qcow2 spec is clear that an incomplete final
cluster (whether or not that last cluster end is also sector-aligned) is
well-formed with the tail reading as zeroes; if the last cluster is
guest-visible, qemu-img will have rounded it up to sector (but not
necessarily cluster) alignment. But if the last cluster is something
else, like the refcount, L1, or L2 table, then yes, it is very common
that the consumed disk size is not sector aligned.
$ qemu-img create -f qcow2 test3 123
Formatting 'test3', fmt=qcow2 size=123 cluster_size=65536
lazy_refcounts=off refcount_bits=16
$ qemu-img info test3
image: test3
file format: qcow2
virtual size: 512 (512 bytes)
disk size: 196K
cluster_size: 65536
Format specific information:
compat: 1.1
lazy refcounts: false
refcount bits: 16
corrupt: false
$ ls -l test3
-rw-r--r--. 1 eblake eblake 196616 Jun 11 20:32 test3
$ echo $((196616/512*512))
196608
So, you can see that the guest-visible size was rounded up (123 became
512), which is now sector-aligned but not cluster-aligned; and that the
final cluster occupies only 8 bytes at the moment (the difference
between 196616 and 196608).
Maybe we should improve qemu to always create cluster-aligned qcow2
images, but that's a bigger audit.
Right. We indeed forbid uploads on unaligned images in the UI.
But we found that qemu-img creates unaligned images:
$ qemu-img create -f qcow2 empty.raw 10g
Formatting 'empty.raw', fmt=qcow2 size=10737418240 cluster_size=65536
lazy_refcounts=off refcount_bits=16
$ ls -l empty.raw
-rw-r--r--. 1 nsoffer nsoffer 196768 Jun 12 01:59 empty.raw
$ python -c "print 196768 % 512"
160
$ qemu-img map -f raw --output json test.raw
Wait; where did test.raw come from; given that your earlier commands
were dealing with empty.raw? And mapping a qcow2 file as though it were
raw is indeed confusing (you don't want to do that for a guest using the
image).
The image becomes aligned once you write anything into it, but I
still find
it
strange that qemu-img create such files. Shouldn't it create always 3
complete
clusters?
Because historical versions of qemu do not, we already have to cope with
unaligned images (when opening them as qcow2; but not when
transliterating and opening them as raw, since it took this long to find
the regression); we could make future versions of qemu always create
cluster-aligned images, but you'd still have to cope with existing
images that aren't.
--
Eric Blake, Principal Software Engineer
Red Hat, Inc. +1-919-301-3266
Virtualization: qemu.org | libvirt.org