Storage migration: Preallocation forced on destination
Hi, We're migrating an existing storage (glusterfs) to a new one (iSCSI). All disks on glusterfs are thin provisioned, but when migrating to iSCSI the following warning is shown: The following disks will become preallocated, and may consume considerably more space on the target: local-disk Why is that? Is there a way to migrate disks so they are thin provisioned on iSCSI as well? Thanks.
On Fri, Mar 4, 2016 at 7:07 PM, Nicolás <nicolas@devels.es> wrote:
Hi,
We're migrating an existing storage (glusterfs) to a new one (iSCSI). All disks on glusterfs are thin provisioned, but when migrating to iSCSI the following warning is shown:
The following disks will become preallocated, and may consume considerably more space on the target: local-disk
Why is that? Is there a way to migrate disks so they are thin provisioned on iSCSI as well?
The issue is that we use raw sparse format for thin provisioned disks on file based storage. The file system provides the thin provisioning, maintaining holes in the files. When we create the destination lv, we use the disk virtual size, so you get practically a preallocated volume. I think we can do better - before copying the disk, we can check the actual used space (e.g. what qemu-img info or stat report), and create the destination lv using the used size (plus additional space for qcow format). I tested this by extending the destination lv manually and then copying data manually using qemu-img convert, and it works. Can you file a bug for this, and explain the use case? Nir
I think it's hard to calculate the additional space for cow format without analysing raw image. It's better to allocate enough space, and then decrease it after qemu-img convert. Please note that while disk moving keeps disk format, disk copying changes format. So when you copy a thin provisioned disk to iSCSI storage it's being converted to cow. The issue is that size of converted lv still looks like preallocated. You can decrease it manually via lvchange, or you can move it to a file based storage and back. Moving disks keeps disk format, but fixes its size. Also please consider qcow compat=1.1 as default disk format both for file and block storages. On 04/03/16 23:23, "users-bounces@ovirt.org on behalf of Nir Soffer" <users-bounces@ovirt.org on behalf of nsoffer@redhat.com> wrote:
On Fri, Mar 4, 2016 at 7:07 PM, Nicolás <nicolas@devels.es> wrote:
Hi,
We're migrating an existing storage (glusterfs) to a new one (iSCSI). All disks on glusterfs are thin provisioned, but when migrating to iSCSI the following warning is shown:
The following disks will become preallocated, and may consume considerably more space on the target: local-disk
Why is that? Is there a way to migrate disks so they are thin provisioned on iSCSI as well?
The issue is that we use raw sparse format for thin provisioned disks on file based storage. The file system provides the thin provisioning, maintaining holes in the files.
When we create the destination lv, we use the disk virtual size, so you get practically a preallocated volume.
I think we can do better - before copying the disk, we can check the actual used space (e.g. what qemu-img info or stat report), and create the destination lv using the used size (plus additional space for qcow format).
I tested this by extending the destination lv manually and then copying data manually using qemu-img convert, and it works.
Can you file a bug for this, and explain the use case?
Nir _______________________________________________ Users mailing list Users@ovirt.org http://lists.ovirt.org/mailman/listinfo/users
On Sat, Mar 5, 2016 at 12:19 AM, Pavel Gashev <Pax@acronis.com> wrote:
I think it's hard to calculate the additional space for cow format without analysing raw image. It's better to allocate enough space, and then decrease it after qemu-img convert.
We use 10% as a rough estimate for additional space when converting from raw to qcow format. Sure it will waste some space, but it is good enough. How to you plan to check the used size on the destination lv?
Please note that while disk moving keeps disk format, disk copying changes format. So when you copy a thin provisioned disk to iSCSI storage it's being converted to cow. The issue is that size of converted lv still looks like preallocated. You can decrease it manually via lvchange, or you can move it to a file based storage and back. Moving disks keeps disk format, but fixes its size.
Yes, this seems to be the way to work around this issue currently: 1. Copy to the disk to block storage - will convert it to qcow format on preallocated lv 2. Move disk from block storage to file storage 3. Move disk back to block storage
Also please consider qcow compat=1.1 as default disk format both for file and block storages.
This will make your disk incompatible with old ovirt versions on el6. In storage domain format v3 we are using comapt=0.10. We plan to move to compat=1.1 in 4.0.
On 04/03/16 23:23, "users-bounces@ovirt.org on behalf of Nir Soffer" <users-bounces@ovirt.org on behalf of nsoffer@redhat.com> wrote:
On Fri, Mar 4, 2016 at 7:07 PM, Nicolás <nicolas@devels.es> wrote:
Hi,
We're migrating an existing storage (glusterfs) to a new one (iSCSI). All disks on glusterfs are thin provisioned, but when migrating to iSCSI the following warning is shown:
The following disks will become preallocated, and may consume considerably more space on the target: local-disk
Why is that? Is there a way to migrate disks so they are thin provisioned on iSCSI as well?
The issue is that we use raw sparse format for thin provisioned disks on file based storage. The file system provides the thin provisioning, maintaining holes in the files.
When we create the destination lv, we use the disk virtual size, so you get practically a preallocated volume.
I think we can do better - before copying the disk, we can check the actual used space (e.g. what qemu-img info or stat report), and create the destination lv using the used size (plus additional space for qcow format).
I tested this by extending the destination lv manually and then copying data manually using qemu-img convert, and it works.
Can you file a bug for this, and explain the use case?
Nir _______________________________________________ Users mailing list Users@ovirt.org http://lists.ovirt.org/mailman/listinfo/users
On 05/03/16 02:23, "Nir Soffer" <nsoffer@redhat.com> wrote:
On Sat, Mar 5, 2016 at 12:19 AM, Pavel Gashev <Pax@acronis.com> wrote:
I think it's hard to calculate the additional space for cow format without analysing raw image. It's better to allocate enough space, and then decrease it after qemu-img convert.
We use 10% as a rough estimate for additional space when converting from raw to qcow format. Sure it will waste some space, but it is good enough.
How to you plan to check the used size on the destination lv?
"qemu-img check" reports image end offset. Also it has --output=json option. # qemu-img check --output=json /rhev/data-center/00000002-0002-0002-0002-0000000002db/2e9250bd-5cf3-40c2-8e85-1e31d4c4d779/images/bc4c68be-0bb0-436a-b18d-014d9ae4e580/9b63d90e-cb1e-4a30-a0b6-0b80fa201f8d { "image-end-offset": 14648672256, "total-clusters": 524288, "check-errors": 0, "allocated-clusters": 223439, "filename": "/rhev/data-center/00000002-0002-0002-0002-0000000002db/2e9250bd-5cf3-40c2-8e85-1e31d4c4d779/images/bc4c68be-0bb0-436a-b18d-014d9ae4e580/9b63d90e-cb1e-4a30-a0b6-0b80fa201f8d", "format": "qcow2", "fragmented-clusters": 6 }
Please note that while disk moving keeps disk format, disk copying changes format. So when you copy a thin provisioned disk to iSCSI storage it's being converted to cow. The issue is that size of converted lv still looks like preallocated. You can decrease it manually via lvchange, or you can move it to a file based storage and back. Moving disks keeps disk format, but fixes its size.
Yes, this seems to be the way to work around this issue currently: 1. Copy to the disk to block storage - will convert it to qcow format on preallocated lv 2. Move disk from block storage to file storage 3. Move disk back to block storage
Yes. This is how I've moved my stuff from M$ NFS to iSCSI. Note this works well since 3.6.3 only, see https://bugzilla.redhat.com/1304405
Also please consider qcow compat=1.1 as default disk format both for file and block storages.
This will make your disk incompatible with old ovirt versions on el6. In storage domain format v3 we are using comapt=0.10.
We plan to move to compat=1.1 in 4.0.
Too long to wait :) Currently it's hardcoded. It would be great, if it was configurable via /etc/vdsm/vdsm.conf at least. It would allow early adopt compat=1.1 in el7 environments.
On Sat, Mar 5, 2016 at 10:52 AM, Pavel Gashev <Pax@acronis.com> wrote:
On 05/03/16 02:23, "Nir Soffer" <nsoffer@redhat.com> wrote:
On Sat, Mar 5, 2016 at 12:19 AM, Pavel Gashev <Pax@acronis.com> wrote:
I think it's hard to calculate the additional space for cow format without analysing raw image. It's better to allocate enough space, and then decrease it after qemu-img convert.
We use 10% as a rough estimate for additional space when converting from raw to qcow format. Sure it will waste some space, but it is good enough.
How to you plan to check the used size on the destination lv?
"qemu-img check" reports image end offset. Also it has --output=json option.
# qemu-img check --output=json /rhev/data-center/00000002-0002-0002-0002-0000000002db/2e9250bd-5cf3-40c2-8e85-1e31d4c4d779/images/bc4c68be-0bb0-436a-b18d-014d9ae4e580/9b63d90e-cb1e-4a30-a0b6-0b80fa201f8d { "image-end-offset": 14648672256, "total-clusters": 524288, "check-errors": 0, "allocated-clusters": 223439, "filename": "/rhev/data-center/00000002-0002-0002-0002-0000000002db/2e9250bd-5cf3-40c2-8e85-1e31d4c4d779/images/bc4c68be-0bb0-436a-b18d-014d9ae4e580/9b63d90e-cb1e-4a30-a0b6-0b80fa201f8d", "format": "qcow2", "fragmented-clusters": 6 }
Right, actually we are already doing this after cold merge, - https://github.com/oVirt/vdsm/blob/d87d031ea52eb76aab29cd0c55d7f3b5f0743ccd/... - https://github.com/oVirt/vdsm/blob/d87d031ea52eb76aab29cd0c55d7f3b5f0743ccd/... We should start doing this after moving and copying thin disks, and after live mege. With live merge, we make sure that the base volume is large enough to include all data from the top volume. This does overallocation and having automatic shrinking after that can be nice.
Also please consider qcow compat=1.1 as default disk format both for file and block storages.
This will make your disk incompatible with old ovirt versions on el6. In storage domain format v3 we are using comapt=0.10.
We plan to move to compat=1.1 in 4.0.
Too long to wait :) Currently it's hardcoded. It would be great, if it was configurable via /etc/vdsm/vdsm.conf at least. It would allow early adopt compat=1.1 in el7 environments.
Good idea, If you file a bug and send a patch it will be available in the next release. Nir
Nir, On 05/03/16 15:35, "Nir Soffer" <nsoffer@redhat.com> wrote:
On Sat, Mar 5, 2016 at 10:52 AM, Pavel Gashev <Pax@acronis.com> wrote:
On 05/03/16 02:23, "Nir Soffer" <nsoffer@redhat.com> wrote:
On Sat, Mar 5, 2016 at 12:19 AM, Pavel Gashev <Pax@acronis.com> wrote:
Also please consider qcow compat=1.1 as default disk format both for file and block storages.
This will make your disk incompatible with old ovirt versions on el6. In storage domain format v3 we are using comapt=0.10.
We plan to move to compat=1.1 in 4.0.
Too long to wait :) Currently it's hardcoded. It would be great, if it was configurable via /etc/vdsm/vdsm.conf at least. It would allow early adopt compat=1.1 in el7 environments.
Good idea, If you file a bug and send a patch it will be available in the next release.
Here you go https://bugzilla.redhat.com/show_bug.cgi?id=1317850 Thanks
On Tue, Mar 15, 2016 at 1:14 PM, Pavel Gashev <Pax@acronis.com> wrote:
Nir,
On 05/03/16 15:35, "Nir Soffer" <nsoffer@redhat.com> wrote:
On Sat, Mar 5, 2016 at 10:52 AM, Pavel Gashev <Pax@acronis.com> wrote:
On 05/03/16 02:23, "Nir Soffer" <nsoffer@redhat.com> wrote:
On Sat, Mar 5, 2016 at 12:19 AM, Pavel Gashev <Pax@acronis.com> wrote:
Also please consider qcow compat=1.1 as default disk format both for file and block storages.
This will make your disk incompatible with old ovirt versions on el6. In storage domain format v3 we are using comapt=0.10.
We plan to move to compat=1.1 in 4.0.
Too long to wait :) Currently it's hardcoded. It would be great, if it was configurable via /etc/vdsm/vdsm.conf at least. It would allow early adopt compat=1.1 in el7 environments.
Good idea, If you file a bug and send a patch it will be available in the next release.
Here you go https://bugzilla.redhat.com/show_bug.cgi?id=1317850
Thanks! I applied the patch to master and uploaded it to our code review system: https://gerrit.ovirt.org/54759 Except moving the new config option to [irs] section, the patch looks fine. Cheers, Nir
El 04/03/16 a las 20:23, Nir Soffer escribió:
Hi,
We're migrating an existing storage (glusterfs) to a new one (iSCSI). All disks on glusterfs are thin provisioned, but when migrating to iSCSI the following warning is shown:
The following disks will become preallocated, and may consume considerably more space on the target: local-disk
Why is that? Is there a way to migrate disks so they are thin provisioned on iSCSI as well? The issue is that we use raw sparse format for thin provisioned disks on file based storage. The file system provides the thin provisioning, maintaining holes in
On Fri, Mar 4, 2016 at 7:07 PM, Nicolás <nicolas@devels.es> wrote: the files.
When we create the destination lv, we use the disk virtual size, so you get practically a preallocated volume.
I think we can do better - before copying the disk, we can check the actual used space (e.g. what qemu-img info or stat report), and create the destination lv using the used size (plus additional space for qcow format).
I tested this by extending the destination lv manually and then copying data manually using qemu-img convert, and it works.
Can you file a bug for this, and explain the use case?
Nir
Done, for the ones interested on it: https://bugzilla.redhat.com/show_bug.cgi?id=1314959 Thanks. Regards.
participants (3)
-
Nicolás -
Nir Soffer -
Pavel Gashev