[ovirt-users] Problem with backing-file, how to fix the backing-chain ?

Adam Litke alitke at redhat.com
Mon Oct 17 19:57:57 UTC 2016 

Have you tried a qemu-img 'unsafe' rebase?

Let's say you have the chain:
    base <- middle <- top

and due to the old backing chain links we have

base parent: None
middle parent: ../uuid/../uuid/../uuid/base
top parent: ../uuid/../uuid/middle

Since the volumes are always stored in the same directory, the
repeated ../uuid/ parts of the path are unnecessary and can always be
replaced with just the last part.  Once making the LVs available on
the system you should be able to fix the paths with the following
commands:

qemu-img rebase -u -b base middle
qemu-img rebase -u -b middle top

You can then verify the whole backing chain by: 

qemu-img info --backing-chain top

Let me know if this helps.


On 17/10/16 10:34 +0200, Dael Maselli wrote:
>Hi all,
>
>We run an ovirt environment before with engine v3.6.5 (if remember good)
>and now with v4.0.4 (we upgraded because we read the bug with
>backing-file was resolved with v4).
>
>We upgraded some of the hosts machines (but not all still) at v4.0.4 too
>to see if this would fix the problem, but nothing.
>
>The problem is that we have several VMs with snapshots, we do daily,
>weekly and monthly snapshots, keep some of them (usually the fresh ones)
>and remove the olds (that in the case they are weekly snapshots, they
>are in the middle of a series of snapshots), this in the time has
>produced the famous
>
>Backing file too long bug.
>
>So we upgraded the engine from 3.6.5 to 4.0.4 (latest available).
>
>We discovered this bug, when we tried to upgrade an host to v4.0.4,
>doing so a VM in the host didn't migrate, so we shutdown it and tried to
>run on another host, but never succeded for the bug.
>
>We don't know if we have more VMs in this situation because we upgraded
>only 2 hosts on 10.
>
>Investigating the problem we discovered that the backing file indicated
>in each of LVM snapshots report a path very long with
>/dev/storage-domain-id/../image-group-id/ with ../image-group-id/
>repeated a lot of times and at the end /parentid.
>
>So to understand which was the right path that it would contain, we
>cloned a VM in the v4.0.4 and then we did 4 snapshots, now the backing
>file path is
>
>/dev/storage-domain-id/parentid
>
>Is there a way to modify the path in the backing-file or a way to
>recover the VM from this state ?
>
>Where do reside the informations about the backing-file path ?
>
>I attach here all the commands we run
>
>On the ovirt manager (host with the engine only) we run
>
>ovirt-shell
>
>[oVirt shell (connected)]# list disks --parent-vm-name vm1
>
>id         : 2df25a13-6958-40a8-832f-9a26ce65de0f
>name       : vm1_Disk2
>
>id         : 8cda0aa6-9e25-4b50-ba00-b877232a1983
>name       : vm1_Disk1
>
>[oVirt shell (connected)]# show disk 8cda0aa6-9e25-4b50-ba00-b877232a1983
>
>id                               : 8cda0aa6-9e25-4b50-ba00-b877232a1983
>name                             : vm1_Disk1
>actual_size                      : 1073741824
>alias                            : vm1_Disk1
>disk_profile-id                  : 1731f79a-5034-4270-9a87-94d93025deac
>format                           : cow
>image_id                         : 7b354e2a-2099-4f2a-80b7-fba7d1fd13ee
>propagate_errors                 : False
>provisioned_size                 : 17179869184
>shareable                        : False
>size                             : 17179869184
>sparse                           : True
>status-state                     : ok
>storage_domains-storage_domain-id: 384f9059-ef2f-4d43-a54f-de71c5d589c8
>storage_type                     : image
>wipe_after_delete                : False
>
>[root at ovc1mgr ~]# su - postgres
>Last login: Fri Oct 14 01:02:14 CEST 2016
>-bash-4.2$ psql -d engine -U postgres
>psql (9.2.15)
>Type "help" for help.
>
>engine=#\x on
>engine=# select * from images where image_group_id =
>'8cda0aa6-9e25-4b50-ba00-b877232a1983' order by creation_date;
>
>-[ RECORD 1 ]---------+-------------------------------------
>image_guid            | 60ba7acf-58cb-475b-b9ee-15b1be99fee6
>creation_date         | 2016-03-29 15:12:34+02
>size                  | 17179869184
>it_guid               | 00000000-0000-0000-0000-000000000000
>parentid              | 00000000-0000-0000-0000-000000000000
>imagestatus           | 4
>lastmodified          | 2016-04-21 11:25:59.972+02
>vm_snapshot_id        | 27c187cd-989f-4f7a-ac05-49c4410de6c2
>volume_type           | 1
>volume_format         | 5
>image_group_id        | 8cda0aa6-9e25-4b50-ba00-b877232a1983
>_create_date          | 2016-03-29 15:12:31.994065+02
>_update_date          | 2016-09-04 01:10:08.773649+02
>active                | f
>volume_classification | 1
>
>-[ RECORD 2 ]---------+-------------------------------------
>image_guid            | 68d764ec-bc2e-4e1d-b8f2-b44afd9fcb2e
>creation_date         | 2016-07-03 01:01:30+02
>size                  | 17179869184
>it_guid               | 00000000-0000-0000-0000-000000000000
>parentid              | 60ba7acf-58cb-475b-b9ee-15b1be99fee6
>imagestatus           | 1
>lastmodified          | 2016-07-04 01:03:33.732+02
>vm_snapshot_id        | 175c2071-a06b-4b0e-a069-5cc4bb236a34
>volume_type           | 2
>volume_format         | 4
>image_group_id        | 8cda0aa6-9e25-4b50-ba00-b877232a1983
>_create_date          | 2016-07-03 01:01:15.069585+02
>_update_date          | 2016-09-11 02:06:04.420965+02
>active                | f
>volume_classification | 1
>
>-[ RECORD 3 ]---------+-------------------------------------
>image_guid            | 37ca6494-e990-44e5-8597-28845a0a19b5
>creation_date         | 2016-08-07 01:06:15+02
>size                  | 17179869184
>it_guid               | 00000000-0000-0000-0000-000000000000
>parentid              | 68d764ec-bc2e-4e1d-b8f2-b44afd9fcb2e
>imagestatus           | 1
>lastmodified          | 2016-08-08 01:00:03.778+02
>vm_snapshot_id        | 4c0e5ac0-2ef3-4996-b3e9-7fd566d97b1a
>volume_type           | 2
>volume_format         | 4
>image_group_id        | 8cda0aa6-9e25-4b50-ba00-b877232a1983
>_create_date          | 2016-08-07 01:06:01.777156+02
>_update_date          | 2016-09-25 01:55:54.090026+02
>active                | f
>volume_classification | 1
>
>I removed other 10 snapshot infos from the list to easily read all
>
>On a host running v4.0.4 we run
>
>root at ovc2n06 ~]# lvchange -aey -pr
>384f9059-ef2f-4d43-a54f-de71c5d589c8/60ba7acf-58cb-475b-b9ee-15b1be99fee6
>   Logical volume "60ba7acf-58cb-475b-b9ee-15b1be99fee6" changed.
>
>This is the base image infact it doesn't contain a backing-file
>
>[root at ovc2n06 ~]# qemu-img info --backing-chain
>/dev/384f9059-ef2f-4d43-a54f-de71c5d589c8/60ba7acf-58cb-475b-b9ee-15b1be99fee6
>image:
>/dev/384f9059-ef2f-4d43-a54f-de71c5d589c8/60ba7acf-58cb-475b-b9ee-15b1be99fee6
>file format: raw
>virtual size: 16G (17179869184 bytes)
>disk size: 0
>
>[root at ovc2n06 ~]# lvchange -aey -pr
>384f9059-ef2f-4d43-a54f-de71c5d589c8/68d764ec-bc2e-4e1d-b8f2-b44afd9fcb2e
>   Logical volume "68d764ec-bc2e-4e1d-b8f2-b44afd9fcb2e" changed.
>
>[root at ovc2n06 ~]# qemu-img info --backing-chain
>/dev/384f9059-ef2f-4d43-a54f-de71c5d589c8/68d764ec-bc2e-4e1d-b8f2-b44afd9fcb2e
>qemu-img: Could not open
>'/dev/384f9059-ef2f-4d43-a54f-de71c5d589c8/../8cda0aa6-9e25-4b50-ba00-b877232a1983/../8cda0aa6-9e25-4b50-ba00-b877232a1983/../8cda0aa6-9e25-4b50-ba00-b877232a1983/../8cda0aa6-9e25-4b50-ba00-b877232a1983/../8cda0aa6-9e25-4b50-ba00-b877232a1983/../8cda0aa6-9e25-4b50-ba00-b877232a1983/../8cda0aa6-9e25-4b50-ba00-b877232a1983/../8cda0aa6-9e25-4b50-ba00-b877232a1983/../8cda0aa6-9e25-4b50-ba00-b877232a1983/../8cda0aa6-9e25-4b50-ba00-b877232a1983/../8cda0aa6-9e25-4b50-ba00-b877232a1983/../8cda0aa6-9e25-4b50-ba00-b877232a1983/../8cda0aa6-9e25-4b50-ba00-b877232a1983/../8cda0aa6-9e25-4b50-ba00-b877232a1983/../8cda0aa6-9e25-4b50-ba00-b877232a1983/../8cda0aa6-9e25-4b50-ba00-b877232a1983/../8cda0aa6-9e25-4b50-ba00-b877232a1983/../8cda0aa6-9e25-4b50-ba00-b877232a1983/../8cda0aa6-9e25-4b50-ba00-b877232a1983/../8cda0aa6-9e25-4b50-ba00-b877232a1983/../8cda0aa6-9e25-4b50-ba00-b877232a1983/../8cda0aa6-9e25-4b50-ba00-b877232a1983/../8cda0aa6-9e25-4b50-ba00-b877232a1983/60ba7acf-58cb-475b-b9ee-15b
>1be99fee6':
>Could not open
>'/dev/384f9059-ef2f-4d43-a54f-de71c5d589c8/../8cda0aa6-9e25-4b50-ba00-b877232a1983/../8cda0aa6-9e25-4b50-ba00-b877232a1983/../8cda0aa6-9e25-4b50-ba00-b877232a1983/../8cda0aa6-9e25-4b50-ba00-b877232a1983/../8cda0aa6-9e25-4b50-ba00-b877232a1983/../8cda0aa6-9e25-4b50-ba00-b877232a1983/../8cda0aa6-9e25-4b50-ba00-b877232a1983/../8cda0aa6-9e25-4b50-ba00-b877232a1983/../8cda0aa6-9e25-4b50-ba00-b877232a1983/../8cda0aa6-9e25-4b50-ba00-b877232a1983/../8cda0aa6-9e25-4b50-ba00-b877232a1983/../8cda0aa6-9e25-4b50-ba00-b877232a1983/../8cda0aa6-9e25-4b50-ba00-b877232a1983/../8cda0aa6-9e25-4b50-ba00-b877232a1983/../8cda0aa6-9e25-4b50-ba00-b877232a1983/../8cda0aa6-9e25-4b50-ba00-b877232a1983/../8cda0aa6-9e25-4b50-ba00-b877232a1983/../8cda0aa6-9e25-4b50-ba00-b877232a1983/../8cda0aa6-9e25-4b50-ba00-b877232a1983/../8cda0aa6-9e25-4b50-ba00-b877232a1983/../8cda0aa6-9e25-4b50-ba00-b877232a1983/../8cda0aa6-9e25-4b50-ba00-b877232a1983/../8cda0aa6-9e25-4b50-ba00-b877232a1983/60ba7acf-58cb-475b-b9ee-15b
>1be99fee6':
>No such file or directory
>
>To fix this problem we made a link
>
>[root at ovc2n06 ~]# lvchange -aey -pr
>384f9059-ef2f-4d43-a54f-de71c5d589c8/68d764ec-bc2e-4e1d-b8f2-b44afd9fcb2e
>   Logical volume "68d764ec-bc2e-4e1d-b8f2-b44afd9fcb2e" changed.
>[root at ovc2n06 ~]# qemu-img info --backing-chain
>/dev/384f9059-ef2f-4d43-a54f-de71c5d589c8/68d764ec-bc2e-4e1d-b8f2-b44afd9fcb2e
>qemu-img: Could not open
>'/dev/384f9059-ef2f-4d43-a54f-de71c5d589c8/../8cda0aa6-9e25-4b50-ba00-b877232a1983/../8cda0aa6-9e25-4b50-ba00-b877232a1983/../8cda0aa6-9e25-4b50-ba00-b877232a1983/../8cda0aa6-9e25-4b50-ba00-b877232a1983/../8cda0aa6-9e25-4b50-ba00-b877232a1983/../8cda0aa6-9e25-4b50-ba00-b877232a1983/../8cda0aa6-9e25-4b50-ba00-b877232a1983/../8cda0aa6-9e25-4b50-ba00-b877232a1983/../8cda0aa6-9e25-4b50-ba00-b877232a1983/../8cda0aa6-9e25-4b50-ba00-b877232a1983/../8cda0aa6-9e25-4b50-ba00-b877232a1983/../8cda0aa6-9e25-4b50-ba00-b877232a1983/../8cda0aa6-9e25-4b50-ba00-b877232a1983/../8cda0aa6-9e25-4b50-ba00-b877232a1983/../8cda0aa6-9e25-4b50-ba00-b877232a1983/../8cda0aa6-9e25-4b50-ba00-b877232a1983/../8cda0aa6-9e25-4b50-ba00-b877232a1983/../8cda0aa6-9e25-4b50-ba00-b877232a1983/../8cda0aa6-9e25-4b50-ba00-b877232a1983/../8cda0aa6-9e25-4b50-ba00-b877232a1983/../8cda0aa6-9e25-4b50-ba00-b877232a1983/../8cda0aa6-9e25-4b50-ba00-b877232a1983/../8cda0aa6-9e25-4b50-ba00-b877232a1983/60ba7acf-58cb-475b-b9ee-15b
>1be99fee6':
>Could not open
>'/dev/384f9059-ef2f-4d43-a54f-de71c5d589c8/../8cda0aa6-9e25-4b50-ba00-b877232a1983/../8cda0aa6-9e25-4b50-ba00-b877232a1983/../8cda0aa6-9e25-4b50-ba00-b877232a1983/../8cda0aa6-9e25-4b50-ba00-b877232a1983/../8cda0aa6-9e25-4b50-ba00-b877232a1983/../8cda0aa6-9e25-4b50-ba00-b877232a1983/../8cda0aa6-9e25-4b50-ba00-b877232a1983/../8cda0aa6-9e25-4b50-ba00-b877232a1983/../8cda0aa6-9e25-4b50-ba00-b877232a1983/../8cda0aa6-9e25-4b50-ba00-b877232a1983/../8cda0aa6-9e25-4b50-ba00-b877232a1983/../8cda0aa6-9e25-4b50-ba00-b877232a1983/../8cda0aa6-9e25-4b50-ba00-b877232a1983/../8cda0aa6-9e25-4b50-ba00-b877232a1983/../8cda0aa6-9e25-4b50-ba00-b877232a1983/../8cda0aa6-9e25-4b50-ba00-b877232a1983/../8cda0aa6-9e25-4b50-ba00-b877232a1983/../8cda0aa6-9e25-4b50-ba00-b877232a1983/../8cda0aa6-9e25-4b50-ba00-b877232a1983/../8cda0aa6-9e25-4b50-ba00-b877232a1983/../8cda0aa6-9e25-4b50-ba00-b877232a1983/../8cda0aa6-9e25-4b50-ba00-b877232a1983/../8cda0aa6-9e25-4b50-ba00-b877232a1983/60ba7acf-58cb-475b-b9ee-15b
>1be99fee6':
>No such file or directory
>[root at ovc2n06 ~]# ln -s /dev/384f9059-ef2f-4d43-a54f-de71c5d589c8
>/dev/8cda0aa6-9e25-4b50-ba00-b877232a1983
>[root at ovc2n06 ~]# qemu-img info --backing-chain
>/dev/384f9059-ef2f-4d43-a54f-de71c5d589c8/68d764ec-bc2e-4e1d-b8f2-b44afd9fcb2e
>image:
>/dev/384f9059-ef2f-4d43-a54f-de71c5d589c8/68d764ec-bc2e-4e1d-b8f2-b44afd9fcb2e
>file format: qcow2
>virtual size: 16G (17179869184 bytes)
>disk size: 0
>cluster_size: 65536
>backing file:
>../8cda0aa6-9e25-4b50-ba00-b877232a1983/../8cda0aa6-9e25-4b50-ba00-b877232a1983/../8cda0aa6-9e25-4b50-ba00-b877232a1983/../8cda0aa6-9e25-4b50-ba00-b877232a1983/../8cda0aa6-9e25-4b50-ba00-b877232a1983/../8cda0aa6-9e25-4b50-ba00-b877232a1983/../8cda0aa6-9e25-4b50-ba00-b877232a1983/../8cda0aa6-9e25-4b50-ba00-b877232a1983/../8cda0aa6-9e25-4b50-ba00-b877232a1983/../8cda0aa6-9e25-4b50-ba00-b877232a1983/../8cda0aa6-9e25-4b50-ba00-b877232a1983/../8cda0aa6-9e25-4b50-ba00-b877232a1983/../8cda0aa6-9e25-4b50-ba00-b877232a1983/../8cda0aa6-9e25-4b50-ba00-b877232a1983/../8cda0aa6-9e25-4b50-ba00-b877232a1983/../8cda0aa6-9e25-4b50-ba00-b877232a1983/../8cda0aa6-9e25-4b50-ba00-b877232a1983/../8cda0aa6-9e25-4b50-ba00-b877232a1983/../8cda0aa6-9e25-4b50-ba00-b877232a1983/../8cda0aa6-9e25-4b50-ba00-b877232a1983/../8cda0aa6-9e25-4b50-ba00-b877232a1983/../8cda0aa6-9e25-4b50-ba00-b877232a1983/../8cda0aa6-9e25-4b50-ba00-b877232a1983/60ba7acf-58cb-475b-b9ee-15b1be99fee6
>(actual path:
>/dev/384f9059-ef2f-4d43-a54f-de71c5d589c8/../8cda0aa6-9e25-4b50-ba00-b877232a1983/../8cda0aa6-9e25-4b50-ba00-b877232a1983/../8cda0aa6-9e25-4b50-ba00-b877232a1983/../8cda0aa6-9e25-4b50-ba00-b877232a1983/../8cda0aa6-9e25-4b50-ba00-b877232a1983/../8cda0aa6-9e25-4b50-ba00-b877232a1983/../8cda0aa6-9e25-4b50-ba00-b877232a1983/../8cda0aa6-9e25-4b50-ba00-b877232a1983/../8cda0aa6-9e25-4b50-ba00-b877232a1983/../8cda0aa6-9e25-4b50-ba00-b877232a1983/../8cda0aa6-9e25-4b50-ba00-b877232a1983/../8cda0aa6-9e25-4b50-ba00-b877232a1983/../8cda0aa6-9e25-4b50-ba00-b877232a1983/../8cda0aa6-9e25-4b50-ba00-b877232a1983/../8cda0aa6-9e25-4b50-ba00-b877232a1983/../8cda0aa6-9e25-4b50-ba00-b877232a1983/../8cda0aa6-9e25-4b50-ba00-b877232a1983/../8cda0aa6-9e25-4b50-ba00-b877232a1983/../8cda0aa6-9e25-4b50-ba00-b877232a1983/../8cda0aa6-9e25-4b50-ba00-b877232a1983/../8cda0aa6-9e25-4b50-ba00-b877232a1983/../8cda0aa6-9e25-4b50-ba00-b877232a1983/../8cda0aa6-9e25-4b50-ba00-b877232a1983/60ba7acf-58cb-475b-b9ee-15b1
>be99fee6)
>backing file format: raw
>Format specific information:
>     compat: 0.10
>     refcount bits: 16
>
>image:
>/dev/384f9059-ef2f-4d43-a54f-de71c5d589c8/../8cda0aa6-9e25-4b50-ba00-b877232a1983/../8cda0aa6-9e25-4b50-ba00-b877232a1983/../8cda0aa6-9e25-4b50-ba00-b877232a1983/../8cda0aa6-9e25-4b50-ba00-b877232a1983/../8cda0aa6-9e25-4b50-ba00-b877232a1983/../8cda0aa6-9e25-4b50-ba00-b877232a1983/../8cda0aa6-9e25-4b50-ba00-b877232a1983/../8cda0aa6-9e25-4b50-ba00-b877232a1983/../8cda0aa6-9e25-4b50-ba00-b877232a1983/../8cda0aa6-9e25-4b50-ba00-b877232a1983/../8cda0aa6-9e25-4b50-ba00-b877232a1983/../8cda0aa6-9e25-4b50-ba00-b877232a1983/../8cda0aa6-9e25-4b50-ba00-b877232a1983/../8cda0aa6-9e25-4b50-ba00-b877232a1983/../8cda0aa6-9e25-4b50-ba00-b877232a1983/../8cda0aa6-9e25-4b50-ba00-b877232a1983/../8cda0aa6-9e25-4b50-ba00-b877232a1983/../8cda0aa6-9e25-4b50-ba00-b877232a1983/../8cda0aa6-9e25-4b50-ba00-b877232a1983/../8cda0aa6-9e25-4b50-ba00-b877232a1983/../8cda0aa6-9e25-4b50-ba00-b877232a1983/../8cda0aa6-9e25-4b50-ba00-b877232a1983/../8cda0aa6-9e25-4b50-ba00-b877232a1983/60ba7acf-58cb-475b-b9ee-15b1
>be99fee6
>file format: raw
>virtual size: 16G (17179869184 bytes)
>disk size: 0
>
>We moved to the next snapshot
>
>[root at ovc2n06 ~]# lvchange -aey -pr
>384f9059-ef2f-4d43-a54f-de71c5d589c8/37ca6494-e990-44e5-8597-28845a0a19b5
>   Logical volume "37ca6494-e990-44e5-8597-28845a0a19b5" changed.
>
>And now the error from the bug
>
>qemu-img info --backing-chain /dev/384f9059-ef2f-4d43-a54f-de71c5d589c8/37ca6494-e990-44e5-8597-28845a0a19b5
>qemu-img: Could not open '/dev/384f9059-ef2f-4d43-a54f-de71c5d589c8/37ca6494-e990-44e5-8597-28845a0a19b5': Backing file name too long
>
>Any snapshot starting from this snapshot is not usable, the VM was
>running ok up to we did shutdown it, it never starts again.
>
>How you can see the path of the backing-file is wrong and too long
>already in the previous snapshots, is there a way to fix it or to edit
>it manually ?
>
>Obviously we tried to clone, export, create a qcow2 image, from all the
>snapshot later 07-august but the operation didn't complete, we can
>recover only from the snapshot of 7-august that is missing 2 months of
>new data.
>
>Please if you have a workaround or a solution, can you write the
>commands we need to run with examples, we searched a lot about
>backing-file but only the manual of the qemu-img command was found with
>no examples on how to recover or change it.
>
>Thank you!
>
>
>
>_______________________________________________
>Users mailing list
>Users at ovirt.org
>http://lists.ovirt.org/mailman/listinfo/users

-- 
Adam Litke

More information about the Users mailing list