Re: [ovirt-users] Import storage domain - disks not listed

On Mon, May 2, 2016 at 5:45 PM, Sahina Bose <sabose(a)redhat.com&gt; wrote: > > > On 05/02/2016 05:57 PM, Maor Lipchuk wrote: > > > > On Mon, May 2, 2016 at 1:08 PM, Sahina Bose <sabose(a)redhat.com&gt; wrote: >> >> >> On 05/02/2016 03:15 PM, Maor Lipchuk wrote: >> >> >> >> On Mon, May 2, 2016 at 12:29 PM, Sahina Bose <sabose(a)redhat.com&gt; wrote: >>> >>> >>> On 05/01/2016 05:33 AM, Maor Lipchuk wrote: >>> >>> Hi Sahina, >>> >>> The disks with snapshots should be part of the VMs, once you will register those VMs you should see those disks in the disks sub tab. >>> >>> >>> Maor, >>> >>> I was unable to import VM which prompted question - I assumed we had to register disks first. So maybe I need to troubleshoot why I could not import VMs from the domain first. >>> It fails with an error "Image does not exist". Where does it look for volume IDs to pass to GetImageInfoVDSCommand - the OVF disk? >>> >>> >>> In engine.log >>> >>> 2016-05-02 04:15:14,812 ERROR [org.ovirt.engine.core.vdsbroker.irsbroker.GetImageInfoVDSCommand] (ajp-/127.0.0.1:8702-1) [32f0b27c] Ir >>> sBroker::getImageInfo::Failed getting image info imageId='6f4da17a-05a2-4d77-8091-d2fca3bbea1c' does not exist on domainName='sahinasl >>> ave', domainId='5e1a37cf-933d-424c-8e3d-eb9e40b690a7', error code: 'VolumeDoesNotExist', message: Volume does not exist: (u'6f4da17a-0 >>> 5a2-4d77-8091-d2fca3bbea1c',) >>> 2016-05-02 04:15:14,814 WARN [org.ovirt.engine.core.vdsbroker.irsbroker.DoesImageExistVDSCommand] (ajp-/127.0.0.1:8702-1) [32f0b27c] >>> executeIrsBrokerCommand: getImageInfo on '6f4da17a-05a2-4d77-8091-d2fca3bbea1c' threw an exception - assuming image doesn't exist: IRS >>> GenericException: IRSErrorException: VolumeDoesNotExist >>> 2016-05-02 04:15:14,814 INFO [org.ovirt.engine.core.vdsbroker.irsbroker.DoesImageExistVDSCommand] (ajp-/127.0.0.1:8702-1) [32f0b27c] >>> FINISH, DoesImageExistVDSCommand, return: false, log id: 3366f39b >>> 2016-05-02 04:15:14,814 WARN [org.ovirt.engine.core.bll.ImportVmFromConfigurationCommand] (ajp-/127.0.0.1:8702-1) [32f0b27c] CanDoAction of action 'ImportVmFromConfiguration' failed for user admin@internal. Reasons: VAR__ACTION__IMPORT,VAR__TYPE__VM,ACTION_TYPE_FAILED_VM_IMAGE_DOES_NOT_EXIST >>> >>> >>> >>> jsonrpc.Executor/2::DEBUG::2016-05-02 13:45:13,903::__init__::503::jsonrpc.JsonRpcServer::(_serveRequest) Calling 'Volume.getInfo' in >>> bridge with {u'imageID': u'c52e4e02-dc6c-4a77-a184-9fcab88106c2', u'storagepoolID': u'46ac4975-a84e-4e76-8e73-7971d0dadf0b', u'volumeI >>> D': u'6f4da17a-05a2-4d77-8091-d2fca3bbea1c', u'storagedomainID': u'5e1a37cf-933d-424c-8e3d-eb9e40b690a7'} >>> >>> jsonrpc.Executor/2::DEBUG::2016-05-02 13:45:13,910::fileVolume::535::Storage.Volume::(validateVolumePath) validate path for 6f4da17a-05a2-4d77-8091-d2fca3bbea1c >>> jsonrpc.Executor/2::ERROR::2016-05-02 13:45:13,914::task::866::Storage.TaskManager.Task::(_setError) Task=`94dba16f-f7eb-439e-95e2-a04b34b92f84`::Unexpected error >>> Traceback (most recent call last): >>> File "/usr/share/vdsm/storage/task.py", line 873, in _run >>> return fn(*args, **kargs) >>> File "/usr/share/vdsm/logUtils.py", line 49, in wrapper >>> res = f(*args, **kwargs) >>> File "/usr/share/vdsm/storage/hsm.py", line 3162, in getVolumeInfo >>> volUUID=volUUID).getInfo() >>> File "/usr/share/vdsm/storage/sd.py", line 457, in produceVolume >>> volUUID) >>> File "/usr/share/vdsm/storage/glusterVolume.py", line 16, in __init__ >>> volUUID) >>> File "/usr/share/vdsm/storage/fileVolume.py", line 58, in __init__ >>> volume.Volume.__init__(self, repoPath, sdUUID, imgUUID, volUUID) >>> File "/usr/share/vdsm/storage/volume.py", line 181, in __init__ >>> self.validate() >>> File "/usr/share/vdsm/storage/volume.py", line 194, in validate >>> self.validateVolumePath() >>> File "/usr/share/vdsm/storage/fileVolume.py", line 540, in validateVolumePath >>> raise se.VolumeDoesNotExist(self.volUUID) >>> VolumeDoesNotExist: Volume does not exist: (u'6f4da17a-05a2-4d77-8091-d2fca3bbea1c',) >>> >>> When I look at the tree output - there's no 6f4da17a-05a2-4d77-8091-d2fca3bbea1c file. >>> >>> >>> ├── c52e4e02-dc6c-4a77-a184-9fcab88106c2 >>> │ │ │ ├── 34e46104-8fad-4510-a5bf-0730b97a6659 >>> │ │ │ ├── 34e46104-8fad-4510-a5bf-0730b97a6659.lease >>> │ │ │ ├── 34e46104-8fad-4510-a5bf-0730b97a6659.meta >>> │ │ │ ├── 766a15b9-57db-417d-bfa0-beadbbb84ad2 >>> │ │ │ ├── 766a15b9-57db-417d-bfa0-beadbbb84ad2.lease >>> │ │ │ ├── 766a15b9-57db-417d-bfa0-beadbbb84ad2.meta >>> │ │ │ ├── 90f1e26a-00e9-4ea5-9e92-2e448b9b8bfa >>> │ │ │ ├── 90f1e26a-00e9-4ea5-9e92-2e448b9b8bfa.lease >>> │ │ │ └── 90f1e26a-00e9-4ea5-9e92-2e448b9b8bfa.meta >> >> >> Usually the "image does not exists" message is prompted once the VM's disk is managed in a different storage domain which were not imported yet. >> >> Few questions: >> 1. Were there any other Storage Domain which are not present in the setup? >> >> >> In the original RHEV instance - there were 3 storage domain >> i) Hosted engine storage domain: engine >> ii) Master data domain: vmstore >> iii) An export domain: expVol (no data here) >> >> To my backup RHEV server, I only imported vmstore. > > Just to be sure, can you look into the server which the engine storage domain resides on. > Is there a chance that 6f4da17a-05a2-4d77-8091-d2fca3bbea1c could be there? > > Also do you have an old engine log, is there a chance to look for image 6f4da17a-05a2-4d77-8091-d2fca3bbea1c in there, to see when it was created and if there were any problems in the process. > > > > I checked the engine logs and it seems that image 6f4da17a-05a2-4d77-8091-d2fca3bbea1c was removed as part of snapshot merge. Could it be that the OVF was not updated? The OVF_STORE is being update every 60 minutes or when a Storage Domain is moved to maintenance. If the DR process was done before the OVF_STORE got updated and the setup was destroyed then that might be the reason for the missing image in the OVF. The problem is that this image id is already part of the images chain configured in the VM's OVF. As a work around maybe you can create a new volume with VdsClient createVolume on the Host with the same UUID. or maybe to merge all the disk's snapshots through the vdsClient and once there will be only one volume you can register this disk as floating disk.

> > engine.log-20160426.gz:2016-04-26 01:54:28,576 INFO [org.ovirt.engine.core.vdsbroker.vdsbroker.MergeVDSCommand] (pool-5-thread-1) [6f94943f] START, MergeVDSCommand(HostName = rhsdev9, MergeVDSCommandParameters:{runAsync='true', hostId='b9a662bf-e05e-4e6a-9dfe-ec1be76d48e7', vmId='e2b89e45-6f99-465d-aa08-fc4f746f0dd0', storagePoolId='00000001-0001-0001-0001-000000000305', storageDomainId='5e1a37cf-933d-424c-8e3d-eb9e40b690a7', imageGroupId='c52e4e02-dc6c-4a77-a184-9fcab88106c2', imageId='6f4da17a-05a2-4d77-8091-d2fca3bbea1c', baseImageId='90f1e26a-00e9-4ea5-9e92-2e448b9b8bfa', topImageId='6f4da17a-05a2-4d77-8091-d2fca3bbea1c', bandwidth='0'}), log id: 5a65a864 > engine.log-20160426.gz:2016-04-26 01:55:03,629 INFO [org.ovirt.engine.core.bll.MergeCommandCallback] (DefaultQuartzScheduler_Worker-57) [6f94943f] Merge command has completed for images '90f1e26a-00e9-4ea5-9e92-2e448b9b8bfa'..'6f4da17a-05a2-4d77-8091-d2fca3bbea1c' > engine.log-20160426.gz:2016-04-26 01:55:06,707 INFO [org.ovirt.engine.core.bll.MergeStatusCommand] (pool-5-thread-2) [3c72e9f8] Successfully removed volume(s): [6f4da17a-05a2-4d77-8091-d2fca3bbea1c] > > > >> >> 2. Can you look for the image id 6f4da17a-05a2-4d77-8091-d2fca3bbea1c in your storage server (Search on all the rest of the storage domains)? >> >> >> No - there was no file with this id (checked both in the original RHEV instance) >> [root@dhcp42-105 mnt]# pwd >> /rhev/data-center/mnt >> [root@dhcp42-105 mnt]# find . -name 6f4da17a-05a2-4d77-8091-d2fca3bbea1c >> [root@dhcp42-105 mnt]# >> >> >> Were there any operations being done on the VM before the recovery such as remove disk, move disk, or a new creation of a disk? >> >> >> No. The only operation done was creation of a snapshot - and it completed before the recovery was attempted. >> >> >> Regards, >> Maor >> >>> >>> Regarding floating disks (without snapshots), you can register them through REST. >>> If you are working on the master branch there should be a sub tab dedicated for those also. >>> >>> Regards, >>> Maor >>> >>> On Tue, Apr 26, 2016 at 1:44 PM, Sahina Bose <sabose(a)redhat.com&gt; wrote: >>>> Hi all, >>>> >>>> I have a gluster volume used as data storage domain which is replicated to a slave gluster volume (say, slavevol) using gluster's geo-replication feature. >>>> >>>> Now, in a new oVirt instance, I use the import storage domain to import the slave gluster volume. The "VM Import" tab correctly lists the VMs that were present in my original gluster volume. However the "Disks" tab is empty. >>>> >>>> GET https://new-ovitt/api/storagedomains/5e1a37cf-933d-424c-8e3d-eb9e40b690a7... --> >>>> <disks/> >>>> >>>> >>>> In the code GetUnregisteredDiskQuery - if volumesList.size() != 1 - the image is skipped with a comment that we can't deal with snapshots. >>>> >>>> How do I recover the disks/images in this case? >>>> >>>> >>>> Further info: >>>> >>>> /rhev/data-center/mnt/glusterSD/10.70.40.112:_slavevol >>>> ├── 5e1a37cf-933d-424c-8e3d-eb9e40b690a7 >>>> │ ├── dom_md >>>> │ │ ├── ids >>>> │ │ ├── inbox >>>> │ │ ├── leases >>>> │ │ ├── metadata >>>> │ │ └── outbox >>>> │ ├── images >>>> │ │ ├── 202efaa6-0d01-40f3-a541-10eee920d221 >>>> │ │ │ ├── eb701046-6ee1-4c9d-b097-e51a8fd283e1 >>>> │ │ │ ├── eb701046-6ee1-4c9d-b097-e51a8fd283e1.lease >>>> │ │ │ └── eb701046-6ee1-4c9d-b097-e51a8fd283e1.meta >>>> │ │ ├── c52e4e02-dc6c-4a77-a184-9fcab88106c2 >>>> │ │ │ ├── 34e46104-8fad-4510-a5bf-0730b97a6659 >>>> │ │ │ ├── 34e46104-8fad-4510-a5bf-0730b97a6659.lease >>>> │ │ │ ├── 34e46104-8fad-4510-a5bf-0730b97a6659.meta >>>> │ │ │ ├── 766a15b9-57db-417d-bfa0-beadbbb84ad2 >>>> │ │ │ ├── 766a15b9-57db-417d-bfa0-beadbbb84ad2.lease >>>> │ │ │ ├── 766a15b9-57db-417d-bfa0-beadbbb84ad2.meta >>>> │ │ │ ├── 90f1e26a-00e9-4ea5-9e92-2e448b9b8bfa >>>> │ │ │ ├── 90f1e26a-00e9-4ea5-9e92-2e448b9b8bfa.lease >>>> │ │ │ └── 90f1e26a-00e9-4ea5-9e92-2e448b9b8bfa.meta >>>> │ │ ├── c75de5b7-aa88-48d7-ba1b-067181eac6ae >>>> │ │ │ ├── ff09e16a-e8a0-452b-b95c-e160e68d09a9 >>>> │ │ │ ├── ff09e16a-e8a0-452b-b95c-e160e68d09a9.lease >>>> │ │ │ └── ff09e16a-e8a0-452b-b95c-e160e68d09a9.meta >>>> │ │ ├── efa94a0d-c08e-4ad9-983b-4d1d76bca865 >>>> │ │ │ ├── 64e3913c-da91-447c-8b69-1cff1f34e4b7 >>>> │ │ │ ├── 64e3913c-da91-447c-8b69-1cff1f34e4b7.lease >>>> │ │ │ ├── 64e3913c-da91-447c-8b69-1cff1f34e4b7.meta >>>> │ │ │ ├── 8174e8b4-3605-4db3-86a1-cb62c3a079f4 >>>> │ │ │ ├── 8174e8b4-3605-4db3-86a1-cb62c3a079f4.lease >>>> │ │ │ ├── 8174e8b4-3605-4db3-86a1-cb62c3a079f4.meta >>>> │ │ │ ├── e79a8821-bb4a-436a-902d-3876f107dd99 >>>> │ │ │ ├── e79a8821-bb4a-436a-902d-3876f107dd99.lease >>>> │ │ │ └── e79a8821-bb4a-436a-902d-3876f107dd99.meta >>>> │ │ └── f5eacc6e-4f16-4aa5-99ad-53ac1cda75b7 >>>> │ │ ├── 476bbfe9-1805-4c43-bde6-e7de5f7bd75d >>>> │ │ ├── 476bbfe9-1805-4c43-bde6-e7de5f7bd75d.lease >>>> │ │ └── 476bbfe9-1805-4c43-bde6-e7de5f7bd75d.meta >>>> │ └── master >>>> │ ├── tasks >>>> │ └── vms >>>> └── __DIRECT_IO_TEST__ >>>> >>>> engine.log: >>>> 2016-04-26 06:37:57,715 INFO [org.ovirt.engine.core.vdsbroker.irsbroker.GetImageInfoVDSCommand] (org.ovirt.thread.pool-6-thread-25) [5e6b7a53] FINISH, GetImageInfoVDSCommand, return: org.ov >>>> irt.engine.core.common.businessentities.storage.DiskImage@d4b3ac2f, log id: 7b693bad >>>> 2016-04-26 06:37:57,724 INFO [org.ovirt.engine.core.vdsbroker.irsbroker.GetVolumesListVDSCommand] (org.ovirt.thread.pool-6-thread-25) [5e6b7a53] START, GetVolumesListVDSCommand( StoragePool >>>> DomainAndGroupIdBaseVDSCommandParameters:{runAsync='true', storagePoolId='ed338557-5995-4634-97e2-15454a9d8800', ignoreFailoverLimit='false', storageDomainId='5e1a37cf-933d-424c-8e3d-eb9e40b >>>> 690a7', imageGroupId='c52e4e02-dc6c-4a77-a184-9fcab88106c2'}), log id: 741b9214 >>>> 2016-04-26 06:37:58,748 INFO [org.ovirt.engine.core.vdsbroker.irsbroker.GetVolumesListVDSCommand] (org.ovirt.thread.pool-6-thread-25) [5e6b7a53] FINISH, GetVolumesListVDSCommand, return: [9 >>>> 0f1e26a-00e9-4ea5-9e92-2e448b9b8bfa, 766a15b9-57db-417d-bfa0-beadbbb84ad2, 34e46104-8fad-4510-a5bf-0730b97a6659], log id: 741b9214 >>>> >>>> _______________________________________________ >>>> Users mailing list >>>> Users(a)ovirt.org >>>> http://lists.ovirt.org/mailman/listinfo/users >>> >>> >> >