<div dir="ltr"><br><div class="gmail_extra"><br><div class="gmail_quote">On Fri, May 6, 2016 at 3:08 PM, Sahina Bose <span dir="ltr"><<a href="mailto:sabose@redhat.com" target="_blank">sabose@redhat.com</a>></span> wrote:<br><blockquote class="gmail_quote" style="margin:0px 0px 0px 0.8ex;border-left-width:1px;border-left-color:rgb(204,204,204);border-left-style:solid;padding-left:1ex">Hi,<br>
<br>
Back to Import Storage domain questions -<br>
<br>
To ensure consistency, we snapshot the running VMs prior to replicating the gluster volume to a central site. The VMs have been setup such that OS disks are on a gluster volume called "vmstore" and non-OS disks are on a gluster volume called "data"<br>
For back up , we are only interested in "data" - so only this storage domain is imported at backup recovery site.<br>
<br>
Since the "data" storage domain does not contain all disks - VMs cannot be imported - that's expected.<br>
<br>
To retrieve the backed up disks - since the disk has snapshot - this does not seem possible either. How do we work around this?<br></blockquote><div><br></div><div>Since the engine can't support floating disks with snapshot you can try to merge those snapshots (by using qemu-img merge and qemu-img rebase.) until there will be only one volume left.</div><div>Once there will be only one volume oVirt can import the disk.</div><div><br></div><div>Another workaround might be that if you still have the setup which could not import the VM because of missing volume, then, you can try manipulate the OVF in the Data Base.</div><div>The VMs' and Template's XML representation are contained in the "unregistered_ovf_of_entities" table.</div><div>You can fetch the VM's OVF from that table:</div><div> SELECT ovf_data FROM unregistered_ovf_of_entities where entity_name ilike '%(the VM name (You can also fetch by id with entity_id))%';<br></div><div><br></div><div>Update the OVF with the correlated volume ids in the storage server. After updating this, the import of the VM should work</div><div><br></div><div> </div><blockquote class="gmail_quote" style="margin:0px 0px 0px 0.8ex;border-left-width:1px;border-left-color:rgb(204,204,204);border-left-style:solid;padding-left:1ex">
<br>
thanks!<br>
<br>
<br>
<br>
On 05/03/2016 03:21 PM, Maor Lipchuk wrote:<br>
<blockquote class="gmail_quote" style="margin:0px 0px 0px 0.8ex;border-left-width:1px;border-left-color:rgb(204,204,204);border-left-style:solid;padding-left:1ex">
There is this bug <a href="https://bugzilla.redhat.com/1270562" rel="noreferrer" target="_blank">https://bugzilla.redhat.com/1270562</a> which forces<br>
OVF_STORE update but it is not yet implemented.<br>
<br>
On Tue, May 3, 2016 at 8:52 AM, Sahina Bose <<a href="mailto:sabose@redhat.com" target="_blank">sabose@redhat.com</a>> wrote:<br>
<blockquote class="gmail_quote" style="margin:0px 0px 0px 0.8ex;border-left-width:1px;border-left-color:rgb(204,204,204);border-left-style:solid;padding-left:1ex">
<br>
On 05/02/2016 09:36 PM, Maor Lipchuk wrote:<br>
<blockquote class="gmail_quote" style="margin:0px 0px 0px 0.8ex;border-left-width:1px;border-left-color:rgb(204,204,204);border-left-style:solid;padding-left:1ex">
On Mon, May 2, 2016 at 5:45 PM, Sahina Bose <<a href="mailto:sabose@redhat.com" target="_blank">sabose@redhat.com</a>> wrote:<br>
<blockquote class="gmail_quote" style="margin:0px 0px 0px 0.8ex;border-left-width:1px;border-left-color:rgb(204,204,204);border-left-style:solid;padding-left:1ex">
<br>
<br>
On 05/02/2016 05:57 PM, Maor Lipchuk wrote:<br>
<br>
<br>
<br>
On Mon, May 2, 2016 at 1:08 PM, Sahina Bose <<a href="mailto:sabose@redhat.com" target="_blank">sabose@redhat.com</a>> wrote:<br>
<blockquote class="gmail_quote" style="margin:0px 0px 0px 0.8ex;border-left-width:1px;border-left-color:rgb(204,204,204);border-left-style:solid;padding-left:1ex">
<br>
<br>
On 05/02/2016 03:15 PM, Maor Lipchuk wrote:<br>
<br>
<br>
<br>
On Mon, May 2, 2016 at 12:29 PM, Sahina Bose <<a href="mailto:sabose@redhat.com" target="_blank">sabose@redhat.com</a>> wrote:<br>
<blockquote class="gmail_quote" style="margin:0px 0px 0px 0.8ex;border-left-width:1px;border-left-color:rgb(204,204,204);border-left-style:solid;padding-left:1ex">
<br>
<br>
On 05/01/2016 05:33 AM, Maor Lipchuk wrote:<br>
<br>
Hi Sahina,<br>
<br>
The disks with snapshots should be part of the VMs, once you will<br>
register those VMs you should see those disks in the disks sub tab.<br>
<br>
<br>
Maor,<br>
<br>
I was unable to import VM which prompted question - I assumed we had to<br>
register disks first. So maybe I need to troubleshoot why I could not import<br>
VMs from the domain first.<br>
It fails with an error "Image does not exist". Where does it look for<br>
volume IDs to pass to GetImageInfoVDSCommand - the OVF disk?<br>
<br>
<br>
In engine.log<br>
<br>
2016-05-02 04:15:14,812 ERROR<br>
[org.ovirt.engine.core.vdsbroker.irsbroker.GetImageInfoVDSCommand]<br>
(ajp-/127.0.0.1:8702-1) [32f0b27c] Ir<br>
sBroker::getImageInfo::Failed getting image info<br>
imageId='6f4da17a-05a2-4d77-8091-d2fca3bbea1c' does not exist on<br>
domainName='sahinasl<br>
ave', domainId='5e1a37cf-933d-424c-8e3d-eb9e40b690a7', error code:<br>
'VolumeDoesNotExist', message: Volume does not exist: (u'6f4da17a-0<br>
5a2-4d77-8091-d2fca3bbea1c',)<br>
2016-05-02 04:15:14,814 WARN<br>
[org.ovirt.engine.core.vdsbroker.irsbroker.DoesImageExistVDSCommand]<br>
(ajp-/127.0.0.1:8702-1) [32f0b27c]<br>
executeIrsBrokerCommand: getImageInfo on<br>
'6f4da17a-05a2-4d77-8091-d2fca3bbea1c' threw an exception - assuming image<br>
doesn't exist: IRS<br>
GenericException: IRSErrorException: VolumeDoesNotExist<br>
2016-05-02 04:15:14,814 INFO<br>
[org.ovirt.engine.core.vdsbroker.irsbroker.DoesImageExistVDSCommand]<br>
(ajp-/127.0.0.1:8702-1) [32f0b27c]<br>
FINISH, DoesImageExistVDSCommand, return: false, log id: 3366f39b<br>
2016-05-02 04:15:14,814 WARN<br>
[org.ovirt.engine.core.bll.ImportVmFromConfigurationCommand]<br>
(ajp-/127.0.0.1:8702-1) [32f0b27c] CanDoAction of action<br>
'ImportVmFromConfiguration' failed for user admin@internal. Reasons:<br>
VAR__ACTION__IMPORT,VAR__TYPE__VM,ACTION_TYPE_FAILED_VM_IMAGE_DOES_NOT_EXIST<br>
<br>
<br>
<br>
jsonrpc.Executor/2::DEBUG::2016-05-02<br>
13:45:13,903::__init__::503::jsonrpc.JsonRpcServer::(_serveRequest) Calling<br>
'Volume.getInfo' in<br>
bridge with {u'imageID': u'c52e4e02-dc6c-4a77-a184-9fcab88106c2',<br>
u'storagepoolID': u'46ac4975-a84e-4e76-8e73-7971d0dadf0b', u'volumeI<br>
D': u'6f4da17a-05a2-4d77-8091-d2fca3bbea1c', u'storagedomainID':<br>
u'5e1a37cf-933d-424c-8e3d-eb9e40b690a7'}<br>
<br>
jsonrpc.Executor/2::DEBUG::2016-05-02<br>
13:45:13,910::fileVolume::535::Storage.Volume::(validateVolumePath) validate<br>
path for 6f4da17a-05a2-4d77-8091-d2fca3bbea1c<br>
jsonrpc.Executor/2::ERROR::2016-05-02<br>
13:45:13,914::task::866::Storage.TaskManager.Task::(_setError)<br>
Task=`94dba16f-f7eb-439e-95e2-a04b34b92f84`::Unexpected error<br>
Traceback (most recent call last):<br>
File "/usr/share/vdsm/storage/task.py", line 873, in _run<br>
return fn(*args, **kargs)<br>
File "/usr/share/vdsm/logUtils.py", line 49, in wrapper<br>
res = f(*args, **kwargs)<br>
File "/usr/share/vdsm/storage/hsm.py", line 3162, in getVolumeInfo<br>
volUUID=volUUID).getInfo()<br>
File "/usr/share/vdsm/storage/sd.py", line 457, in produceVolume<br>
volUUID)<br>
File "/usr/share/vdsm/storage/glusterVolume.py", line 16, in<br>
__init__<br>
volUUID)<br>
File "/usr/share/vdsm/storage/fileVolume.py", line 58, in __init__<br>
volume.Volume.__init__(self, repoPath, sdUUID, imgUUID, volUUID)<br>
File "/usr/share/vdsm/storage/volume.py", line 181, in __init__<br>
self.validate()<br>
File "/usr/share/vdsm/storage/volume.py", line 194, in validate<br>
self.validateVolumePath()<br>
File "/usr/share/vdsm/storage/fileVolume.py", line 540, in<br>
validateVolumePath<br>
raise se.VolumeDoesNotExist(self.volUUID)<br>
VolumeDoesNotExist: Volume does not exist:<br>
(u'6f4da17a-05a2-4d77-8091-d2fca3bbea1c',)<br>
<br>
When I look at the tree output - there's no<br>
6f4da17a-05a2-4d77-8091-d2fca3bbea1c file.<br>
<br>
<br>
├── c52e4e02-dc6c-4a77-a184-9fcab88106c2<br>
│ │ │ ├── 34e46104-8fad-4510-a5bf-0730b97a6659<br>
│ │ │ ├── 34e46104-8fad-4510-a5bf-0730b97a6659.lease<br>
│ │ │ ├── 34e46104-8fad-4510-a5bf-0730b97a6659.meta<br>
│ │ │ ├── 766a15b9-57db-417d-bfa0-beadbbb84ad2<br>
│ │ │ ├── 766a15b9-57db-417d-bfa0-beadbbb84ad2.lease<br>
│ │ │ ├── 766a15b9-57db-417d-bfa0-beadbbb84ad2.meta<br>
│ │ │ ├── 90f1e26a-00e9-4ea5-9e92-2e448b9b8bfa<br>
│ │ │ ├── 90f1e26a-00e9-4ea5-9e92-2e448b9b8bfa.lease<br>
│ │ │ └── 90f1e26a-00e9-4ea5-9e92-2e448b9b8bfa.meta<br>
</blockquote>
<br>
<br>
Usually the "image does not exists" message is prompted once the VM's<br>
disk is managed in a different storage domain which were not imported yet.<br>
<br>
Few questions:<br>
1. Were there any other Storage Domain which are not present in the<br>
setup?<br>
<br>
<br>
In the original RHEV instance - there were 3 storage domain<br>
i) Hosted engine storage domain: engine<br>
ii) Master data domain: vmstore<br>
iii) An export domain: expVol (no data here)<br>
<br>
To my backup RHEV server, I only imported vmstore.<br>
</blockquote>
<br>
Just to be sure, can you look into the server which the engine storage<br>
domain resides on.<br>
Is there a chance that 6f4da17a-05a2-4d77-8091-d2fca3bbea1c could be<br>
there?<br>
<br>
Also do you have an old engine log, is there a chance to look for image<br>
6f4da17a-05a2-4d77-8091-d2fca3bbea1c in there, to see when it was created<br>
and if there were any problems in the process.<br>
<br>
<br>
<br>
I checked the engine logs and it seems that image<br>
6f4da17a-05a2-4d77-8091-d2fca3bbea1c was removed as part of snapshot merge.<br>
Could it be that the OVF was not updated?<br>
</blockquote>
<br>
<br>
The OVF_STORE is being update every 60 minutes or when a Storage<br>
Domain is moved to maintenance.<br>
If the DR process was done before the OVF_STORE got updated and the<br>
setup was destroyed then that might be the reason for the missing<br>
image in the OVF.<br>
<br>
The problem is that this image id is already part of the images chain<br>
configured in the VM's OVF.<br>
As a work around maybe you can create a new volume with VdsClient<br>
createVolume on the Host with the same UUID.<br>
or maybe to merge all the disk's snapshots through the vdsClient and<br>
once there will be only one volume you can register this disk as<br>
floating disk.<br>
</blockquote>
<br>
Is there a way via API to update OVF_STORE?<br>
This way before the DR process we can ensure that the OVF_STORE is referring<br>
to the correct images.<br>
<br>
<br>
<blockquote class="gmail_quote" style="margin:0px 0px 0px 0.8ex;border-left-width:1px;border-left-color:rgb(204,204,204);border-left-style:solid;padding-left:1ex">
<br>
<blockquote class="gmail_quote" style="margin:0px 0px 0px 0.8ex;border-left-width:1px;border-left-color:rgb(204,204,204);border-left-style:solid;padding-left:1ex">
engine.log-20160426.gz:2016-04-26 01:54:28,576 INFO<br>
[org.ovirt.engine.core.vdsbroker.vdsbroker.MergeVDSCommand]<br>
(pool-5-thread-1) [6f94943f] START, MergeVDSCommand(HostName = rhsdev9,<br>
MergeVDSCommandParameters:{runAsync='true',<br>
hostId='b9a662bf-e05e-4e6a-9dfe-ec1be76d48e7',<br>
vmId='e2b89e45-6f99-465d-aa08-fc4f746f0dd0',<br>
storagePoolId='00000001-0001-0001-0001-000000000305',<br>
storageDomainId='5e1a37cf-933d-424c-8e3d-eb9e40b690a7',<br>
imageGroupId='c52e4e02-dc6c-4a77-a184-9fcab88106c2',<br>
imageId='6f4da17a-05a2-4d77-8091-d2fca3bbea1c',<br>
baseImageId='90f1e26a-00e9-4ea5-9e92-2e448b9b8bfa',<br>
topImageId='6f4da17a-05a2-4d77-8091-d2fca3bbea1c', bandwidth='0'}), log id:<br>
5a65a864<br>
engine.log-20160426.gz:2016-04-26 01:55:03,629 INFO<br>
[org.ovirt.engine.core.bll.MergeCommandCallback]<br>
(DefaultQuartzScheduler_Worker-57) [6f94943f] Merge command has completed<br>
for images<br>
'90f1e26a-00e9-4ea5-9e92-2e448b9b8bfa'..'6f4da17a-05a2-4d77-8091-d2fca3bbea1c'<br>
engine.log-20160426.gz:2016-04-26 01:55:06,707 INFO<br>
[org.ovirt.engine.core.bll.MergeStatusCommand] (pool-5-thread-2) [3c72e9f8]<br>
Successfully removed volume(s): [6f4da17a-05a2-4d77-8091-d2fca3bbea1c]<br>
<br>
<br>
<br>
<blockquote class="gmail_quote" style="margin:0px 0px 0px 0.8ex;border-left-width:1px;border-left-color:rgb(204,204,204);border-left-style:solid;padding-left:1ex">
2. Can you look for the image id 6f4da17a-05a2-4d77-8091-d2fca3bbea1c in<br>
your storage server (Search on all the rest of the storage domains)?<br>
<br>
<br>
No - there was no file with this id (checked both in the original RHEV<br>
instance)<br>
[root@dhcp42-105 mnt]# pwd<br>
/rhev/data-center/mnt<br>
[root@dhcp42-105 mnt]# find . -name<br>
6f4da17a-05a2-4d77-8091-d2fca3bbea1c<br>
[root@dhcp42-105 mnt]#<br>
<br>
<br>
Were there any operations being done on the VM before the recovery such<br>
as remove disk, move disk, or a new creation of a disk?<br>
<br>
<br>
No. The only operation done was creation of a snapshot - and it<br>
completed before the recovery was attempted.<br>
<br>
<br>
Regards,<br>
Maor<br>
<br>
<blockquote class="gmail_quote" style="margin:0px 0px 0px 0.8ex;border-left-width:1px;border-left-color:rgb(204,204,204);border-left-style:solid;padding-left:1ex">
Regarding floating disks (without snapshots), you can register them<br>
through REST.<br>
If you are working on the master branch there should be a sub tab<br>
dedicated for those also.<br>
<br>
Regards,<br>
Maor<br>
<br>
On Tue, Apr 26, 2016 at 1:44 PM, Sahina Bose <<a href="mailto:sabose@redhat.com" target="_blank">sabose@redhat.com</a>> wrote:<br>
<blockquote class="gmail_quote" style="margin:0px 0px 0px 0.8ex;border-left-width:1px;border-left-color:rgb(204,204,204);border-left-style:solid;padding-left:1ex">
Hi all,<br>
<br>
I have a gluster volume used as data storage domain which is<br>
replicated to a slave gluster volume (say, slavevol) using gluster's<br>
geo-replication feature.<br>
<br>
Now, in a new oVirt instance, I use the import storage domain to<br>
import the slave gluster volume. The "VM Import" tab correctly lists the VMs<br>
that were present in my original gluster volume. However the "Disks" tab is<br>
empty.<br>
<br>
GET<br>
<a href="https://new-ovitt/api/storagedomains/5e1a37cf-933d-424c-8e3d-eb9e40b690a7/disks;unregistered" rel="noreferrer" target="_blank">https://new-ovitt/api/storagedomains/5e1a37cf-933d-424c-8e3d-eb9e40b690a7/disks;unregistered</a><br>
--><br>
<disks/><br>
<br>
<br>
In the code GetUnregisteredDiskQuery - if volumesList.size() != 1 -<br>
the image is skipped with a comment that we can't deal with snapshots.<br>
<br>
How do I recover the disks/images in this case?<br>
<br>
<br>
Further info:<br>
<br>
/rhev/data-center/mnt/glusterSD/10.70.40.112:_slavevol<br>
├── 5e1a37cf-933d-424c-8e3d-eb9e40b690a7<br>
│ ├── dom_md<br>
│ │ ├── ids<br>
│ │ ├── inbox<br>
│ │ ├── leases<br>
│ │ ├── metadata<br>
│ │ └── outbox<br>
│ ├── images<br>
│ │ ├── 202efaa6-0d01-40f3-a541-10eee920d221<br>
│ │ │ ├── eb701046-6ee1-4c9d-b097-e51a8fd283e1<br>
│ │ │ ├── eb701046-6ee1-4c9d-b097-e51a8fd283e1.lease<br>
│ │ │ └── eb701046-6ee1-4c9d-b097-e51a8fd283e1.meta<br>
│ │ ├── c52e4e02-dc6c-4a77-a184-9fcab88106c2<br>
│ │ │ ├── 34e46104-8fad-4510-a5bf-0730b97a6659<br>
│ │ │ ├── 34e46104-8fad-4510-a5bf-0730b97a6659.lease<br>
│ │ │ ├── 34e46104-8fad-4510-a5bf-0730b97a6659.meta<br>
│ │ │ ├── 766a15b9-57db-417d-bfa0-beadbbb84ad2<br>
│ │ │ ├── 766a15b9-57db-417d-bfa0-beadbbb84ad2.lease<br>
│ │ │ ├── 766a15b9-57db-417d-bfa0-beadbbb84ad2.meta<br>
│ │ │ ├── 90f1e26a-00e9-4ea5-9e92-2e448b9b8bfa<br>
│ │ │ ├── 90f1e26a-00e9-4ea5-9e92-2e448b9b8bfa.lease<br>
│ │ │ └── 90f1e26a-00e9-4ea5-9e92-2e448b9b8bfa.meta<br>
│ │ ├── c75de5b7-aa88-48d7-ba1b-067181eac6ae<br>
│ │ │ ├── ff09e16a-e8a0-452b-b95c-e160e68d09a9<br>
│ │ │ ├── ff09e16a-e8a0-452b-b95c-e160e68d09a9.lease<br>
│ │ │ └── ff09e16a-e8a0-452b-b95c-e160e68d09a9.meta<br>
│ │ ├── efa94a0d-c08e-4ad9-983b-4d1d76bca865<br>
│ │ │ ├── 64e3913c-da91-447c-8b69-1cff1f34e4b7<br>
│ │ │ ├── 64e3913c-da91-447c-8b69-1cff1f34e4b7.lease<br>
│ │ │ ├── 64e3913c-da91-447c-8b69-1cff1f34e4b7.meta<br>
│ │ │ ├── 8174e8b4-3605-4db3-86a1-cb62c3a079f4<br>
│ │ │ ├── 8174e8b4-3605-4db3-86a1-cb62c3a079f4.lease<br>
│ │ │ ├── 8174e8b4-3605-4db3-86a1-cb62c3a079f4.meta<br>
│ │ │ ├── e79a8821-bb4a-436a-902d-3876f107dd99<br>
│ │ │ ├── e79a8821-bb4a-436a-902d-3876f107dd99.lease<br>
│ │ │ └── e79a8821-bb4a-436a-902d-3876f107dd99.meta<br>
│ │ └── f5eacc6e-4f16-4aa5-99ad-53ac1cda75b7<br>
│ │ ├── 476bbfe9-1805-4c43-bde6-e7de5f7bd75d<br>
│ │ ├── 476bbfe9-1805-4c43-bde6-e7de5f7bd75d.lease<br>
│ │ └── 476bbfe9-1805-4c43-bde6-e7de5f7bd75d.meta<br>
│ └── master<br>
│ ├── tasks<br>
│ └── vms<br>
└── __DIRECT_IO_TEST__<br>
<br>
engine.log:<br>
2016-04-26 06:37:57,715 INFO<br>
[org.ovirt.engine.core.vdsbroker.irsbroker.GetImageInfoVDSCommand]<br>
(org.ovirt.thread.pool-6-thread-25) [5e6b7a53] FINISH,<br>
GetImageInfoVDSCommand, return: org.ov<br>
irt.engine.core.common.businessentities.storage.DiskImage@d4b3ac2f,<br>
log id: 7b693bad<br>
2016-04-26 06:37:57,724 INFO<br>
[org.ovirt.engine.core.vdsbroker.irsbroker.GetVolumesListVDSCommand]<br>
(org.ovirt.thread.pool-6-thread-25) [5e6b7a53] START,<br>
GetVolumesListVDSCommand( StoragePool<br>
DomainAndGroupIdBaseVDSCommandParameters:{runAsync='true',<br>
storagePoolId='ed338557-5995-4634-97e2-15454a9d8800',<br>
ignoreFailoverLimit='false',<br>
storageDomainId='5e1a37cf-933d-424c-8e3d-eb9e40b<br>
690a7', imageGroupId='c52e4e02-dc6c-4a77-a184-9fcab88106c2'}), log id:<br>
741b9214<br>
2016-04-26 06:37:58,748 INFO<br>
[org.ovirt.engine.core.vdsbroker.irsbroker.GetVolumesListVDSCommand]<br>
(org.ovirt.thread.pool-6-thread-25) [5e6b7a53] FINISH,<br>
GetVolumesListVDSCommand, return: [9<br>
0f1e26a-00e9-4ea5-9e92-2e448b9b8bfa,<br>
766a15b9-57db-417d-bfa0-beadbbb84ad2, 34e46104-8fad-4510-a5bf-0730b97a6659],<br>
log id: 741b9214<br>
<br>
_______________________________________________<br>
Users mailing list<br>
<a href="mailto:Users@ovirt.org" target="_blank">Users@ovirt.org</a><br>
<a href="http://lists.ovirt.org/mailman/listinfo/users" rel="noreferrer" target="_blank">http://lists.ovirt.org/mailman/listinfo/users</a><br>
</blockquote>
<br>
<br>
</blockquote></blockquote></blockquote></blockquote></blockquote></blockquote>
<br>
</blockquote></div><br></div></div>