Hi Steven,
I suspect this is somehow related to sanlock based on the error ResourceAcqusitionFailed,
Can u please attach the VDSM logs as well.
Liron, Nir do you have any insights about this?
Thanks,
Maor
----- Original Message -----
From: "Steven M. Bellistri" <SBellistri(a)ldirx.com>
To: "Elad Ben Aharon" <ebenahar(a)redhat.com>, "Maor Lipchuk"
<mlipchuk(a)redhat.com>, "Liron Aravot"
<laravot(a)redhat.com>
Cc: users(a)ovirt.org
Sent: Wednesday, February 11, 2015 7:07:00 PM
Subject: RE: [ovirt-users] importing iscsi storage domain
yes, no problem. I am including the logs from yesterday and today.
From: Elad Ben Aharon [mailto:ebenahar@redhat.com]
Sent: Wednesday, February 11, 2015 3:58 AM
To: Steven M. Bellistri; Maor Lipchuk; Liron Aravot
Cc: users(a)ovirt.org
Subject: Re: [ovirt-users] importing iscsi storage domain
Steven - can you please attach engine.log also?
(/var/log/ovirt-engine/engine.log)
Adding Maor and Liron from DEV.
Maor, Liron, can you take a look?
In vdsm.log I see the following:
Thread-77065::WARNING::2015-02-10
09:43:49,917::resourceManager::591::Storage.ResourceManager::(registerResource)
Resource factory failed to create resource
'2e94bbac-1648-49db-8605-ce574d2bc240_imageNS.952af113-8219-43a4-92a9-bfe913ec8b69'.
Canceling request.
Traceback (most recent call last):
File "/usr/share/vdsm/storage/resourceManager.py", line 587, in
registerResource
obj = namespaceObj.factory.createResource(name, lockType)
File "/usr/share/vdsm/storage/resourceFactories.py", line 193, in
createResource
lockType)
File "/usr/share/vdsm/storage/resourceFactories.py", line 119, in
__getResourceCandidatesList
repoPath = os.path.join(self.storage_repository, dom.getPools()[0])
IndexError: list index out of range
Thread-77065::DEBUG::2015-02-10
09:43:49,918::resourceManager::210::Storage.ResourceManager.Request::(cancel)
ResName=`2e94bbac-1648-49db-8605-ce574d2bc240_imageNS.952af113-8219-43a4-92a9-bfe913ec8b69`ReqID=`10e68fbb-ac50-461f-92ac-d8c43060ed43`::Canceled
request
Thread-77065::WARNING::2015-02-10
09:43:49,918::resourceManager::203::Storage.ResourceManager.Request::(cancel)
ResName=`2e94bbac-1648-49db-8605-ce574d2bc240_imageNS.952af113-8219-43a4-92a9-bfe913ec8b69`ReqID=`10e68fbb-ac50-461f-92ac-d8c43060ed43`::Tried
to cancel a processed request
Thread-77065::ERROR::2015-02-10
09:43:49,918::task::866::Storage.TaskManager.Task::(_setError)
Task=`6e023655-23ad-42cd-9b7e-6c5e5e3b08b9`::Unexpected error
Traceback (most recent call last):
File "/usr/share/vdsm/storage/task.py", line 873, in _run
return fn(*args, **kargs)
File "/usr/share/vdsm/logUtils.py", line 45, in wrapper
res = f(*args, **kwargs)
File "/usr/share/vdsm/storage/hsm.py", line 1285, in setVolumeDescription
pool.setVolumeDescription(sdUUID, imgUUID, volUUID, description)
File "/usr/share/vdsm/storage/securable.py", line 77, in wrapper
return method(self, *args, **kwargs)
File "/usr/share/vdsm/storage/sp.py", line 1884, in setVolumeDescription
rm.LockType.exclusive):
File "/usr/share/vdsm/storage/resourceManager.py", line 523, in
acquireResource
raise se.ResourceAcqusitionFailed()
ResourceAcqusitionFailed: Could not acquire resource. Probably resource
factory threw an exception.: ()
________________________________
From: "Steven M. Bellistri"
<SBellistri@ldirx.com<mailto:SBellistri@ldirx.com>>
To: "Elad Ben Aharon"
<ebenahar@redhat.com<mailto:ebenahar@redhat.com>>
Sent: Tuesday, 10 February, 2015 6:35:32 PM
Subject: RE: [ovirt-users] importing iscsi storage domain
Another issue, when I failback the 2 storage domains, the 2nd SD is not
working. The vm that has the DB drives will not boot after I failover (I
checked the virtio-scsi setting and it was checked), but I was getting an
error in the manager saying that: Failed to update OVF disks
952af113-8219-43a4-92a9-bfe913ec8b69, OVF data isn't updated on those OVF
stores (Data Center Default, Storage Domain iSCSI_2). So then I tried
putting the SD2 into maintenance mode and then reactivating it, but it would
not active or would it detach. I had to destroy the SD. The weird thing is
that I had no problems with the 1st SD. I am sending the vdsm.log file from
the host in the PR site.
From: Elad Ben Aharon [mailto:ebenahar@redhat.com]
Sent: Monday, February 9, 2015 1:26 AM
To: Steven M. Bellistri
Cc: Aharon Canan; users@ovirt.org<mailto:users@ovirt.org>
Subject: Re: [ovirt-users] importing iscsi storage domain
The import domain operation consists of several stages which their execution
time can vary.
It would be almost impossible to estimate how long it will take. But, since
the whole import operation, along with the VMs registration in the new
environment, doesn't involve any long time operations, like disks creation,
it shouldn't take more than a few minutes.
________________________________
From: "Steven M. Bellistri"
<SBellistri@ldirx.com<mailto:SBellistri@ldirx.com>>
To: "Elad Ben Aharon"
<ebenahar@redhat.com<mailto:ebenahar@redhat.com>>
Cc: "Aharon Canan" <acanan@redhat.com<mailto:acanan@redhat.com>>,
users@ovirt.org<mailto:users@ovirt.org>
Sent: Thursday, 5 February, 2015 5:41:11 PM
Subject: RE: [ovirt-users] importing iscsi storage domain
I put the storage domain in maintenance mode, then changed the time from
60minutes to 5min, then after I failed the storage over I was able to import
the domain. Thanks so much for the help. I just have one more question, is
there anyway to calculate how long it will take to import a domain, because
our production environment is about 5.5TB with about 40vm’s?
thanks
From: Elad Ben Aharon [mailto:ebenahar@redhat.com]
Sent: Thursday, February 5, 2015 1:49 AM
To: Steven M. Bellistri
Cc: Aharon Canan; users@ovirt.org<mailto:users@ovirt.org>
Subject: Re: [ovirt-users] importing iscsi storage domain
If you can't see the VMs under 'VM Import' subtab under the imported storage
domain, it means that you won't be able to recover those VMs (because their
configuration files weren't uploaded to the domain).
For the next time, you can decrease the time interval of the OVFs upload as
explained below in the thread.
Regarding the upgrade, it is possible to upgrade from 3.4 to 3.5 and once
you'll upgrade the setup and the DC compatibility version to 3.5, the OVFs
of the VMs will be uploaded to the storage domains where the VMs's disks are
located on. (again, every 60 minutes by default).
Elad Ben Aharon
RHEV-QE storage
[cid:image001.jpg@01D045EA.B12E26C0]
________________________________
From: "Steven M. Bellistri"
<SBellistri@ldirx.com<mailto:SBellistri@ldirx.com>>
To: "Elad Ben Aharon"
<ebenahar@redhat.com<mailto:ebenahar@redhat.com>>
Cc: "Aharon Canan" <acanan@redhat.com<mailto:acanan@redhat.com>>,
users@ovirt.org<mailto:users@ovirt.org>
Sent: Wednesday, 4 February, 2015 5:44:37 PM
Subject: RE: [ovirt-users] importing iscsi storage domain
No, I did not wait 60 minutes before I tried to failover. Do, I need to
start over, or can I just failback and wait the hour for the ovfs to upload?
I do have a question, is this feature possible for an environment that was
upgraded from 3.4 to 3.5? My production environment is 3.4, the 3.5
environment is a clean test environment, but if I cant upgrade to 3.5 from
3.4 then this will not be a good DR option for me.
thanks
From: Elad Ben Aharon [mailto:ebenahar@redhat.com]
Sent: Wednesday, February 4, 2015 8:56 AM
To: Steven M. Bellistri
Cc: Aharon Canan; users@ovirt.org<mailto:users@ovirt.org>
Subject: Re: [ovirt-users] importing iscsi storage domain
Hi Steven,
In order to import the VMs from the imported storage domain, the VMs
configuration files (which are called OVFs) have to be uploaded first before
the domain is detached from the setup. This is being done by engine in a
time interval of 60 minutes (it's configurable using rhevm-config --set
OvfUpdateInMinutes=<value> and restart ovirt-engine service). From 3.5.1,
the OVFs are uploaded once you move the storage domain to maintenance. did
you do so in the original RHEV environment? What was the version of this
environment?
If you didn't deactivate the domain in the first environment and then tried
to attach it to the DR environment, the existence of the VMs in the domain
depends on the OVFs upload that was supposed to be done by the first engine,
which is being done every 60 minutes as explained, so if you've created
those VMs and immediately destroyed the setup, those VMs probably won't be
available for you in the DR environment.
Elad Ben Aharon
RHEV-QE storage
[cid:image001.jpg@01D045EA.B12E26C0]
________________________________
From: "Steven M. Bellistri"
<SBellistri@LDIRx.com<mailto:SBellistri@LDIRx.com>>
To: "Aharon Canan" <acanan@redhat.com<mailto:acanan@redhat.com>>
Cc: users@ovirt.org<mailto:users@ovirt.org>
Sent: Tuesday, 3 February, 2015 6:16:01 PM
Subject: Re: [ovirt-users] importing iscsi storage domain
version 3.5.1
From: Aharon Canan [mailto:acanan@redhat.com]
Sent: Tuesday, February 3, 2015 6:22 AM
To: Steven M. Bellistri
Cc: users@ovirt.org<mailto:users@ovirt.org>
Subject: Re: [ovirt-users] importing iscsi storage domain
What version are you using?
Regards,
__________________________________________________
Aharon Canan
int phone - 8272036
ext phone - +97297692036
email - acanan@redhat.com<mailto:acanan@redhat.com>
________________________________
From: "Steven M. Bellistri"
<SBellistri@ldirx.com<mailto:SBellistri@ldirx.com>>
To: "Aharon Canan" <acanan@redhat.com<mailto:acanan@redhat.com>>
Cc: users@ovirt.org<mailto:users@ovirt.org>
Sent: Monday, February 2, 2015 8:12:00 PM
Subject: RE: [ovirt-users] importing iscsi storage domain
Thank you for the response.
I went thru the steps 1-9, and when I get to step 10 there is no import vms
or import templates sub-tabs. I am including the vdsm.log from the host.
Also, when I do a lvs on the host, I can see the disks that reside in the
imported SD, one is 30gb and the other is 40gb. The storage domain I am
importing is called iSCSI.
From: Aharon Canan [mailto:acanan@redhat.com]
Sent: Sunday, February 1, 2015 9:55 AM
To: Steven M. Bellistri
Cc: users@ovirt.org<mailto:users@ovirt.org>
Subject: Re: [ovirt-users] importing iscsi storage domain
Hi
Please check below link for info, ovirt version should be 3.5
http://www.ovirt.org/Features/ImportStorageDomain
please be aware to step 10 under "Work flow for Import block Storage Domain -
UI flow"
In case it is not working, please add exact steps and engine/vdsm logs so we
will be able to check
Regards,
__________________________________________________
Aharon Canan
int phone - 8272036
ext phone - +97297692036
email - acanan@redhat.com<mailto:acanan@redhat.com>
________________________________
From: "Steven M. Bellistri"
<SBellistri@LDIRx.com<mailto:SBellistri@LDIRx.com>>
To: "users@ovirt.org<mailto:users@ovirt.org>"
<users@ovirt.org<mailto:users@ovirt.org>>
Sent: Friday, January 30, 2015 12:18:54 AM
Subject: [ovirt-users] importing iscsi storage domain
I am trying to test failing over to a DR site using the storage domain
import. The DR site shows that it imported the iscsi storage domain, and
activates it in the datacenter, but none of the vm’s or anything shows up.
What am I missing.
thanks
Steven Bellistri
This email and its attachments may contain privileged and confidential
information and/or protected health information (PHI) intended solely for
the use of LDI Integrated Pharmacy Services and the recipient(s) named
above. If you are not the recipient, or the employee or agent responsible
for delivering this message to the intended recipient, you are hereby
notified that any review, dissemination, distribution, printing or copying
of this email message and/or any attachments is strictly prohibited. If you
have received this transmission in error, please notify the sender
immediately at 1-866-516-3121 and permanently delete this email and any
attachments.
_______________________________________________
Users mailing list
Users@ovirt.org<mailto:Users@ovirt.org>
http://lists.ovirt.org/mailman/listinfo/users
This email and its attachments may contain privileged and confidential
information and/or protected health information (PHI) intended solely for
the use of LDI Integrated Pharmacy Services and the recipient(s) named
above. If you are not the recipient, or the employee or agent responsible
for delivering this message to the intended recipient, you are hereby
notified that any review, dissemination, distribution, printing or copying
of this email message and/or any attachments is strictly prohibited. If you
have received this transmission in error, please notify the sender
immediately at 1-866-516-3121 and permanently delete this email and any
attachments.
This email and its attachments may contain privileged and confidential
information and/or protected health information (PHI) intended solely for
the use of LDI Integrated Pharmacy Services and the recipient(s) named
above. If you are not the recipient, or the employee or agent responsible
for delivering this message to the intended recipient, you are hereby
notified that any review, dissemination, distribution, printing or copying
of this email message and/or any attachments is strictly prohibited. If you
have received this transmission in error, please notify the sender
immediately at 1-866-516-3121 and permanently delete this email and any
attachments.
_______________________________________________
Users mailing list
Users@ovirt.org<mailto:Users@ovirt.org>
http://lists.ovirt.org/mailman/listinfo/users
This email and its attachments may contain privileged and confidential
information and/or protected health information (PHI) intended solely for
the use of LDI Integrated Pharmacy Services and the recipient(s) named
above. If you are not the recipient, or the employee or agent responsible
for delivering this message to the intended recipient, you are hereby
notified that any review, dissemination, distribution, printing or copying
of this email message and/or any attachments is strictly prohibited. If you
have received this transmission in error, please notify the sender
immediately at 1-866-516-3121 and permanently delete this email and any
attachments.
This email and its attachments may contain privileged and confidential
information and/or protected health information (PHI) intended solely for
the use of LDI Integrated Pharmacy Services and the recipient(s) named
above. If you are not the recipient, or the employee or agent responsible
for delivering this message to the intended recipient, you are hereby
notified that any review, dissemination, distribution, printing or copying
of this email message and/or any attachments is strictly prohibited. If you
have received this transmission in error, please notify the sender
immediately at 1-866-516-3121 and permanently delete this email and any
attachments.
This email and its attachments may contain privileged and confidential
information and/or protected health information (PHI) intended solely for
the use of LDI Integrated Pharmacy Services and the recipient(s) named
above. If you are not the recipient, or the employee or agent responsible
for delivering this message to the intended recipient, you are hereby
notified that any review, dissemination, distribution, printing or copying
of this email message and/or any attachments is strictly prohibited. If you
have received this transmission in error, please notify the sender
immediately at 1-866-516-3121 and permanently delete this email and any
attachments.
This email and its attachments may contain privileged and confidential
information and/or protected health information (PHI) intended solely for
the use of LDI Integrated Pharmacy Services and the recipient(s) named
above. If you are not the recipient, or the employee or agent responsible
for delivering this message to the intended recipient, you are hereby
notified that any review, dissemination, distribution, printing or copying
of this email message and/or any attachments is strictly prohibited. If you
have received this transmission in error, please notify the sender
immediately at 1-866-516-3121 and permanently delete this email and any
attachments.