6:41 a.m.
Hello
We use oVirt 4.2.5.2-1.el7 (Hosted engine / 4 hosts in cluster / about twenty virtual
machines)
Virtual machine disks are located on the Data Domain from FC SAN.
Snapshots of all virtual machines are created normally. But for one virtual machine, we
can not create a snapshot.
When we try to create a snapshot in the oVirt web console, we see such errors:
Aug 13, 2018, 1:05:06 PM Failed to complete snapshot 'KOM-APP14_BACKUP01' creation
for VM 'KOM-APP14'.
Aug 13, 2018, 1:05:01 PM VDSM KOM-VM14 command HSMGetAllTasksStatusesVDS failed: Could not
acquire resource. Probably resource factory threw an exception.: ()
Aug 13, 2018, 1:05:00 PM Snapshot 'KOM-APP14_BACKUP01' creation for VM
'KOM-APP14' was initiated by petya@sub.holding.com(a)sub.holding.com-authz.
At this time on the server with the role of "SPM" in the vdsm.log we see this:
...
2018-08-13 05:05:06,471-0500 INFO (jsonrpc/0) [jsonrpc.JsonRpcServer] RPC call
VM.getStats succeeded in 0.00 seconds (__init__:573)
2018-08-13 05:05:06,478-0500 INFO (jsonrpc/5) [jsonrpc.JsonRpcServer] RPC call
Image.deleteVolumes succeeded in 0.05 seconds (__init__:573)
2018-08-13 05:05:06,478-0500 INFO (tasks/3) [storage.ThreadPool.WorkerThread] START task
bb45ae7e-77e9-4fec-9ee2-8e1f0ad3d589 (cmd=<bound method Task.commit of
<vdsm.storage.task.Task instance at 0x7f06b85a2128>>, args=None)
(threadPool:208)
2018-08-13 05:05:07,009-0500 WARN (tasks/3) [storage.ResourceManager] Resource factory
failed to create resource
'01_img_6db73566-0f7f-4438-a9ef-6815075f45ea.cdf1751b-64d3-42bc-b9ef-b0174c7ea068'.
Canceling request. (resourceManager:543)
Traceback (most recent call last):
File "/usr/lib/python2.7/site-packages/vdsm/storage/resourceManager.py", line
539, in registerResource
obj = namespaceObj.factory.createResource(name, lockType)
File "/usr/lib/python2.7/site-packages/vdsm/storage/resourceFactories.py",
line 193, in createResource
lockType)
File "/usr/lib/python2.7/site-packages/vdsm/storage/resourceFactories.py",
line 122, in __getResourceCandidatesList
imgUUID=resourceName)
File "/usr/lib/python2.7/site-packages/vdsm/storage/image.py", line 213, in
getChain
if srcVol.isLeaf():
File "/usr/lib/python2.7/site-packages/vdsm/storage/volume.py", line 1430, in
isLeaf
return self._manifest.isLeaf()
File "/usr/lib/python2.7/site-packages/vdsm/storage/volume.py", line 139, in
isLeaf
return self.getVolType() == sc.type2name(sc.LEAF_VOL)
File "/usr/lib/python2.7/site-packages/vdsm/storage/volume.py", line 135, in
getVolType
self.voltype = self.getMetaParam(sc.VOLTYPE)
File "/usr/lib/python2.7/site-packages/vdsm/storage/volume.py", line 119, in
getMetaParam
meta = self.getMetadata()
File "/usr/lib/python2.7/site-packages/vdsm/storage/blockVolume.py", line 112,
in getMetadata
md = VolumeMetadata.from_lines(lines)
File "/usr/lib/python2.7/site-packages/vdsm/storage/volumemetadata.py", line
103, in from_lines
"Missing metadata key: %s: found: %s" % (e, md))
MetaDataKeyNotFoundError: Meta Data key not found error: ("Missing metadata key:
'DOMAIN': found: {'NONE':
'######################################################################################################################################################################################################################################################################################################################################################################################################################################################################################################################'}",)
2018-08-13 05:05:07,010-0500 WARN (tasks/3) [storage.ResourceManager.Request]
(ResName='01_img_6db73566-0f7f-4438-a9ef-6815075f45ea.cdf1751b-64d3-42bc-b9ef-b0174c7ea068',
ReqID='3d924e5e-60d1-47b0-86a7-c63585b56f09') Tried to cancel a processed request
(resourceManager:187)
2018-08-13 05:05:07,010-0500 ERROR (tasks/3) [storage.TaskManager.Task]
(Task='bb45ae7e-77e9-4fec-9ee2-8e1f0ad3d589') Unexpected error (task:875)
Traceback (most recent call last):
File "/usr/lib/python2.7/site-packages/vdsm/storage/task.py", line 882, in
_run
return fn(*args, **kargs)
File "/usr/lib/python2.7/site-packages/vdsm/storage/task.py", line 336, in
run
return self.cmd(*self.argslist, **self.argsdict)
File "/usr/lib/python2.7/site-packages/vdsm/storage/securable.py", line 79, in
wrapper
return method(self, *args, **kwargs)
File "/usr/lib/python2.7/site-packages/vdsm/storage/sp.py", line 1966, in
deleteVolume
with rm.acquireResource(img_ns, imgUUID, rm.EXCLUSIVE):
File "/usr/lib/python2.7/site-packages/vdsm/storage/resourceManager.py", line
1025, in acquireResource
return _manager.acquireResource(namespace, name, lockType, timeout=timeout)
File "/usr/lib/python2.7/site-packages/vdsm/storage/resourceManager.py", line
475, in acquireResource
raise se.ResourceAcqusitionFailed()
ResourceAcqusitionFailed: Could not acquire resource. Probably resource factory threw an
exception.: ()
2018-08-13 05:05:07,059-0500 INFO (tasks/3) [storage.ThreadPool.WorkerThread] FINISH task
bb45ae7e-77e9-4fec-9ee2-8e1f0ad3d589 (threadPool:210)
2018-08-13 05:05:07,246-0500 INFO (jsonrpc/1) [root]
/usr/libexec/vdsm/hooks/after_get_caps/50_openstacknet: rc=0 err= (hooks:110)
2018-08-13 05:05:07,660-0500 INFO (jsonrpc/1) [root]
/usr/libexec/vdsm/hooks/after_get_caps/openstacknet_utils.py: rc=0 err= (hooks:110)
2018-08-13 05:05:08,152-0500 INFO (jsonrpc/1) [root]
/usr/libexec/vdsm/hooks/after_get_caps/ovirt_provider_ovn_hook: rc=0 err= (hooks:110)
Please help us to solve this problem.