Urgent help needed / Snapshot deletion failure

25 Sep 2019

      Dear all we are desperate

We have tried to delete a Snapshot and it hangs at merging the snapshots. We've found out that this is a known bug because the merge process is being called with wrong parameters according to the bug report https://bugzilla.redhat.com/show_bug.cgi?id=1601212. The Snapshot's Disks are in illegal state and the VM is locked. Since it is a very important VM for our non-profit organization, we need to have this machine back online as soon as possible. Is there a way to fix this without updating to qemu-kvm-ev-2.12.0? At least back to the state before the deletion would be fantastic.

Our vdsm.log:
2019-09-25 14:01:41,283+0200 WARN  (vdsm.Scheduler) [Executor] Worker blocked: <Worker name=jsonrpc/0 running <Task <JsonRpcTask {'params': {u'topVolUUID': u'8a9f190f-2725-4535-a69d-c74e4e57d372', u'vmID': u'1422899a-2151-4d5d-9d66-e74f19084542', u'drive': {u'imageID': u'eb2bce92-e758-4bea-93fa-02a56574b932', u'volumeID': u'8a9f190f-2725-4535-a69d-c74e4e57d372', u'domainID': u'022f39ee-eeb8-4b51-9549-9d7e3c88d4a8', u'poolID': u'00000001-0001-0001-0001-000000000307'}, u'bandwidth': u'0', u'jobUUID': u'c3fdc4a9-9d6d-424a-b9df-b96be5622e0a', u'baseVolUUID': u'3e15121b-0795-4056-bafe-448068c9ec71'}, 'jsonrpc': '2.0', 'method': u'VM.merge', 'id': u'9cd540b7-a32f-4f95-9fe2-9ce70d5b6478'} at 0x7f674fec5710> timeout=60, duration=6420 at 0x7f674fec58d0> task#=1896299 at 0x7f674c06ae90>, traceback:
File: "/usr/lib64/python2.7/threading.py", line 785, in __bootstrap
  self.__bootstrap_inner()
File: "/usr/lib64/python2.7/threading.py", line 812, in __bootstrap_inner
  self.run()
File: "/usr/lib64/python2.7/threading.py", line 765, in run
  self.__target(*self.__args, **self.__kwargs)
File: "/usr/lib/python2.7/site-packages/vdsm/common/concurrent.py", line 194, in run
  ret = func(*args, **kwargs)
File: "/usr/lib/python2.7/site-packages/vdsm/executor.py", line 301, in _run
  self._execute_task()
File: "/usr/lib/python2.7/site-packages/vdsm/executor.py", line 315, in _execute_task
  task()
File: "/usr/lib/python2.7/site-packages/vdsm/executor.py", line 391, in __call__
  self._callable()
File: "/usr/lib/python2.7/site-packages/yajsonrpc/__init__.py", line 523, in __call__
  self._handler(self._ctx, self._req)
File: "/usr/lib/python2.7/site-packages/yajsonrpc/__init__.py", line 566, in _serveRequest
  response = self._handle_request(req, ctx)
File: "/usr/lib/python2.7/site-packages/yajsonrpc/__init__.py", line 606, in _handle_request
  res = method(**params)
File: "/usr/lib/python2.7/site-packages/vdsm/rpc/Bridge.py", line 197, in _dynamicMethod
  result = fn(*methodArgs)
File: "<string>", line 2, in merge
File: "/usr/lib/python2.7/site-packages/vdsm/common/api.py", line 48, in method
  ret = func(*args, **kwargs)
File: "<string>", line 2, in merge
File: "/usr/lib/python2.7/site-packages/vdsm/common/api.py", line 122, in method
  ret = func(*args, **kwargs)
File: "/usr/lib/python2.7/site-packages/vdsm/API.py", line 739, in merge
  drive, baseVolUUID, topVolUUID, bandwidth, jobUUID)
File: "/usr/lib/python2.7/site-packages/vdsm/virt/vm.py", line 6041, in merge
  self.updateVmJobs()
File: "/usr/lib/python2.7/site-packages/vdsm/virt/vm.py", line 5818, in updateVmJobs
  self._vmJobs = self.queryBlockJobs()
File: "/usr/lib/python2.7/site-packages/vdsm/virt/vm.py", line 5832, in queryBlockJobs
  with self._jobsLock:
File: "/usr/lib/python2.7/site-packages/pthreading.py", line 60, in __enter__
  self.acquire()
File: "/usr/lib/python2.7/site-packages/pthreading.py", line 68, in acquire
  rc = self.lock() if blocking else self.trylock()
File: "/usr/lib/python2.7/site-packages/pthread.py", line 96, in lock
  return _libpthread.pthread_mutex_lock(self._mutex) (executor:363)

Kind regards
Simon

smirta＠gmx.net

smirta＠gmx.net

tags

participants (1)