[ovirt-devel] Issues on master with releasing a lock

Hi All, We have some issues with a few flows on master where the lock acquired is being released more than once. This could be a real problem in cases where the lock is prematurely released for a second command that had acquired the lock which the first command has released. For instance 1. Command A has acquired the VM lock and executes a child command. 2. The child command releases the lock 3. Command B now acquires the VM lock 4. Command A continues execution and finally releases the lock the second time 5. The Command B's lock has now been released prematurely by Command A. The tell tale sign indicating that a flow is one of the offending flows is the warning message in the logs indicating that a previously released lock is being released again. [org.ovirt.engine.core.bll.lock.InMemoryLockManager] (EE-ManagedThreadFactory-engine-Thread-6) [1435f925] Trying to release exclusive lock which does not exist, lock key: '53cfa6c3-ecd6-4796-84cb-5c53d7c2a77fVDS_FENCE' I have seen issues with Live Merge [1], Moving storage domain to maintenance[2] , VdsNotRespondingTreatmentCommand [3] and Creating a snapshot [4]. I have submitted some patches to fix the issue with Live Merge [5], Moving storage domain to maintenance [6] and VdsNotRespondingTreatmentCommand [7]. The multiple lock release issue needs to fixed and if you are the owner of a flow please make sure the flow does not log a warning message to the logs. There is no single solution to fix the issue, this has be done flow by flow. But below are a few pointers 1. If a command has a callback and the command is executing a child command, which in turn is executing a few child commands. The child command should also have a command callback so that the locks are released by the framework. Example patch [6] 2. If a parent has acquired the lock and is executing a child command with cloned context, to make sure that the child does not release the locks acquired by the parent pass the cloned parent context with out locks to the child. Example patch [7] Thanks Ravi [1] https://bugzilla.redhat.com/1568556 [2] https://bugzilla.redhat.com/1568447 [3] https://bugzilla.redhat.com/1571300 [4] https://bugzilla.redhat.com/1569625 <https://bugzilla.redhat.com/1569625> [5] https://gerrit.ovirt.org/#/c/90482/ [6] https://gerrit.ovirt.org/#/c/90411/ [7] https://gerrit.ovirt.org/#/c/90568/