[Engine-devel] Requirement for Locking Mechanism
Hi All, We are introducing a new short term locking mechanism at engine. The motivation is races which are occurring between different flows, the main problem is : we did not update status of some entity and other flow was started , which is a main cause for different collisions and situation which can not be fixed in appropriate way by engine or vdsm. The current status is a workaround which is trying to reduce a race : in many places in the code, there is additional query to DB in order to check the appropriate status of entity. The proposed solution is internal locking mechanism - which will lock an appropriate entity until its status will not be updated in DB, after that an internal short term lock will be released. The design for locking mechanism can be found here: http://www.ovirt.org/wiki/Features/DetailedLockMechanism The patch which is introduce an implementation for mechanism can be found here: http://gerrit.ovirt.org/#change,403 (Using of new mechanism all around a code and integration with bll will be sent in the future, including cleaning of workaround) Regards Michael
On 11/29/2011 12:13 PM, Michael Kublin wrote:
Hi All,
We are introducing a new short term locking mechanism at engine. The motivation is races which are occurring between different flows, the main problem is : we did not update status of some entity and other flow was started , which is a main cause for different collisions and situation which can not be fixed in appropriate way by engine or vdsm. The current status is a workaround which is trying to reduce a race : in many places in the code, there is additional query to DB in order to check the appropriate status of entity. The proposed solution is internal locking mechanism - which will lock an appropriate entity until its status will not be updated in DB, after that an internal short term lock will be released.
The design for locking mechanism can be found here: http://www.ovirt.org/wiki/Features/DetailedLockMechanism
The page still has many remains of the template.
The patch which is introduce an implementation for mechanism can be found here: http://gerrit.ovirt.org/#change,403 (Using of new mechanism all around a code and integration with bll will be sent in the future, including cleaning of workaround)
Regards Michael
_______________________________________________ Engine-devel mailing list Engine-devel@ovirt.org http://lists.ovirt.org/mailman/listinfo/engine-devel
There's an implicit assumption that there's only going to be a single active backend working against the database? Y.
----- Original Message -----
From: "Yaniv Kaul" <ykaul@redhat.com> To: "Michael Kublin" <mkublin@redhat.com> Cc: engine-devel@ovirt.org Sent: Tuesday, November 29, 2011 12:18:15 PM Subject: Re: [Engine-devel] Requirement for Locking Mechanism
On 11/29/2011 12:13 PM, Michael Kublin wrote:
Hi All,
We are introducing a new short term locking mechanism at engine. The motivation is races which are occurring between different flows, the main problem is : we did not update status of some entity and other flow was started , which is a main cause for different collisions and situation which can not be fixed in appropriate way by engine or vdsm. The current status is a workaround which is trying to reduce a race : in many places in the code, there is additional query to DB in order to check the appropriate status of entity. The proposed solution is internal locking mechanism - which will lock an appropriate entity until its status will not be updated in DB, after that an internal short term lock will be released.
The design for locking mechanism can be found here: http://www.ovirt.org/wiki/Features/DetailedLockMechanism
The page still has many remains of the template.
The patch which is introduce an implementation for mechanism can be found here: http://gerrit.ovirt.org/#change,403 (Using of new mechanism all around a code and integration with bll will be sent in the future, including cleaning of workaround)
Regards Michael
_______________________________________________ Engine-devel mailing list Engine-devel@ovirt.org http://lists.ovirt.org/mailman/listinfo/engine-devel
There's an implicit assumption that there's only going to be a single active backend working against the database? Y.
Yes, the backend will lock some entity, until its status will not be updated in db, after that the lock will be released. So, it will solve a race between canDoAction and beginning of execute of the command
On 11/29/2011 12:41 PM, Michael Kublin wrote:
----- Original Message -----
From: "Yaniv Kaul"<ykaul@redhat.com> To: "Michael Kublin"<mkublin@redhat.com> Cc: engine-devel@ovirt.org Sent: Tuesday, November 29, 2011 12:18:15 PM Subject: Re: [Engine-devel] Requirement for Locking Mechanism
On 11/29/2011 12:13 PM, Michael Kublin wrote:
Hi All,
We are introducing a new short term locking mechanism at engine. The motivation is races which are occurring between different flows, the main problem is : we did not update status of some entity and other flow was started , which is a main cause for different collisions and situation which can not be fixed in appropriate way by engine or vdsm. The current status is a workaround which is trying to reduce a race : in many places in the code, there is additional query to DB in order to check the appropriate status of entity. The proposed solution is internal locking mechanism - which will lock an appropriate entity until its status will not be updated in DB, after that an internal short term lock will be released.
The design for locking mechanism can be found here: http://www.ovirt.org/wiki/Features/DetailedLockMechanism The page still has many remains of the template.
The patch which is introduce an implementation for mechanism can be found here: http://gerrit.ovirt.org/#change,403 (Using of new mechanism all around a code and integration with bll will be sent in the future, including cleaning of workaround)
Regards Michael
_______________________________________________ Engine-devel mailing list Engine-devel@ovirt.org http://lists.ovirt.org/mailman/listinfo/engine-devel There's an implicit assumption that there's only going to be a single active backend working against the database? Y.
Yes, the backend will lock some entity, until its status will not be updated in db, after that the lock will be released. So, it will solve a race between canDoAction and beginning of execute of the command
Michael, what do you think the performance implications on the system?
_______________________________________________ Engine-devel mailing list Engine-devel@ovirt.org http://lists.ovirt.org/mailman/listinfo/engine-devel
----- Original Message -----
From: "Moran Goldboim" <mgoldboi@redhat.com> To: "Michael Kublin" <mkublin@redhat.com> Cc: "Yaniv Kaul" <ykaul@redhat.com>, engine-devel@ovirt.org Sent: Tuesday, November 29, 2011 1:40:03 PM Subject: Re: [Engine-devel] Requirement for Locking Mechanism
On 11/29/2011 12:41 PM, Michael Kublin wrote:
----- Original Message -----
From: "Yaniv Kaul"<ykaul@redhat.com> To: "Michael Kublin"<mkublin@redhat.com> Cc: engine-devel@ovirt.org Sent: Tuesday, November 29, 2011 12:18:15 PM Subject: Re: [Engine-devel] Requirement for Locking Mechanism
On 11/29/2011 12:13 PM, Michael Kublin wrote:
Hi All,
We are introducing a new short term locking mechanism at engine. The motivation is races which are occurring between different flows, the main problem is : we did not update status of some entity and other flow was started , which is a main cause for different collisions and situation which can not be fixed in appropriate way by engine or vdsm. The current status is a workaround which is trying to reduce a race : in many places in the code, there is additional query to DB in order to check the appropriate status of entity. The proposed solution is internal locking mechanism - which will lock an appropriate entity until its status will not be updated in DB, after that an internal short term lock will be released.
The design for locking mechanism can be found here: http://www.ovirt.org/wiki/Features/DetailedLockMechanism The page still has many remains of the template.
The patch which is introduce an implementation for mechanism can be found here: http://gerrit.ovirt.org/#change,403 (Using of new mechanism all around a code and integration with bll will be sent in the future, including cleaning of workaround)
Regards Michael
_______________________________________________ Engine-devel mailing list Engine-devel@ovirt.org http://lists.ovirt.org/mailman/listinfo/engine-devel There's an implicit assumption that there's only going to be a single active backend working against the database? Y.
Yes, the backend will lock some entity, until its status will not be updated in db, after that the lock will be released. So, it will solve a race between canDoAction and beginning of execute of the command
Michael, what do you think the performance implications on the system? Because of locks are implemented in memory and because of today we already has some in memory lock mechanism (which is wrong) and after introducing of new mechanism we actually can reduce number of db queries, I think that performance will not decrease and maybe we will fill some improvements
_______________________________________________ Engine-devel mailing list Engine-devel@ovirt.org http://lists.ovirt.org/mailman/listinfo/engine-devel
On 11/29/2011 12:41 PM, Michael Kublin wrote:
----- Original Message -----
From: "Yaniv Kaul"<ykaul@redhat.com> To: "Michael Kublin"<mkublin@redhat.com> Cc: engine-devel@ovirt.org Sent: Tuesday, November 29, 2011 12:18:15 PM Subject: Re: [Engine-devel] Requirement for Locking Mechanism
On 11/29/2011 12:13 PM, Michael Kublin wrote:
Hi All,
We are introducing a new short term locking mechanism at engine. The motivation is races which are occurring between different flows, the main problem is : we did not update status of some entity and other flow was started , which is a main cause for different collisions and situation which can not be fixed in appropriate way by engine or vdsm. The current status is a workaround which is trying to reduce a race : in many places in the code, there is additional query to DB in order to check the appropriate status of entity. The proposed solution is internal locking mechanism - which will lock an appropriate entity until its status will not be updated in DB, after that an internal short term lock will be released.
The design for locking mechanism can be found here: http://www.ovirt.org/wiki/Features/DetailedLockMechanism
The page still has many remains of the template.
The patch which is introduce an implementation for mechanism can be found here: http://gerrit.ovirt.org/#change,403 (Using of new mechanism all around a code and integration with bll will be sent in the future, including cleaning of workaround)
Regards Michael
_______________________________________________ Engine-devel mailing list Engine-devel@ovirt.org http://lists.ovirt.org/mailman/listinfo/engine-devel
There's an implicit assumption that there's only going to be a single active backend working against the database? Y.
Yes, the backend will lock some entity, until its status will not be updated in db, after that the lock will be released. So, it will solve a race between canDoAction and beginning of execute of the command
so how will distributed engines for scale out later on will look like with this mechanism?
On 11/29/2011 06:32 PM, Itamar Heim wrote:
On 11/29/2011 12:41 PM, Michael Kublin wrote:
----- Original Message -----
From: "Yaniv Kaul"<ykaul@redhat.com> To: "Michael Kublin"<mkublin@redhat.com> Cc: engine-devel@ovirt.org Sent: Tuesday, November 29, 2011 12:18:15 PM Subject: Re: [Engine-devel] Requirement for Locking Mechanism
On 11/29/2011 12:13 PM, Michael Kublin wrote:
Hi All,
We are introducing a new short term locking mechanism at engine. The motivation is races which are occurring between different flows, the main problem is : we did not update status of some entity and other flow was started , which is a main cause for different collisions and situation which can not be fixed in appropriate way by engine or vdsm. The current status is a workaround which is trying to reduce a race : in many places in the code, there is additional query to DB in order to check the appropriate status of entity. The proposed solution is internal locking mechanism - which will lock an appropriate entity until its status will not be updated in DB, after that an internal short term lock will be released.
The design for locking mechanism can be found here: http://www.ovirt.org/wiki/Features/DetailedLockMechanism
The page still has many remains of the template.
The patch which is introduce an implementation for mechanism can be found here: http://gerrit.ovirt.org/#change,403 (Using of new mechanism all around a code and integration with bll will be sent in the future, including cleaning of workaround)
Regards Michael
_______________________________________________ Engine-devel mailing list Engine-devel@ovirt.org http://lists.ovirt.org/mailman/listinfo/engine-devel
There's an implicit assumption that there's only going to be a single active backend working against the database? Y.
Yes, the backend will lock some entity, until its status will not be updated in db, after that the lock will be released. So, it will solve a race between canDoAction and beginning of execute of the command
so how will distributed engines for scale out later on will look like with this mechanism?
We currently have caching in oVirt core in several places, when we'll handle distributed deployment we'll handle all of them. it can be either distributed cache or any other suitable solution. Livnat
_______________________________________________ Engine-devel mailing list Engine-devel@ovirt.org http://lists.ovirt.org/mailman/listinfo/engine-devel
participants (5)
-
Itamar Heim -
Livnat Peer -
Michael Kublin -
Moran Goldboim -
Yaniv Kaul