[PATCH] [Kimchi] Update Kimchi objectstore versioning.
From: Paulo Vital <pvital@linux.vnet.ibm.com> This patch is part of the solution for the Ginger Issue #51 and it depends on Wok's "Upgrade objectstore schema" patch. With the new structure of the objectstore changed in Wok 2.0.0, the schema of the objectstore file needs to be checked and, if necessary, updated to reflect the new structure. This patch removes from Kimchi the methods used to check the objectstore schema and upgrade it if necessary. These methods are now present in wok.plugins and are common to any Wok plugin. Paulo Vital (1): Update Kimchi objectstore versioning. root.py | 4 ++-- utils.py | 40 ---------------------------------------- 2 files changed, 2 insertions(+), 42 deletions(-) -- 2.5.0
From: Paulo Vital <pvital@linux.vnet.ibm.com> With the new structure of the objectstore changed in Wok 2.0.0, the schema of the objectstore file needs to be checked and, if necessary, updated to reflect the new structure. This patch removes from Kimchi the methods used to check the objectstore schema and upgrade it if necessary. These methods are now present in wok.plugins and are common to any Wok plugin. Signed-off-by: Paulo Vital <pvital@linux.vnet.ibm.com> --- root.py | 4 ++-- utils.py | 40 ---------------------------------------- 2 files changed, 2 insertions(+), 42 deletions(-) diff --git a/root.py b/root.py index 4fbe0fc..cee30f6 100644 --- a/root.py +++ b/root.py @@ -26,9 +26,9 @@ from wok.plugins.kimchi.i18n import messages from wok.plugins.kimchi.control import sub_nodes from wok.plugins.kimchi.model import model as kimchiModel from wok.plugins.kimchi.utils import upgrade_objectstore_data -from wok.plugins.kimchi.utils import upgrade_objectstore_schema from wok.plugins.kimchi.utils import upgrade_objectstore_template_disks from wok.root import WokRoot +from wok.utils import upgrade_objectstore_schema class KimchiRoot(WokRoot): @@ -68,7 +68,7 @@ class KimchiRoot(WokRoot): # Some paths or URI's present in the objectstore have changed after # Kimchi 2.0.0 release. Check here if an upgrade in the schema and data # are necessary. - if upgrade_objectstore_schema('version'): + if upgrade_objectstore_schema(config.get_object_store(), 'version'): upgrade_objectstore_data('icon', 'images', 'plugins/kimchi/') upgrade_objectstore_data('storagepool', '/storagepools', '/plugins/kimchi') diff --git a/utils.py b/utils.py index 0997faa..5a3f209 100644 --- a/utils.py +++ b/utils.py @@ -101,46 +101,6 @@ def validate_repo_url(url): raise InvalidParameter("KCHREPOS0002E") -def get_objectstore_fields(): - """ - Return a list with all fields of the objectstore. - """ - conn = sqlite3.connect(config.get_object_store(), timeout=10) - cursor = conn.cursor() - schema_fields = [] - sql = "PRAGMA table_info('objects')" - cursor.execute(sql) - for row in cursor.fetchall(): - schema_fields.append(row[1]) - return schema_fields - - -def upgrade_objectstore_schema(field=None): - """ - Add a new column (of type TEXT) in the objectstore schema. - """ - if field is None: - wok_log.error("Cannot upgrade objectstore schema.") - return False - - if field in get_objectstore_fields(): - return False - try: - conn = sqlite3.connect(config.get_object_store(), timeout=10) - cursor = conn.cursor() - sql = "ALTER TABLE objects ADD COLUMN %s TEXT" % field - cursor.execute(sql) - wok_log.info("Objectstore schema sucessfully upgraded.") - conn.close() - except sqlite3.Error, e: - if conn: - conn.rollback() - conn.close() - wok_log.error("Cannot upgrade objectstore schema: ", e.args[0]) - return False - return True - - def upgrade_objectstore_data(item, old_uri, new_uri): """ Upgrade the value of a given JSON's item of all Template and VM entries -- 2.5.0
This solution works for Kimchi and Ginger now, but I analyzed it a bit deeper and I cannot see it as a good long-term solution. Reasons: - other plugins will never be allowed to have a column named version in its table, for whatever reason they want; - every new record must have the version inserted, today we cannot measure the impact, but, if for some reason objectstore must be heavily used, this can impact; - we will have (or each plugin will have) to change this in future, because all 'object' tables will have 'version' column, then each plugin will have to manage by itself how to check and update the records or schemas; - finally, I think that Wok should not touch plugins 'objectstore'. Like I said, this solution works for Kimchi/Ginger now, and I am ok, once we are running out of time. But keep in mind , that we are going to have rework in future. Aline can decide what to do. Reviewed-by: Rodrigo Trujillo <rodrigo.trujillo@linux.vnet.ibm.com> On 12/14/2015 11:46 AM, pvital@linux.vnet.ibm.com wrote:
From: Paulo Vital <pvital@linux.vnet.ibm.com>
This patch is part of the solution for the Ginger Issue #51 and it depends on Wok's "Upgrade objectstore schema" patch.
With the new structure of the objectstore changed in Wok 2.0.0, the schema of the objectstore file needs to be checked and, if necessary, updated to reflect the new structure.
This patch removes from Kimchi the methods used to check the objectstore schema and upgrade it if necessary. These methods are now present in wok.plugins and are common to any Wok plugin.
Paulo Vital (1): Update Kimchi objectstore versioning.
root.py | 4 ++-- utils.py | 40 ---------------------------------------- 2 files changed, 2 insertions(+), 42 deletions(-)
-- 2.5.0
_______________________________________________ Kimchi-devel mailing list Kimchi-devel@ovirt.org http://lists.ovirt.org/mailman/listinfo/kimchi-devel
On 12/14/2015 01:06 PM, Rodrigo Trujillo wrote:
This solution works for Kimchi and Ginger now, but I analyzed it a bit deeper and I cannot see it as a good long-term solution.
I think the first and most important point you should see here is that Wok provides the objectstore structure for any plugin wants to use it. So, since a plugin is using the Wok's objectstore structure, the solution will works. Second, this change came from a Ginger and GingerBase issue regarding the missing 'version' field on their structure, something solved in Kimchi by commit 50b104a46e65047dfdf299ff8a5f652f2422a949. In addition, this patch depends on Wok's patch I sent a little bit early.
Reasons: - other plugins will never be allowed to have a column named version in its table, for whatever reason they want;
It's false. Once a plugin is using Wok's ObjectStore structure, a column version will be present starting on version 2.0.0. See commit fec97768abc8b407d4eaaca2fc892708da7210c9
- every new record must have the version inserted, today we cannot measure the impact, but, if for some reason objectstore must be heavily used, this can impact;
The complexity to record the version is the same of any other information, since it's done once - when all information is recorded.
- we will have (or each plugin will have) to change this in future, because all 'object' tables will have 'version' column, then each plugin will have to manage by itself how to check and update the records or schemas;
Kimchi already fixed this. Patches to Ginger and GingerBase was already submitted to Ginger ML.
- finally, I think that Wok should not touch plugins 'objectstore'.
Why not? It's a common infra-structure provided and that any plugin can use. All current plugins (kimchi, ginger and gingerbase) that use objectstore have a solution to this problem.
Like I said, this solution works for Kimchi/Ginger now, and I am ok, once we are running out of time. But keep in mind , that we are going to have rework in future. Aline can decide what to do.
Reviewed-by: Rodrigo Trujillo <rodrigo.trujillo@linux.vnet.ibm.com>
On 12/14/2015 11:46 AM, pvital@linux.vnet.ibm.com wrote:
From: Paulo Vital <pvital@linux.vnet.ibm.com>
This patch is part of the solution for the Ginger Issue #51 and it depends on Wok's "Upgrade objectstore schema" patch.
With the new structure of the objectstore changed in Wok 2.0.0, the schema of the objectstore file needs to be checked and, if necessary, updated to reflect the new structure.
This patch removes from Kimchi the methods used to check the objectstore schema and upgrade it if necessary. These methods are now present in wok.plugins and are common to any Wok plugin.
Paulo Vital (1): Update Kimchi objectstore versioning.
root.py | 4 ++-- utils.py | 40 ---------------------------------------- 2 files changed, 2 insertions(+), 42 deletions(-)
-- 2.5.0
_______________________________________________ Kimchi-devel mailing list Kimchi-devel@ovirt.org http://lists.ovirt.org/mailman/listinfo/kimchi-devel
On 12/14/2015 03:20 PM, Paulo Ricardo Paz Vital wrote:
On 12/14/2015 01:06 PM, Rodrigo Trujillo wrote:
This solution works for Kimchi and Ginger now, but I analyzed it a bit deeper and I cannot see it as a good long-term solution. I think the first and most important point you should see here is that Wok provides the objectstore structure for any plugin wants to use it. So, since a plugin is using the Wok's objectstore structure, the solution will works.
Second, this change came from a Ginger and GingerBase issue regarding the missing 'version' field on their structure, something solved in Kimchi by commit 50b104a46e65047dfdf299ff8a5f652f2422a949.
In addition, this patch depends on Wok's patch I sent a little bit early.
Reasons: - other plugins will never be allowed to have a column named version in its table, for whatever reason they want; It's false. Once a plugin is using Wok's ObjectStore structure, a column version will be present starting on version 2.0.0. See commit fec97768abc8b407d4eaaca2fc892708da7210c9 I see. I thought that version was tied to Wok's version...
- every new record must have the version inserted, today we cannot measure the impact, but, if for some reason objectstore must be heavily used, this can impact; The complexity to record the version is the same of any other information, since it's done once - when all information is recorded. I was talking about disk space and time to save/retrieve information.
- we will have (or each plugin will have) to change this in future, because all 'object' tables will have 'version' column, then each plugin will have to manage by itself how to check and update the records or schemas; Kimchi already fixed this. Patches to Ginger and GingerBase was already submitted to Ginger ML.
Thats why I said that works only for Kimchi/Ginger[Base] ... WOK should be independent of plugin, what I see, is that wok is being used to fix problems that the plugins should fix. Actually, plugins will have to check the version of records in the future.
- finally, I think that Wok should not touch plugins 'objectstore'.
Why not? It's a common infra-structure provided and that any plugin can use. All current plugins (kimchi, ginger and gingerbase) that use objectstore have a solution to this problem.
Bacause Wok is modifying what should be a private database. Its ok Wok provides the infra (creates the db, provide functions to add/remove/update, etc), it should not touch the data.
Like I said, this solution works for Kimchi/Ginger now, and I am ok, once we are running out of time. But keep in mind , that we are going to have rework in future. Aline can decide what to do.
Reviewed-by: Rodrigo Trujillo <rodrigo.trujillo@linux.vnet.ibm.com>
On 12/14/2015 11:46 AM, pvital@linux.vnet.ibm.com wrote:
From: Paulo Vital <pvital@linux.vnet.ibm.com>
This patch is part of the solution for the Ginger Issue #51 and it depends on Wok's "Upgrade objectstore schema" patch.
With the new structure of the objectstore changed in Wok 2.0.0, the schema of the objectstore file needs to be checked and, if necessary, updated to reflect the new structure.
This patch removes from Kimchi the methods used to check the objectstore schema and upgrade it if necessary. These methods are now present in wok.plugins and are common to any Wok plugin.
Paulo Vital (1): Update Kimchi objectstore versioning.
root.py | 4 ++-- utils.py | 40 ---------------------------------------- 2 files changed, 2 insertions(+), 42 deletions(-)
-- 2.5.0
_______________________________________________ Kimchi-devel mailing list Kimchi-devel@ovirt.org http://lists.ovirt.org/mailman/listinfo/kimchi-devel
Kimchi-devel mailing list Kimchi-devel@ovirt.org http://lists.ovirt.org/mailman/listinfo/kimchi-devel
On 12/14/2015 04:09 PM, Rodrigo Trujillo wrote:
On 12/14/2015 03:20 PM, Paulo Ricardo Paz Vital wrote:
On 12/14/2015 01:06 PM, Rodrigo Trujillo wrote:
This solution works for Kimchi and Ginger now, but I analyzed it a bit deeper and I cannot see it as a good long-term solution. I think the first and most important point you should see here is that Wok provides the objectstore structure for any plugin wants to use it. So, since a plugin is using the Wok's objectstore structure, the solution will works.
Second, this change came from a Ginger and GingerBase issue regarding the missing 'version' field on their structure, something solved in Kimchi by commit 50b104a46e65047dfdf299ff8a5f652f2422a949.
In addition, this patch depends on Wok's patch I sent a little bit early.
Reasons: - other plugins will never be allowed to have a column named version in its table, for whatever reason they want; It's false. Once a plugin is using Wok's ObjectStore structure, a column version will be present starting on version 2.0.0. See commit fec97768abc8b407d4eaaca2fc892708da7210c9 I see. I thought that version was tied to Wok's version...
- every new record must have the version inserted, today we cannot measure the impact, but, if for some reason objectstore must be heavily used, this can impact; The complexity to record the version is the same of any other information, since it's done once - when all information is recorded. I was talking about disk space and time to save/retrieve information.
- we will have (or each plugin will have) to change this in future, because all 'object' tables will have 'version' column, then each plugin will have to manage by itself how to check and update the records or schemas; Kimchi already fixed this. Patches to Ginger and GingerBase was already submitted to Ginger ML.
Thats why I said that works only for Kimchi/Ginger[Base] ... WOK should be independent of plugin, what I see, is that wok is being used to fix problems that the plugins should fix.
Actually, plugins will have to check the version of records in the future.
Now I understood what you were talking. Your point is correct, and the solution proposed is doing exactly that - plugins is checking if version is present. However, IMO, it's Wok's responsibility to provide the tools to do this, and that is what Wok and Kimchi patches are doing - moving the methods to check and upgrade the objectstore schema, that is the same code for all plugins, to Wok.
- finally, I think that Wok should not touch plugins 'objectstore'.
Why not? It's a common infra-structure provided and that any plugin can use. All current plugins (kimchi, ginger and gingerbase) that use objectstore have a solution to this problem.
Bacause Wok is modifying what should be a private database. Its ok Wok provides the infra (creates the db, provide functions to add/remove/update, etc), it should not touch the data.
But Wok is not doing that. The only modification that Wok will do, but only if plugin request to do, is upgrade the objectstore schema. Not it's data. Data update methods still reside on src/wok/plugin/kimchi/utils.py file.
Like I said, this solution works for Kimchi/Ginger now, and I am ok, once we are running out of time. But keep in mind , that we are going to have rework in future. Aline can decide what to do.
Reviewed-by: Rodrigo Trujillo <rodrigo.trujillo@linux.vnet.ibm.com>
On 12/14/2015 11:46 AM, pvital@linux.vnet.ibm.com wrote:
From: Paulo Vital <pvital@linux.vnet.ibm.com>
This patch is part of the solution for the Ginger Issue #51 and it depends on Wok's "Upgrade objectstore schema" patch.
With the new structure of the objectstore changed in Wok 2.0.0, the schema of the objectstore file needs to be checked and, if necessary, updated to reflect the new structure.
This patch removes from Kimchi the methods used to check the objectstore schema and upgrade it if necessary. These methods are now present in wok.plugins and are common to any Wok plugin.
Paulo Vital (1): Update Kimchi objectstore versioning.
root.py | 4 ++-- utils.py | 40 ---------------------------------------- 2 files changed, 2 insertions(+), 42 deletions(-)
-- 2.5.0
_______________________________________________ Kimchi-devel mailing list Kimchi-devel@ovirt.org http://lists.ovirt.org/mailman/listinfo/kimchi-devel
Kimchi-devel mailing list Kimchi-devel@ovirt.org http://lists.ovirt.org/mailman/listinfo/kimchi-devel
_______________________________________________ Kimchi-devel mailing list Kimchi-devel@ovirt.org http://lists.ovirt.org/mailman/listinfo/kimchi-devel
On 12/14/2015 04:27 PM, Paulo Ricardo Paz Vital wrote:
On 12/14/2015 04:09 PM, Rodrigo Trujillo wrote:
On 12/14/2015 03:20 PM, Paulo Ricardo Paz Vital wrote:
On 12/14/2015 01:06 PM, Rodrigo Trujillo wrote:
This solution works for Kimchi and Ginger now, but I analyzed it a bit deeper and I cannot see it as a good long-term solution. I think the first and most important point you should see here is that Wok provides the objectstore structure for any plugin wants to use it. So, since a plugin is using the Wok's objectstore structure, the solution will works.
Second, this change came from a Ginger and GingerBase issue regarding the missing 'version' field on their structure, something solved in Kimchi by commit 50b104a46e65047dfdf299ff8a5f652f2422a949.
In addition, this patch depends on Wok's patch I sent a little bit early.
Reasons: - other plugins will never be allowed to have a column named version in its table, for whatever reason they want; It's false. Once a plugin is using Wok's ObjectStore structure, a column version will be present starting on version 2.0.0. See commit fec97768abc8b407d4eaaca2fc892708da7210c9 I see. I thought that version was tied to Wok's version... - every new record must have the version inserted, today we cannot measure the impact, but, if for some reason objectstore must be heavily used, this can impact; The complexity to record the version is the same of any other information, since it's done once - when all information is recorded. I was talking about disk space and time to save/retrieve information.
- we will have (or each plugin will have) to change this in future, because all 'object' tables will have 'version' column, then each plugin will have to manage by itself how to check and update the records or schemas; Kimchi already fixed this. Patches to Ginger and GingerBase was already submitted to Ginger ML. Thats why I said that works only for Kimchi/Ginger[Base] ... WOK should be independent of plugin, what I see, is that wok is being used to fix problems that the plugins should fix.
Actually, plugins will have to check the version of records in the future.
Now I understood what you were talking. Your point is correct, and the solution proposed is doing exactly that - plugins is checking if version is present.
However, IMO, it's Wok's responsibility to provide the tools to do this, and that is what Wok and Kimchi patches are doing - moving the methods to check and upgrade the objectstore schema, that is the same code for all plugins, to Wok.
- finally, I think that Wok should not touch plugins 'objectstore'.
Why not? It's a common infra-structure provided and that any plugin can use. All current plugins (kimchi, ginger and gingerbase) that use objectstore have a solution to this problem. Bacause Wok is modifying what should be a private database. Its ok Wok provides the infra (creates the db, provide functions to add/remove/update, etc), it should not touch the data.
But Wok is not doing that. The only modification that Wok will do, but only if plugin request to do, is upgrade the objectstore schema. Not it's data.
Data update methods still reside on src/wok/plugin/kimchi/utils.py file.
You are right Paulo, data is not included (as I firstly thought), just schema is modified by wok.
Like I said, this solution works for Kimchi/Ginger now, and I am ok, once we are running out of time. But keep in mind , that we are going to have rework in future. Aline can decide what to do.
Reviewed-by: Rodrigo Trujillo <rodrigo.trujillo@linux.vnet.ibm.com>
On 12/14/2015 11:46 AM, pvital@linux.vnet.ibm.com wrote:
From: Paulo Vital <pvital@linux.vnet.ibm.com>
This patch is part of the solution for the Ginger Issue #51 and it depends on Wok's "Upgrade objectstore schema" patch.
With the new structure of the objectstore changed in Wok 2.0.0, the schema of the objectstore file needs to be checked and, if necessary, updated to reflect the new structure.
This patch removes from Kimchi the methods used to check the objectstore schema and upgrade it if necessary. These methods are now present in wok.plugins and are common to any Wok plugin.
Paulo Vital (1): Update Kimchi objectstore versioning.
root.py | 4 ++-- utils.py | 40 ---------------------------------------- 2 files changed, 2 insertions(+), 42 deletions(-)
-- 2.5.0
_______________________________________________ Kimchi-devel mailing list Kimchi-devel@ovirt.org http://lists.ovirt.org/mailman/listinfo/kimchi-devel
Kimchi-devel mailing list Kimchi-devel@ovirt.org http://lists.ovirt.org/mailman/listinfo/kimchi-devel
_______________________________________________ Kimchi-devel mailing list Kimchi-devel@ovirt.org http://lists.ovirt.org/mailman/listinfo/kimchi-devel
_______________________________________________ Kimchi-devel mailing list Kimchi-devel@ovirt.org http://lists.ovirt.org/mailman/listinfo/kimchi-devel
Applied. Thanks. Regards, Aline Manera
participants (4)
-
Aline Manera -
Paulo Ricardo Paz Vital -
pvital＠linux.vnet.ibm.com
-
Rodrigo Trujillo