[Engine-devel] Introduce a change to oVirt-engine-core DB
--=_3a2650e5-0f37-4517-8bfb-2025d36037dc Content-Type: text/plain; charset=utf-8 Content-Transfer-Encoding: 7bit Hi, I would like to introduce a change to two tables in oVirt-engine-core. The VM and VM-template share most of their configuration. I am looking to unify the two tables into one table. Currently, if such a change would be made in the DB layer, it would be abstracted by the DAL so no code beyond that layer should be affected, and code changes to that layer should be minimal (since we use views and stored procedures to abstract the DB structure anyway). The major change would be at the DB layer, which will mostly require data migration and changing of the STPs. If anyone has PostgresSQL knowledge and would like to pitch in and help me push this change it would be great. I have more technical details if anyone is interested. Attached is the diff between both tables (vm_static and vm_templates). Regards, Mike --=_3a2650e5-0f37-4517-8bfb-2025d36037dc Content-Type: text/x-patch; name=vm.template.tables.diff Content-Disposition: attachment; filename=vm.template.tables.diff Content-Transfer-Encoding: base64 Miw0YzIsNQo8IGNoaWxkX2NvdW50IGludGVnZXIgTk9UIE5VTEwgREVGQVVMVCAwLAo8IENPTlNU UkFJTlQgcGtfdm1fdGVtcGxhdGVzIFBSSU1BUlkgS0VZICh2bXRfZ3VpZCksCjwgQ09OU1RSQUlO VCB2ZHNfZ3JvdXBzX3ZtX3RlbXBsYXRlcyBGT1JFSUdOIEtFWSAodmRzX2dyb3VwX2lkKSBSRUZF UkVOQ0VTIHZkc19ncm91cHMgKHZkc19ncm91cF9pZCkgTUFUQ0ggU0lNUExFIE9OIFVQREFURSBO TyBBQ1RJT04gT04gREVMRVRFIE5PIEFDVElPTgotLS0KPiBDT05TVFJBSU5UIGZrX3Zkc19zdGF0 aWNfdm1fc3RhdGljIEZPUkVJR04gS0VZIChkZWRpY2F0ZWRfdm1fZm9yX3ZkcykgUkVGRVJFTkNF UyB2ZHNfc3RhdGljICh2ZHNfaWQpIE1BVENIIFNJTVBMRSBPTiBVUERBVEUgTk8gQUNUSU9OIE9O IERFTEVURSBOTyBBQ1RJT04sCj4gQ09OU1RSQUlOVCBwa192bV9zdGF0aWMgUFJJTUFSWSBLRVkg KHZtX2d1aWQpLAo+IENPTlNUUkFJTlQgdmRzX2dyb3Vwc192bV9zdGF0aWMgRk9SRUlHTiBLRVkg KHZkc19ncm91cF9pZCkgUkVGRVJFTkNFUyB2ZHNfZ3JvdXBzICh2ZHNfZ3JvdXBfaWQpIE1BVENI IFNJTVBMRSBPTiBVUERBVEUgTk8gQUNUSU9OIE9OIERFTEVURSBOTyBBQ1RJT04sCj4gQ09OU1RS QUlOVCB2bV90ZW1wbGF0ZXNfdm1fc3RhdGljIEZPUkVJR04gS0VZICh2bXRfZ3VpZCkgUkVGRVJF TkNFUyB2bV90ZW1wbGF0ZXMgKHZtdF9ndWlkKSBNQVRDSCBTSU1QTEUgT04gVVBEQVRFIE5PIEFD VElPTiBPTiBERUxFVEUgTk8gQUNUSU9OCjZjNyw5CjwgY3JlYXRpb25fZGF0ZSB0aW1lc3RhbXAg d2l0aCB0aW1lIHpvbmUgTk9UIE5VTEwsCi0tLQo+IF9jcmVhdGVfZGF0ZSB0aW1lc3RhbXAgd2l0 aCB0aW1lIHpvbmUgREVGQVVMVCAoJ25vdyc6OnRleHQpOjp0aW1lc3RhbXAgd2l0aG91dCB0aW1l IHpvbmUsCj4gY3JlYXRpb25fZGF0ZSB0aW1lc3RhbXAgd2l0aCB0aW1lIHpvbmUsCj4gZGVkaWNh dGVkX3ZtX2Zvcl92ZHMgdXVpZCwKMTRhMTgKPiBpc19pbml0aWFsaXplZCBib29sZWFuLAoyMGMy NCwyNQo8ICJuYW1lIiBjaGFyYWN0ZXIgdmFyeWluZyg0MCkgTk9UIE5VTEwsCi0tLQo+IG1pZ3Jh dGlvbl9zdXBwb3J0IGludGVnZXIgTk9UIE5VTEwgREVGQVVMVCAwLAo+IG1pbl9hbGxvY2F0ZWRf bWVtIGludGVnZXIgTk9UIE5VTEwgREVGQVVMVCAwLAoyNmEzMgo+IHByZWRlZmluZWRfcHJvcGVy dGllcyBjaGFyYWN0ZXIgdmFyeWluZyg0MDAwKSwKMjhkMzMKPCBzdGF0dXMgaW50ZWdlciBOT1Qg TlVMTCwKMzBjMzUKPCBfdXBkYXRlX2RhdGUgdGltZXN0YW1wIHdpdGggdGltZSB6b25lIERFRkFV TFQgbm93KCksCi0tLQo+IF91cGRhdGVfZGF0ZSB0aW1lc3RhbXAgd2l0aCB0aW1lIHpvbmUsCjMx YTM3Cj4gdXNlcmRlZmluZWRfcHJvcGVydGllcyBjaGFyYWN0ZXIgdmFyeWluZyg0MDAwKSwKMzJh MzksNDAKPiB2bV9ndWlkIHV1aWQgTk9UIE5VTEwsCj4gdm1fbmFtZSBjaGFyYWN0ZXIgdmFyeWlu ZygyNTUpIE5PVCBOVUxMLAo= --=_3a2650e5-0f37-4517-8bfb-2025d36037dc--
On 11/15/2011 08:17 AM, Mike Kolesnik wrote:
Hi,
I would like to introduce a change to two tables in oVirt-engine-core.
The VM and VM-template share most of their configuration. I am looking to unify the two tables into one table.
Currently, if such a change would be made in the DB layer, it would be abstracted by the DAL so no code beyond that layer should be affected, and code changes to that layer should be minimal (since we use views and stored procedures to abstract the DB structure anyway). The major change would be at the DB layer, which will mostly require data migration and changing of the STPs.
If anyone has PostgresSQL knowledge and would like to pitch in and help me push this change it would be great. I have more technical details if anyone is interested. Attached is the diff between both tables (vm_static and vm_templates).
Regards, Mike
in general, i think this is beneficial. a general concern with merging tables would be table locking, but i am not sure if relevant here. something to think about - how will the model deal with things which will be different between the two? for example, let's say tomorrow we want to add a new feature around templates versions/generations. - user creates a template for templateX - user creates VMs from tempalteX - security and other patches come out, so user wants to update the template so new VMs created from it will have them - user wants to update the template and re-seal it (template is obviously read only, so user needs to create another template). - today user would have to create templateXv1, v2, v3, etc. - would make much more sense if these would all appear as the same template with a version field, rather than disparate templates. going back to the table unification question - maybe it is not a problem, and the table will have a column with the version field which will be relevant only for templates? i guess the question is how many unrelated fields will we have, and will we keep them in same table or separate them out. looking at the fields different right now, i think a single table would be fine. in the future splitting entity specific fields could be revisited.
----- Original Message -----
On 11/15/2011 08:17 AM, Mike Kolesnik wrote:
Hi,
I would like to introduce a change to two tables in oVirt-engine-core.
The VM and VM-template share most of their configuration. I am looking to unify the two tables into one table.
Currently, if such a change would be made in the DB layer, it would be abstracted by the DAL so no code beyond that layer should be affected, and code changes to that layer should be minimal (since we use views and stored procedures to abstract the DB structure anyway). The major change would be at the DB layer, which will mostly require data migration and changing of the STPs.
If anyone has PostgresSQL knowledge and would like to pitch in and help me push this change it would be great. I have more technical details if anyone is interested. Attached is the diff between both tables (vm_static and vm_templates).
Regards, Mike
in general, i think this is beneficial. a general concern with merging tables would be table locking, but i am not sure if relevant here.
something to think about - how will the model deal with things which will be different between the two?
for example, let's say tomorrow we want to add a new feature around templates versions/generations. - user creates a template for templateX - user creates VMs from tempalteX - security and other patches come out, so user wants to update the template so new VMs created from it will have them - user wants to update the template and re-seal it (template is obviously read only, so user needs to create another template). - today user would have to create templateXv1, v2, v3, etc. - would make much more sense if these would all appear as the same template with a version field, rather than disparate templates.
If the templates in presentaion should be grouped by something other than their ID (ie by name) then it can be done at the logic/presentation layer. In the DB I believe that we will still have a new record for each version, and with different UUID so it doesn't affect the table design.
going back to the table unification question - maybe it is not a problem, and the table will have a column with the version field which will be relevant only for templates? i guess the question is how many unrelated fields will we have, and will we keep them in same table or separate them out.\
In this method, "single table inheritance", the fields which are not in the base type are still kept in the same table. This way you gain simplicity and order in the DB, while you give up constraints which need to be kept at the logic level. It's a tradeoff which I think would be good in this case, since the amount of different fields is small. The differnet types simply map to certain fields that they need, much like a view on the table.
looking at the fields different right now, i think a single table would be fine. in the future splitting entity specific fields could be revisited.
Of course this whole thing can be undone without much work if somewhere along the road we deicde that it wasn't a good idea. Regards, Mike
On 11/15/2011 10:15 AM, Mike Kolesnik wrote:
----- Original Message -----
On 11/15/2011 08:17 AM, Mike Kolesnik wrote:
Hi,
I would like to introduce a change to two tables in oVirt-engine-core.
The VM and VM-template share most of their configuration. I am looking to unify the two tables into one table.
Currently, if such a change would be made in the DB layer, it would be abstracted by the DAL so no code beyond that layer should be affected, and code changes to that layer should be minimal (since we use views and stored procedures to abstract the DB structure anyway). The major change would be at the DB layer, which will mostly require data migration and changing of the STPs.
If anyone has PostgresSQL knowledge and would like to pitch in and help me push this change it would be great. I have more technical details if anyone is interested. Attached is the diff between both tables (vm_static and vm_templates).
Regards, Mike
in general, i think this is beneficial. a general concern with merging tables would be table locking, but i am not sure if relevant here.
something to think about - how will the model deal with things which will be different between the two?
for example, let's say tomorrow we want to add a new feature around templates versions/generations. - user creates a template for templateX - user creates VMs from tempalteX - security and other patches come out, so user wants to update the template so new VMs created from it will have them - user wants to update the template and re-seal it (template is obviously read only, so user needs to create another template). - today user would have to create templateXv1, v2, v3, etc. - would make much more sense if these would all appear as the same template with a version field, rather than disparate templates.
If the templates in presentaion should be grouped by something other than their ID (ie by name) then it can be done at the logic/presentation layer. In the DB I believe that we will still have a new record for each version, and with different UUID so it doesn't affect the table design.
going back to the table unification question - maybe it is not a problem, and the table will have a column with the version field which will be relevant only for templates? i guess the question is how many unrelated fields will we have, and will we keep them in same table or separate them out.\
In this method, "single table inheritance", the fields which are not in the base type are still kept in the same table. This way you gain simplicity and order in the DB, while you give up constraints which need to be kept at the logic level. It's a tradeoff which I think would be good in this case, since the amount of different fields is small.
The differnet types simply map to certain fields that they need, much like a view on the table.
Just to add on this, this is one of the "inheritance" strategies that is provided by Hibernate/JPA. Another approach is to have all common fields in a single table, and have other tables contain only the fields that are different for each one of the types (and hold an FK to the "common fields" table). Although this approach can be treated as a hybrid between the two, the need for FK here may hit us with performance.
looking at the fields different right now, i think a single table would be fine. in the future splitting entity specific fields could be revisited.
Of course this whole thing can be undone without much work if somewhere along the road we deicde that it wasn't a good idea.
Regards, Mike _______________________________________________ Engine-devel mailing list Engine-devel@ovirt.org http://lists.ovirt.org/mailman/listinfo/engine-devel
----- Original Message -----
From: "Yair Zaslavsky" <yzaslavs@redhat.com> To: engine-devel@ovirt.org Sent: Tuesday, November 15, 2011 10:26:57 AM Subject: Re: [Engine-devel] Introduce a change to oVirt-engine-core DB
On 11/15/2011 10:15 AM, Mike Kolesnik wrote:
----- Original Message -----
On 11/15/2011 08:17 AM, Mike Kolesnik wrote:
Hi,
I would like to introduce a change to two tables in oVirt-engine-core.
The VM and VM-template share most of their configuration. I am looking to unify the two tables into one table.
Currently, if such a change would be made in the DB layer, it would be abstracted by the DAL so no code beyond that layer should be affected, and code changes to that layer should be minimal (since we use views and stored procedures to abstract the DB structure anyway). The major change would be at the DB layer, which will mostly require data migration and changing of the STPs.
If anyone has PostgresSQL knowledge and would like to pitch in and help me push this change it would be great. I have more technical details if anyone is interested. Attached is the diff between both tables (vm_static and vm_templates).
Regards, Mike
in general, i think this is beneficial. a general concern with merging tables would be table locking, but i am not sure if relevant here.
something to think about - how will the model deal with things which will be different between the two?
for example, let's say tomorrow we want to add a new feature around templates versions/generations. - user creates a template for templateX - user creates VMs from tempalteX - security and other patches come out, so user wants to update the template so new VMs created from it will have them - user wants to update the template and re-seal it (template is obviously read only, so user needs to create another template). - today user would have to create templateXv1, v2, v3, etc. - would make much more sense if these would all appear as the same template with a version field, rather than disparate templates.
If the templates in presentaion should be grouped by something other than their ID (ie by name) then it can be done at the logic/presentation layer. In the DB I believe that we will still have a new record for each version, and with different UUID so it doesn't affect the table design.
going back to the table unification question - maybe it is not a problem, and the table will have a column with the version field which will be relevant only for templates? i guess the question is how many unrelated fields will we have, and will we keep them in same table or separate them out.\
In this method, "single table inheritance", the fields which are not in the base type are still kept in the same table. This way you gain simplicity and order in the DB, while you give up constraints which need to be kept at the logic level. It's a tradeoff which I think would be good in this case, since the amount of different fields is small.
The differnet types simply map to certain fields that they need, much like a view on the table.
Just to add on this, this is one of the "inheritance" strategies that is provided by Hibernate/JPA. Another approach is to have all common fields in a single table, and have other tables contain only the fields that are different for each one of the types (and hold an FK to the "common fields" table). Although this approach can be treated as a hybrid between the two, the need for FK here may hit us with performance.
I have talked with Mike about this whole issue. It seems that we can handle both in one table. We have talked on using a flag to distinguish between both. In case that we have any performance issue, this flag could be used apply partitioning on the table. If diff grows we can use 1) inheritance (several options as above) 2) Handling in the same table (NULL when not relevant) and encapsulate with a view having only relevant columns. 3) having a diff table for each and having a view that joins the information if diff is reasonable 2) is the cheapest/simple solution. if diff grows 1) should be considered I would not use 3) in any case
looking at the fields different right now, i think a single table would be fine. in the future splitting entity specific fields could be revisited.
Of course this whole thing can be undone without much work if somewhere along the road we deicde that it wasn't a good idea.
Regards, Mike _______________________________________________ Engine-devel mailing list Engine-devel@ovirt.org http://lists.ovirt.org/mailman/listinfo/engine-devel
_______________________________________________ Engine-devel mailing list Engine-devel@ovirt.org http://lists.ovirt.org/mailman/listinfo/engine-devel
On 11/15/2011 10:15 AM, Mike Kolesnik wrote:
In this method, "single table inheritance", the fields which are not in the base type are still kept in the same table. This way you gain simplicity and order in the DB, while you give up constraints which need to be kept at the logic level. It's a tradeoff which I think would be good in this case, since the amount of different fields is small.
The differnet types simply map to certain fields that they need, much like a view on the table.
looking at the fields different right now, i think a single table would be fine. in the future splitting entity specific fields could be revisited.
Of course this whole thing can be undone without much work if somewhere along the road we deicde that it wasn't a good idea.
doesn't have to be undone. you could also just spin off the columns which aren't shared by the two entities. anyway - i think we are agreeing
On 11/15/2011 07:20 AM, Itamar Heim wrote:
On 11/15/2011 10:15 AM, Mike Kolesnik wrote:
In this method, "single table inheritance", the fields which are not in the base type are still kept in the same table. This way you gain simplicity and order in the DB, while you give up constraints which need to be kept at the logic level. It's a tradeoff which I think would be good in this case, since the amount of different fields is small.
The differnet types simply map to certain fields that they need, much like a view on the table.
looking at the fields different right now, i think a single table would be fine. in the future splitting entity specific fields could be revisited.
Of course this whole thing can be undone without much work if somewhere along the road we deicde that it wasn't a good idea.
doesn't have to be undone. you could also just spin off the columns which aren't shared by the two entities. anyway - i think we are agreeing _______________________________________________ Engine-devel mailing list Engine-devel@ovirt.org http://lists.ovirt.org/mailman/listinfo/engine-devel What problem is this attempting to solve? I understand that it is not aesthetically pleasing to have the two split out but unless this is causing undue complexity in the code (which doesn't seem to be the case due to the abstractions) is causing performance problems or is making further development difficult I'm tempted to say leave it as it is.
----- Original Message -----
On 11/15/2011 07:20 AM, Itamar Heim wrote:
On 11/15/2011 10:15 AM, Mike Kolesnik wrote:
In this method, "single table inheritance", the fields which are not in the base type are still kept in the same table. This way you gain simplicity and order in the DB, while you give up constraints which need to be kept at the logic level. It's a tradeoff which I think would be good in this case, since the amount of different fields is small.
The differnet types simply map to certain fields that they need, much like a view on the table.
looking at the fields different right now, i think a single table would be fine. in the future splitting entity specific fields could be revisited.
Of course this whole thing can be undone without much work if somewhere along the road we deicde that it wasn't a good idea.
doesn't have to be undone. you could also just spin off the columns which aren't shared by the two entities. anyway - i think we are agreeing _______________________________________________ Engine-devel mailing list Engine-devel@ovirt.org http://lists.ovirt.org/mailman/listinfo/engine-devel What problem is this attempting to solve? I understand that it is not aesthetically pleasing to have the two split out but unless this is causing undue complexity in the code (which doesn't seem to be the case due to the abstractions) is causing performance problems or is making further development difficult I'm tempted to say leave it as it is.
The main benefit I see is keeping it DRY. This will help us manage the DB structure more easily (less tables and mapping tables which all just duplicate each other). I don't think this change is something that heavy that it's not worth doing, and for me keeping it DRY is an advantage, just as you would refactor code to do the same.
_______________________________________________ Engine-devel mailing list Engine-devel@ovirt.org http://lists.ovirt.org/mailman/listinfo/engine-devel
Regards, Mike
On 11/15/2011 05:08 PM, Mike Kolesnik wrote:
----- Original Message -----
On 11/15/2011 07:20 AM, Itamar Heim wrote:
On 11/15/2011 10:15 AM, Mike Kolesnik wrote:
In this method, "single table inheritance", the fields which are not in the base type are still kept in the same table. This way you gain simplicity and order in the DB, while you give up constraints which need to be kept at the logic level. It's a tradeoff which I think would be good in this case, since the amount of different fields is small.
The differnet types simply map to certain fields that they need, much like a view on the table.
looking at the fields different right now, i think a single table would be fine. in the future splitting entity specific fields could be revisited.
Of course this whole thing can be undone without much work if somewhere along the road we deicde that it wasn't a good idea.
doesn't have to be undone. you could also just spin off the columns which aren't shared by the two entities. anyway - i think we are agreeing _______________________________________________ Engine-devel mailing list Engine-devel@ovirt.org http://lists.ovirt.org/mailman/listinfo/engine-devel What problem is this attempting to solve? I understand that it is not aesthetically pleasing to have the two split out but unless this is causing undue complexity in the code (which doesn't seem to be the case due to the abstractions) is causing performance problems or is making further development difficult I'm tempted to say leave it as it is.
The main benefit I see is keeping it DRY. This will help us manage the DB structure more easily (less tables and mapping tables which all just duplicate each other). I don't think this change is something that heavy that it's not worth doing, and for me keeping it DRY is an advantage, just as you would refactor code to do the same.
+1 On that - I think it will also help new community members to get in our code quicker.
_______________________________________________ Engine-devel mailing list Engine-devel@ovirt.org http://lists.ovirt.org/mailman/listinfo/engine-devel
Regards, Mike _______________________________________________ Engine-devel mailing list Engine-devel@ovirt.org http://lists.ovirt.org/mailman/listinfo/engine-devel
----- Original Message -----
From: "Mike Kolesnik" <mkolesni@redhat.com> To: "Jon Choate" <jchoate@redhat.com> Cc: engine-devel@ovirt.org Sent: Tuesday, November 15, 2011 5:08:01 PM Subject: Re: [Engine-devel] Introduce a change to oVirt-engine-core DB
----- Original Message -----
On 11/15/2011 07:20 AM, Itamar Heim wrote:
On 11/15/2011 10:15 AM, Mike Kolesnik wrote:
In this method, "single table inheritance", the fields which are not in the base type are still kept in the same table. This way you gain simplicity and order in the DB, while you give up constraints which need to be kept at the logic level. It's a tradeoff which I think would be good in this case, since the amount of different fields is small.
The differnet types simply map to certain fields that they need, much like a view on the table.
looking at the fields different right now, i think a single table would be fine. in the future splitting entity specific fields could be revisited.
Of course this whole thing can be undone without much work if somewhere along the road we deicde that it wasn't a good idea.
doesn't have to be undone. you could also just spin off the columns which aren't shared by the two entities. anyway - i think we are agreeing _______________________________________________ Engine-devel mailing list Engine-devel@ovirt.org http://lists.ovirt.org/mailman/listinfo/engine-devel What problem is this attempting to solve? I understand that it is not aesthetically pleasing to have the two split out but unless this is causing undue complexity in the code (which doesn't seem to be the case due to the abstractions) is causing performance problems or is making further development difficult I'm tempted to say leave it as it is.
The main benefit I see is keeping it DRY. This will help us manage the DB structure more easily (less tables and mapping tables which all just duplicate each other). I don't think this change is something that heavy that it's not worth doing, and for me keeping it DRY is an advantage, just as you would refactor code to do the same.
the advantage i see is when adding a feature to vm, and it need to be added in vm_static, it usually need also to be added to the template object as well, it will help remembering and make it easier.. (although it is not so common)
_______________________________________________ Engine-devel mailing list Engine-devel@ovirt.org http://lists.ovirt.org/mailman/listinfo/engine-devel
Regards, Mike _______________________________________________ Engine-devel mailing list Engine-devel@ovirt.org http://lists.ovirt.org/mailman/listinfo/engine-devel
participants (6)
-
Eli Mesika -
Itamar Heim -
Jon Choate -
Mike Kolesnik -
Omer Frenkel -
Yair Zaslavsky