Eli Mesika has submitted this change and it was merged. Change subject: db: refactoring upgrade process ...................................................................... db: refactoring upgrade process Up to now, each version, I had squashed (version - 3) upgrade scripts into the base database manually. For example in 3.4 , I had squashed 3.1 upgrade scripts and in 3.5 I had squashed 3.2 upgrade scripts. The squashing was done by creating a new schema including all the squashed version changes and applying all inserts into the insert_data.sql and insert_pre_defined_roles.sql manually This was a time consuming and error-prone process that sometimes lead to regressions since it was not easy to compare databases data using pg_dump because PG does not guarantee the order in which data is dumped even if the databases compared are identical. This patch introduces a new approach when all data up to the squashed version, is auto generated per table by using pg_dump under the data directory Another complementary tool (not part of this patch) enables to dump PG databases and forcing the data being ordered by table PK, the tool also compares databases schema and data and make it very easy to check if databases are identical. Since some entities like DC & Cluster need a new generated UUID per installation, I wrote a uuidgen.sh script that does exactly that and generates new UUIDs for each installation for those entities on the data before it is inserted to the database. The process of squashing, from now on, will be : 1) creating a git branch with only the squashed version upgrade scripts 2) generating schema and insert files (automatically by pg_dump) 3) restoring all deleted upgrade files with version > squashed version that were deleted in 1) 4) creating a database based on 1) to 3) 5) creating a database on master 6) using the database comparison tool to verify both databases are identical (except some generated UUIDs) Change-Id: I89b8ecc31ca3a179886d84ae7f725282c5571c37 Signed-off-by: emesika <emesika@redhat.com> --- A packaging/dbscripts/data/0000_insert_schema_version.sql A packaging/dbscripts/data/0010_insert_users.sql A packaging/dbscripts/data/0020_insert_ad_groups.sql A packaging/dbscripts/data/0030_insert_storage_pool.sql A packaging/dbscripts/data/0040_insert_vds_groups.sql A packaging/dbscripts/data/0050_insert_roles.sql A packaging/dbscripts/data/0060_insert_permissions.sql A packaging/dbscripts/data/0070_insert_roles_groups.sql A packaging/dbscripts/data/0080_insert_images.sql A packaging/dbscripts/data/0090_insert_network.sql A packaging/dbscripts/data/0100_insert_network_cluster.sql A packaging/dbscripts/data/0110_insert_vm_static.sql A packaging/dbscripts/data/0120_insert_vm_device.sql A packaging/dbscripts/data/0130_insert_vm_ovf_generations.sql A packaging/dbscripts/data/0140_insert_action_version_map.sql A packaging/dbscripts/data/0150_insert_event_map.sql A packaging/dbscripts/data/0160_insert_dwh_history_timekeeping.sql M packaging/dbscripts/dbfunc-custom.sh D packaging/dbscripts/insert_data.sql D packaging/dbscripts/insert_predefined_roles.sql A packaging/dbscripts/uuidgen.sh 21 files changed, 783 insertions(+), 913 deletions(-) Approvals: Eli Mesika: Verified; Looks good to me, approved Lior Vernia: Looks good to me, but someone else must approve -- To view, visit http://gerrit.ovirt.org/37256 To unsubscribe, visit http://gerrit.ovirt.org/settings Gerrit-MessageType: merged Gerrit-Change-Id: I89b8ecc31ca3a179886d84ae7f725282c5571c37 Gerrit-PatchSet: 6 Gerrit-Project: ovirt-engine Gerrit-Branch: master Gerrit-Owner: Eli Mesika <emesika@redhat.com> Gerrit-Reviewer: Allon Mureinik <amureini@redhat.com> Gerrit-Reviewer: Eli Mesika <emesika@redhat.com> Gerrit-Reviewer: Lior Vernia <lvernia@redhat.com> Gerrit-Reviewer: Oved Ourfali <oourfali@redhat.com> Gerrit-Reviewer: Ravi Nori <rnori@redhat.com> Gerrit-Reviewer: Yair Zaslavsky <yzaslavs@redhat.com> Gerrit-Reviewer: automation@ovirt.org Gerrit-Reviewer: oVirt Jenkins CI Server