Change in ovirt-engine[master]: db: refactoring upgrade process

[emesika at redhat.com]

Eli Mesika has submitted this change and it was merged.

Change subject: db: refactoring upgrade process
......................................................................


db: refactoring upgrade process

Up to now, each version, I had squashed (version - 3)  upgrade scripts into the base database
manually.
For example in 3.4 , I had squashed  3.1 upgrade scripts and in 3.5 I had
squashed 3.2 upgrade scripts.

The squashing was done by creating a new schema including all the
squashed version changes and applying all inserts into the
insert_data.sql and insert_pre_defined_roles.sql manually

This was a time consuming and error-prone process that sometimes lead
to regressions since it was not easy to compare databases data using
pg_dump because PG does not guarantee the order in which data is dumped
even if the databases compared are identical.

This patch introduces a new approach when all data up to the  squashed
version, is auto generated per table by using pg_dump under the data
directory

Another complementary tool (not part of this patch) enables to dump PG
databases and forcing the data being ordered by table PK, the tool also
compares databases schema and data and make it very easy to check if
databases are identical.

Since some entities like DC & Cluster need a new generated UUID per
installation, I wrote a uuidgen.sh script that does exactly that and
generates new UUIDs for each installation for those entities on the data
before it is inserted to the database.

The process of squashing, from now on, will be :

1) creating a git branch with only the squashed version upgrade scripts
2) generating schema and insert files (automatically by pg_dump)
3) restoring all deleted upgrade files with version > squashed version
that were deleted in 1)
4) creating a database based on 1) to 3)
5) creating a database on master
6) using the database comparison tool to verify both databases are identical
(except some generated UUIDs)

Change-Id: I89b8ecc31ca3a179886d84ae7f725282c5571c37
Signed-off-by: emesika <emesika at redhat.com>
---
A packaging/dbscripts/data/0000_insert_schema_version.sql
A packaging/dbscripts/data/0010_insert_users.sql
A packaging/dbscripts/data/0020_insert_ad_groups.sql
A packaging/dbscripts/data/0030_insert_storage_pool.sql
A packaging/dbscripts/data/0040_insert_vds_groups.sql
A packaging/dbscripts/data/0050_insert_roles.sql
A packaging/dbscripts/data/0060_insert_permissions.sql
A packaging/dbscripts/data/0070_insert_roles_groups.sql
A packaging/dbscripts/data/0080_insert_images.sql
A packaging/dbscripts/data/0090_insert_network.sql
A packaging/dbscripts/data/0100_insert_network_cluster.sql
A packaging/dbscripts/data/0110_insert_vm_static.sql
A packaging/dbscripts/data/0120_insert_vm_device.sql
A packaging/dbscripts/data/0130_insert_vm_ovf_generations.sql
A packaging/dbscripts/data/0140_insert_action_version_map.sql
A packaging/dbscripts/data/0150_insert_event_map.sql
A packaging/dbscripts/data/0160_insert_dwh_history_timekeeping.sql
M packaging/dbscripts/dbfunc-custom.sh
D packaging/dbscripts/insert_data.sql
D packaging/dbscripts/insert_predefined_roles.sql
A packaging/dbscripts/uuidgen.sh
21 files changed, 783 insertions(+), 913 deletions(-)

Approvals:
  Eli Mesika: Verified; Looks good to me, approved
  Lior Vernia: Looks good to me, but someone else must approve



-- 
To view, visit http://gerrit.ovirt.org/37256
To unsubscribe, visit http://gerrit.ovirt.org/settings

Gerrit-MessageType: merged
Gerrit-Change-Id: I89b8ecc31ca3a179886d84ae7f725282c5571c37
Gerrit-PatchSet: 6
Gerrit-Project: ovirt-engine
Gerrit-Branch: master
Gerrit-Owner: Eli Mesika <emesika at redhat.com>
Gerrit-Reviewer: Allon Mureinik <amureini at redhat.com>
Gerrit-Reviewer: Eli Mesika <emesika at redhat.com>
Gerrit-Reviewer: Lior Vernia <lvernia at redhat.com>
Gerrit-Reviewer: Oved Ourfali <oourfali at redhat.com>
Gerrit-Reviewer: Ravi Nori <rnori at redhat.com>
Gerrit-Reviewer: Yair Zaslavsky <yzaslavs at redhat.com>
Gerrit-Reviewer: automation at ovirt.org
Gerrit-Reviewer: oVirt Jenkins CI Server