Currently, our upgrade to 4.4 fails with error: FATAL: Existing database 'ovirt_engine_history' or user 'ovirt_engine_history' found and temporary ones created We have upgraded the running 4.3 installation to the latest version and also use the latest packages for the upgrade on the new CentOS 8.2 installation. The back-up is made following the Hosted Engine upgrade steps in the manual, using: `engine-backup --scope=all --mode=backup --file=backup.bck --log=backuplog.log` The upgrade is performed after copying the backup.bck file to the new server and using `hosted-engine --deploy --restore-from-file=backup.bck` After creating the Engine VM, the installation process hangs when the backup is restored. We tried it several times, using a complete or a partial back-up. Old/current oVirt version: 4.3.10.4-1.el7 New version: 4.4.1.8 ovirt-ansible-hosted-engine-setup: 1.1.6 Did anyone get the same error while upgrading an existing installation? Thanks! Error log Ansible on Host: 2020-07-15 12:34:09,361+0200 INFO otopi.ovirt_hosted_engine_setup.ansible_utils ansible_utils._process_output:109 TASK [ovirt.hosted_engine_setup : Run engine-backup] 2020-07-15 12:35:28,778+0200 DEBUG otopi.ovirt_hosted_engine_setup.ansible_utils ansible_utils._process_output:103 {'msg': 'non-zero return code', 'cmd': 'engine-backup --mode=restore --log=/var/log/ovirt-engine/setup/restore-backup-$(date -u +%Y%m%d%H%M%S).log --file=/root/engine_backup --provision-all-databases --restore-permissions', 'stdout': "Start of engine-backup with mode 'restore'\nscope: all\narchive file: /root/engine_backup\nlog file: /var/log/ovirt-engine/setup/restore-backup-20200715103410.log\nPreparing to restore:\n- Unpacking file '/root/engine_backup'\nRestoring:\n- Files\n------------------------------------------------------------------------------\nPlease note:\n\nOperating system is different from the one used during backup.\nCurrent operating system: centos8\nOperating system at backup: centos7\n\nApache httpd configuration will not be restored.\nYou will be asked about it on the next engine-setup run.\n---------------------------------------------------------- --------------------\nProvisioning PostgreSQL users/databases:\n- user 'engine', database 'engine'\n- extra user 'ovirt_engine_history' having grants on database engine, created with a random password\n- user 'ovirt_engine_history', database 'ovirt_engine_history'", 'stderr': "FATAL: Existing database 'ovirt_engine_history' or user 'ovirt_engine_history' found and temporary ones created - Please clean up everything and try again", 'rc': 1, 'start': '2020-07-15 12:34:10.824630', 'end': '2020-07-15 12:35:28.488261', 'delta': '0:01:17.663631', 'changed': True, 'invocation': {'module_args': {'_raw_params': 'engine-backup --mode=restore --log=/var/log/ovirt-engine/setup/restore-backup-$(date -u +%Y%m%d%H%M%S).log --file=/root/engine_backup --provision-all-databases --restore-permissions', '_uses_shell': True, 'warn': True, 'stdin_add_newline': True, 'strip_empty_ends': True, 'argv': None, 'chdir': None, 'executable': None, 'creates': None, 'removes': None, 'stdin': None}}, 'stdout_lines' : ["Start of engine-backup with mode 'restore'", 'scope: all', 'archive file: /root/engine_backup', 'log file: /var/log/ovirt-engine/setup/restore-backup-20200715103410.log', 'Preparing to restore:', "- Unpacking file '/root/engine_backup'", 'Restoring:', '- Files', '------------------------------------------------------------------------------', 'Please note:', '', 'Operating system is different from the one used during backup.', 'Current operating system: centos8', 'Operating system at backup: centos7', '', 'Apache httpd configuration will not be restored.', 'You will be asked about it on the next engine-setup run.', '------------------------------------------------------------------------------', 'Provisioning PostgreSQL users/databases:', "- user 'engine', database 'engine'", "- extra user 'ovirt_engine_history' having grants on database engine, created with a random password", "- user 'ovirt_engine_history', database 'ovirt_engine_history'"], 'stderr_lines': ["FATAL: Existing d atabase 'ovirt_engine_history' or user 'ovirt_engine_history' found and temporary ones created - Please clean up everything and try again"], '_ansible_no_log': False, '_ansible_delegated_vars': {'ansible_host': 'ovirt-management.dc1.triplon', 'ansible_port': None, 'ansible_user': 'root'}} 2020-07-15 12:35:28,879+0200 ERROR otopi.ovirt_hosted_engine_setup.ansible_utils ansible_utils._process_output:107 fatal: [localhost -> ovirt-management.dc1.triplon]: FAILED! => {"changed": true, "cmd": "engine-backup --mode=restore --log=/var/log/ovirt-engine/setup/restore-backup-$(date -u +%Y%m%d%H%M%S).log --file=/root/engine_backup --provision-all-databases --restore-permissions", "delta": "0:01:17.663631", "end": "2020-07-15 12:35:28.488261", "msg": "non-zero return code", "rc": 1, "start": "2020-07-15 12:34:10.824630", "stderr": "FATAL: Existing database 'ovirt_engine_history' or user 'ovirt_engine_history' found and temporary ones created - Please clean up everything and try again", "stderr_lines": ["FATAL: Existing database 'ovirt_engine_history' or user 'ovirt_engine_history' found and temporary ones created - Please clean up everything and try again"], "stdout": "Start of engine-backup with mode 'restore'\nscope: all\narchive file: /root/engine_backup\nlog file: /var/log/ov irt-engine/setup/restore-backup-20200715103410.log\nPreparing to restore:\n- Unpacking file '/root/engine_backup'\nRestoring:\n- Files\n------------------------------------------------------------------------------\nPlease note:\n\nOperating system is different from the one used during backup.\nCurrent operating system: centos8\nOperating system at backup: centos7\n\nApache httpd configuration will not be restored.\nYou will be asked about it on the next engine-setup run.\n------------------------------------------------------------------------------\nProvisioning PostgreSQL users/databases:\n- user 'engine', database 'engine'\n- extra user 'ovirt_engine_history' having grants on database engine, created with a random password\n- user 'ovirt_engine_history', database 'ovirt_engine_history'", "stdout_lines": ["Start of engine-backup with mode 'restore'", "scope: all", "archive file: /root/engine_backup", "log file: /var/log/ovirt-engine/setup/restore-backup-20200715103410.log", "Prep aring to restore:", "- Unpacking file '/root/engine_backup'", "Restoring:", "- Files", "------------------------------------------------------------------------------", "Please note:", "", "Operating system is different from the one used during backup.", "Current operating system: centos8", "Operating system at backup: centos7", "", "Apache httpd configuration will not be restored.", "You will be asked about it on the next engine-setup run.", "------------------------------------------------------------------------------", "Provisioning PostgreSQL users/databases:", "- user 'engine', database 'engine'", "- extra user 'ovirt_engine_history' having grants on database engine, created with a random password", "- user 'ovirt_engine_history', database 'ovirt_engine_history'"]} restore-backup-xx.log file on the created Hosted Engine (retrieved by Ansible after failing): 2020-07-15 15:57:36 6437: Start of engine-backup mode restore scope all file /root/engine_backup 2020-07-15 15:57:36 6437: OUTPUT: Start of engine-backup with mode 'restore' 2020-07-15 15:57:36 6437: OUTPUT: scope: all 2020-07-15 15:57:36 6437: OUTPUT: archive file: /root/engine_backup 2020-07-15 15:57:36 6437: OUTPUT: log file: /var/log/ovirt-engine/setup/restore-backup-20200715135736.log 2020-07-15 15:57:36 6437: OUTPUT: Preparing to restore: 2020-07-15 15:57:36 6437: OUTPUT: - Unpacking file '/root/engine_backup' 2020-07-15 15:57:36 6437: Opening tarball /root/engine_backup to /tmp/engine-backup.5tvwGDx3qs 2020-07-15 15:57:37 6437: Verifying hash 2020-07-15 15:57:37 6437: Verifying version 2020-07-15 15:57:37 6437: Reading config 2020-07-15 15:57:37 6437: OUTPUT: Restoring: 2020-07-15 15:57:37 6437: OUTPUT: - Files 2020-07-15 15:57:37 6437: Restoring files 2020-07-15 15:57:38 6437: Reloading configuration 2020-07-15 15:57:38 6437: OUTPUT: Provisioning PostgreSQL users/databases: 2020-07-15 15:57:38 6437: provisionDB: user engine host localhost port 5432 database engine secured False secured_host_validation False 2020-07-15 15:57:38 6437: OUTPUT: - user 'engine', database 'engine' [ INFO ] Stage: Initializing [ INFO ] Stage: Environment setup Configuration files: /etc/ovirt-engine-setup.conf.d/10-packaging-jboss.conf, /etc/ovirt-engine-setup.conf.d/10-packaging.conf, /etc/ovirt-engine-setup.conf.d/20-setup-ovirt-post.conf, /tmp/engine-backup.5tvwGDx3qs/pg-provision-answer-file Log file: /var/log/ovirt-engine/setup/ovirt-engine-provisiondb-20200715155739-3o20a7.log Version: otopi-1.9.2 (otopi-1.9.2-1.el8) [ INFO ] Stage: Environment packages setup [ INFO ] Stage: Programs detection [ INFO ] Stage: Environment customization [ INFO ] Stage: Setup validation [ INFO ] Stage: Transaction setup [ INFO ] Stage: Misc configuration (early) [ INFO ] Stage: Package installation [ INFO ] Stage: Misc configuration [ INFO ] Initializing PostgreSQL [ INFO ] Creating PostgreSQL 'engine' database [ INFO ] Configuring PostgreSQL [ INFO ] Install selinux module /usr/share/ovirt-engine/selinux/ansible-runner-service.cil [ INFO ] Stage: Transaction commit [ INFO ] Stage: Closing up [ INFO ] Stage: Clean up Log file is located at /var/log/ovirt-engine/setup/ovirt-engine-provisiondb-20200715155739-3o20a7.log [ INFO ] Stage: Pre-termination [ INFO ] Stage: Termination [ INFO ] Execution of provisiondb completed successfully 2020-07-15 15:58:14 6437: OUTPUT: - extra user 'ovirt_engine_history' having grants on database engine, created with a random password [ INFO ] Stage: Initializing [ INFO ] Stage: Environment setup Configuration files: /etc/ovirt-engine-setup.conf.d/10-packaging-jboss.conf, /etc/ovirt-engine-setup.conf.d/10-packaging.conf, /etc/ovirt-engine-setup.conf.d/20-setup-ovirt-post.conf, /tmp/engine-backup.5tvwGDx3qs/pg-provision-answer-file Log file: /var/log/ovirt-engine/setup/ovirt-engine-provisiondb-20200715155815-j4ba3l.log Version: otopi-1.9.2 (otopi-1.9.2-1.el8) [ INFO ] Stage: Environment packages setup [ INFO ] Stage: Programs detection [ INFO ] Stage: Environment customization [ INFO ] Stage: Setup validation [ INFO ] Stage: Transaction setup [ INFO ] Stage: Misc configuration (early) [ INFO ] Stage: Package installation [ INFO ] Stage: Misc configuration [ INFO ] Install selinux module /usr/share/ovirt-engine/selinux/ansible-runner-service.cil [ INFO ] Stage: Transaction commit [ INFO ] Stage: Closing up [ INFO ] Stage: Clean up Log file is located at /var/log/ovirt-engine/setup/ovirt-engine-provisiondb-20200715155815-j4ba3l.log [ INFO ] Stage: Pre-termination [ INFO ] Stage: Termination [ INFO ] Execution of provisiondb completed successfully 2020-07-15 15:58:46 6437: provisionDB: user ovirt_engine_history host localhost port 5432 database ovirt_engine_history secured False secured_host_validation False 2020-07-15 15:58:46 6437: OUTPUT: - user 'ovirt_engine_history', database 'ovirt_engine_history' [ INFO ] Stage: Initializing [ INFO ] Stage: Environment setup Configuration files: /etc/ovirt-engine-setup.conf.d/10-packaging-jboss.conf, /etc/ovirt-engine-setup.conf.d/10-packaging.conf, /etc/ovirt-engine-setup.conf.d/20-setup-ovirt-post.conf, /tmp/engine-backup.5tvwGDx3qs/pg-provision-answer-file Log file: /var/log/ovirt-engine/setup/ovirt-engine-provisiondb-20200715155847-qk7ipy.log Version: otopi-1.9.2 (otopi-1.9.2-1.el8) [ INFO ] Stage: Environment packages setup [ INFO ] Stage: Programs detection [ INFO ] Stage: Environment customization [ INFO ] Stage: Setup validation [ INFO ] Stage: Transaction setup [ INFO ] Stage: Misc configuration (early) [ INFO ] Stage: Package installation [ INFO ] Stage: Misc configuration [ INFO ] Creating PostgreSQL 'ovirt_engine_history' database [ INFO ] Configuring PostgreSQL [ ERROR ] Failed to execute stage 'Misc configuration': Existing resources found, new ones created: database ovirt_engine_history_20200715155850 user ovirt_engine_history_20200715155850 [ INFO ] Stage: Clean up Log file is located at /var/log/ovirt-engine/setup/ovirt-engine-provisiondb-20200715155847-qk7ipy.log [ INFO ] Stage: Pre-termination [ INFO ] Stage: Termination [ ERROR ] Execution of provisiondb failed 2020-07-15 15:58:52 6437: FATAL: Existing database 'ovirt_engine_history' or user 'ovirt_engine_history' found and temporary ones created - Please clean up everything and try again