New subject: External engine CentOS 8.4 and 4.4.8 to Rocky Linux 8.5 and 4.4.9

Friday, 14 January 2022

Just wanted to share what in subject and feedback/comments welcome.
Starting env is an external engine on CentOS 8.4, updated up to late
September, with 4.4.8 async 2 (ovirt-engine-4.4.8.6-1.el8.noarch.rpm) and
kernel 4.18.0-305.19.1.el8_4

[g.cecchi@ovmgr1 ~]$ cat /etc/centos-release
CentOS Linux release 8.4.2105

I migrated to Rocky Linux 8.5 and then run engine-setup to take it to
current 4.4.9 level.
The system is a VM on a vSphere environment.
Currently there are 3 hosts with CentOS 8.4, that I'm going then to migrate
to Rocky Linux too.

Downloaded the current migration script from

https://raw.githubusercontent.com/rocky-linux/rocky-tools/main/migrate2ro...
chmod +x migrate2rocky.sh

My engine needs a proxy to exit on the internet and the script doesn't
check my /etc/yum.conf configuration

so I set
export https_proxy=http://my_proxy:my_proxy_port
export http_proxy=http://my_proxy:my_proxy_port (not sure if this
necessary...)

otherwise when you run the script you get
curl: (7) Failed to connect to dl.rockylinux.org port 443: Connection timed
out
Error downloading the Rocky Linux signing key.

then as root

./migrate2rocky.sh -r

notable things observed:

. some packages from ovirt repos are gathered too... I don't know if
correct in this stage...

Running dnf update before we attempt the migration.
Last metadata expiration check: 0:00:47 ago on Thu Jan 13 11:56:21 2022.
Dependencies resolved.
========================================================================
. . .
 otopi-common                                       noarch  1.9.6-1.el8
                         ovirt-4.4                           94 k
 ovirt-ansible-collection                           noarch  1.6.5-1.el8
                         ovirt-4.4                          289 k
 ovirt-engine-dwh-grafana-integration-setup         noarch  4.4.9.1-1.el8
                         ovirt-4.4                           88 k
 ovirt-engine-dwh-setup                             noarch  4.4.9.1-1.el8
                         ovirt-4.4                           95 k
 ovirt-engine-extension-aaa-ldap                    noarch  1.4.5-1.el8
                         ovirt-4.4                          130 k
 ovirt-engine-metrics                               noarch  1.4.4-1.el8
                         ovirt-4.4                           92 k
 ovirt-engine-setup                                 noarch  4.4.9.5-1.el8
                         ovirt-4.4                           18 k
 ovirt-engine-setup-base                            noarch  4.4.9.5-1.el8
                         ovirt-4.4                          116 k
 ovirt-engine-setup-plugin-cinderlib                noarch  4.4.9.5-1.el8
                         ovirt-4.4                           39 k
 ovirt-engine-setup-plugin-imageio                  noarch  4.4.9.5-1.el8
                         ovirt-4.4                           27 k
 ovirt-engine-setup-plugin-ovirt-engine             noarch  4.4.9.5-1.el8
                         ovirt-4.4                          203 k
 ovirt-engine-setup-plugin-ovirt-engine-common      noarch  4.4.9.5-1.el8
                         ovirt-4.4                          122 k
 ovirt-engine-setup-plugin-vmconsole-proxy-helper   noarch  4.4.9.5-1.el8
                         ovirt-4.4                           38 k
 ovirt-engine-setup-plugin-websocket-proxy          noarch  4.4.9.5-1.el8
                         ovirt-4.4                           39 k
 ovirt-engine-vmconsole-proxy-helper                noarch  4.4.9.5-1.el8
                         ovirt-4.4                           26 k
 ovirt-engine-websocket-proxy                       noarch  4.4.9.5-1.el8
                         ovirt-4.4                           33 k
 ovirt-imageio-client                               x86_64  2.3.0-1.el8
                         ovirt-4.4                           21 k
 ovirt-imageio-common                               x86_64  2.3.0-1.el8
                         ovirt-4.4                          158 k
 ovirt-imageio-daemon                               x86_64  2.3.0-1.el8
                         ovirt-4.4                           15 k
 ovirt-release44                                    noarch  4.4.9.3-1.el8
                         ovirt-4.4                           20 k
 ovirt-web-ui                                       noarch  1.7.2-1.el8
                         ovirt-4.4                           11 M
. . .
 qemu-img                                           x86_64  15:6.0.0-33.el8
                         ovirt-4.4-advanced-virtualization  1.9 M
. . .
Transaction Summary
=========================================================================================

Install    5 Packages
Upgrade  393 Packages
Remove     3 Packages

Total download size: 689 M
. . .
Generating grub configuration file ...
File descriptor 3 (/dev/pts/1) leaked on vgs invocation. Parent PID
4056644: /usr/sbin/grub2-probe
File descriptor 4 (/dev/pts/1) leaked on vgs invocation. Parent PID
4056644: /usr/sbin/grub2-probe
File descriptor 5 (/var/log/migrate2rocky.log) leaked on vgs invocation.
Parent PID 4056644: /usr/sbin/grub2-probe
File descriptor 3 (/dev/pts/1) leaked on vgs invocation. Parent PID
4056644: /usr/sbin/grub2-probe
. . .
Adding boot menu entry for EFI firmware configuration
done
BootCurrent: 0003
BootOrder: 0004,0003,0000,0001,0002
Boot0000* EFI Virtual disk (0.0)
Boot0001* EFI VMware Virtual SATA CDROM Drive (0.0)
Boot0002* EFI Network
Boot0003* CentOS Linux
Boot0004* Rocky Linux

Done, please reboot your system.
A log of this installation can be found at /var/log/migrate2rocky.log

Due to the oVirt related packages update, I then executed the upgrade,
running the "engine-setup" command, without rebooting the server at this
stage

In the output I got these things with many packages to be installed; I
don't know if it is something depending on 4.4.8 -> 4.4.9 or on Rocky
migration:

. . .
Checking for product updates...
          Setup needs to install or update the following packages:
          [install] adobe-mappings-cmap-20171205-3.el8.noarch will be
installed
          [install] adobe-mappings-cmap-deprecated-20171205-3.el8.noarch
will be installed
          [install] adobe-mappings-pdf-20180407-1.el8.noarch will be
installed
          [install] atk-2.28.1-1.el8.x86_64 will be installed
          [install] blosc-1.17.0-1.el8.x86_64 will be installed
          [install] ceph-common-16.2.7-1.el8.x86_64 will be installed
          [install] dejavu-sans-fonts-2.35-7.el8.noarch will be installed
          [install] fribidi-1.0.4-8.el8.x86_64 will be installed
          [install] gd-2.2.5-7.el8.x86_64 will be installed
          [install] gdk-pixbuf2-modules-2.36.12-5.el8.x86_64 will be
installed
          [install] google-droid-sans-fonts-20120715-13.el8.noarch will be
installed
          [install] gperftools-libs-2.9.1-1.el8.x86_64 will be installed
          [install] graphviz-2.40.1-43.el8.x86_64 will be installed
          [install] gtk-update-icon-cache-3.22.30-8.el8.x86_64 will be
installed
          [install] gtk2-2.24.32-5.el8.x86_64 will be installed
          [install] hdf5-1.10.5-5.el8.x86_64 will be installed
          [install] hicolor-icon-theme-0.17-2.el8.noarch will be installed
          [install] jasper-libs-2.0.14-5.el8.x86_64 will be installed
          [install] jbig2dec-libs-0.16-1.el8.x86_64 will be installed
          [install] jbigkit-libs-2.1-14.el8.x86_64 will be installed
          [install] leveldb-1.20-1.el8.x86_64 will be installed
          [install] libICE-1.0.9-15.el8.x86_64 will be installed
          [install] libSM-1.2.3-1.el8.x86_64 will be installed
          [install] libXaw-1.0.13-10.el8.x86_64 will be installed
          [install] libXcomposite-0.4.4-14.el8.x86_64 will be installed
          [install] libXcursor-1.1.15-3.el8.x86_64 will be installed
          [install] libXdamage-1.1.4-14.el8.x86_64 will be installed
          [install] libXfixes-5.0.3-7.el8.x86_64 will be installed
          [install] libXft-2.3.3-1.el8.x86_64 will be installed
          [install] libXi-1.7.10-1.el8.x86_64 will be installed
          [install] libXinerama-1.1.4-1.el8.x86_64 will be installed
          [install] libXmu-1.1.3-1.el8.x86_64 will be installed
          [install] libXpm-3.5.12-8.el8.x86_64 will be installed
          [install] libXrandr-1.5.2-1.el8.x86_64 will be installed
          [install] libXt-1.1.5-12.el8.x86_64 will be installed
          [install] libXxf86misc-1.0.4-1.el8.x86_64 will be installed
          [install] libXxf86vm-1.1.4-9.el8.x86_64 will be installed
          [install] libaec-1.0.2-3.el8.x86_64 will be installed
          [install] libbabeltrace-1.5.4-3.el8.x86_64 will be installed
          [install] libcephfs2-16.2.7-1.el8.x86_64 will be installed
. . .
          [install] urw-base35-nimbus-roman-fonts-20170801-10.el8.noarch
will be installed
          [install] urw-base35-nimbus-sans-fonts-20170801-10.el8.noarch
will be installed
          [install] urw-base35-p052-fonts-20170801-10.el8.noarch will be
installed
          [install]
urw-base35-standard-symbols-ps-fonts-20170801-10.el8.noarch will be
installed
          [install] urw-base35-z003-fonts-20170801-10.el8.noarch will be
installed
          [install] xorg-x11-font-utils-7.5-41.el8.x86_64 will be installed
          [install] xorg-x11-fonts-ISO8859-1-100dpi-7.5-19.el8.noarch will
be installed
          [install] xorg-x11-server-utils-7.7-27.el8.x86_64 will be
installed
          Replying "No" will abort Setup. You can pass the option
"--offline" to prevent installing or updating packages.
          Do you wish to update them now? (Yes, No) [Yes]:
. . . I went ahead pressing Return button

DNF Downloaded Ceph packages for x86_64
          [ ERROR ] DNF Importing GPG key 0xE451E5B5:
          Userid     : "CentOS Storage SIG (
http://wiki.centos.org/SpecialInterestGroup/S
torage) <security(a)centos.org&gt;&quot;
          Fingerprint: 7412 9C0B 173B 071A 3775 951A D4A2 E50B E451 E5B5
          From       :
https://www.centos.org/keys/RPM-GPG-KEY-CentOS-SIG-Storage

          Please confirm Confirm use of GPG Key userid=CentOS Storage SIG (
http://wiki.ce
ntos.org/SpecialInterestGroup/Storage) <security(a)centos.org&gt;
hexkeyid=E451E5B5 [yes/no]: yes
          [ INFO  ] DNF Installed:
urw-base35-fonts-common-20170801-10.el8.noarch
. . .
          --== END OF SUMMARY ==--

          [ INFO  ] Stage: Clean up
          Log file is located at
/var/log/ovirt-engine/setup/ovirt-engine-setup-20220113133933-7l9rq4.log
          [ INFO  ] Generating answer file
'/var/lib/ovirt-engine/setup/answers/20220113134624-setup.conf'
          [ INFO  ] Stage: Pre-termination
          [ INFO  ] Stage: Termination
          [ INFO  ] Execution of setup completed successfully

so it ended ok

at this stage, before reboot

# yum update
Last metadata expiration check: 1:38:48 ago on Thu 13 Jan 2022 12:07:45 PM
CET.
Dependencies resolved.
Nothing to do.
Complete!

reboot

But after reboot

[root@ovmgr1 g.cecchi]# dnf update
error: rpmdb: BDB0113 Thread/process 2771/140020593752960 failed: BDB1507
Thread died in Berkeley DB library
error: db5 error(-30973) from dbenv->failchk: BDB0087 DB_RUNRECOVERY: Fatal
error, run database recovery
error: cannot open Packages index using db5 -  (-30973)
error: cannot open Packages database in /var/lib/rpm
Error: Error: rpmdb open failed

[root@ovmgr1 g.cecchi]# ls -lrt /var/lib/rpm
total 241400
-rw-r--r--. 1 root root      8192 Jan 13 12:07 Enhancename
-rw-r--r--. 1 root root      8192 Jan 13 12:14 Filetriggername
-rw-r--r--. 1 root root      8192 Jan 13 12:14 Supplementname
-rw-r--r--. 1 root root      8192 Jan 13 13:44 Triggername
-rw-r--r--. 1 root root      8192 Jan 13 13:44 Suggestname
-rw-r--r--. 1 root root      8192 Jan 13 13:44 Recommendname
-rw-r--r--. 1 root root      8192 Jan 13 13:44 Transfiletriggername
-rw-r--r--. 1 root root     32768 Jan 13 13:44 Obsoletename
-rw-r--r--. 1 root root     16384 Jan 13 13:44 Conflictname
-rw-r--r--. 1 root root 232259584 Jan 13 13:44 Packages
-rw-r--r--. 1 root root     69632 Jan 13 13:44 Name
-rw-r--r--. 1 root root   6549504 Jan 13 13:44 Basenames
-rw-r--r--. 1 root root     32768 Jan 13 13:44 Group
-rw-r--r--. 1 root root    466944 Jan 13 13:44 Requirename
-rw-r--r--. 1 root root   3350528 Jan 13 13:44 Providename
-rw-r--r--. 1 root root   2478080 Jan 13 13:44 Dirnames
-rw-r--r--. 1 root root     98304 Jan 13 13:45 Sigmd5
-rw-r--r--. 1 root root     36864 Jan 13 13:45 Installtid
-rw-r--r--. 1 root root    167936 Jan 13 13:45 Sha1header
-rw-r--r--. 1 root root   1318912 Jan 13 13:50 __db.003
-rw-r--r--. 1 root root     90112 Jan 13 13:50 __db.002
-rw-r--r--. 1 root root    286720 Jan 13 13:50 __db.001
[root@ovmgr1 g.cecchi]# date

I followed what discussed here to rebuild the rpm database:
https://cloudlinux.zendesk.com/hc/en-us/articles/115004075294-Fix-rpmdb-T...

[root@ovmgr1 g.cecchi]# mkdir /var/lib/rpm/backup
[root@ovmgr1 g.cecchi]# cp -a /var/lib/rpm/__db* /var/lib/rpm/backup/
[root@ovmgr1 g.cecchi]# rm -f /var/lib/rpm/__db.[0-9][0-9]*
[root@ovmgr1 g.cecchi]# rpm --quiet -qa
[root@ovmgr1 g.cecchi]# rpm --rebuilddb
error: could not delete old database at /var/lib/rpmold.2997

[root@ovmgr1 g.cecchi]# yum clean all
163 files removed
[root@ovmgr1 g.cecchi]#

[root@ovmgr1 g.cecchi]# ls -lrt /var/lib/rpm
total 155112
-rw-r--r--. 1 root root      8192 Jan 13 13:53 Enhancename
-rw-r--r--. 1 root root      8192 Jan 13 13:53 Filetriggername
-rw-r--r--. 1 root root      8192 Jan 13 13:53 Supplementname
-rw-r--r--. 1 root root      8192 Jan 13 13:53 Suggestname
-rw-r--r--. 1 root root      8192 Jan 13 13:53 Transfiletriggername
-rw-r--r--. 1 root root      8192 Jan 13 13:53 Recommendname
-rw-r--r--. 1 root root      8192 Jan 13 13:53 Triggername
-rw-r--r--. 1 root root     73728 Jan 13 13:53 Sigmd5
-rw-r--r--. 1 root root    114688 Jan 13 13:53 Sha1header
-rw-r--r--. 1 root root    356352 Jan 13 13:53 Requirename
-rw-r--r--. 1 root root   3325952 Jan 13 13:53 Providename
-rw-r--r--. 1 root root 146223104 Jan 13 13:53 Packages
-rw-r--r--. 1 root root     32768 Jan 13 13:53 Obsoletename
-rw-r--r--. 1 root root     69632 Jan 13 13:53 Name
-rw-r--r--. 1 root root     20480 Jan 13 13:53 Installtid
-rw-r--r--. 1 root root     28672 Jan 13 13:53 Group
-rw-r--r--. 1 root root   2342912 Jan 13 13:53 Dirnames
-rw-r--r--. 1 root root     16384 Jan 13 13:53 Conflictname
-rw-r--r--. 1 root root   5820416 Jan 13 13:53 Basenames
-rw-r--r--. 1 root root     90112 Jan 13 13:53 __db.002
-rw-r--r--. 1 root root    286720 Jan 13 13:53 __db.001
-rw-r--r--. 1 root root    263800 Jan 13 13:53 __db.003
[root@ovmgr1 g.cecchi]#

[root@ovmgr1 g.cecchi]# dnf update
Ceph packages for x86_64                                 1.0 MB/s | 412 kB
    00:00
Rocky Linux 8 - AppStream                                8.9 MB/s | 8.7 MB
    00:00
Rocky Linux 8 - BaseOS                                   4.3 MB/s | 4.6 MB
    00:01
Rocky Linux 8 - Extras                                   6.2 kB/s |  10 kB
    00:01
Rocky Linux 8 - PowerTools                               3.5 MB/s | 2.5 MB
    00:00
Latest oVirt 4.4 Release                                 388 kB/s | 2.6 MB
    00:06
Extra Packages for Enterprise Linux 8 - x86_64           6.9 MB/s |  11 MB
    00:01
CentOS-8 - Gluster 8                                     133 kB/s |  77 kB
    00:00
virtio-win builds roughly matching what will be shipped   92 kB/s | 143 kB
    00:01
Copr repo for EL8_collection owned by sbonazzo           226 kB/s | 246 kB
    00:01
Copr repo for gluster-ansible owned by sac                10 kB/s | 7.3 kB
    00:00
Advanced Virtualization packages for x86_64              1.2 MB/s | 255 kB
    00:00
CentOS-8 - oVirt 4.4                                     5.3 MB/s | 2.3 MB
    00:00
CentOS-8 - OpsTools - collectd                           1.3 MB/s | 149 kB
    00:00
CentOS-8 - NFV OpenvSwitch                               533 kB/s |  67 kB
    00:00
OpenStack Victoria Repository                            6.2 MB/s | 3.7 MB
    00:00
Dependencies resolved.
Nothing to do.
Complete!
[root@ovmgr1 g.cecchi]#

Is the above repos' list supposed to be ok?

Also, currently I have these 3 files

[root@ovmgr1 g.cecchi]# cat /etc/rocky-release
Rocky Linux release 8.5 (Green Obsidian)

[root@ovmgr1 g.cecchi]# cat /etc/centos-release
Rocky Linux release 8.5 (Green Obsidian)

[root@ovmgr1 g.cecchi]# cat /etc/redhat-release
Rocky Linux release 8.5 (Green Obsidian)
[root@ovmgr1 g.cecchi]#

I don't know if I have to remove any of those....

On manager now, connecting to the web admin UI:
oVirt Open Virtualization Manager
Software Version:4.4.9.5-1.el8

The hosts are there and so the VMs. More tests and checks to be done before
updating hosts.
In /etc/alternatives directory some java links were missing as detailed in
Rocky migration page
https://github.com/rocky-linux/rocky-tools/tree/main/migrate2rocky

followed their instructions

[root@ovmgr1 g.cecchi]# rpm -qa --scripts
java-{1.8.0,11}-openjdk-{headless,devel} | sed -n '/postinstall/, /exit/{
/postinstall/! { /exit/ ! p} }' | sh
[root@ovmgr1 g.cecchi]#

Gianluca

2024

2023

2022

2021

2020

2019

2018

2017

2016

2015

2014

2013

2012

2011

External engine CentOS 8.4 and 4.4.8 to Rocky Linux 8.5 and 4.4.9