oVirt 2.5.1.2 HCI Gluster Recovery Options
by Clint Boggio
Greetings oVirt community;
I've got a three-node HCI cluster with hosted engine and on one of the three nodes I lost the underlying RAID array under the Gluster bricks. It looks like I'm going to have to rebuild the RAID array and thus likely lose the Gluster configuration that lives there. The host OS (oVirt Node) is fine and it's just the Gluster/RAID piece that is affected.
Can one of you please point me to the best documentation link to the process of re-configuring that failed node to be identical to the rest of the nodes please ? It's configured for replica-three and I don't want to have a problem with nodes 2 and 3 before I get node 1 back up and healthy.
Any help will be greatly appreciated.
1 year, 11 months
Detach Button Greyed Out After Putting Storage Domain Into Maintenance
by mark.matlock＠oneildata.com
Trying to safely remove some storage domains in oVirt and am not getting errors or log messages explaining why the <Detach> button is greyed out. It succeeds in deactivating but I can't safely remove it. The only button available is <Activate>
Also, I've noticed unless I checkmark 'Ignore OVF Update Failures', the storage domain will exit Maintenance and go right back to Active status.
Software Version: 4.3.10.4-1.el7
Logged in as admin
1 year, 11 months
Host kernel command line incorrectly quoted in oVirt 4.5.3.2
by Gianluca Amato
Hi all.
I recently changed the kernel command line of one of my hosts using oVirt Manager (I use a custom kernel command line, not one obtained by selecting the standard features). This did not work as expected. The entire kernel command line was saved in /boot/loader/entries surrounded by single quotes, making it useless for the kernel. I may later fix the problem by connecting to the host and removing the single quotes by the files in /boot/loader/entries. I am sure the same procedure used to work in previous versions of oVirt.
Has someone else encountered this problem ?
I would be glad to submit a bug on bugzilla, but submitting bugs for oVirt seems to be blocked.
1 year, 11 months
Combining vNUMA and dedicated CPU Pinning Policy
by Gianluca Amato
Hello,
as stated in the subject, I am trying to combine the "dedicated" CPU Pinning Policy with vNUMA. My host has 2 sockets, 28 cores per socket and 2 threads per core. In each socket I allocated 128 hugepages of 1GB each. This is the output of "numactl --hardware":
------------
available: 2 nodes (0-1)
node 0 cpus: 0 2 4 6 8 10 12 14 16 18 20 22 24 26 28 30 32 34 36 38 40 42 44 46 48 50 52 54 56 58 60 62 64 66 68 70 72 74 76 78 80 82 84 86 88 90 92 94 96 98 100 102 104 106 108 110
node 0 size: 257333 MB
node 0 free: 122518 MB
node 1 cpus: 1 3 5 7 9 11 13 15 17 19 21 23 25 27 29 31 33 35 37 39 41 43 45 47 49 51 53 55 57 59 61 63 65 67 69 71 73 75 77 79 81 83 85 87 89 91 93 95 97 99 101 103 105 107 109 111
node 1 size: 257993 MB
node 1 free: 124972 MB
node distances:
node 0 1
0: 10 21
1: 21 10
-----------
I would like to dedicate one half of all the cores to a single guest. I created a VM with 2 sockets, 14 cores per socket and 2 threads per core, using the "dedicated" CPU Pinning Policy. If a set "NUMA Node Count" to zero, the VM starts and works flawlessy. However, all virtual cores are allocated in the first physical socket. If I set "NUMA Node Count" to 2 (without pinning vNUMA nodes to host NUMA nodes) I cannot start the VM since I get the error message: "The host xyz did not satisfy internal filter CpuPinning because it doesn't have enough CPUs for the dedicated CPU policy that the VM is set with.. ". Same thing happens if I set "NUMA Node Count" to 1.
Am I doing something wrong ?
1 year, 11 months
Re: Recombining disks under a single VM
by Matthew.Stier＠fujitsu.com
Scratch this query. It appears we had two identical images. It wouldn't permit importing due to matching disk id's.
From: Matthew.Stier(a)fujitsu.com <Matthew.Stier(a)fujitsu.com>
Sent: Saturday, December 10, 2022 10:24 PM
To: users(a)ovirt.org
Subject: [ovirt-users] Recombining disks under a single VM
Up until now, we've been using iSCSI as the storage on all of our datacenters, and when we need to bulk move VMs, we moved iSCSI storage domains.
We've just setup our first datacenter with Fiber Channel storage, and no connectivity to our iSCSI storage. For this move we have fallen back to using NFS storage domains, to move the necessary VMs from datacenter ONE to datacenter TWO.
The bulk of the transfer went fine, but for about one dozen two disk VMs both disks did not get moved at the same time. So now I have one dozen VMs with only one drive on datacenter TWO and the other drive as an un-importable VM on the NFS storage partition, mounted on datacenter TWO.
Any hints on how I "merge" these two, back into one function VM?
== Matthew.Stier(a)fujitsu.com<mailto:Matthew.Stier@fujitsu.com>
1 year, 11 months
Recombining disks under a single VM
by Matthew.Stier＠fujitsu.com
Up until now, we've been using iSCSI as the storage on all of our datacenters, and when we need to bulk move VMs, we moved iSCSI storage domains.
We've just setup our first datacenter with Fiber Channel storage, and no connectivity to our iSCSI storage. For this move we have fallen back to using NFS storage domains, to move the necessary VMs from datacenter ONE to datacenter TWO.
The bulk of the transfer went fine, but for about one dozen two disk VMs both disks did not get moved at the same time. So now I have one dozen VMs with only one drive on datacenter TWO and the other drive as an un-importable VM on the NFS storage partition, mounted on datacenter TWO.
Any hints on how I "merge" these two, back into one function VM?
== Matthew.Stier(a)fujitsu.com
1 year, 11 months
Migrating to keycloak
by eric.j.gillingham＠jpl.nasa.gov
I have an existing ovirt cluster, and I'm trying to migrate it from the internal sso and LDAP over to keycloak but am kind of at a loss.
I followed the Activation procedures on https://github.com/oVirt/ovirt-engine-keycloak/blob/master/keycloak_usage... and am able to login to the keycloak console fine, but when I try to access the ovirt-engine admin panel I just get an internal server error.
httpd log contains "oidc_util_json_string_print: oidc_util_check_json_error: response contained an "error" entry with value: ""Realm does not exist"""
Does engine-setup not configure the keycloak it creates with the proper configuration for ovirt? The apache config seems to have some password and other settings for oidc, so that end got configured, but not the keycloak side. There's no ovirt-engine or other ovirt related clients inside the newly created keycloak.
1 year, 11 months
centos-stream-release-8.6-1.el8.noarch vs redhat-release-8.7-0.3.el8.x86_64
by eshwayri＠gmail.com
What is the best way to resolve the below error. All other EL8 packages are now at 8.7. (--nobest) but the release file(s) aren't updating because this package won't upgrade. The blocking centos-stream-release-8.6-1.el8.noarch package is a pre-req for all the oVirt files, so I can't remove it. I suspect all oVirt is using from it are the repos. Do I force install redhat-release-8.7-0.3.el8.x86_64? Is there a way to do that?
[root@kvmo-el8 /]# yum update
Updating Subscription Management repositories.
Last metadata expiration check: 2:28:57 ago on Wed 07 Dec 2022 03:25:45 PM EST.
Error:
Problem: installed package centos-stream-release-8.6-1.el8.noarch obsoletes redhat-release < 9 provided by redhat-release-8.7-0.3.el8.x86_64
- cannot install the best update candidate for package redhat-release-8.6-0.1.el8.x86_64
- problem with installed package centos-stream-release-8.6-1.el8.noarch
(try to add '--skip-broken' to skip uninstallable packages or '--nobest' to use not only best candidate packages)
1 year, 11 months
Failed to execute stage 'Closing up': Command '/usr/share/ovirt-engine-keycloak/bin/kk_cli.sh' failed to execute
by yp414＠163.com
engine-setup
[ INFO ] Stage: Initializing
[ INFO ] Stage: Environment setup
Configuration files: /etc/ovirt-engine-setup.conf.d/10-packaging-jboss.conf, /etc/ovirt-engine-setup.conf.d/10-packaging.conf
Log file: /var/log/ovirt-engine/setup/ovirt-engine-setup-20221204031123-60y1er.log
Version: otopi-1.10.3 (otopi-1.10.3-1.el8)
[ INFO ] Stage: Environment packages setup
[ INFO ] Stage: Programs detection
[ INFO ] Stage: Environment setup (late)
[ INFO ] Stage: Environment customization
--== PRODUCT OPTIONS ==--
Configure Cinderlib integration (Currently in tech preview) (Yes, No) [No]:
Configure Engine on this host (Yes, No) [Yes]:
Configuring ovirt-provider-ovn also sets the Default cluster's default network provider to ovirt-provider-ovn.
Non-Default clusters may be configured with an OVN after installation.
Configure ovirt-provider-ovn (Yes, No) [Yes]:
Configure WebSocket Proxy on this host (Yes, No) [Yes]:
* Please note * : Data Warehouse is required for the engine.
If you choose to not configure it on this host, you have to configure
it on a remote host, and then configure the engine on this host so
that it can access the database of the remote Data Warehouse host.
Configure Data Warehouse on this host (Yes, No) [Yes]:
* Please note * : Keycloak is now deprecating AAA/JDBC authentication module.
It is highly recommended to install Keycloak based authentication.
Configure Keycloak on this host (Yes, No) [Yes]:
Configure VM Console Proxy on this host (Yes, No) [Yes]:
Configure Grafana on this host (Yes, No) [Yes]:
--== PACKAGES ==--
[ INFO ] Checking for product updates...
[ INFO ] No product updates found
--== NETWORK CONFIGURATION ==--
Host fully qualified DNS name of this server [pm.local]:
[WARNING] Failed to resolve pm.local using DNS, it can be resolved only locally
Setup can automatically configure the firewall on this system.
Note: automatic configuration of the firewall may overwrite current settings.
Do you want Setup to configure the firewall? (Yes, No) [Yes]:
[ INFO ] firewalld will be configured as firewall manager.
--== DATABASE CONFIGURATION ==--
Where is the DWH database located? (Local, Remote) [Local]:
Setup can configure the local postgresql server automatically for the DWH to run. This may conflict with existing applications.
Would you like Setup to automatically configure postgresql and create DWH database, or prefer to perform that manually? (Automatic, Manual) [Automatic]:
Where is the Keycloak database located? (Local, Remote) [Local]:
Setup can configure the local postgresql server automatically for the Keycloak to run. This may conflict with existing applications.
Would you like Setup to automatically configure postgresql and create Keycloak database, or prefer to perform that manually? (Automatic, Manual) [Automatic]:
Where is the Engine database located? (Local, Remote) [Local]:
Setup can configure the local postgresql server automatically for the engine to run. This may conflict with existing applications.
Would you like Setup to automatically configure postgresql and create Engine database, or prefer to perform that manually? (Automatic, Manual) [Automatic]:
--== OVIRT ENGINE CONFIGURATION ==--
Engine admin password:
Confirm engine admin password:
[WARNING] Password is weak: The password is shorter than 8 characters
Use weak password? (Yes, No) [No]:yes
Application mode (Virt, Gluster, Both) [Both]:
Use Engine admin password as initial keycloak admin password (Yes, No) [Yes]:
--== STORAGE CONFIGURATION ==--
Default SAN wipe after delete (Yes, No) [No]:
--== PKI CONFIGURATION ==--
Organization name for certificate [local]:
--== APACHE CONFIGURATION ==--
Setup can configure the default page of the web server to present the application home page. This may conflict with existing applications.
Do you wish to set the application as the default page of the web server? (Yes, No) [Yes]:
Setup can configure apache to use SSL using a certificate issued from the internal CA.
Do you wish Setup to configure that, or prefer to perform that manually? (Automatic, Manual) [Automatic]:
--== SYSTEM CONFIGURATION ==--
--== MISC CONFIGURATION ==--
Please choose Data Warehouse sampling scale:
(1) Basic
(2) Full
(1, 2)[1]:
Use Engine admin password as initial Grafana admin password (Yes, No) [Yes]:
--== END OF CONFIGURATION ==--
[ INFO ] Stage: Setup validation
[WARNING] Less than 16384MB of memory is available
--== CONFIGURATION PREVIEW ==--
Application mode : both
Default SAN wipe after delete : False
Host FQDN : pm.local
Firewall manager : firewalld
Update Firewall : True
Set up Cinderlib integration : False
Configure local Engine database : True
Set application as default page : True
Configure Apache SSL : True
Keycloak installation : True
Engine database host : localhost
Engine database port : 5432
Engine database secured connection : False
Engine database host name validation : False
Engine database name : engine
Engine database user name : engine
Engine installation : True
PKI organization : local
Set up ovirt-provider-ovn : True
DWH installation : True
DWH database host : localhost
DWH database port : 5432
DWH database secured connection : False
DWH database host name validation : False
DWH database name : ovirt_engine_history
Configure local DWH database : True
Grafana integration : True
Grafana database user name : ovirt_engine_history_grafana
Keycloak database host : localhost
Keycloak database port : 5432
Keycloak database secured connection : False
Keycloak database host name validation : False
Keycloak database name : ovirt_engine_keycloak
Keycloak database user name : ovirt_engine_keycloak
Configure local Keycloak database : True
Configure VMConsole Proxy : True
Configure WebSocket Proxy : True
Please confirm installation settings (OK, Cancel) [OK]:
[ INFO ] Stage: Transaction setup
[ INFO ] Stopping engine service
[ INFO ] Stopping ovirt-fence-kdump-listener service
[ INFO ] Stopping dwh service
[ INFO ] Stopping vmconsole-proxy service
[ INFO ] Stopping websocket-proxy service
[ INFO ] Stage: Misc configuration (early)
[ INFO ] Stage: Package installation
[ INFO ] Stage: Misc configuration
[ INFO ] Upgrading CA
[ INFO ] Creating PostgreSQL 'engine' database
[ INFO ] Configuring PostgreSQL
[ INFO ] Creating PostgreSQL 'ovirt_engine_history' database
[ INFO ] Configuring PostgreSQL
[ INFO ] Creating PostgreSQL 'ovirt_engine_keycloak' database
[ INFO ] Configuring PostgreSQL
[ INFO ] Creating CA: /etc/pki/ovirt-engine/ca.pem
[ INFO ] Creating CA: /etc/pki/ovirt-engine/qemu-ca.pem
[ INFO ] Creating a user for Grafana
[ INFO ] Setting up ovirt-vmconsole proxy helper PKI artifacts
[ INFO ] Setting up ovirt-vmconsole SSH PKI artifacts
[ INFO ] Configuring WebSocket Proxy
[ INFO ] Creating/refreshing Engine database schema
[ INFO ] Creating/refreshing DWH database schema
[ INFO ] Updating OVN SSL configuration
[ INFO ] Updating OVN timeout configuration
[ INFO ] Creating/refreshing Engine 'internal' domain database schema
[ INFO ] Creating default mac pool range
[ INFO ] Adding default OVN provider to database
[ INFO ] Adding OVN provider secret to database
[ INFO ] Setting a password for internal user admin
[ INFO ] Creating initial Keycloak admin user
[ INFO ] Generating post install configuration file '/etc/ovirt-engine-setup.conf.d/20-setup-ovirt-post.conf'
[ INFO ] Stage: Transaction commit
[ INFO ] Stage: Closing up
--== SUMMARY ==--
[ INFO ] No need to restart fapolicyd because it is not running.
[ INFO ] Starting dwh service
[ INFO ] Starting Grafana service
[ INFO ] Restarting ovirt-vmconsole proxy service
To login to oVirt using Keycloak SSO, enter 'admin@ovirt' as username and the password provided during Setup
To login to Keycloak Administration Console enter 'admin' as username and the password provided during Setup
Web access for Keycloak Administration Console is enabled at:
https://pm.local/ovirt-engine-auth/admin
Web access is enabled at:
http://pm.local:80/ovirt-engine
https://pm.local:443/ovirt-engine
Internal CA fingerprint: SHA256: F6:6C:CF:41:58:64:D1:84:25:10:A6:6B:4D:96:8B:EB:F5:F2:DA:FB:BD:CF:B4:2C:02:62:0B:0A:B3:15:14:33
SSH fingerprint: SHA256:Xnov0hwwe6/DN5udn3MypHx9EU5CelG6eYMHlaUZJFQ
[ INFO ] Starting engine service
[WARNING] Less than 16384MB of memory is available
Web access for grafana is enabled at:
https://pm.local/ovirt-engine-grafana/
Please run the following command on the engine machine pm.local, for SSO to work:
systemctl restart ovirt-engine
--== END OF SUMMARY ==--
[ INFO ] Restarting httpd
[ INFO ] Start with setting up Keycloak for Ovirt Engine
[ ERROR ] Failed to execute stage 'Closing up': Command '/usr/share/ovirt-engine-keycloak/bin/kk_cli.sh' failed to execute
[ INFO ] Stage: Clean up
Log file is located at /var/log/ovirt-engine/setup/ovirt-engine-setup-20221204031123-60y1er.log
[ INFO ] Generating answer file '/var/lib/ovirt-engine/setup/answers/20221204031509-setup.conf'
[ INFO ] Stage: Pre-termination
[ INFO ] Stage: Termination
[ ERROR ] Execution of setup failed
1 year, 11 months
