January 2024 - Users - oVirt List Archives

Cannot remove Snapshot. The VM is during a backup operation.
by and＠missme.ro 30 Jan '24

30 Jan '24

Hello! Running ovirt Version 4.5.5-1.el8 I had an issue with the iscsi server during the backup and I have two VMs that cannot be backed up anymore by Veeam. In the ovirt event log i have the following errors: Snapshot 'Auto-generated for Backup VM' creation for VM 'dns-a' has been completed. VDSM ovirt1-02 command StartNbdServerVDS failed: Bitmap does not exist: "{'reason': 'Bitmap does not exist in /rhev/data-center/mnt/blockSD/b2fa3469-a380-4180-a89a-43d65085d1b9/images/6a4de98a-b544-4df8-beb1-e560fd61c0e6/cdb26b8b-c447-48de-affa-d7f778aebac7', 'bitmap': '12d2fb20-74da-4e63-b240-f1a42210760c'}" Transfer was stopped by system. Reason: failed to create a signed image ticket. Image Download with disk dns-a_Disk1 was initiated by veeam@internal-authz Image Download with disk dns-a_Disk1 was cancelled. The error on the Veeam backup proxy: dns-a: Unable to create image transfer: Reason: 'Operation Failed', Detail: '[]' When trying to delete the snapshot from the administration interface I receive the following error in the web interface (and nothing gets logged in the event log) Cannot remove Snapshot. The VM is during a backup operation. How should I go about fixing this issue?

2 1

HE Storage Domain Path Config Setting - Where?
by Matthew J Black 30 Jan '24

30 Jan '24

Hey Guys, Quick Q: In which file (on a Hosted-Engine or Hosted-Engine Host) is the configuration for the path to a Storage Domain kept - in particular, the "hosted-engine" Storage Domain? I've got something "funny" going on: the logs (as far as I can see) are reporting that 2 of my 3 HE-Hosts can't connect to the HE Storage Domain (but don't explain why), and the OVE GUI is reporting an "odd" (ie incorrect; non-existent) path to the HE Storage Domain. Via CLI I have confirmed that all three HE Hosts *can* reach (ie have the correct "findmnt" mappings) to the HE Storage Domain's actual file location, and I can't locate the "ghost" HE Storage Domain path or its config setting anywhere - so I don't even know if that's the issue, but I'd like to eliminate it from my trouble-shooting process. Anyway, if someone could get back to me, please, I'd really appreciate it. Cheers Dulux-Oz

2 1

Geo-replication configuration problem
by Ismet Sonmez 28 Jan '24

28 Jan '24

Hello, Newly installed node version 4.5.5 2 clusters and three nodes each every time I try to replicate the geo in cluster 1 it gives an error 3 errors like this: VDSM node4 command UpdateGlusterGeoRepKeysVDS failed: Internal JSON-RPC error: {'cause': 'Attempting to call function: GlusterVolume.geoRepKeysUpdate bound method of object <vdsm.gluster.apiwrapper.GlusterVolume at address 0x7f1524534f28>> with arguments: (\'root\', \'command="/usr/libexec/glusterfs/gsyncd" ssh-rsa AAAAB3N**** \'command="tar ${SSH_ORIGINAL_COMMAND#* }" ssh-rsa AAAA**** \'command="/usr/libexec/glusterfs/gsyncd" ssh-rsa AAAAB3**** \'command="tar ${SSH_ORIGINAL_COMMAND#* }" ssh-rsa AAAA*** \'command="/usr/libexec/glusterfs/gsyncd" ssh-rsa AAAA******* \'command="tar ${SSH_ORIGINAL_COMMAND#* }" ssh-rsa AAAA**** ]) error: bytes-like object required, not \'str\' } I understand that when creating the rsa key, or sending it to the node. How can I solve it?

1 0

Forbid hosts/nodes from assembling a soft-raid that was created inside a VM
by Vladislav Solovei 27 Jan '24

27 Jan '24

When I create a soft-raid (md-raid) inside a virtual machine (yes, sometimes it's necessary to do this to get a disk inside the VM with a capacity of more than 8TB), the host/node detects this array and assembles it automatically. I tried adding the parameter 'raid=noautodetect' to the kernel boot parameters, but it doesn't help. Is it possible to prevent hosts from doing this? :)

1 0

Low confirmed free space on gluster volume
by Jonas 27 Jan '24

27 Jan '24

Hello list I am regularly getting the following error on a Gluster volume hosted on a three node hyperconverged oVirt-Cluster: Warning! Low confirmed free space on gluster volume tier1-owncloud-users-01 This volume is configured with 3TiB (2.5TIB used) and is used as the name implies to store data of an ownCloud instance. It is not used as a storage domain in oVirt. I don't really want to resize the volume just to make the warning go away. While clicking through the oVirt web interface i found the configuration "Warning Low Confirmed Space Indicator" in the advanced parameters of the storage domains but did not find any similar setting in the configuration of the gluster volumes. Do you know of way to configure this setting for a gluster volume which is not used as a storage domain? Thank you Jonas

1 0

Change Hosted Engine VM MAC address
by Sergei Panchenko 26 Jan '24

26 Jan '24

Good morning, colleagues! Due some network issues I need to change HostedEngine VM MAC address. The additional diffeculty is unaccessebility of the HE Web-interface via the network (the cause of - those network issues -)). Is there any way to change the HE vNIC's MAC using command line (on the host where HE VM works or on the HE VM)? Thanks in advance, Sergei.

2 2

Need help again, new issue with VM import
by Michaal R 24 Jan '24

24 Jan '24

I have a VM that's a little over 4TB in size that won't import for whatever reason. I've tried changing the export from the streaming disk format to a flat vmdk,thinking that might be it, but it didn't work. I've gone through the OVF and don't see anything that stands out. Same for the logs, I see where it's having issues importing the 4TB drive from the VM, but I can't decipher the error messages well enough to know how to fix it. The drives had chkdsk run on them before they were exported, and the VM was cleanly shut down, so I don't understand what the issue is. Not even sure at this point how to run a filesystem repair on the vmdk, in case the export corrupted something. And I can't export it directly from ESXi either, for some reason. Each attempt fails with an unspecified error. Below is a snip from the import log where it's reading the drives. It has one of the first errors: [ 2.492229] scsi host0: Virtio SCSI HBA [ 2.501365] scsi 0:0:0:0: Direct-Access QEMU QEMU HARDDISK 2.5+ PQ: 0 ANSI: 5 [ 2.504080] scsi 0:0:1:0: Direct-Access QEMU QEMU HARDDISK 2.5+ PQ: 0 ANSI: 5 [ 2.506035] scsi 0:0:2:0: Direct-Access QEMU QEMU HARDDISK 2.5+ PQ: 0 ANSI: 5 [ 2.540737] sd 0:0:0:0: Power-on or device reset occurred [ 2.540790] sd 0:0:1:0: Power-on or device reset occurred [ 2.540899] sd 0:0:2:0: Power-on or device reset occurred [ 2.541130] sd 0:0:0:0: [sda] 8589934592 512-byte logical blocks: (4.40 TB/4.00 TiB) [ 2.541168] sd 0:0:2:0: [sdc] 8388608 512-byte logical blocks: (4.29 GB/4.00 GiB) [ 2.541319] sd 0:0:2:0: [sdc] Write Protect is off [ 2.541391] sd 0:0:0:0: [sda] Write Protect is off [ 2.541592] sd 0:0:2:0: [sdc] Write cache: enabled, read cache: enabled, doesn't support DPO or FUA [ 2.542019] sd 0:0:0:0: [sda] Write cache: enabled, read cache: enabled, doesn't support DPO or FUA [ 2.544060] sd 0:0:2:0: [sdc] Attached SCSI disk [ 2.545294] sd 0:0:1:0: [sdb] 251658240 512-byte logical blocks: (129 GB/120 GiB) [ 2.545437] sd 0:0:1:0: [sdb] Write Protect is off [ 2.545966] sd 0:0:1:0: [sdb] Write cache: enabled, read cache: enabled, doesn't support DPO or FUA [ 2.551250] sdb: sdb1 sdb2 [ 2.556698] sd 0:0:1:0qemu-nbd: Disconnect client, due to: Failed to send reply: reading from file failed: Invalid argument : [sdb] Attached SCSI disk qemu-nbd: Disconnect client, due to: Failed to send reply: reading from file failed: Invalid argument qemu-nbd: Disconnect client, due to: Failed to send reply: reading from file failed: Invalid argument [ 2.618883] sd 0:0:0:0: [sda] tag#157 FAILED Result: hostbyte=DID_OK driverbyte=DRIVER_OK cmd_age=0s [ 2.618890] sd 0:0:0:0: [sda] tag#157 Sense Key : Aborted Command [current] [ 2.618892] sd 0:0:0:0: [sda] tag#157 Add. Sense: I/O process terminated [ 2.618894] sd 0:0:0:0: [sda] tag#157 CDB: Read(16) 88 00 00 00 00 01 ff ff ff f8 00 00 00 08 00 00 [ 2.618897] I/O error, dev sda, sector 8589934584 op 0x0:(READ) flags 0x0 phys_seg 1 prio class 2 supermin: intern[ 2.618902] Buffer I/O error on dev sda, logical block 1073741823, async page read al insmod virtio[ 2.618927] Alternate GPT is invalid, using primary GPT. The rest of the log is peppered with Buffer I/O error entries on /dev/sda (the 4TB vmdk). Could someone please help? I think I've been looking at these logs and trying to fix the drive for so long I've gone logic blind and can't see the answer right in front of my eyes. Here's a link to the logs pulled from the host: https://www.dropbox.com/scl/fi/570t279s0k3pfgafuvv01/felicity-import-1.18.2…

1 0

Re: [External] : Re: can hosted engine deploy use local repository mirrors instead of internet ones?
by iucounu＠gmail.com 24 Jan '24

24 Jan '24

Hi Marcos, > > The dnsmasq service running on the KVM host manages the IP assignment during the first > deployment phase. > How did you deploy your KVM host? Which configurations have you done on it before running > the hosted-engine --deploy? > Also, what is your full hosted-engine deployment command? > I deployed the KVM hosts using the guide at: https://www.ovirt.org/documentation/installing_ovirt_as_a_self-hosted_engin… (section 4.2 Installing Enterprise Linux Hosts) and https://www.ovirt.org/download/install_on_rhel.html Most of the ovirt packages were installed from the installation of the ovirt-engine-appliance. The KVM hosts are using EL 9.3. I've tried on two different EL hosts: one with the standard ovirt-45 repos, and the other had the nightly builds enabled. I observed the same issues on both. I haven't changed any specific settings, such as networking or storage. The full command to deploy the engine VM was done via: hosted-engine --deploy --4 I have just setup glusterfs just as a temporary storage option now (yet to run the deploy again), though I'm not sure how to get ovirt to use it. As mentioned I don't know whether this or the networking is causing it to fail. In case it is important, I notice that virtnetworkd.socket systemd unit gets killed during the deployment, and has to be restarted otherwise the deployment fails prematurely. It also masks all the libvirtd systemd units as part of the cleanup, and these have to manually unmasked and several need manually restarting (virtnetword.socket, virtqemud.socket and virtstoraged.sock) before the deployment is run again or the deploy will fail on trying to communicate with these. Thanks very much for the help, let me know if any further information needed. Cam > > -----Original Message----- > From: iucounu(a)gmail.com <iucounu(a)gmail.com> > Sent: Tuesday, January 23, 2024 12:49 PM > To: users(a)ovirt.org > Subject: [ovirt-users] Re: [External] : Re: can hosted engine deploy use local repository > mirrors instead of internet ones? > > Thanks very for the reply Marcos. I tried another deployment, just to see if "wait > for the host to be up" would time out, and I saw a couple of errors in the log: > > From the ovirt-hosted-engine-setup-ansible-final_clean log, it mentions that the VM IP is > undefined: > > 2024-01-23 12:50:19,554+0000 DEBUG otopi.ovirt_hosted_engine_setup.ansible_utils > ansible_utils._process_output:109 {'msg': "The task includes an option with > an undefined variable. The error was: 'local_vm_ip' is undefined. > 'local_vm_ip' is undefined\n\nThe error appears to be in > '/usr/share/ansible/collections/ansible_collections/ovirt/ovirt/roles/hosted_engine_setup/tasks/sync_on_engine_machine.yml': > line 2, column 3, but may\nbe elsewhere in the file depending on the exact syntax > problem.\n\nThe offending line appears to be:\n\n---\n- name: Set the name for add_host\n > ^ here\n", '_ansible_no_log': False} > > In the ovirt-hosted-engine-setuplog, it mentions not being able to get the storage pool: > > > 2024-01-23 12:50:35,787+0000 DEBUG otopi.ovirt_hosted_engine_setup.ansible_utils > ansible_utils._process_output:109 {'changed': True, 'stdout': '', > 'stderr': "error: failed to get pool 'localvmvy8whst5'\nerror: > Storage pool not found: no storage pool with matching name > 'localvmvy8whst5'", 'rc': 1, 'cmd': ['virsh', > '-c', 'qemu:///system?authfile=/etc/ovirt-hosted-engine/virsh_auth.conf', > 'pool-destroy', 'localvmvy8whst5'], 'start': '2024-01-23 > 12:50:35.558666', 'end': '2024-01-23 12:50:35.611808', > 'delta': '0:00:00.053142', 'msg': 'non-zero return code', > 'invocation': {'module_args': {'_raw_params': 'virsh -c > qemu:///system?authfile=/etc/ovirt-hosted-engine/virsh_auth.conf pool-destroy > localvmvy8whst5', '_uses_shell': False, 'stdin_add_newline': True, > 'strip_empty_ends': True, 'argv': None, 'chdir': None, > 'executable': None, 'creates': None, 'removes': None, > 'stdin': None}}, 'stdout_lines': [], 'stderr_lines': ["error: > failed to get pool 'localvmvy8whst5'", "error: Storage pool not fou > nd: no storage pool with matching name 'localvmvy8whst5'"], > '_ansible_no_log': None} > > I set the IP to the one I have assigned in DNS, but when I attach to the console of the VM > (which is still running, though the disk image has been deleted) via virsh, it shows me a > completely different IP: In hosted-engine --deploy, I set a 10.0.0.x address, however, it > shows a 192.168.1.x address on the VM. Do I need to set this somewhere else, e.g., with > '--ansible-extra-vars=he_ipv4_subnet_prefix='? > > As for the storage pool, is that for later VM deployment? The deploy script did not ask me > for a storage location. If I need to specify this, where do I do this? > > Thanks again for any help, > > Kind regards, > > Cam > > PS: is there a simple way to have the answers saved so I don't have to keep running > through all the questions every time I try a deployment > _______________________________________________ > Users mailing list -- users(a)ovirt.org > To unsubscribe send an email to users-leave(a)ovirt.org Privacy Statement: > https://urldefense.com/v3/__https://www.ovirt.org/privacy-policy.html__;!... > oVirt Code of Conduct: > https://urldefense.com/v3/__https://www.ovirt.org/community/about/communi... > List Archives: > https://urldefense.com/v3/__https://lists.ovirt.org/archives/list/users@o...

1 0

Re: [External] : Re: can hosted engine deploy use local repository mirrors instead of internet ones?
by iucounu＠gmail.com 24 Jan '24

24 Jan '24

Thanks very for the reply Marcos. I tried another deployment, just to see if "wait for the host to be up" would time out, and I saw a couple of errors in the log: From the ovirt-hosted-engine-setup-ansible-final_clean log, it mentions that the VM IP is undefined: 2024-01-23 12:50:19,554+0000 DEBUG otopi.ovirt_hosted_engine_setup.ansible_utils ansible_utils._process_output:109 {'msg': "The task includes an option with an undefined variable. The error was: 'local_vm_ip' is undefined. 'local_vm_ip' is undefined\n\nThe error appears to be in '/usr/share/ansible/collections/ansible_collections/ovirt/ovirt/roles/hosted_engine_setup/tasks/sync_on_engine_machine.yml': line 2, column 3, but may\nbe elsewhere in the file depending on the exact syntax problem.\n\nThe offending line appears to be:\n\n---\n- name: Set the name for add_host\n ^ here\n", '_ansible_no_log': False} In the ovirt-hosted-engine-setuplog, it mentions not being able to get the storage pool: 2024-01-23 12:50:35,787+0000 DEBUG otopi.ovirt_hosted_engine_setup.ansible_utils ansible_utils._process_output:109 {'changed': True, 'stdout': '', 'stderr': "error: failed to get pool 'localvmvy8whst5'\nerror: Storage pool not found: no storage pool with matching name 'localvmvy8whst5'", 'rc': 1, 'cmd': ['virsh', '-c', 'qemu:///system?authfile=/etc/ovirt-hosted-engine/virsh_auth.conf', 'pool-destroy', 'localvmvy8whst5'], 'start': '2024-01-23 12:50:35.558666', 'end': '2024-01-23 12:50:35.611808', 'delta': '0:00:00.053142', 'msg': 'non-zero return code', 'invocation': {'module_args': {'_raw_params': 'virsh -c qemu:///system?authfile=/etc/ovirt-hosted-engine/virsh_auth.conf pool-destroy localvmvy8whst5', '_uses_shell': False, 'stdin_add_newline': True, 'strip_empty_ends': True, 'argv': None, 'chdir': None, 'executable': None, 'creates': None, 'removes': None, 'stdin': None}}, 'stdout_lines': [], 'stderr_lines': ["error: failed to get pool 'localvmvy8whst5'", "error: Storage pool not fou nd: no storage pool with matching name 'localvmvy8whst5'"], '_ansible_no_log': None} I set the IP to the one I have assigned in DNS, but when I attach to the console of the VM (which is still running, though the disk image has been deleted) via virsh, it shows me a completely different IP: In hosted-engine --deploy, I set a 10.0.0.x address, however, it shows a 192.168.1.x address on the VM. Do I need to set this somewhere else, e.g., with '--ansible-extra-vars=he_ipv4_subnet_prefix='? As for the storage pool, is that for later VM deployment? The deploy script did not ask me for a storage location. If I need to specify this, where do I do this? Thanks again for any help, Kind regards, Cam PS: is there a simple way to have the answers saved so I don't have to keep running through all the questions every time I try a deployment

2 1

Apple Mac Pro 2013 install hangs with oVirt Node installer 4.5 but ok with 4.3.10
by john＠alwayson.net.au 24 Jan '24

24 Jan '24

This hardware (6 Core Xeon E5-1650v2) successfully runs Fedora 39 and oVirt 4.3.10 (CentOS 7) but freezes immediately when attempting to boot from either of the latest node installer ISOs: ovirt-node-ng-installer-latest-el8.iso ovirt-node-ng-installer-latest-el9.iso I suspect it will require tweaks to the kernel parameters which will need to be made to the ISO installer image prior to booting. Any suggestions would be appreciated. Thanks

2 2

Upgrade host from 4.5.4 to 4.5.5 failed
by Jacob M. Nielsen 23 Jan '24

23 Jan '24

Upgrade host from Ovirt manager - Result Install Failed Here log 2024-01-23 23:08:38 CET - TASK [ovirt-host-upgrade : Upgrade packages] *********************************** 2024-01-23 23:11:47 CET - { "uuid" : "a7ac6634-4644-4416-a818-183b42b3c925", "counter" : 191, "stdout" : "", "start_line" : 186, "end_line" : 186, "runner_ident" : "0b7adac9-6ff4-4934-a9c1-ad0785877978", "event" : "runner_on_failed", "pid" : 135486, "created" : "2024-01-23T22:11:45.029493", "parent_uuid" : "000c2930-20d1-ae10-b6a7-000000000042", "event_data" : { "playbook" : "ovirt-host-upgrade.yml", "playbook_uuid" : "89cebab6-f73e-4a2f-bb30-b8de7e04367a", "play" : "all", "play_uuid" : "000c2930-20d1-ae10-b6a7-000000000002", "play_pattern" : "all", "task" : "Upgrade packages", "task_uuid" : "000c2930-20d1-ae10-b6a7-000000000042", "task_action" : "ansible.builtin.yum", "task_args" : "", "task_path" : "/usr/share/ovirt-engine/ansible-runner-service-project/project/roles/ovirt-host-upgrade/tasks/main.yml:69", "role" : "ovirt-host-upgrade", "host" : "10.63.0.6", "remote_addr" : "10.63.0.6", "res" : { "results" : [ { "failed" : true, "msg" : "Failed to validate GPG signature for ovirt-node-ng-image-update-4.5.5-1.el8.noarch: Public key for ovirt-node-ng-image-update-4.5.5-1.el8.noarch.rpm is not installed", "invocation" : { "module_args" : { Tried to just do the yum update on the host Got this result Tried to upgrade host from Ovirt manager , and got InstallFailed 2024-01-23 23:08:38 CET - TASK [ovirt-host-upgrade : Upgrade packages] *********************************** 2024-01-23 23:11:47 CET - { "uuid" : "a7ac6634-4644-4416-a818-183b42b3c925", "counter" : 191, "stdout" : "", "start_line" : 186, "end_line" : 186, "runner_ident" : "0b7adac9-6ff4-4934-a9c1-ad0785877978", "event" : "runner_on_failed", "pid" : 135486, "created" : "2024-01-23T22:11:45.029493", "parent_uuid" : "000c2930-20d1-ae10-b6a7-000000000042", "event_data" : { "playbook" : "ovirt-host-upgrade.yml", "playbook_uuid" : "89cebab6-f73e-4a2f-bb30-b8de7e04367a", "play" : "all", "play_uuid" : "000c2930-20d1-ae10-b6a7-000000000002", "play_pattern" : "all", "task" : "Upgrade packages", "task_uuid" : "000c2930-20d1-ae10-b6a7-000000000042", "task_action" : "ansible.builtin.yum", "task_args" : "", "task_path" : "/usr/share/ovirt-engine/ansible-runner-service-project/project/roles/ovirt-host-upgrade/tasks/main.yml:69", "role" : "ovirt-host-upgrade", "host" : "10.63.0.6", "remote_addr" : "10.63.0.6", "res" : { "results" : [ { "failed" : true, "msg" : "Failed to validate GPG signature for ovirt-node-ng-image-update-4.5.5-1.el8.noarch: Public key for ovirt-node-ng-image-update-4.5.5-1.el8.noarch.rpm is not installed", "invocation" : { "module_args" : { Tried to make the yum update Last metadata expiration check: 0:55:29 ago on 2024-01-23T22:43:18 CET. Dependencies resolved. =========================================================================================================================================== Package Architecture Version Repository Size =========================================================================================================================================== Installing: ovirt-node-ng-image-update noarch 4.5.5-1.el8 ovirt-45-upstream 1.3 G replacing ovirt-node-ng-image-update-placeholder.noarch 4.5.4-1.el8 Transaction Summary =========================================================================================================================================== Install 1 Package Total download size: 1.3 G Is this ok [y/N]: y Downloading Packages: ovirt-node-ng-image-update-4.5.5-1.el8.noarch.rpm 7.3 MB/s | 1.3 GB 02:59 ------------------------------------------------------------------------------------------------------------------------------------------- Total 7.2 MB/s | 1.3 GB 03:02 oVirt upstream for CentOS Stream 8 - oVirt 4.5 2.8 MB/s | 2.9 kB 00:00 Importing GPG key 0xFE590CB7: Userid : "oVirt <infra(a)ovirt.org>" Fingerprint: 31A5 D783 7FAD 7CB2 86CD 3469 AB8C 4F9D FE59 0CB7 From : /etc/pki/rpm-gpg/RPM-GPG-KEY-oVirt-4.5 Is this ok [y/N]: y Key imported successfully Import of key(s) didn't help, wrong key(s)? Public key for ovirt-node-ng-image-update-4.5.5-1.el8.noarch.rpm is not installed. Failing package is: ovirt-node-ng-image-update-4.5.5-1.el8.noarch GPG Keys are configured as: file:///etc/pki/rpm-gpg/RPM-GPG-KEY-oVirt-4.5 The downloaded packages were saved in cache until the next successful transaction. You can remove cached packages by executing 'yum clean packages'. Error: GPG check FAILED Anyone seen this and found a solution ??

1 0

oVirt nodes with local storage
by Wild Star 21 Jan '24

21 Jan '24

On the weekend I upgraded my self-hosted oVirt engine to 4.56. All went well with that! I also spotted that there was an update for all my 4.54 nodes, but something has changed with the repo overnight because none of my nodes see updates anymore, though the update remains available here … https://resources.ovirt.org/pub/ovirt-4.5/iso/ovirt-node-ng-installer/. Yesterday, I attempted to update just one of my nodes and ran into this snag… “Local storage domains were found on the same filesystem as / ! Please migrate the data to a new LV before upgrading, or you will lose the VMs”. I’ve always stored my VMs in a separate /DATA directory off the root filesystem, and shared the local storage using NFS. I know it’s not ideal, but with good frequent regular backups, it has served me well for many years. A Google search revealed, that others have also had similar local storage issues, and suggestions on ways to mitigate the issue, including the oVirt documentation found here… https://www.ovirt.org/documentation/upgrade_guide/index.html#Upgrading_hype…, is not my preferred fix. In the past node updates with my local storage were not a problem and were easy peasy! From some of the discussions I saw at Red Hat, I have deduced (maybe wrongly) that there was an issue that necessitated a fix, which introduced a required check for local storage during the upgrade process. I probably should move all the local storage off the oVirt nodes, but at this time, that is easier said than done. I’m posting here only to see if there are perhaps other ideas or perspectives I may not have thought of and should consider with my local storage. Thank you all in advance, much appreciated, and of course, thank you all for supporting oVirt!

3 5

Odd behavior after upgrade to 4.5
by Seann G. Clark 19 Jan '24

19 Jan '24

All, I just recently upgraded from 4.3 to 4.5, and while most of the upgrade went smoothly and I didn't see any major issues, I have noticed a few things that are causing problems. The one piece that I had to sneak in to make the install successful, is push my CA and intermediate CA certs into the engine after it came up the first time, but before it fully installed, otherwise it would fail restoring the backup. This is to address using a custom Apache cert for the hosted engine, instead of the default self-signed CA. The biggest issue is with keycloak. To make the upgrade succeed I had to skip keycloak installation, after doing some digging related to this error: "Failed to execute stage 'Misc configuration': 'OVESETUP_OVN/ovirtProviderOvnSecret'" I don't actively use OVN in my environment right now, and I think it is all pretty much in the default install state. This is the same error I get when I loop back to install keycloak now that the engine is up. This is what is in the logs for the last few lines preceding the error and failure event: 2024-01-19 19:09:32,602+0000 DEBUG otopi.plugins.ovirt_engine_setup.ovirt_engine.network.ovirtproviderovn plugin.execute:921 execute-output: ('ovn-sbctl', 'set-connection', 'pssl:6642:[::]') stdout: 2024-01-19 19:09:32,602+0000 DEBUG otopi.plugins.ovirt_engine_setup.ovirt_engine.network.ovirtproviderovn plugin.execute:926 execute-output: ('ovn-sbctl', 'set-connection', 'pssl:6642:[::]') stderr: 2024-01-19 19:09:32,626+0000 DEBUG otopi.context context._executeMethod:127 Stage misc METHOD otopi.plugins.ovirt_engine_setup.ovirt_engine.network.ovirtproviderovn.Plugin._misc_configure_ovn_timeout 2024-01-19 19:09:32,627+0000 INFO otopi.plugins.ovirt_engine_setup.ovirt_engine.network.ovirtproviderovn ovirtproviderovn._misc_configure_ovn_timeout:1076 Updating OVN timeout configuration 2024-01-19 19:09:32,628+0000 DEBUG otopi.plugins.ovirt_engine_setup.ovirt_engine.network.ovirtproviderovn plugin.executeRaw:813 execute: ('ovn-sbctl', 'set', 'connection', '.', 'inactivity_probe=60000'), executable='None', cwd='None', env=None 2024-01-19 19:09:32,652+0000 DEBUG otopi.plugins.ovirt_engine_setup.ovirt_engine.network.ovirtproviderovn plugin.executeRaw:863 execute-result: ('ovn-sbctl', 'set', 'connection', '.', 'inactivity_probe=60000'), rc=0 2024-01-19 19:09:32,653+0000 DEBUG otopi.plugins.ovirt_engine_setup.ovirt_engine.network.ovirtproviderovn plugin.execute:921 execute-output: ('ovn-sbctl', 'set', 'connection', '.', 'inactivity_probe=60000') stdout: 2024-01-19 19:09:32,653+0000 DEBUG otopi.plugins.ovirt_engine_setup.ovirt_engine.network.ovirtproviderovn plugin.execute:926 execute-output: ('ovn-sbctl', 'set', 'connection', '.', 'inactivity_probe=60000') stderr: 2024-01-19 19:09:32,677+0000 DEBUG otopi.context context._executeMethod:127 Stage misc METHOD otopi.plugins.ovirt_engine_setup.ovirt_engine.network.ovirtproviderovn.Plugin._misc_configure_provider 2024-01-19 19:09:32,678+0000 DEBUG otopi.context context._executeMethod:145 method exception Traceback (most recent call last): File "/usr/lib/python3.6/site-packages/otopi/context.py", line 132, in _executeMethod method['method']() File "/usr/share/ovirt-engine/setup/bin/../plugins/ovirt-engine-setup/ovirt-engine/network/ovirtproviderovn.py", line 1113, in _misc_configure_provider self._configure_ovirt_provider_ovn() File "/usr/share/ovirt-engine/setup/bin/../plugins/ovirt-engine-setup/ovirt-engine/network/ovirtproviderovn.py", line 796, in _configure_ovirt_provider_ovn content = self._create_config_content() File "/usr/share/ovirt-engine/setup/bin/../plugins/ovirt-engine-setup/ovirt-engine/network/ovirtproviderovn.py", line 761, in _create_config_content OvnEnv.OVIRT_PROVIDER_OVN_SECRET KeyError: 'OVESETUP_OVN/ovirtProviderOvnSecret' 2024-01-19 19:09:32,682+0000 ERROR otopi.context context._executeMethod:154 Failed to execute stage 'Misc configuration': 'OVESETUP_OVN/ovirtProviderOvnSecret' 2024-01-19 19:09:32,683+0000 DEBUG otopi.transaction transaction.abort:124 aborting 'DNF Transaction' I have spent a fair amount of time researching this but haven't found a specific root cause, or solution on this issue. Thank you in advance, Seann

1 0

Cannot update oVirt Node
by Gianluca Amato 19 Jan '24

19 Jan '24

Hello, I am trying to update my oVirt installation to 4.5.5, but while I had no problems upgrading the self-hosted ovirt-engine, I am not able to upgrade any node (running oVirt Node NG 4.5.4 or older). I just click Upgrade in the oVirt Manager, and I get a bunch of events telling me that everything was OK and the upgrade succeeded, but in reality nothing happened on the nodes: the result of "nodectl info" before and after the upgrade process is exactly the same. I gave a look to the engine.log and to the relevant ovirt-host-mgmt-ansible log in the ovirt-engine VM, but I cannot find the problem. Do any of you have any ideas ? I am attaching some relevant files. Note that the upgrade consistently fails in the same way on other nodes, which have other versions of oVirt Node NG (always in the 4.5 family). Thanks for your help --gianluca

5 7

hosted-engine --deploy: No module named 'he_ansible'
by iucounu＠gmail.com 19 Jan '24

19 Jan '24

Hi, I'm trying to install hosted engine on a server (4.5.5-1.el9) and it reports 'No module named 'he_ansible'. I have what could an he_ansible module under /usr/share/ovirt-hosted-engine-setup: /usr/share/ovirt-hosted-engine-setup/he_ansible /usr/share/ovirt-hosted-engine-setup/he_ansible/__pycache__ /usr/share/ovirt-hosted-engine-setup/he_ansible/ansible.cfg /usr/share/ovirt-hosted-engine-setup/he_ansible/callback_plugins /usr/share/ovirt-hosted-engine-setup/he_ansible/constants.py /usr/share/ovirt-hosted-engine-setup/he_ansible/trigger_role.yml /usr/share/ovirt-hosted-engine-setup/he_ansible/__pycache__/constants.cpython-39.opt-1.pyc /usr/share/ovirt-hosted-engine-setup/he_ansible/__pycache__/constants.cpython-39.pyc /usr/share/ovirt-hosted-engine-setup/he_ansible/callback_plugins/1_otopi_json.py /usr/share/ovirt-hosted-engine-setup/he_ansible/callback_plugins/2_ovirt_logger.py Is there a missing include/path somewhere? Thanks for any help. Full error: bash# hosted-engine --deploy <string>:1: DeprecationWarning: distro.linux_distribution() is deprecated. It should only be used as a compatibility shim with Python's platform.linux_distribution(). Please use distro.id(), distro.version() and distro.name() instead. <string>:1: DeprecationWarning: distro.linux_distribution() is deprecated. It should only be used as a compatibility shim with Python's platform.linux_distribution(). Please use distro.id(), distro.version() and distro.name() instead. ***L:ERROR Internal error: No module named 'he_ansible' Traceback (most recent call last): File "/usr/lib/python3.9/site-packages/otopi/main.py", line 141, in execute self.context.loadPlugins() File "/usr/lib/python3.9/site-packages/otopi/context.py", line 803, in loadPlugins self._loadPluginGroups(plugindir, needgroups, loadedgroups) File "/usr/lib/python3.9/site-packages/otopi/context.py", line 112, in _loadPluginGroups self._loadPlugins(path, path, groupname) File "/usr/lib/python3.9/site-packages/otopi/context.py", line 69, in _loadPlugins self._loadPlugins(base, d, groupname) File "/usr/lib/python3.9/site-packages/otopi/context.py", line 95, in _loadPlugins util.loadModule( File "/usr/lib/python3.9/site-packages/otopi/util.py", line 110, in loadModule spec.loader.exec_module(module) File "<frozen importlib._bootstrap_external>", line 850, in exec_module File "<frozen importlib._bootstrap>", line 228, in _call_with_frames_removed File "/usr/share/ovirt-hosted-engine-setup/scripts/../plugins/gr-he-ansiblesetup/core/__init__.py", line 25, in <module> from . import misc File "/usr/share/ovirt-hosted-engine-setup/scripts/../plugins/gr-he-ansiblesetup/core/misc.py", line 34, in <module> from ovirt_hosted_engine_setup import ansible_utils File "/usr/lib/python3.9/site-packages/ovirt_hosted_engine_setup/ansible_utils.py", line 33, in <module> from he_ansible.constants import AnsibleCallback ModuleNotFoundError: No module named 'he_ansible' During handling of the above exception, another exception occurred: Traceback (most recent call last): File "/usr/lib/python3.9/site-packages/otopi/__main__.py", line 88, in main installer.execute() File "/usr/lib/python3.9/site-packages/otopi/main.py", line 143, in execute util.raiseExceptionInformation( File "/usr/lib/python3.9/site-packages/otopi/util.py", line 85, in raiseExceptionInformation raise info[1].with_traceback(info[2]) File "/usr/lib/python3.9/site-packages/otopi/main.py", line 141, in execute self.context.loadPlugins() File "/usr/lib/python3.9/site-packages/otopi/context.py", line 803, in loadPlugins self._loadPluginGroups(plugindir, needgroups, loadedgroups) File "/usr/lib/python3.9/site-packages/otopi/context.py", line 112, in _loadPluginGroups self._loadPlugins(path, path, groupname) File "/usr/lib/python3.9/site-packages/otopi/context.py", line 69, in _loadPlugins self._loadPlugins(base, d, groupname) File "/usr/lib/python3.9/site-packages/otopi/context.py", line 95, in _loadPlugins util.loadModule( File "/usr/lib/python3.9/site-packages/otopi/util.py", line 110, in loadModule spec.loader.exec_module(module) File "<frozen importlib._bootstrap_external>", line 850, in exec_module File "<frozen importlib._bootstrap>", line 228, in _call_with_frames_removed File "/usr/share/ovirt-hosted-engine-setup/scripts/../plugins/gr-he-ansiblesetup/core/__init__.py", line 25, in <module> from . import misc File "/usr/share/ovirt-hosted-engine-setup/scripts/../plugins/gr-he-ansiblesetup/core/misc.py", line 34, in <module> from ovirt_hosted_engine_setup import ansible_utils File "/usr/lib/python3.9/site-packages/ovirt_hosted_engine_setup/ansible_utils.py", line 33, in <module> from he_ansible.constants import AnsibleCallback otopi.main.PluginLoadException: No module named 'he_ansible' root@lonovirt1 /u/cmcl bash#

1 1

virt-viewer - lost partialy connectivity
by paf1＠email.cz 19 Jan '24

19 Jan '24

Hello, after update centos9-stream to latest packages with ovirt 4.5.5 I lost VMs console connectivity . For html5 option I lost all ( can't connect anyway ) For virt-viewer option only partially, can view output screen, but keyboard has no input ( sometimes accept enter ) I tried to reinstall ovirt-vmconsole-host-1.0.9-3.el9.noarch and ovirt- vmconsole-1.0.9-3.el9.noarch , but with no success, the same situation with reinstall virt-viewer 11.0 locally ( ubuntu 23.04 last updates ). This stopped my VMs deploying :( . Exists any fix or workaround about this issue ?? thx. a lot regs. Pavel

1 0

Re: [EXT]Re: Re: [SOLVED] Re: Re: How to re-enroll (or renew) host certificates for a single-host hosted-engine deployment?
by Strahil Nikolov 18 Jan '24

18 Jan '24

Actually , the vdsmd trick is needed only to fool the engine that the host has failed (you need to disable the fencing or prevent it somehow - like changing the ipmi user's password).The engine itself uses ssh to distribute the new certificate to the host. Best Regards,Strahil Nikolov On Wed, Jan 17, 2024 at 18:07, Michael Douglass<mikedoug(a)certida.com> wrote: Ah, the steps I list here work for non-single node environments as well. The problem with vdsm-tool doing the certificate renewal in that setup is that the certificate needs to be signed by the engine host and not the local host. Apologies if this causes any confusion for single-node users. From: Strahil Nikolov <hunter86_bg(a)yahoo.com> Sent: Saturday, January 13, 2024 6:57 PM To: Michael Douglass <mikedoug(a)certida.com>; mikedoug--- via Users <users(a)ovirt.org> Subject: [EXT]Re: [ovirt-users] Re: [SOLVED] Re: Re: How to re-enroll (or renew) host certificates for a single-host hosted-engine deployment? CAUTION: Be careful of links - This email originated from outside of Certida.Hi, Recently I found another way to renew the certificates on a one-node self-hosted environment.The trick is to stop vdsmd and wait till the engine shows the system unresponsive. Then you can trigger a certificate renewal and just power on vdsmd again. Best Regards,Strahil Nikolov On Sun, Jan 14, 2024 at 2:54, mikedoug--- via Users<users(a)ovirt.org> wrote:This is the only place we found the answer we needed concerning how to sign a host key from the engine! Saved our bacon! The only tweak I would make is that there were a few more destinations at the end to copy the files into, and instead of rebooting the host we found that just restarting vdsmd and libvirtd got everything working without any existing VMs having to stop. Here's a complete update of what you have above intermingling some extra bits for anyone who has to do this in the future. If the certs on your oVirt host expire, it can be a PITA to figure out how to fix it. It's actually simple, but takes a LOT of manual work. Make sure every part of every script makes sense as I made some modifications as I documented it! I do not warrant that I didn't make a mistake somewhere. :) ########### On the ENGINE HOST ########### ## Check the CA Cert on the ENGINE openssl x509 -in /etc/pki/ovirt-engine/ca.pem -noout -dates ## If it is expired, then on the ENGINE, rebuild the CA Cert with this in a script: ## NOTE: these are steps from another post I didn't need to do them -- could be dangerous... ``` set -x ## Make the script echo everything out, so if it fails you know where set -e ## Make the script STOP on any error set my_date="$(date +"%Y%m%d%H%M%S")" # Backup the existing CA files /bin/cp -p /etc/pki/ovirt-engine/private/ca.pem /etc/pki/ovirt-engine/private/ca.pem.$my_date /bin/cp -p /etc/pki/ovirt-engine/ca.pem{,.$my_date} /bin/mv /etc/pki/ovirt-engine/certs/ca.der{,.$my_date} # Sign the key openssl x509 -signkey /etc/pki/ovirt-engine/private/ca.pem -in /etc/pki/ovirt-engine/ca.pem -out /etc/pki/ovirt-engine/ca.pem.new -days 3650 -sha256 openssl x509 -in /etc/pki/ovirt-engine/ca.pem.new -text > /etc/pki/ovirt-engine/ca.pem.new.full # Put the files into place /bin/mv -f /etc/pki/ovirt-engine/ca.pem.new.full /etc/pki/ovirt-engine/ca.pem /bin/cp -p /etc/pki/ovirt-engine/ca.pem.new /etc/pki/ovirt-engine/certs/ca.der ``` Now you need to copy the new CA file over to the host: Source: ENGINE `/etc/pki/ovirt-engine/ca.pem` Dest: HOST `/tmp/new-ca.pem` ########### On the oVirt Host ########### # Create a CSR using the information from the existing certificate and the existing key: openssl x509 -x509toreq -in /etc/pki/libvirt/clientcert.pem -out /tmp/HOST.csr -signkey /etc/pki/libvirt/private/clientkey.pem Now you need to copy the new CA file over to the host: Source: HOST `/tmp/HOST.csr` Dest: ENGINE `/etc/pki/ovirt-engine/requests/full.hostname.com.req` ########### On the ENGINE HOST ########### # Now sign it: /usr/share/ovirt-engine/bin/pki-enroll-request.sh --name=full.hostname.com # NB -- adding --san results in an error: --san=host.na.me (So no Subject Alternate Names) Now you need to copy the new Certificate file over to the host: Source: ENGINE /etc/pki/ovirt-engine/certs/full.hostname.com.cer Dest: HOST /tmp/new-cert.pem ########### On the oVirt Host ########### Run this script to put the cert and CA in place. Note if you don't put a ca into /tmp/new-ca.pem it skips that step. ``` set -x set -e set my_date="$(date +"%Y%m%d%H%M%S")" for x in /etc/pki/libvirt/clientcert.pem /etc/pki/vdsm/certs/vdsmcert.pem /etc/pki/vdsm/libvirt-migrate/server-cert.pem /etc/pki/vdsm/libvirt-spice/server-cert.pem /etc/pki/vdsm/libvirt-vnc/server-cert.pem; do /bin/mv -n $x ${x}.${mydate} /bin/cp /tmp/new-cert.pem ${x} chmod 644 ${x} chown root:kvm ${x} done if -f /tmp/new-ca.pem; then for x in /etc/pki/vdsm/libvirt-migrate/ca-cert.pem /etc/pki/vdsm/certs/cacert.pem /etc/pki/vdsm/libvirt-vnc/ca-cert.pem /etc/pki/vdsm/libvirt-spice/ca-cert.pem /etc/pki/CA/cacert.pem; do /bin/mv -n $x ${x}.${mydate} /bin/cp /tmp/new-ca.pem ${x} chmod 644 ${x} chown root:kvm ${x} done fi ``` Now you're ready to restart two vital services -- some people say "reboot the host" -- but we found that unecessary. Running this restart was safe for us and didn't cause any running VMs to crash or reboot -- they kept going without issue. Once we did this, waited a few minutes, the host came back up on the engine and everything was happy. Specifically a VM we were unable to get running was a click away from full functionality again! # Restart the two services affected by the key/cert changes systemctl restart vdsmd libvirtd _______________________________________________ Users mailing list -- users(a)ovirt.org To unsubscribe send an email to users-leave(a)ovirt.org Privacy Statement: https://www.ovirt.org/privacy-policy.html oVirt Code of Conduct: https://www.ovirt.org/community/about/community-guidelines/ List Archives: https://lists.ovirt.org/archives/list/users@ovirt.org/message/2EBSBWXFCEAND…

2 1

gluster on iSCSI devices in ovirt environment
by paf1＠email.cz 18 Jan '24

18 Jan '24

hello dears, can anybody explain me HOWTO realize 2 nodes + aribiter gluster from two (three) locations on block iSCSI devices ? Something like this: gluster volume create TEST replica 3 arbiter 1 <location-one-host1 - iSCSI target > <location-two-host2 - iSCSI target> < location-three-host3 - /dev/sda5 e.g. > - ALL applied on multinode ovirt cluster thx a lot for any help regs. Pa.

3 4

NVIDIA vGPU driver for Ovirt 4.5.4
by michael.a.silveira3.ctr＠us.navy.mil 18 Jan '24

18 Jan '24

Hello, Does anyone know which, if any, NVIDIA GRID driver supports Ovirt 4.5.4 on Ovirt-node (kernel 4.18.0-408.el8.x86_64)? I've recently upgraded to Ovirt 4.5 and can't find a NVIDIA GRID driver that will connect to my Tesla v100 on the new kernel. nvidia-smi returns the following no matter what driver I install: NVIDIA-SMI has failed because it couldn't communicate with the NVIDIA driver. Make sure that the latest NVIDIA driver is installed and running.

5 4

Cannot restart ovirt after massive failure.
by Gilboa Davara 18 Jan '24

18 Jan '24

Hello all, During the night, one of my (smaller) setups, a single node self hosted engine (localhost NFS) crashed due to what-looks-like a massive disk failure (Software RAID6, with 10 drives + spare). After a reboot, I let the RAID resync with a fresh drive) and went on to start oVirt. However, no such luck. Two issues: 1. ovirt-ha-broker fails due to broken hosted engine state (log attached). 2. ovirt-ha-agent fails due to network test (tcp) even though both remote-host and DNS servers are active. (log attached). Two questions: 1. Can I somehow force the agent to disable the network liveliness test? 2. Can I somehow force the broker to rebuild / fix the hosted engine state? - Gilboa

4 17

Cannot open Vm Console
by Vittorio 18 Jan '24

18 Jan '24

I got few Vm on my ovirte nodes, but i cant acess to console. When i downloaded it, the error is the following : "Failed to complete haandshake Errore in the pull function."

2 1

VDSM Command DetachStorageDomainVDS Failed
by Matthew J Black 18 Jan '24

18 Jan '24

Hi All, So after having an oVirt cluster crash (no data loss (at this stage), thankfully) and rebuilding from scratch, I'm trying to import the old Storage Domains. I've been successful with three (automatically "detaching" from the old, now non-existent hosts) but one is giving me the following error from the hosted-engine GUI: ~~~ VDSM command DetachStorageDomainVDS failed: Cannot acquire host id: ('e311ddf1-7f2c-49ef-a618-050d9a2b947f', SanlockException(19, 'Sanlock lockspace add failure', 'No such device')) ~~~ I am going to assume ("make an ass out for you and me") that I can run the required command from the cli with a "-force" flag (I hope), so my Q is: What is the command, because I can't seem to find it in any doco (most probably because I'm old and my eyes are feeble :-) ). Any help gratefully appreciated - FTR: that particular Storage Domain have a bunch of VM Images on it which I'd rather import/recover than have to create from scratch. Cheers Dulux-Oz

1 0

Deployment Error: Host is not up - Looking For Some Advice/Pointers
by Matthew J Black 18 Jan '24

18 Jan '24

Hi All, So, on a fresh install on RL v9.3, we're getting a `Host is not up, please check logs, perhaps also on the engine machine` error. This comes up right after the 20min timeout (the 120sec * 10 one). No, the hosted-engine is not deployed (ie hosted-engine --check-deployed). Obviously I need to check the logs, but which ones in particular (the hosted-engine-setup logs, obviously, but which other ones), where are they if the hosted-engine is not running, and, in an effort to narrow down the volume of info to a more manageable level, what should I be looking for? Thanks in advance

1 1

Please, Please Help - New oVirt Install/Deployment Failing - "Host is not up..."
by Matthew J Black 18 Jan '24

18 Jan '24

Hi Everyone, Could someone please help me - I've been trying to do an install of oVirt for *weeks* (including false starts and self-inflicted wounds/errors) and it is still not working. My setup: - oVirt v4.5.3 - A brand new fresh vanilla install of RockyLinux 8.6 - all working AOK - 2*NICs in a bond (802.3ad) with a couple of sub-Interfaces/VLANs - all working AOK - All relevant IPv4 Address in DNS with Reverse Lookups - all working AOK - All relevant IPv4 Address in "/etc/hosts" file - all working AOK - IPv6 (using "method=auto" in the interface config file) enabled on the relevant sub-Interface/VLAN - I'm not using IPv6 on the network, only IPv4, but I'm trying to cover all the bases. - All relevant Ports (as per the oVirt documentation) set up on the firewall - ie firewall-cmd --add-service={{ libvirt-tls | ovirt-imageio | ovirt-vmconsole | vdsm }} - All the relevant Repositories installed (ie RockyLinux BaseOS, AppStream, & PowerTools, and the EPEL, plus the ones from the oVirt documentation) I have followed the oVirt documentation (including the special RHEL-instructions and RockyLinux-instructions) to the letter - no deviations, no special settings, exactly as they are written. All the dnf installs, etc, went off without a hitch, including the "dnf install centos-release-ovirt45", "dnf install ovirt-engine-appliance", and "dnf install ovirt-hosted-engine-setup" - no errors anywhere. Here is the results of a "dnf repolist": - appstream Rocky Linux 8 - AppStream - baseos Rocky Linux 8 - BaseOS - centos-ceph-pacific CentOS-8-stream - Ceph Pacific - centos-gluster10 CentOS-8-stream - Gluster 10 - centos-nfv-openvswitch CentOS-8 - NFV OpenvSwitch - centos-opstools CentOS-OpsTools - collectd - centos-ovirt45 CentOS Stream 8 - oVirt 4.5 - cs8-extras CentOS Stream 8 - Extras - cs8-extras-common CentOS Stream 8 - Extras common packages - epel Extra Packages for Enterprise Linux 8 - x86_64 - epel-modular Extra Packages for Enterprise Linux Modular 8 - x86_64 - ovirt-45-centos-stream-openstack-yoga CentOS Stream 8 - oVirt 4.5 - OpenStack Yoga Repository - ovirt-45-upstream oVirt upstream for CentOS Stream 8 - oVirt 4.5 - powertools Rocky Linux 8 - PowerTools So I kicked-off the oVirt deployment with: "hosted-engine --deploy --4 --ansible-extra-vars=he_offline_deployment=true". I used "--ansible-extra-vars=he_offline_deployment=true" because without that flag I was getting "DNF timout" issues (see my previous post `Local (Deployment) VM Can't Reach "centos-ceph-pacific" Repo`). I answer the defaults to all of questions the script asked, or entered the deployment-relevant answers where appropriate. In doing this I double-checked every answer before hitting <Enter>. Everything progressed smoothly until the deployment reached the "Wait for the host to be up" task... which then hung for more than 30 minutes before failing. From the ovirt-hosted-engine-setup... log file: - 2022-10-20 17:54:26,285+1100 ERROR otopi.ovirt_hosted_engine_setup.ansible_utils ansible_utils._process_output:113 fatal: [localhost]: FAILED! => {"changed": false, "msg": "Host is not up, please check logs, perhaps also on the engine machine"} I checked the following log files and found all of the relevant ERROR lines, then checked several 10s of proceeding and succeeding lines trying to determine what was going wrong, but I could not determine anything. - ovirt-hosted-engine-setup... - ovirt-hosted-engine-setup-ansible-bootstrap_local_vm... - ovirt-hosted-engine-setup-ansible-final_clean... - not really relevant, I believe I can include the log files (or the relevant parts of the log files) if people want - but that are very large: several 100 kilobytes each. I also googled "oVirt Host is not up" and found several entries, but after reading them all the most relevant seems to be a thread from these mailing list: `Install of RHV 4.4 failing - "Host is not up, please check logs, perhaps also on the engine machine"` - but this seems to be talking about an upgrade and I didn't gleam anything useful from it - I could, of course, be wrong about that. So my questions are: - Where else should I be looking (ie other log files, etc, and possible where to find them)? - Does anyone have any idea why this isn't working? - Does anyone have a work-around (including a completely manual process to get things working - I don't mind working in the CLI with virsh, etc)? - What am I doing wrong? Please, I'm really stumped with this, and I really do need help. Cheers Dulux-Oz

8 22

ovirt node ng 4.5.5 fresh install fails
by Levi Wilbert 17 Jan '24

17 Jan '24

I'm attempting to update our oVirt cluster to 4.5.5 from 4.5.4, running oVirt Node NG on the hosts. When I tried updating a host through the oVirt Manager GUI, after the host reboots, it fails to start up and goes into emergency recovery mode: [ 4.534872] localhost systemd[1]: Reached target Local File Systems. [ 4.535119] localhost systemd[1]: Reached target System Initialization. [ 4.535343] localhost systemd[1]: Reached target Basic System. [ 4.536759] localhost systemd[1]: Started Hardware RNG Entropy Gatherer Daemon. [ 4.541801] localhost rngd[1512]: Disabling 7: PKCS11 Entropy generator (pkcs11) [ 4.541801] localhost rngd[1512]: Disabling 5: NIST Network Entropy Beacon (nist) [ 4.541801] localhost rngd[1512]: Disabling 9: Qrypt quantum entropy beacon (qrypt) [ 4.541801] localhost rngd[1512]: Initializing available sources [ 4.542073] localhost rngd[1512]: [hwrng ]: Initialization Failed [ 4.542073] localhost rngd[1512]: [rdrand]: Enabling RDSEED rng support [ 4.542073] localhost rngd[1512]: [rdrand]: Initialized [ 4.542073] localhost rngd[1512]: [jitter]: JITTER timeout set to 5 sec [ 4.582381] localhost rngd[1512]: [jitter]: Initializing AES buffer [ 8.309063] localhost rngd[1512]: [jitter]: Enabling JITTER rng support [ 8.309063] localhost rngd[1512]: [jitter]: Initialized [ 133.884355] localhost dracut-initqueue[1095]: Warning: dracut-initqueue: timeout, still waiting for following initqueue hooks: [ 133.885349] localhost dracut-initqueue[1095]: Warning: /lib/dracut/hooks/initqueue/finished/devexists-\x2fdev\x2fdisk\x2fby-id\x2fmd-uuid-3f47cad8:fecb96ea:0ea37615:4e5dec4e.sh: "[ -e "/dev/disk/by-id/md-uuid-3f47cad8:fecb96ea:0ea37615:4e5dec4e" ]" [ 133.886485] localhost dracut-initqueue[1095]: Warning: /lib/dracut/hooks/initqueue/finished/devexists-\x2fdev\x2fdisk\x2fby-id\x2fmd-uuid-d446b801:d515c112:116ff07f:9ae52466.sh: "[ -e "/dev/disk/by-id/md-uuid-d446b801:d515c112:116ff07f:9ae52466" ]" [ 133.887619] localhost dracut-initqueue[1095]: Warning: /lib/dracut/hooks/initqueue/finished/devexists-\x2fdev\x2fonn\x2fovirt-node-ng-4.5.5-0.20231130.0+1.sh: "if ! grep -q After=remote-fs-pre.target /run/systemd/generator/systemd-cryptsetup(a)*.service 2>/dev/null; then [ 133.887619] localhost dracut-initqueue[1095]: [ -e "/dev/onn/ovirt-node-ng-4.5.5-0.20231130.0+1" ] [ 133.887619] localhost dracut-initqueue[1095]: fi" [ 133.888667] localhost dracut-initqueue[1095]: Warning: /lib/dracut/hooks/initqueue/finished/devexists-\x2fdev\x2fonn\x2fswap.sh: "[ -e "/dev/onn/swap" ]" [ 133.890050] localhost dracut-initqueue[1095]: Warning: dracut-initqueue: starting timeout scripts [ 133.969228] localhost dracut-initqueue[7366]: Scanning devices md126p2 for LVM logical volumes onn/ovirt-node-ng-4.5.5-0.20231130.0+1 [ 133.969228] localhost dracut-initqueue[7366]: onn/swap [ 134.001560] localhost dracut-initqueue[7366]: onn/ovirt-node-ng-4.5.5-0.20231130.0+1 thin [ 134.001560] localhost dracut-initqueue[7366]: onn/swap linear [ 134.014259] localhost dracut-initqueue[7381]: /etc/lvm/profile/imgbased-pool.profile: stat failed: No such file or directory [ 134.532608] localhost dracut-initqueue[7381]: Check of pool onn/pool00 failed (status:64). Manual repair required! I then attempted installing the oVirt Node NG 4.5.5 iso to a USB stick and tried installing that way, however, after going through the GUI and setting up storage, network, hostname, etc, the install fails shortly after clicking "Begin". 22:11:32,671 WARNING org.fedoraproject.Anaconda.Modules.Storage:INFO:blivet:executing action: [468] destroy device lvmthinlv onn-var_log_audit (id 216) 22:11:32,672 WARNING org.fedoraproject.Anaconda.Modules.Storage:DEBUG:blivet: LVMLogicalVolumeDevice.destroy: onn-var_log_audit ; status: False ; 22:11:32,673 WARNING org.fedoraproject.Anaconda.Modules.Storage:DEBUG:blivet: LVMLogicalVolumeDevice.teardown: onn-var_log_audit ; status: False ; controllable: False ; 22:11:32,674 WARNING org.fedoraproject.Anaconda.Modules.Storage:DEBUG:blivet: LVMVolumeGroupDevice.setup_parents: name: onn ; orig: True ; 22:11:32,674 WARNING org.fedoraproject.Anaconda.Modules.Storage:DEBUG:blivet: PartitionDevice.setup: Volume0_0p2 ; orig: True ; status: True ; controllable: True ; 22:11:32,675 WARNING org.fedoraproject.Anaconda.Modules.Storage:DEBUG:blivet: LVMPhysicalVolume.setup: device: /dev/md/Volume0_0p2 ; type: lvmpv ; status: False ; 22:11:32,676 WARNING org.fedoraproject.Anaconda.Modules.Storage:DEBUG:blivet: LVMLogicalVolumeDevice._destroy: onn-var_log_audit ; status: False ; 22:11:32,676 WARNING org.fedoraproject.Anaconda.Modules.Storage:INFO:program:Running [97] lvm lvremove --yes onn/var_log_audit --config= log {level=7 file=/tmp/lvm.log syslog=0} --devices=/dev/md/Volume0_0p2 ... 22:11:33,104 ERR rsyslogd:imjournal: open() failed for path: '/var/lib/rsyslog/imjournal.state.tmp': Operation not permitted [v8.2310.0-3.el9 try https://www.rsyslog.com/e/2433 ] 22:11:33,105 ERR rsyslogd:imjournal: open() failed for path: '/var/lib/rsyslog/imjournal.state.tmp': Operation not permitted [v8.2310.0-3.el9 try https://www.rsyslog.com/e/2433 ] 22:11:33,105 ERR rsyslogd:imjournal: open() failed for path: '/var/lib/rsyslog/imjournal.state.tmp': Operation not permitted [v8.2310.0-3.el9 try https://www.rsyslog.com/e/2433 ] 22:11:33,106 ERR rsyslogd:imjournal: open() failed for path: '/var/lib/rsyslog/imjournal.state.tmp': Operation not permitted [v8.2310.0-3.el9 try https://www.rsyslog.com/e/2433 ] 22:11:33,106 ERR rsyslogd:imjournal: open() failed for path: '/var/lib/rsyslog/imjournal.state.tmp': Operation not permitted [v8.2310.0-3.el9 try https://www.rsyslog.com/e/2433 ] 22:11:33,107 ERR rsyslogd:imjournal: open() failed for path: '/var/lib/rsyslog/imjournal.state.tmp': Operation not permitted [v8.2310.0-3.el9 try https://www.rsyslog.com/e/2433 ] 22:11:33,309 WARNING org.fedoraproject.Anaconda.Modules.Storage:INFO:program:stdout[97]: 22:11:33,310 WARNING org.fedoraproject.Anaconda.Modules.Storage:INFO:program:stderr[97]: /etc/lvm/profile/imgbased-pool.profile: stat failed: No such file or directory 22:11:33,310 WARNING org.fedoraproject.Anaconda.Modules.Storage: Check of pool onn/pool00 failed (status:64). Manual repair required! I'm wondering if it has to do with installing oVirt node on a RAID mirror?

1 0