June 2022 - Users - oVirt List Archives

Host fails to activate
by David Johnson 12 Jun '22

12 Jun '22

Good afternoon all, Ovirt version: 4.14.4.10.7-1.el8 Centos version: Linux version 4.18.0-365.el8.x86_64 ( mockbuild(a)kbuilder.bsys.centos.org) (gcc version 8.5.0 20210514 (Red Hat 8.5.0-10) (GCC)) #1 SMP Thu Feb 10 16:11:23 UTC 2022 Background: We had a mother board fail in our storage device. I was able to migrate the storage domain to the backup device before it failed completely, and have been running on the backup device for several weeks while we purchased a replacement main storage. Today I shut everything down cleanly, replaced the main storage, and restarted the cluster. We did disconnect and reconnect the network on all of the devices as we shuffled equipment in the rack. One of the hosts in the cluster refuses to come back up.I am able to connect to the host via putty. Ovirt gui reporting: Setting Host ovirt-host-03.maxisinc.net to Non-Operational mode. Completed: Jun 11, 2022, 4:59:57 PM Activating Host ovirt-host-03.maxisinc.net Completed: Jun 11, 2022, 4:59:57 PM Invoking Activate Host ovirt-host-03.maxisinc.net Completed: Jun 11, 2022, 4:57:40 PM Installing Host ovirt-host-03.maxisinc.net log from host is 5:09 PM GetManagedObjects() failed: org.freedesktop.DBus.Error.NoReply: Did not receive a reply. Possible causes include: the remote application did not send a reply, the message bus security policy blocked the reply, the reply timeout expired, or the network connection was broken. pulseaudio 4:55 PM bondscan-DGwC1l: option lacp_active: mode dependency failed, not supported in mode balance-alb(6) kernel 4:55 PM bondscan-DGwC1l: option arp_all_targets: invalid value (2) kernel 4:55 PM bondscan-DGwC1l: option fail_over_mac: invalid value (3) kernel 4:55 PM bondscan-DGwC1l: option primary_reselect: invalid value (3) kernel 4:55 PM bondscan-DGwC1l: option ad_select: invalid value (3) kernel 4:55 PM

2 6

oVirt SSH rate limit and disable SSH passwd auth
by tasnadi.peter＠kifu.gov.hu 10 Jun '22

10 Jun '22

Hello, 1. Is it possible to disable ssh root password authentication in a working oVirt cluster without any problems? (host and ovirt-engine) /etc/ssh/sshd_config PasswordAuthentication no The SSH public authentication key is set on the host. 2. I tried setting ssh rate limit using firewall-cmd but it doesn't work for some reason. I can log in more than once. firewall-cmd --permanent --add-rich-rule='rule family="ipv4" priority="-1" service name=ssh limit value=3/m accept' There is best practice for this? Thanks Peter

2 1

[ANN] oVirt 4.5.1 First Release Candidate is now available for testing
by Lev Veyde 09 Jun '22

09 Jun '22

oVirt 4.5.1 First Release Candidate is now available for testing The oVirt Project is pleased to announce the availability of oVirt 4.5.1 First Release Candidate for testing, as of June 9th, 2022. This update is the first in a series of stabilization updates to the 4.5 series. Documentation - If you want to try oVirt as quickly as possible, follow the instructions on the Download <https://ovirt.org/download/> page. - For complete installation, administration, and usage instructions, see the oVirt Documentation <https://ovirt.org/documentation/>. - For upgrading from a previous version, see the oVirt Upgrade Guide <https://ovirt.org/documentation/upgrade_guide/>. - For a general overview of oVirt, see About oVirt <https://ovirt.org/community/about.html>. Important notes before you try it Please note this is a pre-release build. The oVirt Project makes no guarantees as to its suitability or usefulness. This pre-release must not be used in production. Installation instructions For installation instructions and additional information please refer to: https://ovirt.org/documentation/ This release is available now on x86_64 architecture for: - CentOS Stream 8 - RHEL 8.6 Beta and derivatives This release supports Hypervisor Hosts on x86_64: - oVirt Node NG (based on CentOS Stream 8) - CentOS Stream 8 - RHEL 8.6 Beta and derivatives Builds are also available for ppc64le and aarch64. Experimental builds for CentOS Stream 9 are also provided for Hypervisor Hosts. See the release notes [1] for installation instructions and a list of new features and bugs fixed. Notes: - oVirt Appliance is already available based on CentOS Stream 8 - oVirt Node NG is already available based on CentOS Stream 8 Additional Resources: * Read more about the oVirt 4.5.1 pre-release highlights: http://www.ovirt.org/release/4.5.1/ * Get more oVirt project updates on Twitter: https://twitter.com/ovirt * Check out the latest project news on the oVirt blog: http://www.ovirt.org/blog/ [1] http://www.ovirt.org/release/4.5.1/ [2] http://resources.ovirt.org/pub/ovirt-4.5-pre/iso/ Thanks in advance, -- Lev Veyde Senior Software Engineer, RHCE | RHCVA | MCITP Red Hat Israel <https://www.redhat.com> lev(a)redhat.com | lveyde(a)redhat.com <https://red.ht/sig> TRIED. TESTED. TRUSTED. <https://redhat.com/trusted>

1 0

VmMediatedDevices help for vGPU in overt 4.5
by Don Dupuis 09 Jun '22

09 Jun '22

Hello I am looking for an example on how to use the new VmMediatedDevices service to add an Nvidia vGPU in ovirt to guest vms. I had it working just fine in oVirt 4.4 when using the custom_properties method. Just need to understand the new method/way of doing it correctly using the python3-ovirt-engine-sdk. Thanks Don

2 2

Re: list-view instead of tiled-view in oVirt VM Portal?
by Frank Coons 09 Jun '22

09 Jun '22

Please note that (a) there are people that use more than 20 VM's that do not need admin access, and (b) some people do not LIKE looking at big gaudy buttons, even if there are only 15 of them. I put in an RFE to bring back the list view YEARS ago and was basically told that "we know what you want better than you do." I am willing to bet that many more people want the list view than you realize, but you don't seem to be willing to listen. Disgruntled.

2 1

ovirt 4.4.10 ansible version
by Kapetanakis Giannis 09 Jun '22

09 Jun '22

Could someone verify the correct ansible version for ovirt 4.4.10 ? I'm having dependencies problem: # rpm -q ansible ansible-2.9.27-3.el8.noarch # dnf update Last metadata expiration check: 1:14:38 ago on Thu 09 Jun 2022 10:59:53 EEST. Error: Problem: package ovirt-engine-4.4.10.7-1.el8.noarch requires ansible < 2.10.0, but none of the providers can be installed - cannot install both ansible-5.4.0-2.el8.noarch and ansible-2.9.27-3.el8.noarch - cannot install both ansible-2.9.17-1.el8.noarch and ansible-5.4.0-2.el8.noarch - cannot install both ansible-2.9.18-2.el8.noarch and ansible-5.4.0-2.el8.noarch - cannot install both ansible-2.9.20-2.el8.noarch and ansible-5.4.0-2.el8.noarch - cannot install both ansible-2.9.21-2.el8.noarch and ansible-5.4.0-2.el8.noarch - cannot install both ansible-2.9.23-2.el8.noarch and ansible-5.4.0-2.el8.noarch - cannot install both ansible-2.9.24-2.el8.noarch and ansible-5.4.0-2.el8.noarch - cannot install both ansible-2.9.27-2.el8.noarch and ansible-5.4.0-2.el8.noarch - cannot install the best update candidate for package ovirt-engine-4.4.10.7-1.el8.noarch - cannot install the best update candidate for package ansible-2.9.27-3.el8.noarch - package ansible-2.9.20-1.el8.noarch is filtered out by exclude filtering - package ansible-2.9.16-1.el8.noarch is filtered out by exclude filtering - package ansible-2.9.19-1.el8.noarch is filtered out by exclude filtering - package ansible-2.9.23-1.el8.noarch is filtered out by exclude filtering (try to add '--allowerasing' to command line to replace conflicting packages or '--skip-broken' to skip uninstallable packages or '--nobest' to use not only best candidate packages) # dnf list --showduplicates ansible Last metadata expiration check: 1:15:11 ago on Thu 09 Jun 2022 10:59:53 EEST. Installed Packages ansible.noarch 2.9.27-3.el8 @epel Available Packages ansible.noarch 2.9.17-1.el8 ovirt-4.4-centos-ovirt44 ansible.noarch 2.9.18-2.el8 ovirt-4.4-centos-ovirt44 ansible.noarch 2.9.20-2.el8 ovirt-4.4-centos-ovirt44 ansible.noarch 2.9.21-2.el8 ovirt-4.4-centos-ovirt44 ansible.noarch 2.9.23-2.el8 ovirt-4.4-centos-ovirt44 ansible.noarch 2.9.24-2.el8 ovirt-4.4-centos-ovirt44 ansible.noarch 2.9.27-2.el8 ovirt-4.4-centos-ovirt44 ansible.noarch 5.4.0-2.el8 epel ansible.noarch 5.4.0-2.el8 ovirt-4.4-epel thanks, G

2 1

Re: Self-hosted engine failing liveliness check
by McNamara, Bradley 09 Jun '22

09 Jun '22

When a run "hosted-engine --check-liveliness" it returns "Hosted Engine is not up!" When I run this command and can see it hitting the httpd server, with success, in the httpd logs. Accessing the URL returns this: "DB Up!Welcome to Health Status!". ________________________________ From: McNamara, Bradley <Bradley.McNamara(a)seattle.gov> Sent: Wednesday, June 8, 2022 12:54 PM To: users(a)ovirt.org <users(a)ovirt.org> Subject: [ovirt-users] Self-hosted engine failing liveliness check CAUTION: External Email Hello, and thank you all for your help. I'm running Oracle's rebranded oVirt 4.3.10. All has been good until I patched my self-hosted engine. I ran through the normal process: backup, global maintenance mode, update the oVirt packages, run engine-setup, etc. All completed normally without issues. I rebooted the self-hosted engine VM, and now it constantly fails liveliness checks and the HA agent reboots it every five minutes, or so. I put it in back in global maintenance so the HA agent would not reboot it. The VM is up and works correctly. I can do everything normally. From what I can tell the HA agent liveliness check is just a http get to the web portal. I can see that happening with success. What is the lilveliness check actually doing? All services on the VM are up and running without issue. Where can I look to figure this out? Here is the output of hosted-engine --vm-status: [root@itdlolv101 ~]# hosted-engine --vm-status !! Cluster is in GLOBAL MAINTENANCE mode !! --== Host itdlolv100.ci.seattle.wa.us (id: 1) status ==-- conf_on_shared_storage : True Status up-to-date : True Hostname : itdlolv100.ci.seattle.wa.us Host ID : 1 Engine status : {"reason": "vm not running on this host", "health": "bad", "vm": "down", "detail": "unknown"} Score : 3400 stopped : False Local maintenance : False crc32 : 855e161f local_conf_timestamp : 55128 Host timestamp : 55128 Extra metadata (valid at timestamp): metadata_parse_version=1 metadata_feature_version=1 timestamp=55128 (Wed Jun 8 12:52:20 2022) host-id=1 score=3400 vm_conf_refresh_time=55128 (Wed Jun 8 12:52:20 2022) conf_on_shared_storage=True maintenance=False state=GlobalMaintenance stopped=False --== Host itdlolv101.ci.seattle.wa.us (id: 2) status ==-- conf_on_shared_storage : True Status up-to-date : True Hostname : itdlolv101.ci.seattle.wa.us Host ID : 2 Engine status : {"reason": "failed liveliness check", "health": "bad", "vm": "up", "detail": "Up"} Score : 3400 stopped : False Local maintenance : False crc32 : cc1c2261 local_conf_timestamp : 45453 Host timestamp : 45453 Extra metadata (valid at timestamp): metadata_parse_version=1 metadata_feature_version=1 timestamp=45453 (Wed Jun 8 12:55:15 2022) host-id=2 score=3400 vm_conf_refresh_time=45453 (Wed Jun 8 12:55:15 2022) conf_on_shared_storage=True maintenance=False state=GlobalMaintenance stopped=False !! Cluster is in GLOBAL MAINTENANCE mode !! [root@itdlolv101 ~]#

1 0

Re: storage high latency, sanlock errors, cluster instability
by Nir Soffer 08 Jun '22

08 Jun '22

On Sun, May 29, 2022 at 9:03 PM Jonathan Baecker <jonbae77(a)gmail.com> wrote: > > Am 29.05.22 um 19:24 schrieb Nir Soffer: > > On Sun, May 29, 2022 at 7:50 PM Jonathan Baecker <jonbae77(a)gmail.com> wrote: > > Hello everybody, > > we run a 3 node self hosted cluster with GlusterFS. I had a lot of problem upgrading ovirt from 4.4.10 to 4.5.0.2 and now we have cluster instability. > > First I will write down the problems I had with upgrading, so you get a bigger picture: > > engine update when fine > But nodes I could not update because of wrong version of imgbase, so I did a manual update to 4.5.0.1 and later to 4.5.0.2. First time after updating it was still booting into 4.4.10, so I did a reinstall. > Then after second reboot I ended up in the emergency mode. After a long searching I figure out that lvm.conf using use_devicesfile now but there it uses the wrong filters. So I comment out this and add the old filters back. This procedure I have done on all 3 nodes. > > When use_devicesfile (default in 4.5) is enabled, lvm filter is not > used. During installation > the old lvm filter is removed. > > Can you share more info on why it does not work for you? > > The problem was, that the node could not mount the gluster volumes anymore and ended up in emergency mode. > > - output of lsblk > > NAME MAJ:MIN RM SIZE RO TYPE MOUNTPOINT > sda 8:0 0 1.8T 0 disk > `-XA1920LE10063_HKS028AV 253:0 0 1.8T 0 mpath > |-gluster_vg_sda-gluster_thinpool_gluster_vg_sda_tmeta 253:16 0 9G 0 lvm > | `-gluster_vg_sda-gluster_thinpool_gluster_vg_sda-tpool 253:18 0 1.7T 0 lvm > | |-gluster_vg_sda-gluster_thinpool_gluster_vg_sda 253:19 0 1.7T 1 lvm > | |-gluster_vg_sda-gluster_lv_data 253:20 0 100G 0 lvm /gluster_bricks/data > | `-gluster_vg_sda-gluster_lv_vmstore 253:21 0 1.6T 0 lvm /gluster_bricks/vmstore > `-gluster_vg_sda-gluster_thinpool_gluster_vg_sda_tdata 253:17 0 1.7T 0 lvm > `-gluster_vg_sda-gluster_thinpool_gluster_vg_sda-tpool 253:18 0 1.7T 0 lvm > |-gluster_vg_sda-gluster_thinpool_gluster_vg_sda 253:19 0 1.7T 1 lvm > |-gluster_vg_sda-gluster_lv_data 253:20 0 100G 0 lvm /gluster_bricks/data > `-gluster_vg_sda-gluster_lv_vmstore 253:21 0 1.6T 0 lvm /gluster_bricks/vmstore > sr0 11:0 1 1024M 0 rom > nvme0n1 259:0 0 238.5G 0 disk > |-nvme0n1p1 259:1 0 1G 0 part /boot > |-nvme0n1p2 259:2 0 134G 0 part > | |-onn-pool00_tmeta 253:1 0 1G 0 lvm > | | `-onn-pool00-tpool 253:3 0 87G 0 lvm > | | |-onn-ovirt--node--ng--4.5.0.2--0.20220513.0+1 253:4 0 50G 0 lvm / > | | |-onn-pool00 253:7 0 87G 1 lvm > | | |-onn-home 253:8 0 1G 0 lvm /home > | | |-onn-tmp 253:9 0 1G 0 lvm /tmp > | | |-onn-var 253:10 0 15G 0 lvm /var > | | |-onn-var_crash 253:11 0 10G 0 lvm /var/crash > | | |-onn-var_log 253:12 0 8G 0 lvm /var/log > | | |-onn-var_log_audit 253:13 0 2G 0 lvm /var/log/audit > | | |-onn-ovirt--node--ng--4.5.0.1--0.20220511.0+1 253:14 0 50G 0 lvm > | | `-onn-var_tmp 253:15 0 10G 0 lvm /var/tmp > | |-onn-pool00_tdata 253:2 0 87G 0 lvm > | | `-onn-pool00-tpool 253:3 0 87G 0 lvm > | | |-onn-ovirt--node--ng--4.5.0.2--0.20220513.0+1 253:4 0 50G 0 lvm / > | | |-onn-pool00 253:7 0 87G 1 lvm > | | |-onn-home 253:8 0 1G 0 lvm /home > | | |-onn-tmp 253:9 0 1G 0 lvm /tmp > | | |-onn-var 253:10 0 15G 0 lvm /var > | | |-onn-var_crash 253:11 0 10G 0 lvm /var/crash > | | |-onn-var_log 253:12 0 8G 0 lvm /var/log > | | |-onn-var_log_audit 253:13 0 2G 0 lvm /var/log/audit > | | |-onn-ovirt--node--ng--4.5.0.1--0.20220511.0+1 253:14 0 50G 0 lvm > | | `-onn-var_tmp 253:15 0 10G 0 lvm /var/tmp > | `-onn-swap 253:5 0 20G 0 lvm [SWAP] > `-nvme0n1p3 259:3 0 95G 0 part > `-gluster_vg_nvme0n1p3-gluster_lv_engine 253:6 0 94G 0 lvm /gluster_bricks/engine > > - The old lvm filter used, and why it was needed > > filter = ["a|^/dev/disk/by-id/lvm-pv-uuid-Nn7tZl-TFdY-BujO-VZG5-EaGW-5YFd-Lo5pwa$|", "a|^/dev/disk/by-id/lvm-pv-uuid-Wcbxnx-2RhC-s1Re-s148-nLj9-Tr3f-jj4VvE$|", "a|^/dev/disk/by-id/lvm-pv-uuid-lX51wm-H7V4-3CTn-qYob-Rkpx-Tptd-t94jNL$|", "r|.*|"] > > I don't remember exactly any more why it was needed, but without the node was not working correctly. I think I even used vdsm-tool config-lvm-filter. I think that if you list the devices in this filter: ls -lh /dev/disk/by-id/lvm-pv-uuid-Nn7tZl-TFdY-BujO-VZG5-EaGW-5YFd-Lo5pwa \ /dev/disk/by-id/lvm-pv-uuid-Wcbxnx-2RhC-s1Re-s148-nLj9-Tr3f-jj4VvE \ /dev/disk/by-id/lvm-pv-uuid-lX51wm-H7V4-3CTn-qYob-Rkpx-Tptd-t94jNL You will see that these are the devices used by these vgs: gluster_vg_sda, gluster_vg_nvme0n1p3, onn > > - output of vdsm-tool config-lvm-filter > > Analyzing host... > Found these mounted logical volumes on this host: > > logical volume: /dev/mapper/gluster_vg_nvme0n1p3-gluster_lv_engine > mountpoint: /gluster_bricks/engine > devices: /dev/nvme0n1p3 > > logical volume: /dev/mapper/gluster_vg_sda-gluster_lv_data > mountpoint: /gluster_bricks/data > devices: /dev/mapper/XA1920LE10063_HKS028AV > > logical volume: /dev/mapper/gluster_vg_sda-gluster_lv_vmstore > mountpoint: /gluster_bricks/vmstore > devices: /dev/mapper/XA1920LE10063_HKS028AV > > logical volume: /dev/mapper/onn-home > mountpoint: /home > devices: /dev/nvme0n1p2 > > logical volume: /dev/mapper/onn-ovirt--node--ng--4.5.0.2--0.20220513.0+1 > mountpoint: / > devices: /dev/nvme0n1p2 > > logical volume: /dev/mapper/onn-swap > mountpoint: [SWAP] > devices: /dev/nvme0n1p2 > > logical volume: /dev/mapper/onn-tmp > mountpoint: /tmp > devices: /dev/nvme0n1p2 > > logical volume: /dev/mapper/onn-var > mountpoint: /var > devices: /dev/nvme0n1p2 > > logical volume: /dev/mapper/onn-var_crash > mountpoint: /var/crash > devices: /dev/nvme0n1p2 > > logical volume: /dev/mapper/onn-var_log > mountpoint: /var/log > devices: /dev/nvme0n1p2 > > logical volume: /dev/mapper/onn-var_log_audit > mountpoint: /var/log/audit > devices: /dev/nvme0n1p2 > > logical volume: /dev/mapper/onn-var_tmp > mountpoint: /var/tmp > devices: /dev/nvme0n1p2 > > Configuring LVM system.devices. > Devices for following VGs will be imported: > > gluster_vg_sda, gluster_vg_nvme0n1p3, onn > > To properly configure the host, we need to add multipath > blacklist in /etc/multipath/conf.d/vdsm_blacklist.conf: > > blacklist { > wwid "eui.0025388901b1e26f" > } > > > Configure host? [yes,NO] If you run "vdsm-tool config-lvm-filter" and confirm with "yes", I think all the vgs will be imported properly into lvm devices file. I don't think it will solve the storage issues you have since Feb 2022, but at least you will have a standard configuration and the next upgrade will not revert your local settings. > If using lvm devices does not work for you, you can enable the lvm > filter in vdsm configuration > by adding a drop-in file: > > $ cat /etc/vdsm/vdsm.conf.d/99-local.conf > [lvm] > config_method = filter > > And run: > > vdsm-tool config-lvm-filter > > to configure the lvm filter in the best way for vdsm. If this does not create > the right filter we would like to know why, but in general you should use > lvm devices since it avoids the trouble of maintaining the filter and dealing > with upgrades and user edited lvm filter. > > If you disable use_devicesfile, the next vdsm upgrade will enable it > back unless > you change the configuration. > > I would be happy to just use the default, when there is a way to make use_devicesfile to wok. > > Also even if you disable use_devicesfile in lvm.conf, vdsm still use > --devices instead > of filter when running lvm commands, and lvm commands run by vdsm ignore your > lvm filter since the --devices option overrides the system settings. > > ... > > I notice some unsync volume warning, but because I had this in the past to, after upgrading, I though after some time they will disappear. The next day there still where there, so I decided to put the nodes again in the maintenance mode and restart the glusterd service. After some time the sync warnings where gone. > > Not clear what these warnings are, I guess Gluster warning? > > Yes was Gluster warnings under Storage -> Volumes it was saying that some entries are unsync. > > So now the actual problem: > > Since this time the cluster is unstable. I get different errors and warning, like: > > VM [name] is not responding > out of nothing HA VM gets migrated > VM migration can fail > VM backup with snapshoting and export take very long > > How do you backup the vms? do you sue a backup application? how is it > configured? > > I use a self made plython script, which uses the rest api. I create a snapshot from the VM, build a new VM from that snapshot and move the new one to the export domain. This is not very efficient - this copy the entire vm at the point of time of the snapshot and then copy it again to the export domain. If you use a backup application supporting the incremental backup API, the first full backup will copy the entire vm once, but later incremental backup will copy only the changes since the last backup. > > VMs are getting very slow some times > Storage domain vmstore experienced a high latency of 9.14251 > ovs|00001|db_ctl_base|ERR|no key "dpdk-init" in Open_vSwitch record "." column other_config > 489279 [1064359]: s8 renewal error -202 delta_length 10 last_success 489249 > 444853 [2243175]: s27 delta_renew read timeout 10 sec offset 0 /rhev/data-center/mnt/glusterSD/onode1.example.org:_vmstore/3cf83851-1cc8-4f97-8960-08a60b9e25db/dom_md/ids > 471099 [2243175]: s27 delta_renew read timeout 10 sec offset 0 /rhev/data-center/mnt/glusterSD/onode1.example.org:_vmstore/3cf83851-1cc8-4f97-8960-08a60b9e25db/dom_md/ids > many of: 424035 [2243175]: s27 delta_renew long write time XX sec > > All these issues tell use that your storage is not working correctly. > > sanlock.log is full of renewal errors form May: > > $ grep 2022-05- sanlock.log | wc -l > 4844 > > $ grep 2022-05- sanlock.log | grep 'renewal error' | wc -l > 631 > > But there is lot of trouble from earlier months: > > $ grep 2022-04- sanlock.log | wc -l > 844 > $ grep 2022-04- sanlock.log | grep 'renewal error' | wc -l > 29 > > $ grep 2022-03- sanlock.log | wc -l > 1609 > $ grep 2022-03- sanlock.log | grep 'renewal error' | wc -l > 483 > > $ grep 2022-02- sanlock.log | wc -l > 826 > $ grep 2022-02- sanlock.log | grep 'renewal error' | wc -l > 242 > > Here sanlock log looks healthy: > > $ grep 2022-01- sanlock.log | wc -l > 3 > $ grep 2022-01- sanlock.log | grep 'renewal error' | wc -l > 0 > > $ grep 2021-12- sanlock.log | wc -l > 48 > $ grep 2021-12- sanlock.log | grep 'renewal error' | wc -l > 0 > > vdsm log shows that 2 domains are not accessible: > > $ grep ERROR vdsm.log > 2022-05-29 15:07:19,048+0200 ERROR (check/loop) [storage.monitor] > Error checking path > /rhev/data-center/mnt/glusterSD/onode1.example.org:_data/de5f4123-0fac-4238-abcf-a329c142bd47/dom_md/metadata > (monitor:511) > 2022-05-29 16:33:59,049+0200 ERROR (check/loop) [storage.monitor] > Error checking path > /rhev/data-center/mnt/glusterSD/onode1.example.org:_data/de5f4123-0fac-4238-abcf-a329c142bd47/dom_md/metadata > (monitor:511) > 2022-05-29 16:34:39,049+0200 ERROR (check/loop) [storage.monitor] > Error checking path > /rhev/data-center/mnt/glusterSD/onode1.example.org:_data/de5f4123-0fac-4238-abcf-a329c142bd47/dom_md/metadata > (monitor:511) > 2022-05-29 17:21:39,050+0200 ERROR (check/loop) [storage.monitor] > Error checking path > /rhev/data-center/mnt/glusterSD/onode1.example.org:_data/de5f4123-0fac-4238-abcf-a329c142bd47/dom_md/metadata > (monitor:511) > 2022-05-29 17:55:59,712+0200 ERROR (check/loop) [storage.monitor] > Error checking path > /rhev/data-center/mnt/glusterSD/onode1.example.org:_vmstore/3cf83851-1cc8-4f97-8960-08a60b9e25db/dom_md/metadata > (monitor:511) > 2022-05-29 17:56:19,711+0200 ERROR (check/loop) [storage.monitor] > Error checking path > /rhev/data-center/mnt/glusterSD/onode1.example.org:_vmstore/3cf83851-1cc8-4f97-8960-08a60b9e25db/dom_md/metadata > (monitor:511) > 2022-05-29 17:56:39,050+0200 ERROR (check/loop) [storage.monitor] > Error checking path > /rhev/data-center/mnt/glusterSD/onode1.example.org:_data/de5f4123-0fac-4238-abcf-a329c142bd47/dom_md/metadata > (monitor:511) > 2022-05-29 17:56:39,711+0200 ERROR (check/loop) [storage.monitor] > Error checking path > /rhev/data-center/mnt/glusterSD/onode1.example.org:_vmstore/3cf83851-1cc8-4f97-8960-08a60b9e25db/dom_md/metadata > (monitor:511) > > You need to find what is the issue with your Gluster storage. > > I hope that Ritesh can help debug the issue with Gluster. > > Nir > > I'm worry that I do something, that it makes it even more worst, and I hove not idea what's the problem. To me it looks not exactly like a problem with data inconsistencies. The problem is that your Gluster storage is not healthy, and reading and writing to it times out. Please keep users(a)ovirt.org CC when you reply. Gluster storage is very popular in this mailing list and you may get useful help from other users. Nir

4 3

Self-hosted engine failing liveliness check
by McNamara, Bradley 08 Jun '22

08 Jun '22

Hello, and thank you all for your help. I'm running Oracle's rebranded oVirt 4.3.10. All has been good until I patched my self-hosted engine. I ran through the normal process: backup, global maintenance mode, update the oVirt packages, run engine-setup, etc. All completed normally without issues. I rebooted the self-hosted engine VM, and now it constantly fails liveliness checks and the HA agent reboots it every five minutes, or so. I put it in back in global maintenance so the HA agent would not reboot it. The VM is up and works correctly. I can do everything normally. From what I can tell the HA agent liveliness check is just a http get to the web portal. I can see that happening with success. What is the lilveliness check actually doing? All services on the VM are up and running without issue. Where can I look to figure this out? Here is the output of hosted-engine --vm-status: [root@itdlolv101 ~]# hosted-engine --vm-status !! Cluster is in GLOBAL MAINTENANCE mode !! --== Host itdlolv100.ci.seattle.wa.us (id: 1) status ==-- conf_on_shared_storage : True Status up-to-date : True Hostname : itdlolv100.ci.seattle.wa.us Host ID : 1 Engine status : {"reason": "vm not running on this host", "health": "bad", "vm": "down", "detail": "unknown"} Score : 3400 stopped : False Local maintenance : False crc32 : 855e161f local_conf_timestamp : 55128 Host timestamp : 55128 Extra metadata (valid at timestamp): metadata_parse_version=1 metadata_feature_version=1 timestamp=55128 (Wed Jun 8 12:52:20 2022) host-id=1 score=3400 vm_conf_refresh_time=55128 (Wed Jun 8 12:52:20 2022) conf_on_shared_storage=True maintenance=False state=GlobalMaintenance stopped=False --== Host itdlolv101.ci.seattle.wa.us (id: 2) status ==-- conf_on_shared_storage : True Status up-to-date : True Hostname : itdlolv101.ci.seattle.wa.us Host ID : 2 Engine status : {"reason": "failed liveliness check", "health": "bad", "vm": "up", "detail": "Up"} Score : 3400 stopped : False Local maintenance : False crc32 : cc1c2261 local_conf_timestamp : 45453 Host timestamp : 45453 Extra metadata (valid at timestamp): metadata_parse_version=1 metadata_feature_version=1 timestamp=45453 (Wed Jun 8 12:55:15 2022) host-id=2 score=3400 vm_conf_refresh_time=45453 (Wed Jun 8 12:55:15 2022) conf_on_shared_storage=True maintenance=False state=GlobalMaintenance stopped=False !! Cluster is in GLOBAL MAINTENANCE mode !! [root@itdlolv101 ~]#

1 0

How to run virt-sysprep / How to troubleshoot template creation failure
by jeremy_tourville＠hotmail.com 08 Jun '22

08 Jun '22

When attempting to create a template the process fails. It has been suggested to run virt-sysprep manually and see why it it failed. Specifically, I have an Ubuntu 20.04 machine that doesn't boot properly even if the the template process does finish without error. I have tried creating the template several times as a test. about 80% of the time the template creation fails outright for qemu errors. The other 20% "appear" to work but the system boots to a grub emergency prompt Can someone clarify the process? I know you can run the command: virt-sysprep -d <name_of_vm> 1. Where do you run it from? The hypervisor host or the management engine? 2. What account do you need to use? What is the authentication username and password? I also think I read somewhere that you should refrain from using root. Anything else to know? Anything else to try regarding the grub prompt? Thanks.

2 5

Multiple displays
by mblecha＠flagshipbio.com 08 Jun '22

08 Jun '22

Our lab environment was set up on 4.4.10, then upgraded to 4.5, and our production environment was a clean install of 4.5. In our lab environment, I remember having to alter a setting back in 4.4.10 through 'engine-config' or other to get the multiple monitor option to cooperate in the VM settings. I googled around for the original article that allowed me to fix this on 4.4.10, but I cannot find what I had earlier. On the lab environment, the VM displays behave as expected; X starts and the second display becomes available in virt-viewer. In our production environment, same configurations, OS, etcetera on the VM, but X starts and the second display is still not available.

1 0

looking for some ISV backup software which integrates the backup API
by Nathanaël Blanchet 08 Jun '22

08 Jun '22

Hello, We are about to change our backup provider, and I find it is a great chance to choose a full supported ovirt backup solution. I currently use this python script vm-backup-scheduler (https://github.com/wefixit-AT/oVirtBackup) but it is not the workflow officially suggested by the community (https://www.ovirt.org/develop/release-management/features/storage/backup-re…) I've been looking for a long time an ISV who supports such an API, but the only one I found is this one : Acronis Backup Advanced suggested here https://access.redhat.com/ecosystem/search/#/ecosystem/Red%20Hat%20Enterpri… I ran the trial version, but it doesn't seem to do better than the vm-backup-scheduler script, and it doesn't seem to use the backup API (attach a clone as a disk to an existing vm). Can you suggest me some other ISV solutions, if they ever exist... or share me your backup experience?

5 4

nec-xhci: Win7 guest no response after ejecting usb storage
by lizhijian＠fujitsu.com 08 Jun '22

08 Jun '22

Hi folks, It's observed that win7 guest will become no response if i eject usb storage "Eject MiniKing" [cid:part1.8pN8Bh9e.qrkuS06h@fujitsu.com] the last screen after i clicked "Eject MingKing": [cid:part2.3Z0eywnJ.XOYVtZy1@fujitsu.com] After that, i have to hard reset the guest. Thanks Zhijian

3 4

Announcement: The Terraform provider is now available in the Terraform registry
by Janos Bonic 08 Jun '22

08 Jun '22

Hey everyone, You may still remember the email we sent last year about reworking the Terraform provider on the basis of go-ovirt-client. Thanks to the hard work of a lot of people, I'm happy to announce that the first version of the new Terraform provider is now available in the Terraform registry: https://registry.terraform.io/providers/oVirt/ovirt/latest/docs If you want to learn more about the background of this change, or start contributing to the Terraform provider, please see these links: https://blogs.ovirt.org/2021/10/important-changes-to-the-ovirt-terraform-pr… https://github.com/oVirt/terraform-provider-ovirt/blob/main/CONTRIBUTING.md Happy Terraforming! Janos

2 2

oVirt keeps crashing
by Reznikov Alexei 08 Jun '22

08 Jun '22

Hello Paul. I have same issue with "ovirt-engine: ERROR run:542 Error: process terminated with status code 1". How you resolved it? -- AlexR

1 0

ClassNotFoundException after upgrade to 4.5
by astro.alireza＠gmail.com 07 Jun '22

07 Jun '22

Hi all, I tried to upgrade my ovirt engine from 4.3.10 to 4.5.0. After restore backup, everything seems ok. I can login to admin panel. But when I want to add new host, I get this exception: 2022-06-01 12:13:04,987-04 ERROR [org.jboss.as.ejb3.invocation] (default task-32) WFLYEJB0034: Jakarta Enterprise Beans Invocation failed on component Backend for method public abstract org.ovirt.engine.core.common.queries.QueryReturnValue org.ovirt.engine.core.common.interfaces.BackendLocal.runQuery(org.ovirt.engine.core.common.queries.QueryType,org.ovirt.engine.core.common.queries.QueryParametersBase): javax.ejb.EJBException: WFLYEJB0442: Unexpected Error at org.jboss.as.ejb3@24.0.1.Final//org.jboss.as.ejb3.tx.CMTTxInterceptor.invokeInNoTx(CMTTxInterceptor.java:234) at org.jboss.as.ejb3@24.0.1.Final//org.jboss.as.ejb3.tx.CMTTxInterceptor.supports(CMTTxInterceptor.java:446) at org.jboss.as.ejb3@24.0.1.Final//org.jboss.as.ejb3.tx.CMTTxInterceptor.processInvocation(CMTTxInterceptor.java:164) at org.jboss.invocation@1.6.0.Final//org.jboss.invocation.InterceptorContext.proceed(InterceptorContext.java:422) at org.jboss.invocation@1.6.0.Final//org.jboss.invocation.InterceptorContext$Invocation.proceed(InterceptorContext.java:509) at org.jboss.weld.core@3.1.7.SP1//org.jboss.weld.module.ejb.AbstractEJBRequestScopeActivationInterceptor.aroundInvoke(AbstractEJBRequestScopeActivationInterceptor.java:81) at org.jboss.as.weld.common@24.0.1.Final//org.jboss.as.weld.ejb.EjbRequestScopeActivationInterceptor.processInvocation(EjbRequestScopeActivationInterceptor.java:89) at org.jboss.invocation@1.6.0.Final//org.jboss.invocation.InterceptorContext.proceed(InterceptorContext.java:422) at org.jboss.as.ejb3@24.0.1.Final//org.jboss.as.ejb3.component.interceptors.CurrentInvocationContextInterceptor.processInvocation(CurrentInvocationContextInterceptor.java:41) at org.jboss.invocation@1.6.0.Final//org.jboss.invocation.InterceptorContext.proceed(InterceptorContext.java:422) at org.jboss.as.ejb3@24.0.1.Final//org.jboss.as.ejb3.component.invocationmetrics.WaitTimeInterceptor.processInvocation(WaitTimeInterceptor.java:47) at org.jboss.invocation@1.6.0.Final//org.jboss.invocation.InterceptorContext.proceed(InterceptorContext.java:422) at org.jboss.as.ejb3@24.0.1.Final//org.jboss.as.ejb3.security.SecurityContextInterceptor.processInvocation(SecurityContextInterceptor.java:100) at org.jboss.invocation@1.6.0.Final//org.jboss.invocation.InterceptorContext.proceed(InterceptorContext.java:422) at org.jboss.as.ejb3@24.0.1.Final//org.jboss.as.ejb3.deployment.processors.StartupAwaitInterceptor.processInvocation(StartupAwaitInterceptor.java:22) at org.jboss.invocation@1.6.0.Final//org.jboss.invocation.InterceptorContext.proceed(InterceptorContext.java:422) at org.jboss.as.ejb3@24.0.1.Final//org.jboss.as.ejb3.component.interceptors.ShutDownInterceptorFactory$1.processInvocation(ShutDownInterceptorFactory.java:64) at org.jboss.invocation@1.6.0.Final//org.jboss.invocation.InterceptorContext.proceed(InterceptorContext.java:422) at org.jboss.as.ejb3@24.0.1.Final//org.jboss.as.ejb3.component.interceptors.LoggingInterceptor.processInvocation(LoggingInterceptor.java:67) at org.jboss.invocation@1.6.0.Final//org.jboss.invocation.InterceptorContext.proceed(InterceptorContext.java:422) at org.jboss.as.ee@24.0.1.Final//org.jboss.as.ee.component.NamespaceContextInterceptor.processInvocation(NamespaceContextInterceptor.java:50) at org.jboss.invocation@1.6.0.Final//org.jboss.invocation.InterceptorContext.proceed(InterceptorContext.java:422) at org.jboss.invocation@1.6.0.Final//org.jboss.invocation.ContextClassLoaderInterceptor.processInvocation(ContextClassLoaderInterceptor.java:60) at org.jboss.invocation@1.6.0.Final//org.jboss.invocation.InterceptorContext.proceed(InterceptorContext.java:422) at org.jboss.invocation@1.6.0.Final//org.jboss.invocation.InterceptorContext.run(InterceptorContext.java:438) at org.wildfly.security.elytron-base@1.16.1.Final//org.wildfly.security.manager.WildFlySecurityManager.doChecked(WildFlySecurityManager.java:633) at org.jboss.invocation@1.6.0.Final//org.jboss.invocation.AccessCheckingInterceptor.processInvocation(AccessCheckingInterceptor.java:57) at org.jboss.invocation@1.6.0.Final//org.jboss.invocation.InterceptorContext.proceed(InterceptorContext.java:422) at org.jboss.invocation@1.6.0.Final//org.jboss.invocation.ChainedInterceptor.processInvocation(ChainedInterceptor.java:53) at org.jboss.as.ee@24.0.1.Final//org.jboss.as.ee.component.ViewService$View.invoke(ViewService.java:198) at org.jboss.as.ee@24.0.1.Final//org.jboss.as.ee.component.ViewDescription$1.processInvocation(ViewDescription.java:191) at org.jboss.as.ee@24.0.1.Final//org.jboss.as.ee.component.ProxyInvocationHandler.invoke(ProxyInvocationHandler.java:81) at deployment.engine.ear.bll.jar//org.ovirt.engine.core.common.interfaces.BackendLocal$$$view4.runQuery(Unknown Source) at deployment.engine.ear.webadmin.war//org.ovirt.engine.ui.frontend.server.gwt.GenericApiGWTServiceImpl.runQuery(GenericApiGWTServiceImpl.java:90) at deployment.engine.ear.webadmin.war//org.ovirt.engine.ui.frontend.server.gwt.GenericApiGWTServiceImpl.runMultipleQueries(GenericApiGWTServiceImpl.java:121) at jdk.internal.reflect.GeneratedMethodAccessor91.invoke(Unknown Source) at java.base/jdk.internal.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:43) at java.base/java.lang.reflect.Method.invoke(Method.java:566) at deployment.engine.ear.webadmin.war//com.google.gwt.user.server.rpc.RPC.invokeAndEncodeResponse(RPC.java:587) at deployment.engine.ear.webadmin.war//com.google.gwt.user.server.rpc.RemoteServiceServlet.processCall(RemoteServiceServlet.java:333) at deployment.engine.ear.webadmin.war//com.google.gwt.user.server.rpc.RemoteServiceServlet.processCall(RemoteServiceServlet.java:303) at deployment.engine.ear.webadmin.war//com.google.gwt.user.server.rpc.RemoteServiceServlet.processPost(RemoteServiceServlet.java:373) at deployment.engine.ear.webadmin.war//com.google.gwt.user.server.rpc.AbstractRemoteServiceServlet.doPost(AbstractRemoteServiceServlet.java:62) at javax.servlet.api@2.0.0.Final//javax.servlet.http.HttpServlet.service(HttpServlet.java:523) at deployment.engine.ear.webadmin.war//org.ovirt.engine.ui.frontend.server.gwt.GenericApiGWTServiceImpl.service(GenericApiGWTServiceImpl.java:78) at javax.servlet.api@2.0.0.Final//javax.servlet.http.HttpServlet.service(HttpServlet.java:590) at io.undertow.servlet@2.2.8.Final//io.undertow.servlet.handlers.ServletHandler.handleRequest(ServletHandler.java:74) at io.undertow.servlet@2.2.8.Final//io.undertow.servlet.handlers.FilterHandler$FilterChainImpl.doFilter(FilterHandler.java:129) at org.ovirt.engine.core.utils//org.ovirt.engine.core.utils.servlet.HeaderFilter.doFilter(HeaderFilter.java:94) at io.undertow.servlet@2.2.8.Final//io.undertow.servlet.core.ManagedFilter.doFilter(ManagedFilter.java:61) at io.undertow.servlet@2.2.8.Final//io.undertow.servlet.handlers.FilterHandler$FilterChainImpl.doFilter(FilterHandler.java:131) at org.ovirt.engine.core.utils//org.ovirt.engine.core.utils.servlet.CachingFilter.doFilter(CachingFilter.java:133) at io.undertow.servlet@2.2.8.Final//io.undertow.servlet.core.ManagedFilter.doFilter(ManagedFilter.java:61) at io.undertow.servlet@2.2.8.Final//io.undertow.servlet.handlers.FilterHandler$FilterChainImpl.doFilter(FilterHandler.java:131) at deployment.engine.ear.webadmin.war//org.ovirt.engine.core.branding.BrandingFilter.doFilter(BrandingFilter.java:73) at io.undertow.servlet@2.2.8.Final//io.undertow.servlet.core.ManagedFilter.doFilter(ManagedFilter.java:61) at io.undertow.servlet@2.2.8.Final//io.undertow.servlet.handlers.FilterHandler$FilterChainImpl.doFilter(FilterHandler.java:131) at org.ovirt.engine.core.utils//org.ovirt.engine.core.utils.servlet.LocaleFilter.doFilter(LocaleFilter.java:65) at io.undertow.servlet@2.2.8.Final//io.undertow.servlet.core.ManagedFilter.doFilter(ManagedFilter.java:61) at io.undertow.servlet@2.2.8.Final//io.undertow.servlet.handlers.FilterHandler$FilterChainImpl.doFilter(FilterHandler.java:131) at io.undertow.servlet@2.2.8.Final//io.undertow.servlet.handlers.FilterHandler.handleRequest(FilterHandler.java:84) at io.undertow.servlet@2.2.8.Final//io.undertow.servlet.handlers.security.ServletSecurityRoleHandler.handleRequest(ServletSecurityRoleHandler.java:62) at io.undertow.servlet@2.2.8.Final//io.undertow.servlet.handlers.ServletChain$1.handleRequest(ServletChain.java:68) at io.undertow.servlet@2.2.8.Final//io.undertow.servlet.handlers.ServletDispatchingHandler.handleRequest(ServletDispatchingHandler.java:36) at org.wildfly.extension.undertow@24.0.1.Final//org.wildfly.extension.undertow.security.SecurityContextAssociationHandler.handleRequest(SecurityContextAssociationHandler.java:78) at io.undertow.core@2.2.8.Final//io.undertow.server.handlers.PredicateHandler.handleRequest(PredicateHandler.java:43) at io.undertow.servlet@2.2.8.Final//io.undertow.servlet.handlers.RedirectDirHandler.handleRequest(RedirectDirHandler.java:68) at io.undertow.servlet@2.2.8.Final//io.undertow.servlet.handlers.security.SSLInformationAssociationHandler.handleRequest(SSLInformationAssociationHandler.java:117) at io.undertow.servlet@2.2.8.Final//io.undertow.servlet.handlers.security.ServletAuthenticationCallHandler.handleRequest(ServletAuthenticationCallHandler.java:57) at io.undertow.core@2.2.8.Final//io.undertow.server.handlers.PredicateHandler.handleRequest(PredicateHandler.java:43) at io.undertow.core@2.2.8.Final//io.undertow.security.handlers.AuthenticationConstraintHandler.handleRequest(AuthenticationConstraintHandler.java:53) at io.undertow.core@2.2.8.Final//io.undertow.security.handlers.AbstractConfidentialityHandler.handleRequest(AbstractConfidentialityHandler.java:46) at io.undertow.servlet@2.2.8.Final//io.undertow.servlet.handlers.security.ServletConfidentialityConstraintHandler.handleRequest(ServletConfidentialityConstraintHandler.java:64) at io.undertow.servlet@2.2.8.Final//io.undertow.servlet.handlers.security.ServletSecurityConstraintHandler.handleRequest(ServletSecurityConstraintHandler.java:59) at io.undertow.core@2.2.8.Final//io.undertow.security.handlers.AuthenticationMechanismsHandler.handleRequest(AuthenticationMechanismsHandler.java:60) at io.undertow.servlet@2.2.8.Final//io.undertow.servlet.handlers.security.CachedAuthenticatedSessionHandler.handleRequest(CachedAuthenticatedSessionHandler.java:77) at io.undertow.core@2.2.8.Final//io.undertow.security.handlers.NotificationReceiverHandler.handleRequest(NotificationReceiverHandler.java:50) at io.undertow.core@2.2.8.Final//io.undertow.security.handlers.AbstractSecurityContextAssociationHandler.handleRequest(AbstractSecurityContextAssociationHandler.java:43) at io.undertow.core@2.2.8.Final//io.undertow.server.handlers.PredicateHandler.handleRequest(PredicateHandler.java:43) at org.wildfly.extension.undertow@24.0.1.Final//org.wildfly.extension.undertow.security.jacc.JACCContextIdHandler.handleRequest(JACCContextIdHandler.java:61) at io.undertow.core@2.2.8.Final//io.undertow.server.handlers.PredicateHandler.handleRequest(PredicateHandler.java:43) at io.undertow.servlet@2.2.8.Final//io.undertow.servlet.handlers.SendErrorPageHandler.handleRequest(SendErrorPageHandler.java:52) at io.undertow.core@2.2.8.Final//io.undertow.server.handlers.PredicateHandler.handleRequest(PredicateHandler.java:43) at io.undertow.servlet@2.2.8.Final//io.undertow.servlet.handlers.ServletInitialHandler.handleFirstRequest(ServletInitialHandler.java:269) at io.undertow.servlet@2.2.8.Final//io.undertow.servlet.handlers.ServletInitialHandler.access$100(ServletInitialHandler.java:78) at io.undertow.servlet@2.2.8.Final//io.undertow.servlet.handlers.ServletInitialHandler$2.call(ServletInitialHandler.java:133) at io.undertow.servlet@2.2.8.Final//io.undertow.servlet.handlers.ServletInitialHandler$2.call(ServletInitialHandler.java:130) at io.undertow.servlet@2.2.8.Final//io.undertow.servlet.core.ServletRequestContextThreadSetupAction$1.call(ServletRequestContextThreadSetupAction.java:48) at io.undertow.servlet@2.2.8.Final//io.undertow.servlet.core.ContextClassLoaderSetupAction$1.call(ContextClassLoaderSetupAction.java:43) at org.wildfly.extension.undertow@24.0.1.Final//org.wildfly.extension.undertow.security.SecurityContextThreadSetupAction.lambda$create$0(SecurityContextThreadSetupAction.java:105) at org.wildfly.extension.undertow@24.0.1.Final//org.wildfly.extension.undertow.deployment.UndertowDeploymentInfoService$UndertowThreadSetupAction.lambda$create$0(UndertowDeploymentInfoService.java:1535) at org.wildfly.extension.undertow@24.0.1.Final//org.wildfly.extension.undertow.deployment.UndertowDeploymentInfoService$UndertowThreadSetupAction.lambda$create$0(UndertowDeploymentInfoService.java:1535) at org.wildfly.extension.undertow@24.0.1.Final//org.wildfly.extension.undertow.deployment.UndertowDeploymentInfoService$UndertowThreadSetupAction.lambda$create$0(UndertowDeploymentInfoService.java:1535) at io.undertow.servlet@2.2.8.Final//io.undertow.servlet.handlers.ServletInitialHandler.dispatchRequest(ServletInitialHandler.java:249) at io.undertow.servlet@2.2.8.Final//io.undertow.servlet.handlers.ServletInitialHandler.access$000(ServletInitialHandler.java:78) at io.undertow.servlet@2.2.8.Final//io.undertow.servlet.handlers.ServletInitialHandler$1.handleRequest(ServletInitialHandler.java:99) at io.undertow.core@2.2.8.Final//io.undertow.server.Connectors.executeRootHandler(Connectors.java:387) at io.undertow.core@2.2.8.Final//io.undertow.server.HttpServerExchange$1.run(HttpServerExchange.java:841) at org.jboss.threads@2.4.0.Final//org.jboss.threads.ContextClassLoaderSavingRunnable.run(ContextClassLoaderSavingRunnable.java:35) at org.jboss.threads@2.4.0.Final//org.jboss.threads.EnhancedQueueExecutor.safeRun(EnhancedQueueExecutor.java:1990) at org.jboss.threads@2.4.0.Final//org.jboss.threads.EnhancedQueueExecutor$ThreadBody.doRunTask(EnhancedQueueExecutor.java:1486) at org.jboss.threads@2.4.0.Final//org.jboss.threads.EnhancedQueueExecutor$ThreadBody.run(EnhancedQueueExecutor.java:1377) at org.jboss.xnio@3.8.4.Final//org.xnio.XnioWorker$WorkerThreadFactory$1$1.run(XnioWorker.java:1280) at java.base/java.lang.Thread.run(Thread.java:829) Caused by: java.lang.NoClassDefFoundError: org/apache/sshd/common/util/io/output/SecureByteArrayOutputStream at org.ovirt.engine.core.utils//org.ovirt.engine.core.utils.crypt.EngineEncryptionUtils.getEngineSSHPublicKey(EngineEncryptionUtils.java:137) at deployment.engine.ear.bll.jar//org.ovirt.engine.core.bll.hostdeploy.GetEngineSSHPublicKeyQuery.executeQueryCommand(GetEngineSSHPublicKeyQuery.java:19) at deployment.engine.ear.bll.jar//org.ovirt.engine.core.bll.QueriesCommandBase.executeCommand(QueriesCommandBase.java:106) at org.ovirt.engine.core.dal//org.ovirt.engine.core.dal.VdcCommandBase.execute(VdcCommandBase.java:31) at deployment.engine.ear.bll.jar//org.ovirt.engine.core.bll.executor.DefaultBackendQueryExecutor.execute(DefaultBackendQueryExecutor.java:14) at deployment.engine.ear.bll.jar//org.ovirt.engine.core.bll.Backend.runQueryImpl(Backend.java:520) at deployment.engine.ear.bll.jar//org.ovirt.engine.core.bll.Backend.runQuery(Backend.java:489) at jdk.internal.reflect.GeneratedMethodAccessor71.invoke(Unknown Source) at java.base/jdk.internal.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:43) at java.base/java.lang.reflect.Method.invoke(Method.java:566) at org.jboss.as.ee@24.0.1.Final//org.jboss.as.ee.component.ManagedReferenceMethodInterceptor.processInvocation(ManagedReferenceMethodInterceptor.java:52) at org.jboss.invocation@1.6.0.Final//org.jboss.invocation.InterceptorContext.proceed(InterceptorContext.java:422) at org.jboss.invocation@1.6.0.Final//org.jboss.invocation.InterceptorContext$Invocation.proceed(InterceptorContext.java:509) at org.jboss.as.weld.ejb@24.0.1.Final//org.jboss.as.weld.ejb.DelegatingInterceptorInvocationContext.proceed(DelegatingInterceptorInvocationContext.java:92) at org.jboss.weld.core@3.1.7.SP1//org.jboss.weld.interceptor.proxy.WeldInvocationContextImpl.interceptorChainCompleted(WeldInvocationContextImpl.java:107) at org.jboss.weld.core@3.1.7.SP1//org.jboss.weld.interceptor.proxy.WeldInvocationContextImpl.proceed(WeldInvocationContextImpl.java:126) at org.ovirt.engine.core.common//org.ovirt.engine.core.common.di.interceptor.LoggingInterceptor.apply(LoggingInterceptor.java:12) at jdk.internal.reflect.GeneratedMethodAccessor61.invoke(Unknown Source) at java.base/jdk.internal.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:43) at java.base/java.lang.reflect.Method.invoke(Method.java:566) at org.jboss.weld.core@3.1.7.SP1//org.jboss.weld.interceptor.reader.SimpleInterceptorInvocation$SimpleMethodInvocation.invoke(SimpleInterceptorInvocation.java:73) at org.jboss.weld.core@3.1.7.SP1//org.jboss.weld.interceptor.proxy.WeldInvocationContextImpl.invokeNext(WeldInvocationContextImpl.java:92) at org.jboss.weld.core@3.1.7.SP1//org.jboss.weld.interceptor.proxy.WeldInvocationContextImpl.proceed(WeldInvocationContextImpl.java:124) at org.jboss.weld.core@3.1.7.SP1//org.jboss.weld.bean.InterceptorImpl.intercept(InterceptorImpl.java:105) at org.jboss.as.weld.ejb@24.0.1.Final//org.jboss.as.weld.ejb.DelegatingInterceptorInvocationContext.proceed(DelegatingInterceptorInvocationContext.java:82) at org.jboss.as.weld.ejb@24.0.1.Final//org.jboss.as.weld.interceptors.EjbComponentInterceptorSupport.delegateInterception(EjbComponentInterceptorSupport.java:60) at org.jboss.as.weld.common@24.0.1.Final//org.jboss.as.weld.interceptors.Jsr299BindingsInterceptor.delegateInterception(Jsr299BindingsInterceptor.java:77) at org.jboss.as.weld.common@24.0.1.Final//org.jboss.as.weld.interceptors.Jsr299BindingsInterceptor.doMethodInterception(Jsr299BindingsInterceptor.java:89) at org.jboss.as.weld.common@24.0.1.Final//org.jboss.as.weld.interceptors.Jsr299BindingsInterceptor.processInvocation(Jsr299BindingsInterceptor.java:102) at org.jboss.as.ee@24.0.1.Final//org.jboss.as.ee.component.interceptors.UserInterceptorFactory$1.processInvocation(UserInterceptorFactory.java:63) at org.jboss.invocation@1.6.0.Final//org.jboss.invocation.InterceptorContext.proceed(InterceptorContext.java:422) at org.jboss.invocation@1.6.0.Final//org.jboss.invocation.InterceptorContext$Invocation.proceed(InterceptorContext.java:509) at deployment.engine.ear.bll.jar//org.ovirt.engine.core.bll.interceptors.CorrelationIdTrackerInterceptor.aroundInvoke(CorrelationIdTrackerInterceptor.java:13) at jdk.internal.reflect.GeneratedMethodAccessor67.invoke(Unknown Source) at java.base/jdk.internal.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:43) at java.base/java.lang.reflect.Method.invoke(Method.java:566) at org.jboss.as.ee@24.0.1.Final//org.jboss.as.ee.component.ManagedReferenceLifecycleMethodInterceptor.processInvocation(ManagedReferenceLifecycleMethodInterceptor.java:89) at org.jboss.invocation@1.6.0.Final//org.jboss.invocation.InterceptorContext.proceed(InterceptorContext.java:422) at org.jboss.as.ejb3@24.0.1.Final//org.jboss.as.ejb3.component.invocationmetrics.ExecutionTimeInterceptor.processInvocation(ExecutionTimeInterceptor.java:43) at org.jboss.invocation@1.6.0.Final//org.jboss.invocation.InterceptorContext.proceed(InterceptorContext.java:422) at org.jboss.as.ee@24.0.1.Final//org.jboss.as.ee.concurrent.ConcurrentContextInterceptor.processInvocation(ConcurrentContextInterceptor.java:45) at org.jboss.invocation@1.6.0.Final//org.jboss.invocation.InterceptorContext.proceed(InterceptorContext.java:422) at org.jboss.invocation@1.6.0.Final//org.jboss.invocation.InitialInterceptor.processInvocation(InitialInterceptor.java:40) at org.jboss.invocation@1.6.0.Final//org.jboss.invocation.InterceptorContext.proceed(InterceptorContext.java:422) at org.jboss.invocation@1.6.0.Final//org.jboss.invocation.ChainedInterceptor.processInvocation(ChainedInterceptor.java:53) at org.jboss.as.ee@24.0.1.Final//org.jboss.as.ee.component.interceptors.ComponentDispatcherInterceptor.processInvocation(ComponentDispatcherInterceptor.java:52) at org.jboss.invocation@1.6.0.Final//org.jboss.invocation.InterceptorContext.proceed(InterceptorContext.java:422) at org.jboss.as.ejb3@24.0.1.Final//org.jboss.as.ejb3.component.singleton.SingletonComponentInstanceAssociationInterceptor.processInvocation(SingletonComponentInstanceAssociationInterceptor.java:53) at org.jboss.invocation@1.6.0.Final//org.jboss.invocation.InterceptorContext.proceed(InterceptorContext.java:422) at org.jboss.as.ejb3@24.0.1.Final//org.jboss.as.ejb3.tx.CMTTxInterceptor.invokeInNoTx(CMTTxInterceptor.java:232) ... 103 more Caused by: java.lang.ClassNotFoundException: org.apache.sshd.common.util.io.output.SecureByteArrayOutputStream from [Module "org.ovirt.engine.core.uutils" from local module loader @172b013 (finder: local module finder @56673b2c (roots: /usr/share/ovirt-engine-wildfly-overlay/modules,/usr/share/ovirt-engine-wildfly/modules/system/layers/base,/usr/share/ovirt-engine/modules/common,/usr/share/ovirt-engine-extension-aaa-jdbc/modules,/usr/share/ovirt-engine-wildfly/modules,/usr/share/ovirt-engine-wildfly/modules/system/layers/base))] at org.jboss.modules.ModuleClassLoader.findClass(ModuleClassLoader.java:255) at org.jboss.modules.ConcurrentClassLoader.performLoadClassUnchecked(ConcurrentClassLoader.java:410) at org.jboss.modules.ConcurrentClassLoader.performLoadClass(ConcurrentClassLoader.java:398) at org.jboss.modules.ConcurrentClassLoader.loadClass(ConcurrentClassLoader.java:116) ... 153 more

3 3

local on host storage domain full, how to clean up
by Gianluca Cecchi 07 Jun '22

07 Jun '22

Hello, I have a local based storage domain that has got full, so that it is now inactive and virtual machines that are on it are paused (vm paused due to lack of storage space). Any advice on how to clean up, eventually deleting some of them? # df -h /2t_2 Filesystem Size Used Avail Use% Mounted on /dev/mapper/vg_2t_2-2t_2 1.8T 1.7T 0 100% /2t_2 [root@ovirt01 fdf9546c-68fa-42c9-8a10-78ef3ee534b8]# Is there any easy way to map disk-to-vm of directories inside /2t_2/images/dbf9611d-9090-42d6-81e0-58105bc20011/images/ so that I can "sacrifice" some VMs deleting the corresponding disks' directories, to be able at least to activate the storage domain again and make a cleaner check? Or any other suggestions? I'm not able to expand it. It is not directly managed by me and I suppose too much storage over-provisioning has been done. Thanks in advance. Gianluca

4 6

Hyperconverged Ovirt 4.3.10.4: Unresponsive Host
by ralf＠os-s.de 07 Jun '22

07 Jun '22

Hi, I have a hyperconverged ovirt 4.3.10 installation. One host is unresponsive. This has happened in the past but most often resolved itself within a few minutes. This time it is a different beast. The host does not come back. Putting the host in maintenance is not possible because "Host is unresponsive but has running virtual machines". Putting the host into local maintenance on the local CLI does not help. Global maintenance does not have any effect either. The host has been rebooted several times both locally and via the web interface on the engine, which worked but did not resolve the issue. Logs: /var/log/ovirt-engine/engine.log 2022-06-07 06:40:17,606+02 ERROR [org.ovirt.engine.core.dal.dbbroker.auditloghandling.AuditLogDirector] (default task-34) [26f286ba-58d0-44be-96e0-f874057fe603] EVENT_ID: GENERIC_ERROR_MESSAGE(14,001), Cannot switch Host to Maintenance mode. Host still has running VMs on it and is in Non Responsive state. 2022-06-07 06:40:17,606+02 WARN [org.ovirt.engine.core.bll.MaintenanceNumberOfVdssCommand] (default task-34) [26f286ba-58d0-44be-96e0-f874057fe603] Validation of action 'MaintenanceNumberOfVdss' failed for user admin@internal-authz. Reasons: VAR__TYPE__HOST,VAR__ACTION__MAINTENANCE,VDS_CANNOT_MAINTENANCE_VDS_IS_NOT_RESPONDING_WITH_VMS,$HostNotResponding ovirt3.os-s.de 2022-06-07 06:41:23,974+02 ERROR [org.ovirt.engine.core.dal.dbbroker.auditloghandling.AuditLogDirector] (EE-ManagedThreadFactory-engineScheduled-Thread-33) [] EVENT_ID: VDS_BROKER_COMMAND_FAILURE(10,802), VDSM ovirt3.os-s.de command Get Host Capabilities failed: Message timeout which can be caused by communication issues 2022-06-07 06:41:23,974+02 ERROR [org.ovirt.engine.core.vdsbroker.monitoring.HostMonitoring] (EE-ManagedThreadFactory-engineScheduled-Thread-33) [] Unable to RefreshCapabilities: VDSNetworkException: VDSGenericException: VDSNetworkException: Message timeout which can be caused by communication issues 2022-06-07 06:44:47,033+02 ERROR [org.ovirt.engine.core.dal.dbbroker.auditloghandling.AuditLogDirector] (EE-ManagedThreadFactory-engineScheduled-Thread-4) [] EVENT_ID: VDS_BROKER_COMMAND_FAILURE(10,802), VDSM ovirt3.os-s.de command Get Host Capabilities failed: Message timeout which can be caused by communication issues 2022-06-07 06:44:47,034+02 ERROR [org.ovirt.engine.core.vdsbroker.monitoring.HostMonitoring] (EE-ManagedThreadFactory-engineScheduled-Thread-4) [] Unable to RefreshCapabilities: VDSNetworkException: VDSGenericException: VDSNetworkException: Message timeout which can be caused by communication issues vdsmd status: Jun 06 08:51:09 ovirt3.os-s.de vdsm[3567]: WARN Worker blocked: <Worker name=jsonrpc/1 running <Task <JsonRpcTask {'params': {}, 'jsonrpc': '2.0', 'method': u'Host.getStats', 'id': u'ce7ab5d1-3034-4b5a-91c5-9446e17341a2'} at 0x7fe1646563d0> timeout=60, duration=120.00 at 0x7fe1646562d0> task#=15 at 0x7fe165f47e50>, traceback: File: "/usr/lib64/python2.7/threading.py", line 785, in __bootstrap self.__bootstrap_inner() File: "/usr/lib64/python2.7/threading.py", line 812, in __bootstrap_inner self.run() File: "/usr/lib64/python2.7/threading.py", line 765, in run self.__target(*self.__args, **self.__kwargs) File: "/usr/lib/python2.7/site-packages/vdsm/common/concurrent.py", line 260, in run ret = func(*args, **kwargs) File: "/usr/lib/python2.7/site-packages/vdsm/executor.py", line 301, in _run self._execute_task() File: "/usr/lib/python2.7/site-packages/vdsm/executor.py", line 315, in _execute_task task() File: "/usr/lib/python2.7/site-packages/vdsm/executor.py", line 391, in __call__ self._callable() File: "/usr/lib/python2.7/site-packages/yajsonrpc/__init__.py", line 262, in __call__ self._handler(self._ctx, self._req) File: "/usr/lib/python2.7/site-packages/yajsonrpc/__init__.py", line 305, in _serveRequest response = self._handle_request(req, ctx) File: "/usr/lib/python2.7/site-packages/yajsonrpc/__init__.py", line 345, in _handle_request res = method(**params) File: "/usr/lib/python2.7/site-packages/vdsm/rpc/Bridge.py", line 194, in _dynamicMethod result = fn(*methodArgs) File: "<string>", line 2, in getStats File: "/usr/lib/python2.7/site-packages/vdsm/common/api.py", line 50, in method ret = func(*args, **kwargs) File: "/usr/lib/python2.7/site-packages/vdsm/API.py", line 1435, in getStats multipath=True)} File: "/usr/lib/python2.7/site-packages/vdsm/host/api.py", line 79, in get_stats ret['haStats'] = _getHaInfo() File: "/usr/lib/python2.7/site-packages/vdsm/host/api.py", line 182, in _getHaInfo stats = instance.get_all_stats(timeout=5) File: "/usr/lib/python2.7/site-packages/ovirt_hosted_engine_ha/client/client.py", line 94, in get_all_stats stats = broker.get_stats_from_storage() File: "/usr/lib/python2.7/site-packages/ovirt_hosted_engine_ha/lib/brokerlink.py", line 146, in get_stats_from_storage result = self._proxy.get_stats() File: "/usr/lib64/python2.7/xmlrpclib.py", line 1233, in __call__ return self.__send(self.__name, args) File: "/usr/lib64/python2.7/xmlrpclib.py", line 1591, in __request verbose=self.__verbose File: "/usr/lib64/python2.7/xmlrpclib.py", line 1273, in request return self.single_request(host, handler, request_body, verbose) File: "/usr/lib64/python2.7/xmlrpclib.py", line 1301, in single_request self.send_content(h, request_body) File: "/usr/lib64/python2.7/xmlrpclib.py", line 1448, in send_content connection.endheaders(request_body) File: "/usr/lib64/python2.7/httplib.py", line 1052, in endheaders self._send_output(message_body) File: "/usr/lib64/python2.7/httplib.py", line 890, in _send_output self.send(msg) File: "/usr/lib64/python2.7/httplib.py", line 852, in send self.connect() File: "/usr/lib/python2.7/site-packages/ovirt_hosted_engine_ha/lib/unixrpc.py", line 60, in connect self.sock.connect(base64.b16decode(self.host)) File: "/usr/lib64/python2.7/socket.py", line 224, in meth return getattr(self._sock,name)(*args) Jun 06 08:51:14 ovirt3.os-s.de vdsm[3567]: WARN Worker blocked: <Worker name=periodic/0 running <Task <Operation action=<vdsm.virt.sampling.HostMonitor object at 0x7fe16457ba90> at 0x7fe16457bad0> timeout=15, duration=120.00 at 0x7fe1644c3910> task#=22 at 0x7fe1646e4550>, traceback: File: "/usr/lib64/python2.7/threading.py", line 785, in __bootstrap self.__bootstrap_inner() File: "/usr/lib64/python2.7/threading.py", line 812, in __bootstrap_inner self.run() File: "/usr/lib64/python2.7/threading.py", line 765, in run self.__target(*self.__args, **self.__kwargs) File: "/usr/lib/python2.7/site-packages/vdsm/common/concurrent.py", line 260, in run ret = func(*args, **kwargs) File: "/usr/lib/python2.7/site-packages/vdsm/executor.py", line 301, in _run self._execute_task() File: "/usr/lib/python2.7/site-packages/vdsm/executor.py", line 315, in _execute_task task() File: "/usr/lib/python2.7/site-packages/vdsm/executor.py", line 391, in __call__ self._callable() File: "/usr/lib/python2.7/site-packages/vdsm/virt/periodic.py", line 186, in __call__ self._func() File: "/usr/lib/python2.7/site-packages/vdsm/virt/sampling.py", line 481, in __call__ stats = hostapi.get_stats(self._cif, self._samples.stats()) File: "/usr/lib/python2.7/site-packages/vdsm/host/api.py", line 79, in get_stats ret['haStats'] = _getHaInfo() File: "/usr/lib/python2.7/site-packages/vdsm/host/api.py", line 182, in _getHaInfo stats = instance.get_all_stats(timeout=5) File: "/usr/lib/python2.7/site-packages/ovirt_hosted_engine_ha/client/client.py", line 94, in get_all_stats stats = broker.get_stats_from_storage() File: "/usr/lib/python2.7/site-packages/ovirt_hosted_engine_ha/lib/brokerlink.py", line 146, in get_stats_from_storage result = self._proxy.get_stats() File: "/usr/lib64/python2.7/xmlrpclib.py", line 1233, in __call__ return self.__send(self.__name, args) File: "/usr/lib64/python2.7/xmlrpclib.py", line 1591, in __request verbose=self.__verbose File: "/usr/lib64/python2.7/xmlrpclib.py", line 1273, in request return self.single_request(host, handler, request_body, verbose) File: "/usr/lib64/python2.7/xmlrpclib.py", line 1301, in single_request self.send_content(h, request_body) File: "/usr/lib64/python2.7/xmlrpclib.py", line 1448, in send_content connection.endheaders(request_body) File: "/usr/lib64/python2.7/httplib.py", line 1052, in endheaders self._send_output(message_body) File: "/usr/lib64/python2.7/httplib.py", line 890, in _send_output self.send(msg) File: "/usr/lib64/python2.7/httplib.py", line 852, in send self.connect() File: "/usr/lib/python2.7/site-packages/ovirt_hosted_engine_ha/lib/unixrpc.py", line 60, in connect self.sock.connect(base64.b16decode(self.host)) File: "/usr/lib64/python2.7/socket.py", line 224, in meth return getattr(self._sock,name)(*args) vdsmd log: 2022-06-07 05:02:13,717+0000 INFO (jsonrpc/3) [vdsm.api] FINISH multipath_health return={} from=::1,54704, task_id=169b447f-fd52-4a6f-9277-eff9870ffb81 (api:54) 2022-06-07 05:02:13,725+0000 INFO (jsonrpc/3) [api.host] FINISH getStats return={'status': {'message': 'Done', 'code': 0}, 'info': {'cpuStatistics': {'20': {'cpuUser': '0.07', 'nodeIndex': 1, 'cpuSys': '0.00', 'cpuIdle': '99.93'}, '21': {'cpuUser': '0.00', 'nodeIndex': 1, 'cpuSys': '0.00', 'cpuIdle': '100.00'}, '22': {'cpuUser': '0.00', 'nodeIndex': 1, 'cpuSys': '0.00', 'cpuIdle': '100.00'}, '23': {'cpuUser': '0.00', 'nodeIndex': 1, 'cpuSys': '0.07', 'cpuIdle': '99.93'}, '1': {'cpuUser': '0.87', 'nodeIndex': 0, 'cpuSys': '0.60', 'cpuIdle': '98.53'}, '0': {'cpuUser': '0.40', 'nodeIndex': 0, 'cpuSys': '0.27', 'cpuIdle': '99.33'}, '3': {'cpuUser': '0.27', 'nodeIndex': 0, 'cpuSys': '0.33', 'cpuIdle': '99.40'}, '2': {'cpuUser': '0.53', 'nodeIndex': 0, 'cpuSys': '0.40', 'cpuIdle': '99.07'}, '5': {'cpuUser': '0.33', 'nodeIndex': 0, 'cpuSys': '0.33', 'cpuIdle': '99.34'}, '4': {'cpuUser': '0.33', 'nodeIndex': 0, 'cpuSys': '0.40', 'cpuIdle': '99.27'}, '7': {'cpuUser': '0.13', 'nodeIndex': 1, 'cpuSys': '0.07', 'cpuIdle': '99.80'}, '6': {'cpuUser': '0.13', 'nodeIndex': 1, 'cpuSys': '0.07', 'cpuIdle': '99.80'}, '9': {'cpuUser': '0.13', 'nodeIndex': 1, 'cpuSys': '0.13', 'cpuIdle': '99.74'}, '8': {'cpuUser': '0.07', 'nodeIndex': 1, 'cpuSys': '0.13', 'cpuIdle': '99.80'}, '11': {'cpuUser': '0.20', 'nodeIndex': 1, 'cpuSys': '0.13', 'cpuIdle': '99.67'}, '10': {'cpuUser': '0.20', 'nodeIndex': 1, 'cpuSys': '0.27', 'cpuIdle': '99.53'}, '13': {'cpuUser': '0.80', 'nodeIndex': 0, 'cpuSys': '0.47', 'cpuIdle': '98.73'}, '12': {'cpuUser': '0.00', 'nodeIndex': 0, 'cpuSys': '0.00', 'cpuIdle': '100.00'}, '15': {'cpuUser': '0.07', 'nodeIndex': 0, 'cpuSys': '0.00', 'cpuIdle': '99.93'}, '14': {'cpuUser': '0.20', 'nodeIndex': 0, 'cpuSys': '0.27', 'cpuIdle': '99.53'}, '17': {'cpuUser': '0.07', 'nodeIndex': 0, 'cpuSys': '0.00', 'cpuIdle': '99.93'}, '16': {'cpuUser': '0.07', 'nodeIndex': 0, 'cpuSys': '0.00', 'cpuIdle': '99.93'}, '19': {'cpuUser': '1.46', 'nodeIndex': 1, 'cpuSys': '0.33', 'cpuIdle' : '98.21'}, '18': {'cpuUser': '0.27', 'nodeIndex': 1, 'cpuSys': '0.07', 'cpuIdle': '99.66'}}, 'numaNodeMemFree': {'1': {'memPercent': 18, 'memFree': '52935'}, '0': {'memPercent': 17, 'memFree': '53551'}}, 'memShared': 0, 'haScore': 0, 'thpState': 'always', 'ksmMergeAcrossNodes': True, 'vmCount': 3, 'memUsed': '3', 'storageDomains': {}, 'incomingVmMigrations': 0, 'network': {'bond0': {'rxErrors': '0', 'txErrors': '0', 'speed': '2000', 'rxDropped': '4', 'name': 'bond0', 'tx': '84766340728', 'txDropped': '0', 'duplex': 'full', 'sampleTime': 1654578133.717023, 'rx': '102755561238', 'state': 'up'}, 'eno50': {'rxErrors': '0', 'txErrors': '0', 'speed': '1000', 'rxDropped': '0', 'name': 'eno50', 'tx': '0', 'txDropped': '0', 'duplex': 'unknown', 'sampleTime': 1654578133.717023, 'rx': '0', 'state': 'down'}, ';vdsmdummy;': {'rxErrors': '0', 'txErrors': '0', 'speed': '1000', 'rxDropped': '0', 'name': ';vdsmdummy;', 'tx': '0', 'txDropped': '0', 'duplex': 'unknown', 'sampleTime': 1654578133.71702 3, 'rx': '0', 'state': 'down'}, 'ovirtmgmt': {'rxErrors': '0', 'txErrors': '0', 'speed': '1000', 'rxDropped': '122', 'name': 'ovirtmgmt', 'tx': '81789466633', 'txDropped': '0', 'duplex': 'unknown', 'sampleTime': 1654578133.717023, 'rx': '99028291927', 'state': 'up'}, 'lo': {'rxErrors': '0', 'txErrors': '0', 'speed': '1000', 'rxDropped': '0', 'name': 'lo', 'tx': '10214215104', 'txDropped': '0', 'duplex': 'unknown', 'sampleTime': 1654578133.717023, 'rx': '10214215104', 'state': 'up'}, 'ovs-system': {'rxErrors': '0', 'txErrors': '0', 'speed': '1000', 'rxDropped': '0', 'name': 'ovs-system', 'tx': '0', 'txDropped': '0', 'duplex': 'unknown', 'sampleTime': 1654578133.717023, 'rx': '0', 'state': 'down'}, 'eno49': {'rxErrors': '0', 'txErrors': '0', 'speed': '1000', 'rxDropped': '0', 'name': 'eno49', 'tx': '0', 'txDropped': '0', 'duplex': 'unknown', 'sampleTime': 1654578133.717023, 'rx': '0', 'state': 'down'}, 'br-int': {'rxErrors': '0', 'txErrors': '0', 'speed': '1000', 'rxDropped': '0', 'na me': 'br-int', 'tx': '0', 'txDropped': '0', 'duplex': 'unknown', 'sampleTime': 1654578133.717023, 'rx': '0', 'state': 'down'}, 'eno1': {'rxErrors': '0', 'txErrors': '0', 'speed': '1000', 'rxDropped': '0', 'name': 'eno1', 'tx': '0', 'txDropped': '0', 'duplex': 'unknown', 'sampleTime': 1654578133.717023, 'rx': '0', 'state': 'down'}, 'eno2': {'rxErrors': '0', 'txErrors': '0', 'speed': '1000', 'rxDropped': '0', 'name': 'eno2', 'tx': '0', 'txDropped': '0', 'duplex': 'unknown', 'sampleTime': 1654578133.717023, 'rx': '0', 'state': 'down'}, 'eno3': {'rxErrors': '0', 'txErrors': '0', 'speed': '1000', 'rxDropped': '1', 'name': 'eno3', 'tx': '42493183527', 'txDropped': '0', 'duplex': 'full', 'sampleTime': 1654578133.717023, 'rx': '20793574', 'state': 'up'}, 'eno4': {'rxErrors': '0', 'txErrors': '0', 'speed': '1000', 'rxDropped': '1', 'name': 'eno4', 'tx': '42273157201', 'txDropped': '0', 'duplex': 'full', 'sampleTime': 1654578133.717023, 'rx': '102734767664', 'state': 'up'}, 'genev_sys_6081': {'rxErrors': '0', 'txErrors': '8', 'speed': '1000', 'rxDropped': '0', 'name': 'genev_sys_6081', 'tx': '0', 'txDropped': '0', 'duplex': 'unknown', 'sampleTime': 1654578133.717023, 'rx': '0', 'state': 'up'}}, 'txDropped': '128', 'anonHugePages': '326', 'ksmPages': 100, 'elapsedTime': '391.12', 'cpuLoad': '0.16', 'cpuSys': '0.18', 'diskStats': {'/var/log': {'free': '6478'}, '/var/run/vdsm/': {'free': '64294'}, '/tmp': {'free': '906'}}, 'cpuUserVdsmd': '0.47', 'netConfigDirty': 'False', 'memCommitted': 28672, 'ksmState': False, 'vmMigrating': 0, 'ksmCpu': 0, 'memAvailable': 126975, 'bootTime': '1654505219', 'haStats': {'active': True, 'configured': True, 'score': 0, 'localMaintenance': True, 'globalMaintenance': False}, 'momStatus': 'active', 'multipathHealth': {}, 'rxDropped': '0', 'outgoingVmMigrations': 0, 'swapTotal': 32191, 'swapFree': 32191, 'hugepages': defaultdict(<type 'dict'>, {1048576: {'resv_hugepages': 0, 'free_hugepages': 0, 'nr_overcommit_hugepages': 0, 'surplus_hugepages ': 0, 'vm.free_hugepages': 0, 'nr_hugepages': 0, 'nr_hugepages_mempolicy': 0}, 2048: {'resv_hugepages': 0, 'free_hugepages': 0, 'nr_overcommit_hugepages': 0, 'surplus_hugepages': 0, 'vm.free_hugepages': 0, 'nr_hugepages': 0, 'nr_hugepages_mempolicy': 0}}), 'dateTime': '2022-06-07T05:02:13 GMT', 'cpuUser': '0.27', 'memFree': 126719, 'cpuIdle': '99.55', 'vmActive': 0, 'v2vJobs': {}, 'cpuSysVdsmd': '0.13'}} from=::1,54704 (api:54) 2022-06-07 05:02:13,726+0000 INFO (jsonrpc/3) [jsonrpc.JsonRpcServer] RPC call Host.getStats succeeded in 0.03 seconds (__init__:312) 2022-06-07 05:02:13,973+0000 INFO (vmrecovery) [vdsm.api] START getConnectedStoragePoolsList(options=None) from=internal, task_id=65629949-c74d-4f66-846a-1e2c40be9157 (api:48) 2022-06-07 05:02:13,973+0000 INFO (vmrecovery) [vdsm.api] FINISH getConnectedStoragePoolsList return={'poollist': []} from=internal, task_id=65629949-c74d-4f66-846a-1e2c40be9157 (api:54) 2022-06-07 05:02:13,973+0000 INFO (vmrecovery) [vds] recovery: waiting for storage pool to go up (clientIF:723) 2022-06-07 05:02:14,468+0000 INFO (jsonrpc/4) [jsonrpc.JsonRpcServer] RPC call Host.ping2 succeeded in 0.00 seconds (__init__:312) 2022-06-07 05:02:14,471+0000 INFO (jsonrpc/5) [vdsm.api] START repoStats(domains=[u'5d6276cc-08ab-47f6-81e7-4e64aac3d386']) from=::1,54704, task_id=6918f715-bbf4-4c7f-af00-1764294ab665 (api:48) 2022-06-07 05:02:14,472+0000 INFO (jsonrpc/5) [vdsm.api] FINISH repoStats return={} from=::1,54704, task_id=6918f715-bbf4-4c7f-af00-1764294ab665 (api:54) 2022-06-07 05:02:14,472+0000 INFO (jsonrpc/5) [jsonrpc.JsonRpcServer] RPC call Host.getStorageRepoStats succeeded in 0.00 seconds (__init__:312) Gluster volumes are all online. systemctl status ovirt-ha-broker: Jun 06 08:47:08 ovirt3.os-s.de systemd[1]: Started oVirt Hosted Engine High Availability Communications Broker. Jun 06 08:47:59 ovirt3.os-s.de ovirt-ha-broker[1524]: ovirt-ha-broker mgmt_bridge.MgmtBridge ERROR Failed to getVdsStats: No 'network' in result Any ideas? Ralf

1 0

Q: How to Fix Frozen "Reboot in progress" VM Status
by Andrei Verovski 06 Jun '22

06 Jun '22

Hi, I have VM which have restarted successfully yet in oVirt web it is being shown with “Rebooting” status for a very long time. I did: su - postgres psql engine select vm_guid from vm_static where vm_name='WInServerTerminal-2022’; engine=# select status from vm_dynamic where vm_guid='7871067f-221c-48ed-a046-f49499ce9be4'; status -------- 10 (1 row) How to properly correct status from "Rebooting”? Thanks in advance Andrei

3 7

Engine Certificate renewal caused "Error while executing action InstallVds: Internal Engine Error"
by Patrick Hibbs 06 Jun '22

06 Jun '22

OK, while I was debugging the other problem, the engine complained about the internal engine cert expiring in three months. As I had to completely shutdown the entire data center (all hosts including the engine itself) at once, I figured I'd go ahead and renew the engine cert so I wouldn't have to do it later. First problem: engine-setup won't renew the engine certificate (engine.cer / engine.p12) at three months out despite the scary warnings in the notifications tab in the Admin WebUI. It can be forced to by renaming the files however. Second problem: After having renewed the engine certificate, the engine can no longer update a host certificate nor (re-)install a host. Giving me the following error in the Admin WebUI: "Error while executing action InstallVds: Internal Engine Error" I've attached the logs from the engine. -Patrick Hibbs

2 2

how to force engine certificate renewal
by Gianluca Cecchi 06 Jun '22

06 Jun '22

Hello, I'm currently still on 4.4.x. Suppose I have an engine certificate expiring on mid August and I want to force renew it now using "engine-setup --offline" command. How can I do it if possible? How many days before expiration I get the message that it is expiring soon with a proposal of renewing it when running "engine-setup"? Thanks, Gianluca

3 6

nec-xhci: Win7 guest no response after ejecting usb storage
by lizhijian＠fujitsu.com 06 Jun '22

06 Jun '22

Hi folks, It's observed that win7 guest will become no response if i eject usb storage "Eject MiniKing" the last screen after i clicked "Eject MingKing": After that, i have to reset the guest. I wonder if it's a known bug, if not any idea to disable this "Eject" . Thanks Zhijian

1 0

Win Server 2022 and virtio-win 0.1.217-2 update failure
by Andrei Verovski 06 Jun '22

06 Jun '22

Hi, I'm trying to update to latest virtio-win 0.1.217-2, and install fails no matter what, even with real-time protection off. Log file attached. Currently installed: QEMU Guest Agent 103.00 Virtio-guest-tools 0.1.215 How to fix this? Thanks in advance. # ------------------------------------------ [1B1C:1B10][2022-06-03T21:05:41]i001: Burn v3.11.1.2318, Windows v10.0 (Build 20348: Service Pack 0), path: C:\Windows\Temp\{1EACBA77-7C5A-4A47-AAD4-09AFFBAD6EB5}\.cr\virtio-win-guest-tools.exe [1B1C:1B10][2022-06-03T21:05:41]i009: Command Line: '-burn.clean.room=E:\virtio-win-guest-tools.exe -burn.filehandle.attached=568 -burn.filehandle.self=556' [1B1C:1B10][2022-06-03T21:05:41]i000: Setting string variable 'WixBundleOriginalSource' to value 'E:\virtio-win-guest-tools.exe' [1B1C:1B10][2022-06-03T21:05:41]i000: Setting string variable 'WixBundleOriginalSourceFolder' to value 'E:\' [1B1C:1B10][2022-06-03T21:05:41]i000: Setting string variable 'WixBundleLog' to value 'C:\Users\ADMINI~1\AppData\Local\Temp\Virtio-win-guest-tools_20220603210541.log' [1B1C:1B10][2022-06-03T21:05:41]i000: Setting string variable 'WixBundleName' to value 'Virtio-win-guest-tools' [1B1C:1B10][2022-06-03T21:05:41]i000: Setting string variable 'WixBundleManufacturer' to value 'Red Hat, Inc.' [1B1C:1AFC][2022-06-03T21:05:41]i000: Setting numeric variable 'WixStdBALanguageId' to value 1033 [1B1C:1AFC][2022-06-03T21:05:41]i000: Setting version variable 'WixBundleFileVersion' to value '0.1.217.0' [1B1C:1B10][2022-06-03T21:05:41]i100: Detect begin, 8 packages [1B1C:1B10][2022-06-03T21:05:41]i102: Detected related bundle: {e2e8124d-6dd0-4247-bde0-6a6bccde383d}, type: Upgrade, scope: PerMachine, version: 0.1.215.0, operation: MajorUpgrade [1B1C:1B10][2022-06-03T21:05:41]i103: Detected related package: {32A54EE9-1358-4C7B-8468-C5BB42F4F849}, scope: PerMachine, version: 0.1.215.0, language: 0 operation: MajorUpgrade [1B1C:1B10][2022-06-03T21:05:41]w120: Detected partially cached package: vwi.Ix1j, invalid payload: vwi.Ix1j, reason: 0x80070002 [1B1C:1B10][2022-06-03T21:05:41]w120: Detected partially cached package: vwi.lFj1, invalid payload: vwi.lFj1, reason: 0x80070002 [1B1C:1B10][2022-06-03T21:05:41]i103: Detected related package: {EE3877E4-07B0-41F2-ADB8-B45133DDCE37}, scope: PerMachine, version: 0.10.5.0, language: 0 operation: Downgrade [1B1C:1B10][2022-06-03T21:05:41]w120: Detected partially cached package: vwi.yP23, invalid payload: vwi.yP23, reason: 0x80070570 [1B1C:1B10][2022-06-03T21:05:41]i101: Detected package: virtio_win_gt_x64.msi, state: Absent, cached: None [1B1C:1B10][2022-06-03T21:05:41]i104: Detected package: virtio_win_gt_x64.msi, feature: FE_balloon_driver, state: Absent [1B1C:1B10][2022-06-03T21:05:41]i104: Detected package: virtio_win_gt_x64.msi, feature: FE_network_driver, state: Absent [1B1C:1B10][2022-06-03T21:05:41]i104: Detected package: virtio_win_gt_x64.msi, feature: FE_pvpanic_driver, state: Absent [1B1C:1B10][2022-06-03T21:05:41]i104: Detected package: virtio_win_gt_x64.msi, feature: FE_qemufwcfg_driver, state: Absent [1B1C:1B10][2022-06-03T21:05:41]i104: Detected package: virtio_win_gt_x64.msi, feature: FE_qemupciserial_driver, state: Absent [1B1C:1B10][2022-06-03T21:05:41]i104: Detected package: virtio_win_gt_x64.msi, feature: FE_vioinput_driver, state: Absent [1B1C:1B10][2022-06-03T21:05:41]i104: Detected package: virtio_win_gt_x64.msi, feature: FE_viorng_driver, state: Absent [1B1C:1B10][2022-06-03T21:05:41]i104: Detected package: virtio_win_gt_x64.msi, feature: FE_vioscsi_driver, state: Absent [1B1C:1B10][2022-06-03T21:05:41]i104: Detected package: virtio_win_gt_x64.msi, feature: FE_vioserial_driver, state: Absent [1B1C:1B10][2022-06-03T21:05:41]i104: Detected package: virtio_win_gt_x64.msi, feature: FE_viostor_driver, state: Absent [1B1C:1B10][2022-06-03T21:05:41]i104: Detected package: virtio_win_gt_x64.msi, feature: FE_viofs_driver, state: Absent [1B1C:1B10][2022-06-03T21:05:41]i104: Detected package: virtio_win_gt_x64.msi, feature: FE_viogpudo_driver, state: Absent [1B1C:1B10][2022-06-03T21:05:41]i101: Detected package: virtio_win_gt_x86.msi, state: Absent, cached: None [1B1C:1B10][2022-06-03T21:05:41]i104: Detected package: virtio_win_gt_x86.msi, feature: FE_balloon_driver, state: Absent [1B1C:1B10][2022-06-03T21:05:41]i104: Detected package: virtio_win_gt_x86.msi, feature: FE_network_driver, state: Absent [1B1C:1B10][2022-06-03T21:05:41]i104: Detected package: virtio_win_gt_x86.msi, feature: FE_pvpanic_driver, state: Absent [1B1C:1B10][2022-06-03T21:05:41]i104: Detected package: virtio_win_gt_x86.msi, feature: FE_qemufwcfg_driver, state: Absent [1B1C:1B10][2022-06-03T21:05:41]i104: Detected package: virtio_win_gt_x86.msi, feature: FE_qemupciserial_driver, state: Absent [1B1C:1B10][2022-06-03T21:05:41]i104: Detected package: virtio_win_gt_x86.msi, feature: FE_vioinput_driver, state: Absent [1B1C:1B10][2022-06-03T21:05:41]i104: Detected package: virtio_win_gt_x86.msi, feature: FE_viorng_driver, state: Absent [1B1C:1B10][2022-06-03T21:05:41]i104: Detected package: virtio_win_gt_x86.msi, feature: FE_vioscsi_driver, state: Absent [1B1C:1B10][2022-06-03T21:05:41]i104: Detected package: virtio_win_gt_x86.msi, feature: FE_vioserial_driver, state: Absent [1B1C:1B10][2022-06-03T21:05:41]i104: Detected package: virtio_win_gt_x86.msi, feature: FE_viostor_driver, state: Absent [1B1C:1B10][2022-06-03T21:05:41]i104: Detected package: virtio_win_gt_x86.msi, feature: FE_viofs_driver, state: Absent [1B1C:1B10][2022-06-03T21:05:41]i104: Detected package: virtio_win_gt_x86.msi, feature: FE_viogpudo_driver, state: Absent [1B1C:1B10][2022-06-03T21:05:41]i101: Detected package: vwi.Ix1j, state: Present, cached: Partial [1B1C:1B10][2022-06-03T21:05:41]i101: Detected package: vwi.0WRk, state: Absent, cached: None [1B1C:1B10][2022-06-03T21:05:41]i101: Detected package: vwi.lFj1, state: Present, cached: Partial [1B1C:1B10][2022-06-03T21:05:41]i101: Detected package: vwi.LdF5, state: Obsolete, cached: None [1B1C:1B10][2022-06-03T21:05:41]i101: Detected package: vwi.yP23, state: Present, cached: Partial [1B1C:1B10][2022-06-03T21:05:41]i101: Detected package: vwi.KAWK, state: Absent, cached: None [1B1C:1B10][2022-06-03T21:05:41]i199: Detect complete, result: 0x0 [1B1C:1AFC][2022-06-03T21:05:43]i000: Setting numeric variable 'EulaAcceptCheckbox' to value 1 [1B1C:1B10][2022-06-03T21:05:43]i200: Plan begin, 8 packages, action: Install [1B1C:1B10][2022-06-03T21:05:43]i052: Condition 'VersionNT64' evaluates to true. [1B1C:1B10][2022-06-03T21:05:43]i204: Plan 12 msi features for package: virtio_win_gt_x64.msi [1B1C:1B10][2022-06-03T21:05:43]i203: Planned feature: FE_balloon_driver, state: Absent, default requested: Unknown, ba requested: Unknown, execute action: None, rollback action: None [1B1C:1B10][2022-06-03T21:05:43]i203: Planned feature: FE_network_driver, state: Absent, default requested: Unknown, ba requested: Unknown, execute action: None, rollback action: None [1B1C:1B10][2022-06-03T21:05:43]i203: Planned feature: FE_pvpanic_driver, state: Absent, default requested: Unknown, ba requested: Unknown, execute action: None, rollback action: None [1B1C:1B10][2022-06-03T21:05:43]i203: Planned feature: FE_qemufwcfg_driver, state: Absent, default requested: Unknown, ba requested: Unknown, execute action: None, rollback action: None [1B1C:1B10][2022-06-03T21:05:43]i203: Planned feature: FE_qemupciserial_driver, state: Absent, default requested: Unknown, ba requested: Unknown, execute action: None, rollback action: None [1B1C:1B10][2022-06-03T21:05:43]i203: Planned feature: FE_vioinput_driver, state: Absent, default requested: Unknown, ba requested: Unknown, execute action: None, rollback action: None [1B1C:1B10][2022-06-03T21:05:43]i203: Planned feature: FE_viorng_driver, state: Absent, default requested: Unknown, ba requested: Unknown, execute action: None, rollback action: None [1B1C:1B10][2022-06-03T21:05:43]i203: Planned feature: FE_vioscsi_driver, state: Absent, default requested: Unknown, ba requested: Unknown, execute action: None, rollback action: None [1B1C:1B10][2022-06-03T21:05:43]i203: Planned feature: FE_vioserial_driver, state: Absent, default requested: Unknown, ba requested: Unknown, execute action: None, rollback action: None [1B1C:1B10][2022-06-03T21:05:43]i203: Planned feature: FE_viostor_driver, state: Absent, default requested: Unknown, ba requested: Unknown, execute action: None, rollback action: None [1B1C:1B10][2022-06-03T21:05:43]i203: Planned feature: FE_viofs_driver, state: Absent, default requested: Unknown, ba requested: Unknown, execute action: None, rollback action: None [1B1C:1B10][2022-06-03T21:05:43]i203: Planned feature: FE_viogpudo_driver, state: Absent, default requested: Unknown, ba requested: Unknown, execute action: None, rollback action: None [1B1C:1B10][2022-06-03T21:05:43]i000: Setting string variable 'WixBundleRollbackLog_virtio_win_gt_x64.msi' to value 'C:\Users\ADMINI~1\AppData\Local\Temp\Virtio-win-guest-tools_20220603210541_000_virtio_win_gt_x64.msi_rollback.log' [1B1C:1B10][2022-06-03T21:05:43]i000: Setting string variable 'WixBundleLog_virtio_win_gt_x64.msi' to value 'C:\Users\ADMINI~1\AppData\Local\Temp\Virtio-win-guest-tools_20220603210541_000_virtio_win_gt_x64.msi.log' [1B1C:1B10][2022-06-03T21:05:43]i052: Condition 'NOT VersionNT64' evaluates to false. [1B1C:1B10][2022-06-03T21:05:43]i204: Plan 12 msi features for package: virtio_win_gt_x86.msi [1B1C:1B10][2022-06-03T21:05:43]i203: Planned feature: FE_balloon_driver, state: Absent, default requested: Unknown, ba requested: Unknown, execute action: None, rollback action: None [1B1C:1B10][2022-06-03T21:05:43]i203: Planned feature: FE_network_driver, state: Absent, default requested: Unknown, ba requested: Unknown, execute action: None, rollback action: None [1B1C:1B10][2022-06-03T21:05:43]i203: Planned feature: FE_pvpanic_driver, state: Absent, default requested: Unknown, ba requested: Unknown, execute action: None, rollback action: None [1B1C:1B10][2022-06-03T21:05:43]i203: Planned feature: FE_qemufwcfg_driver, state: Absent, default requested: Unknown, ba requested: Unknown, execute action: None, rollback action: None [1B1C:1B10][2022-06-03T21:05:43]i203: Planned feature: FE_qemupciserial_driver, state: Absent, default requested: Unknown, ba requested: Unknown, execute action: None, rollback action: None [1B1C:1B10][2022-06-03T21:05:43]i203: Planned feature: FE_vioinput_driver, state: Absent, default requested: Unknown, ba requested: Unknown, execute action: None, rollback action: None [1B1C:1B10][2022-06-03T21:05:43]i203: Planned feature: FE_viorng_driver, state: Absent, default requested: Unknown, ba requested: Unknown, execute action: None, rollback action: None [1B1C:1B10][2022-06-03T21:05:43]i203: Planned feature: FE_vioscsi_driver, state: Absent, default requested: Unknown, ba requested: Unknown, execute action: None, rollback action: None [1B1C:1B10][2022-06-03T21:05:43]i203: Planned feature: FE_vioserial_driver, state: Absent, default requested: Unknown, ba requested: Unknown, execute action: None, rollback action: None [1B1C:1B10][2022-06-03T21:05:43]i203: Planned feature: FE_viostor_driver, state: Absent, default requested: Unknown, ba requested: Unknown, execute action: None, rollback action: None [1B1C:1B10][2022-06-03T21:05:43]i203: Planned feature: FE_viofs_driver, state: Absent, default requested: Unknown, ba requested: Unknown, execute action: None, rollback action: None [1B1C:1B10][2022-06-03T21:05:43]i203: Planned feature: FE_viogpudo_driver, state: Absent, default requested: Unknown, ba requested: Unknown, execute action: None, rollback action: None [1B1C:1B10][2022-06-03T21:05:43]i052: Condition 'VersionNT64 AND VersionNT64 > v8' evaluates to true. [1B1C:1B10][2022-06-03T21:05:43]i052: Condition 'NOT VersionNT64 AND VersionNT > v8' evaluates to false. [1B1C:1B10][2022-06-03T21:05:43]i052: Condition 'VersionNT64' evaluates to true. [1B1C:1B10][2022-06-03T21:05:43]i052: Condition 'VersionNT64' evaluates to true. [1B1C:1B10][2022-06-03T21:05:43]i052: Condition 'NOT VersionNT64' evaluates to false. [1B1C:1B10][2022-06-03T21:05:43]i201: Planned package: virtio_win_gt_x64.msi, state: Absent, default requested: Present, ba requested: Present, execute: Install, rollback: Uninstall, cache: Yes, uncache: No, dependency: Register [1B1C:1B10][2022-06-03T21:05:43]i201: Planned package: virtio_win_gt_x86.msi, state: Absent, default requested: Absent, ba requested: Absent, execute: None, rollback: None, cache: No, uncache: No, dependency: None [1B1C:1B10][2022-06-03T21:05:43]i201: Planned package: vwi.Ix1j, state: Present, default requested: Present, ba requested: Present, execute: None, rollback: None, cache: No, uncache: No, dependency: Register [1B1C:1B10][2022-06-03T21:05:43]i201: Planned package: vwi.0WRk, state: Absent, default requested: Absent, ba requested: Absent, execute: None, rollback: None, cache: No, uncache: No, dependency: None [1B1C:1B10][2022-06-03T21:05:43]i201: Planned package: vwi.lFj1, state: Present, default requested: Present, ba requested: Present, execute: None, rollback: None, cache: No, uncache: No, dependency: Register [1B1C:1B10][2022-06-03T21:05:43]i201: Planned package: vwi.LdF5, state: Obsolete, default requested: None, ba requested: None, execute: None, rollback: None, cache: No, uncache: No, dependency: None [1B1C:1B10][2022-06-03T21:05:43]i201: Planned package: vwi.yP23, state: Present, default requested: Present, ba requested: Present, execute: None, rollback: None, cache: No, uncache: No, dependency: Register [1B1C:1B10][2022-06-03T21:05:43]i201: Planned package: vwi.KAWK, state: Absent, default requested: Absent, ba requested: Absent, execute: None, rollback: None, cache: No, uncache: No, dependency: None [1B1C:1B10][2022-06-03T21:05:43]i207: Planned related bundle: {e2e8124d-6dd0-4247-bde0-6a6bccde383d}, type: Upgrade, default requested: Absent, ba requested: Absent, execute: Uninstall, rollback: Install, dependency: None [1B1C:1B10][2022-06-03T21:05:43]i299: Plan complete, result: 0x0 [1B1C:1B10][2022-06-03T21:05:43]i300: Apply begin [1B1C:1B10][2022-06-03T21:05:43]i010: Launching elevated engine process. [1B1C:1B10][2022-06-03T21:05:44]i011: Launched elevated engine process. [1B1C:1B10][2022-06-03T21:05:44]i012: Connected to elevated engine. [0AFC:2EFC][2022-06-03T21:05:44]i358: Pausing automatic updates. [0AFC:2EFC][2022-06-03T21:05:44]i359: Paused automatic updates. [0AFC:2EFC][2022-06-03T21:05:44]i360: Creating a system restore point. [0AFC:2EFC][2022-06-03T21:05:44]i362: System restore disabled, system restore point not created. [0AFC:2EFC][2022-06-03T21:05:44]i370: Session begin, registration key: SOFTWARE\Microsoft\Windows\CurrentVersion\Uninstall\{a7780eac-f2df-4623-bd96-3727ef667bba}, options: 0x7, disable resume: No [0AFC:2EFC][2022-06-03T21:05:44]i000: Caching bundle from: 'C:\Windows\Temp\{296E0C9D-39FC-415E-A527-2A9568D0E6F4}\.be\virtio-win-guest-tools.exe' to: 'C:\ProgramData\Package Cache\{a7780eac-f2df-4623-bd96-3727ef667bba}\virtio-win-guest-tools.exe' [0AFC:2EFC][2022-06-03T21:05:44]i320: Registering bundle dependency provider: {a7780eac-f2df-4623-bd96-3727ef667bba}, version: 0.1.217.0 [0AFC:2EFC][2022-06-03T21:05:44]i371: Updating session, registration key: SOFTWARE\Microsoft\Windows\CurrentVersion\Uninstall\{a7780eac-f2df-4623-bd96-3727ef667bba}, resume: Active, restart initiated: No, disable resume: No [0AFC:1868][2022-06-03T21:05:45]i305: Verified acquired payload: virtio_win_gt_x64.msi at path: C:\ProgramData\Package Cache\.unverified\virtio_win_gt_x64.msi, moving to: C:\ProgramData\Package Cache\{FFFCF18E-E05C-49F1-9995-6658638EE25D}v0.1.217\virtio-win-gt-x64.msi. [0AFC:2EFC][2022-06-03T21:05:45]i323: Registering package dependency provider: {FFFCF18E-E05C-49F1-9995-6658638EE25D}, version: 0.1.217, package: virtio_win_gt_x64.msi [0AFC:2EFC][2022-06-03T21:05:45]i301: Applying execute package: virtio_win_gt_x64.msi, action: Install, path: C:\ProgramData\Package Cache\{FFFCF18E-E05C-49F1-9995-6658638EE25D}v0.1.217\virtio-win-gt-x64.msi, arguments: ' ARPSYSTEMCOMPONENT="1" MSIFASTINSTALL="7"' [0AFC:2EFC][2022-06-03T21:06:11]e000: Error 0x80070643: Failed to install MSI package. [0AFC:2EFC][2022-06-03T21:06:11]e000: Error 0x80070643: Failed to execute MSI package. [1B1C:1B10][2022-06-03T21:06:11]e000: Error 0x80070643: Failed to configure per-machine MSI package. [1B1C:1B10][2022-06-03T21:06:11]i319: Applied execute package: virtio_win_gt_x64.msi, result: 0x80070643, restart: None [1B1C:1B10][2022-06-03T21:06:11]e000: Error 0x80070643: Failed to execute MSI package. [0AFC:2EFC][2022-06-03T21:06:11]i318: Skipped rollback of package: virtio_win_gt_x64.msi, action: Uninstall, already: Absent [1B1C:1B10][2022-06-03T21:06:11]i319: Applied rollback package: virtio_win_gt_x64.msi, result: 0x0, restart: None [0AFC:2EFC][2022-06-03T21:06:11]i329: Removed package dependency provider: {FFFCF18E-E05C-49F1-9995-6658638EE25D}, package: virtio_win_gt_x64.msi [0AFC:2EFC][2022-06-03T21:06:11]i351: Removing cached package: virtio_win_gt_x64.msi, from path: C:\ProgramData\Package Cache\{FFFCF18E-E05C-49F1-9995-6658638EE25D}v0.1.217\ [0AFC:2EFC][2022-06-03T21:06:11]i372: Session end, registration key: SOFTWARE\Microsoft\Windows\CurrentVersion\Uninstall\{a7780eac-f2df-4623-bd96-3727ef667bba}, resume: None, restart: None, disable resume: No [0AFC:2EFC][2022-06-03T21:06:11]i330: Removed bundle dependency provider: {a7780eac-f2df-4623-bd96-3727ef667bba} [0AFC:2EFC][2022-06-03T21:06:11]i352: Removing cached bundle: {a7780eac-f2df-4623-bd96-3727ef667bba}, from path: C:\ProgramData\Package Cache\{a7780eac-f2df-4623-bd96-3727ef667bba}\ [0AFC:2EFC][2022-06-03T21:06:11]i371: Updating session, registration key: SOFTWARE\Microsoft\Windows\CurrentVersion\Uninstall\{a7780eac-f2df-4623-bd96-3727ef667bba}, resume: None, restart initiated: No, disable resume: No [1B1C:1B10][2022-06-03T21:06:11]i399: Apply complete, result: 0x80070643, restart: None, ba requested restart: No

2 1

CPU_TYPE_UNSUPPORTED_IN_THIS_CLUSTER_VERSION
by Pavel Mlčoch 06 Jun '22

06 Jun '22

I'm trying install ovirt cluster with ovirt-engine as VM on server1(currently rocky8.6) and ovirt-node as server2. I create cluster and try add server2, but I always get unsupported CPU type error. I tried edit cluster with CPU Types AMD opterons, Intel IvyB, Intel Nahalem and error stays the same. Know anybody which CPU type can be used for Athlon5350 or how to solve this error? oVirt Open Virtualization Manager Software Version:4.5.0.8-1.el8 VARIANT="oVirt Node 4.5.0.3" INFO [org.ovirt.engine.core.vdsbroker.monitoring.HostMonitoring] (EE- ManagedScheduledExecutorService-engineScheduledThreadPool-Thread-45) [7501128a] Host 'server2.pavkamlc.cz'(f8c2d5e5-5177-42cc-a406- 599efbe3b32a) is already in NonOperational status for reason 'CPU_TYPE_UNSUPPORTED_IN_THIS_CLUSTER_VERSION'. SetNonOperationalVds command is skipped. model name : AMD Athlon(tm) 5350 APU with Radeon(tm) R3 flags : fpu vme de pse tsc msr pae mce cx8 apic sep mtrr pge mca cmov pat pse36 clflush mmx fxsr sse sse2 ht syscall nx mmxext fxsr_opt pdpe1gb rdtscp lm constant_tsc rep_good nopl nonstop_tsc cpuid extd_apicid aperfmperf pni pclmulqdq monitor ssse3 cx16 sse4_1 sse4_2 movbe popcnt aes xsave avx f16c lahf_lm cmp_legacy svm extapic cr8_legacy abm sse4a misalignsse 3dnowprefetch osvw ibs skinit wdt topoext perfctr_nb bpext perfctr_llc hw_pstate proc_feedback ssbd vmmcall bmi1 xsaveopt arat npt lbrv svm_lock nrip_save tsc_scale flushbyasid decodeassists pausefilter pfthreshold overflow_recov

3 3

Cannot access ovirt portal behind port-forwarding
by ayman＠tarf.io 06 Jun '22

06 Jun '22

Hi, I have installed a standalone ovirt VM on KVM , I made port-forwarding to the VM so when I use http://name:9998 it will forward to the VM on port 80, the issue is that i get forwarded agian to the default name "http://name/ovirt-engine and I cannot access the portal. is there away to resolve this issue or change this behavior ?

1 0

before_vdsm_start hook will be saved ?
by lizhijian＠fujitsu.com 06 Jun '22

06 Jun '22

Hey folks I have a ovirt-4.3 environment, where i have a before_vdsm_start hook to change default usb2 device to usb3(qemu-xhci). But i noticed that the vm still uses the usb3 device after i removed the before_vdsm_start hook. So i wonder if the xml updated by before_vdsm_start hook will be saved the disk/databse, if so how i can reset it to the original. Thanks

1 1

install new ovirt 4.4 host
by Jiří Sléžka 04 Jun '22

04 Jun '22

Hi, I am around to install new ovirt 4.4 host on RockyLinux8 based server. I just install yum install http://mirror.slu.cz/ovirt/yum-repo/ovirt-release44.rpm but when I search for (for example) qemu-kvm package yum info qemu-kvm I can see only qemu-kvm-6.2.0-11.module+el8.6.0+847+b490afdd from appstream (RockyLinux slips to 8.6). It looks like ovirt-4.4-centos-advanced-virtualization repo does not have qemu-kvm at all yum list | grep ovirt-4.4-centos-advanced-virtualization Shoud I try to install new host from manager and hope or should I do some magic before? Thanks in advance, Jiri

1 2

why ovirtmgmt requires untagged network
by ravi k 03 Jun '22

03 Jun '22

Hello, I tried searching the archives but couldn't find anything related, so posting a new thread. When adding a new host to the cluster, why do we need to assign the IP on an untagged network in the bond? It is failing with error in setupnetworks when we tried it with the IP assigned to a vlan interface over the bond. For context, the cluster is an ovs-switch based cluster. It works when added to a traditional linux bridge based cluster. Can someone throw some light on this please? Regards, Ravi

1 0

HA VM Lease failure with full data storage domain
by Patrick Hibbs 02 Jun '22

02 Jun '22

OK, so the data storage domain on a cluster filled up to the point that the OS refused to allocate any more space. This happened because I tried to create a new prealloc'd disk from the Admin WebUI. The disk creation claims to be completed successfully, I've not tried to use that disk yet, but due to a timeout with the storage domain in question the engine began trying to fence all of the HA VMs. The fencing failed for all of the HA VMs leaving them in a powered off state. Despite all of the HA VMs being up at the time, so no reallocation of the leases should have been necessary. Attempting to restart them manually from the Admin WebUI failed. With the original host they were running on complaining about "no space left on device", and the other hosts claiming that the original host still held the VM lease. After cleaning up some old snapshots, the HA VMs would still not boot. Toggling the High Availability setting for each one and allowing the lease to be removed from the storage domain was required to get the VMs to start again. Re-enabling the High Availability setting there after fixed the lease issue. But now some, not all, of the HA VMs are still throwing "no space left on device" errors when attempting to start them. The others are working just fine even with their HA lease enabled. My questions are: 1. Why does oVirt claim to have a constantly allocated HA VM lease on the storage domain when it's clearly only done while the VM is running? 2. Why does oVirt deallocate the HA VM lease when performing a fencing operation? 3. Why can't oVirt clear the old HA VM lease when the VM is down and the storage pool has space available? (How much space is even needed? The leases section of the storage domain in the Admin WebUI doesn't contain any useful info beyond the fact that a lease should exist for a VM even when it's off.) 4. Is there a better way to force start a HA VM when the lease is old and the VM is powered off? 5. Should I file a bug on the whole HA VM failing to reacquire a lease on a full storage pool? -Patrick Hibbs

2 6

qemu-xhci + win7
by lizhijian＠fujitsu.com 02 Jun '22

02 Jun '22

Hi folks I'm trying to enable usb3.0 for all guest. Currently, win10 and linux guest work well with qemu-xhci device while win7sp1 doesn't I have googled usb3 drivers for win7, but none of them match qemu-xhci. So any body know where i can download this driver for win7 [cid:part1.0dnShC07.J0KywPNb@fujitsu.com] [cid:part2.5HjJSU84.6EjOQ0mv@fujitsu.com] Thanks Zhijian

3 2

4.4.10 -> 4.5.0 upgrade fails
by Sketch 02 Jun '22

02 Jun '22

Just tried to upgrade my engine from 4.4.10.6 -> 4.5.0 on CentOS Stream 8. I relized I missed the step of updating to the very latest version of 4.4.10 around the same time it failed and left things in somewhat of a bad state, so I just built a new Rocky 8.6 host and restored my backup there. Update from 4.4.10.6 to 4.4.10.7 there went fine. 4.4.10.7 -> 4.5.0 fails with the same error, however: [ ERROR ] schema.sh: FATAL: Cannot execute sql command: --file=/usr/share/ovirt-engine/dbscripts/upgrade/pre_upgrade/0000_config.sql [ ERROR ] Failed to execute stage 'Misc configuration': Engine schema refresh failed This appears to be the relevant section of the log: psql:/usr/share/ovirt-engine/dbscripts/upgrade/pre_upgrade/0000_config.sql:22: NOTICE: column "default_value" of relation "vdc_options" already exists, skipping psql:/usr/share/ovirt-engine/dbscripts/upgrade/pre_upgrade/0000_config.sql:1454: ERROR: column "default_value" contains null values CONTEXT: SQL statement "ALTER TABLE vdc_options ALTER COLUMN default_value SET NOT NULL" PL/pgSQL function fn_db_change_column_null(character varying,character varying,boolean) line 10 at EXECUTE FATAL: Cannot execute sql command: --file=/usr/share/ovirt-engine/dbscripts/upgrade/pre_upgrade/0000_config.sql 2022-06-01 01:10:03,015-0700 ERROR otopi.plugins.ovirt_engine_setup.ovirt_engine.db.schema schema._misc:530 schema.sh: FATAL: Cannot execute sql command: --file=/usr/share/ovirt-engine/dbscripts/upgrade/pre_upgrade/0000_config.sql 2022-06-01 01:10:03,016-0700 DEBUG otopi.context context._executeMethod:145 method exception Traceback (most recent call last): File "/usr/lib/python3.6/site-packages/otopi/context.py", line 132, in _executeMethod method['method']() File "/usr/share/ovirt-engine/setup/bin/../plugins/ovirt-engine-setup/ovirt-engine/db/schema.py", line 532, in _misc raise RuntimeError(_('Engine schema refresh failed')) RuntimeError: Engine schema refresh failed 2022-06-01 01:10:03,017-0700 ERROR otopi.context context._executeMethod:154 Failed to execute stage 'Misc configuration': Engine schema refresh failed Any ideas?

2 2

Updating oVirt Node
by Jonas 02 Jun '22

02 Jun '22

Hello all For Updates to the oVirt Node hosts I have seen two options in the documentation: - https://ovirt.org/documentation/administration_guide/index.html#Updating_al… - https://ovirt.org/documentation/administration_guide/index.html#Updating_In… Does anyone here have experiences with one or the other strategy or any recommendation of one over the other? I would like to update the hosts in intervals and not all at once, but as far as I understand it is only possible to update to the latest packages at the time of the update. Or is there a way to define something like a baseline for nodes using oVirt Engine? Thank you for your valuable input! Jonas

1 0

New Template dialog, clicking OK in one environment results in template creation, the other environment does nothing.
by mblecha＠flagshipbio.com 01 Jun '22

01 Jun '22

We have two oVirt environments configured, one for our Lab and one for Production. In the Lab, we can create templates. Clicking OK results in the New Template dialog disappearing and the VM's disk becomes locked and template creation begins. In the production environment, we cannot create templates. Clicking OK results in nothing, the dialog box remains. Nothing indicates missing/incorrect information. Both environments are on hosted engines, and oVirt Engine version 4.5. I have monitored the logs in the production environment and nothing comes up when the OK button is clicked. I have tested both environments in both Firefox and Chrome, and the behavior is also the same, our lab will progress, but production will simply not budge.

2 3