- Users - oVirt List Archives

hosted-engine -vm-status show a ghost node that is not anymore in the cluster: how to remove?
by Diego Ercolani 08 Sep '22

08 Sep '22

engine 4.5.2.4 The issue is that in my cluster when I use the: [root@ovirt-node3 ~]# hosted-engine --vm-status --== Host ovirt-node3.ovirt (id: 1) status ==-- Host ID : 1 Host timestamp : 1633143 Score : 3400 Engine status : {"vm": "down", "health": "bad", "detail": "unknown", "reason": "vm not running on this host"} Hostname : ovirt-node3.ovirt Local maintenance : False stopped : False crc32 : 1cbfcd19 conf_on_shared_storage : True local_conf_timestamp : 1633143 Status up-to-date : True Extra metadata (valid at timestamp): metadata_parse_version=1 metadata_feature_version=1 timestamp=1633143 (Wed Aug 31 14:37:53 2022) host-id=1 score=3400 vm_conf_refresh_time=1633143 (Wed Aug 31 14:37:53 2022) conf_on_shared_storage=True maintenance=False state=EngineDown stopped=False --== Host ovirt-node1.ovirt (id: 2) status ==-- Host ID : 2 Host timestamp : 373629 Score : 0 Engine status : unknown stale-data Hostname : ovirt-node1.ovirt Local maintenance : True stopped : False crc32 : 12a6eb81 conf_on_shared_storage : True local_conf_timestamp : 373630 Status up-to-date : False Extra metadata (valid at timestamp): metadata_parse_version=1 metadata_feature_version=1 timestamp=373629 (Tue Jun 14 16:48:50 2022) host-id=2 score=0 vm_conf_refresh_time=373630 (Tue Jun 14 16:48:50 2022) conf_on_shared_storage=True maintenance=True state=LocalMaintenance stopped=False --== Host ovirt-node2.ovirt (id: 3) status ==-- Host ID : 3 Host timestamp : 434247 Score : 3400 Engine status : {"vm": "down", "health": "bad", "detail": "unknown", "reason": "vm not running on this host"} Hostname : ovirt-node2.ovirt Local maintenance : False stopped : False crc32 : badb3751 conf_on_shared_storage : True local_conf_timestamp : 434247 Status up-to-date : True Extra metadata (valid at timestamp): metadata_parse_version=1 metadata_feature_version=1 timestamp=434247 (Wed Aug 31 14:37:45 2022) host-id=3 score=3400 vm_conf_refresh_time=434247 (Wed Aug 31 14:37:45 2022) conf_on_shared_storage=True maintenance=False state=EngineDown stopped=False --== Host ovirt-node4.ovirt (id: 4) status ==-- Host ID : 4 Host timestamp : 1646655 Score : 3400 Engine status : {"vm": "up", "health": "good", "detail": "Up"} Hostname : ovirt-node4.ovirt Local maintenance : False stopped : False crc32 : 1a16027e conf_on_shared_storage : True local_conf_timestamp : 1646655 Status up-to-date : True Extra metadata (valid at timestamp): metadata_parse_version=1 metadata_feature_version=1 timestamp=1646655 (Wed Aug 31 14:37:43 2022) host-id=4 score=3400 vm_conf_refresh_time=1646655 (Wed Aug 31 14:37:43 2022) conf_on_shared_storage=True maintenance=False state=EngineUp stopped=False The problem is that ovirt-node1.ovirt is not anymore in ther cluster, in the host list presented by the ui there is correctly no ovirt-node1, the ovirt-node1 appears only in the commandline. I did a full text search in the engine DB, but node1 doesn't appear anywhere, even in the filesystem, a grep doesn't find anything

2 5

Ovirt Windows VM Provisioning with Terraform and Sysprep
by venkatar＠emiratesnbd.com 08 Sep '22

08 Sep '22

Hi All, Did anyone tried automated deployment of windows VM in ovirt using terraform ovirt provider (https://github.com/oVirt/terraform-provider-ovirt) We are able to provision the Linux VM, want to check if anyone tried windows VM. I have sample sysprep script which will assign IP,Set hostname and password. I couldn't find anything related over internet for the same. Thanks Ravi

1 0

Engine and host certificates expired
by simon＠justconnect.ie 08 Sep '22

08 Sep '22

I have an environment where the engine wouldn’t start and the certificate expiry dates were as follows. Host1 - 25th Sep 2022 Host2 - 11th Aug 2022 Host3 - 11th Aug 2022 I copied the vdsm certs from Host1 to Host2 & Host3 Engine then started on Host1 and then backed up. Engine cert expiry 11th Aug 2022 I put the cluster into Global Maintenance mode and then tried: ‘engine-setup —offline’ Which failed as the validation check said the engine wasn’t in Global Maintenance mode even though ‘hosted-engine —vm-status said it was. None of the Hosts are ‘GREEN’ (can’t remember what the status was as I’m writing this from memory) but their status is ’RED’. There are VMs running on the 3 Hosts and I’m reluctant to restart anything at the moment. Is there a way to refresh the engine certificate to get past this or do I need to restart vdsm service on each host to bring them back online as far as the engine is concerned? The environment is currently at 4.4.6 and is to be upgraded to 4.5.2 next month. Any help as always will be greatly appreciated. Kind regards Simon

3 9

oVirt 4.5.2.4 api access with keycloak
by Don Dupuis 07 Sep '22

07 Sep '22

Hello I have installed ovirt 4.5.2.4 and keycloak authentication is now standard. On previous installs it could be accessed via a web browser as https://engine_FQDN/ovirt-engine/api. What is the proper url syntax to work with the new keycloak way and I know the username is now admin@ovirt @internalsso? Thanks Don

4 3

VDSNetworkException: protocol version not accepted by client
by Kilian Ries 07 Sep '22

07 Sep '22

Hi, im running an oVirt 4.2.8 cluster with two nodes. A few days ago, my SSL certificates expired. After that, i changed all the certificates on the engine via "engine-upgrade" command and issued new vdsm client certificates. Then i copied the new certificates to my ovirt nodes and restarted vdsmd (systemctl restart vdsmd). Now i'm still not able to connect to my ovirt nodes. In the engine log i can see the following error: ### 2022-09-01 18:25:51,822+02 INFO [org.ovirt.vdsm.jsonrpc.client.reactors.ReactorClient] (SSL Stomp Reactor) [] Connecting to /192.168.xx.xx 2022-09-01 18:25:51,827+02 ERROR [org.ovirt.vdsm.jsonrpc.client.reactors.Reactor] (SSL Stomp Reactor) [] Unable to process messages The server selected protocol version TLS10 is not accepted by client preferences [TLS12] 2022-09-01 18:25:51,829+02 ERROR [org.ovirt.engine.core.vdsbroker.monitoring.HostMonitoring] (EE-ManagedThreadFactory-engineScheduled-Thread-88) [] Unable to RefreshCapabilities: VDSNetworkException: VDSGenericException: VDSNetworkException: The server selected protocol version TLS10 is not accepted by client preferences [TLS12] ### I searched my vdsm client config but i cannot see any specific TLS version set (every option with TLS is commented - seems to be the default): ### $grep -R -i TLS /etc/vdsm/ /etc/vdsm/vdsm.conf:# ssl_protocol = tlsv1 /etc/vdsm/vdsm.conf:# https://docs.python.org/2/library/ssl.html. e.g. OP_NO_TLSv1, /etc/vdsm/vdsm.conf:# OP_NO_TLSv1_1 By default tlv1, tlsv1.1 and tlsv1.2 are enabled. ### On the engine i didn't find any setting to set a specific TLS version - there seems to have been a setting (VdsmSSLProtocol) but that got deprecated years ago. Does anybody know why my engine is still not able to connect to the client vdsmd? I also tried to set "ssl_protocol = tlsv1" via vdsm.conf but that didn't work ... Thanks Regards, Kilian PS: Name : vdsm Architektur : x86_64 Version : 4.19.37 Ausgabe : 1.el7.centos Name : ovirt-engine Architektur : noarch Version : 4.2.8.2 Ausgabe : 1.el7

2 2

oVirt 4.5.2 - ovirt-hosted-engine-setup fails with "error: Must be number, not str"}]" when creating ovirtmgmt network
by Thomas Simmons 06 Sep '22

06 Sep '22

Hello, I am trying to deploy the latest oVirt (4.5.2), on a fully patched Rocky 8.6 system and am having and issue where "ovirt-hosted-engine-setup" is failing when it tries to create the ovirtmgmt network with the error "error: Must be number, not str"}]". When this happens, the engine setup pauses and if I can login to the bootstrap engine UI and when I attempt to manually assign the ovirtmgmt network to the correct nic on the host, I get the same error message. This server has (2) active network interfaces - a gigabit NIC that will be a VM network for all networks except gluster and migration and a 40Gbps Infiniband adapter in connected mode (IPoIB) for gluster and migration (I previously had these servers in the same hardware configuration running oVirt 4.3 on CentOS 7 and would like to have the same setup again - just with latest versions of EL and oVirt). I don't believe it's related, however for transparency I should note that the server is running kernel-lt from elrepo (5.4.212-1.el8.elrepo.x86_64) because both native EL and elrepo support for my Infiniband HBA was dropped in the standard EL8 kernel due to known bugs with that version of the kernel. Thanks in advance for any assistance. Here is the specific error from engine.log on the bootstrap engine. I see similar messages in vdsm.log on the host. 2022-09-04 18:01:10,725-04 INFO [org.ovirt.engine.core.vdsbroker.vdsbroker.HostSetupNetworksVDSCommand] (EE-ManagedThreadFactory-engine-Thread-1) [2a6921b2] START, HostSetupNetworksVDSCommand(HostName = vmh1.my.domain.com, HostSetupNetworksVdsCommandParameters:{hostId='1def9b77-b268-4a64-bac0-3e51c1d16b10', vds='Host[vmh1.my.domain.com,1def9b77-b268-4a64-bac0-3e51c1d16b10]', rollbackOnFailure='true', commitOnSuccess='true', connectivityTimeout='120', networks='[HostNetwork:{defaultRoute='true', bonding='false', networkName='ovirtmgmt', vdsmName='ovirtmgmt', nicName='enp3s0', vlan='null', vmNetwork='true', stp='false', properties='null', ipv4BootProtocol='STATIC_IP', ipv4Address='10.10.65.101', ipv4Netmask='255.255.255.0', ipv4Gateway='10.10.65.1', ipv6BootProtocol='NONE', ipv6Address='null', ipv6Prefix='null', ipv6Gateway='null', nameServers='null'}]', removedNetworks='[]', bonds='[]', removedBonds='[]', clusterSwitchType='LEGACY', managementNetworkChanged='true'}), log id: 6bc2c376 2022-09-04 18:01:10,726-04 INFO [org.ovirt.engine.core.vdsbroker.vdsbroker.HostSetupNetworksVDSCommand] (EE-ManagedThreadFactory-engine-Thread-1) [2a6921b2] FINISH, HostSetupNetworksVDSCommand, return: , log id: 6bc2c376 2022-09-04 18:01:11,251-04 WARN [org.ovirt.engine.core.vdsbroker.vdsbroker.HostSetupNetworksVDSCommand] (EE-ManagedThreadFactory-engine-Thread-1) [2a6921b2] Unexpected return value: Status [code=-32603, message=Internal JSON-RPC error: {'reason': "Attempt to call function: <bound method Global.setupNetworks of <vdsm.API.Global object at 0x7fb24046f240>> with arguments: ({'ovirtmgmt': {'netmask': '255.255.255.0', 'ipv6autoconf': False, 'nic': 'enp3s0', 'bridged': 'true', 'ipaddr': '10.10.65.101', 'defaultRoute': True, 'dhcpv6': False, 'STP': 'no', 'gateway': '10.10.65.1', 'mtu': 1500, 'switch': 'legacy'}}, {}, {'connectivityTimeout': 120, 'commitOnSuccess': True, 'connectivityCheck': 'true'}) error: Must be number, not str"}] 2022-09-04 18:01:11,252-04 ERROR [org.ovirt.engine.core.vdsbroker.vdsbroker.HostSetupNetworksVDSCommand] (EE-ManagedThreadFactory-engine-Thread-1) [2a6921b2] Failed in 'HostSetupNetworksVDS' method 2022-09-04 18:01:11,252-04 WARN [org.ovirt.engine.core.vdsbroker.vdsbroker.HostSetupNetworksVDSCommand] (EE-ManagedThreadFactory-engine-Thread-1) [2a6921b2] Unexpected return value: Status [code=-32603, message=Internal JSON-RPC error: {'reason': "Attempt to call function: <bound method Global.setupNetworks of <vdsm.API.Global object at 0x7fb24046f240>> with arguments: ({'ovirtmgmt': {'netmask': '255.255.255.0', 'ipv6autoconf': False, 'nic': 'enp3s0', 'bridged': 'true', 'ipaddr': '10.10.65.101', 'defaultRoute': True, 'dhcpv6': False, 'STP': 'no', 'gateway': '10.10.65.1', 'mtu': 1500, 'switch': 'legacy'}}, {}, {'connectivityTimeout': 120, 'commitOnSuccess': True, 'connectivityCheck': 'true'}) error: Must be number, not str"}] 2022-09-04 18:01:11,261-04 ERROR [org.ovirt.engine.core.dal.dbbroker.auditloghandling.AuditLogDirector] (EE-ManagedThreadFactory-engine-Thread-1) [2a6921b2] EVENT_ID: VDS_BROKER_COMMAND_FAILURE(10,802), VDSM vmh1.my.domain.com command HostSetupNetworksVDS failed: Internal JSON-RPC error: {'reason': "Attempt to call function: <bound method Global.setupNetworks of <vdsm.API.Global object at 0x7fb24046f240>> with arguments: ({'ovirtmgmt': {'netmask': '255.255.255.0', 'ipv6autoconf': False, 'nic': 'enp3s0', 'bridged': 'true', 'ipaddr': '10.10.65.101', 'defaultRoute': True, 'dhcpv6': False, 'STP': 'no', 'gateway': '10.10.65.1', 'mtu': 1500, 'switch': 'legacy'}}, {}, {'connectivityTimeout': 120, 'commitOnSuccess': True, 'connectivityCheck': 'true'}) error: Must be number, not str"} 2022-09-04 18:01:11,261-04 ERROR [org.ovirt.engine.core.vdsbroker.vdsbroker.HostSetupNetworksVDSCommand] (EE-ManagedThreadFactory-engine-Thread-1) [2a6921b2] Error: VDSGenericException: VDSErrorException: Failed to HostSetupNetworksVDS, error = Internal JSON-RPC error: {'reason': "Attempt to call function: <bound method Global.setupNetworks of <vdsm.API.Global object at 0x7fb24046f240>> with arguments: ({'ovirtmgmt': {'netmask': '255.255.255.0', 'ipv6autoconf': False, 'nic': 'enp3s0', 'bridged': 'true', 'ipaddr': '10.10.65.101', 'defaultRoute': True, 'dhcpv6': False, 'STP': 'no', 'gateway': '10.10.65.1', 'mtu': 1500, 'switch': 'legacy'}}, {}, {'connectivityTimeout': 120, 'commitOnSuccess': True, 'connectivityCheck': 'true'}) error: Must be number, not str"}, code = -32603 2022-09-04 18:01:11,261-04 ERROR [org.ovirt.engine.core.vdsbroker.vdsbroker.HostSetupNetworksVDSCommand] (EE-ManagedThreadFactory-engine-Thread-1) [2a6921b2] Command 'HostSetupNetworksVDSCommand(HostName = vmh1.my.domain.com, HostSetupNetworksVdsCommandParameters:{hostId='1def9b77-b268-4a64-bac0-3e51c1d16b10', vds='Host[vmh1.my.domain.com,1def9b77-b268-4a64-bac0-3e51c1d16b10]', rollbackOnFailure='true', commitOnSuccess='true', connectivityTimeout='120', networks='[HostNetwork:{defaultRoute='true', bonding='false', networkName='ovirtmgmt', vdsmName='ovirtmgmt', nicName='enp3s0', vlan='null', vmNetwork='true', stp='false', properties='null', ipv4BootProtocol='STATIC_IP', ipv4Address='10.10.65.101', ipv4Netmask='255.255.255.0', ipv4Gateway='10.10.65.1', ipv6BootProtocol='NONE', ipv6Address='null', ipv6Prefix='null', ipv6Gateway='null', nameServers='null'}]', removedNetworks='[]', bonds='[]', removedBonds='[]', clusterSwitchType='LEGACY', managementNetworkChanged='true'})' execution failed: VDSGenericException: VDSErrorException: Failed to HostSetupNetworksVDS, error = Internal JSON-RPC error: {'reason': "Attempt to call function: <bound method Global.setupNetworks of <vdsm.API.Global object at 0x7fb24046f240>> with arguments: ({'ovirtmgmt': {'netmask': '255.255.255.0', 'ipv6autoconf': False, 'nic': 'enp3s0', 'bridged': 'true', 'ipaddr': '10.10.65.101', 'defaultRoute': True, 'dhcpv6': False, 'STP': 'no', 'gateway': '10.10.65.1', 'mtu': 1500, 'switch': 'legacy'}}, {}, {'connectivityTimeout': 120, 'commitOnSuccess': True, 'connectivityCheck': 'true'}) error: Must be number, not str"}, code = -32603 2022-09-04 18:01:11,262-04 INFO [org.ovirt.engine.core.bll.network.host.HostSetupNetworksCommand] (EE-ManagedThreadFactory-engine-Thread-1) [2a6921b2] Host setup networks finished. Lock released. Monitoring can run now for host 'vmh1.my.domain.com' from data-center 'Default' 2022-09-04 18:01:11,262-04 ERROR [org.ovirt.engine.core.bll.network.host.HostSetupNetworksCommand] (EE-ManagedThreadFactory-engine-Thread-1) [2a6921b2] Command 'org.ovirt.engine.core.bll.network.host.HostSetupNetworksCommand' failed: EngineException: org.ovirt.engine.core.vdsbroker.vdsbroker.VDSErrorException: VDSGenericException: VDSErrorException: Failed to HostSetupNetworksVDS, error = Internal JSON-RPC error: {'reason': "Attempt to call function: <bound method Global.setupNetworks of <vdsm.API.Global object at 0x7fb24046f240>> with arguments: ({'ovirtmgmt': {'netmask': '255.255.255.0', 'ipv6autoconf': False, 'nic': 'enp3s0', 'bridged': 'true', 'ipaddr': '10.10.65.101', 'defaultRoute': True, 'dhcpv6': False, 'STP': 'no', 'gateway': '10.10.65.1', 'mtu': 1500, 'switch': 'legacy'}}, {}, {'connectivityTimeout': 120, 'commitOnSuccess': True, 'connectivityCheck': 'true'}) error: Must be number, not str"}, code = -32603 (Failed with error unexpected and code 16) 2022-09-04 18:01:11,263-04 INFO [org.ovirt.engine.core.bll.network.host.HostSetupNetworksCommand] (EE-ManagedThreadFactory-engine-Thread-1) [2a6921b2] Lock freed to object 'EngineLock:{exclusiveLocks='[HOST_NETWORK1def9b77-b268-4a64-bac0-3e51c1d16b10=HOST_NETWORK]', sharedLocks=''}' 2022-09-04 18:01:11,269-04 ERROR [org.ovirt.engine.core.dal.dbbroker.auditloghandling.AuditLogDirector] (EE-ManagedThreadFactory-engine-Thread-1) [2a6921b2] EVENT_ID: SETUP_NETWORK_FAILED_FOR_MANAGEMENT_NETWORK_CONFIGURATION(1,120), Failed to configure management network on host vmh1.my.domain.com due to setup networks failure.

1 1

VM has been paused due to no Storage space error on ovirt 4.5
by Jirka Simon 06 Sep '22

06 Sep '22

Hi there, last days we are facing issues with paused VMs (in past it was for few second to resize lv device), but now it doesn't resume. we migrated to 4.5.2 cluster, this never happened before with the same storage. there is almost notning in engine log 2022-09-06 09:47:11,160+02 INFO [org.ovirt.engine.core.vdsbroker.monitoring.VmAnalyzer] (ForkJoinPool-1-worker-9) [51eb7178] VM 'cfff0648-6502-4977-95a8-c6f95c723f6d'(cm1.util.prod.hq.slde v.cz) moved from 'Up' --> 'Paused' 2022-09-06 09:47:11,264+02 INFO [org.ovirt.engine.core.dal.dbbroker.auditloghandling.AuditLogDirector] (ForkJoinPool-1-worker-9) [51eb7178] EVENT_ID: VM_PAUSED(1,025), VM cm1.util.prod.hq. sldev.cz has been paused. 2022-09-06 09:47:11,271+02 ERROR [org.ovirt.engine.core.dal.dbbroker.auditloghandling.AuditLogDirector] (ForkJoinPool-1-worker-9) [51eb7178] EVENT_ID: VM_PAUSED_ENOSPC(138), VM cm1.util.pro d.hq.sldev.cz has been paused due to no Storage space error. but there are erros with LVM in vdsmlog. (attached) ovirt 4.5.2 OS - ovirt-node-ng. Thank you for any hint. Jirka

1 0

storage domain
by izidor.stokin＠gmail.com 05 Sep '22

05 Sep '22

Hello, One of our admins accidentally added one of available LUN disks to storage domain. Now we are not able to remove it as additional storage from it. There was no data written on it. on host it was removed from vg group (vgreduce) , but on site it is still in use (greyed out) (vgreduce had no issues removing pvdisk from it) multipath -ll shows it as unused, dmsetup status shows some values related to this lun: OS: rhel 7.9 package ovirt 4.3 cluster: yes Any idea how to remove it without corrupting datadomain ? is it safe to remove it with dmsetum remove? BRI

1 0

Some problems with ovirt
by אריה קלטר 03 Sep '22

03 Sep '22

Hi, I have two ovirt clusters. 1. 4.3 cluster with single node (was 3 nodes in the past) 2. 4.5.2 cluster with 3 nodes. the problem was also on oVirt-4.5 node with a matching engine version but the cluster configured on version 4.6 because I did have a problem upgrading one of the nodes, I also lost the engine because of gluster volume sync issue between the hosts, so I formatted the problematic node using the 4.5.2 node iso and created a new engine on it, and attached 2 of the existing nodes and did upgrade on the nodes using the new engine. About the new cluster: 1. migration between hosts - almost all the time looks like it succeeds, but the VM is not responding after the successful migration. 2. run reboot inside of the VM - in a high percent of the time it just hung instead of reboot. With 100% CPU. In status "Reboot in Progress", but after some time the status changed to "Up" incorrectly. I tried on the clean images of ubuntu and centos from ovirt-image-repository. I imported a server from the 4.3 cluster to the new cluster, on the old cluster it reboots correctly but in the new cluster it doesn't. I tried it both with gluster and with nfs. Any idea how to solve both problems? Regards, Arye

2 1

Re: How many oVirt cluster, hosts and VMs do you have running?
by Jonas Lindholm 02 Sep '22

02 Sep '22

I was looking at those studies but not sure how old they are. What I can see is that our oVirt deployment seems bigger than any of those studies but I can be wrong. ------ Original Message ------ Received: 01:08 AM MDT, 09/02/2022 From: Sandro Bonazzola <sbonazzo(a)redhat.com> To: jlhm(a)usa.net Cc: oVirt Users <users(a)ovirt.org> Subject: Re: [ovirt-users] How many oVirt cluster, hosts and VMs do you have running? > Would something like an oVirt counter help? > https://github.com/oVirt/ovirt-site/pull/3056 > > > Il giorno ven 2 set 2022 alle ore 08:26 ha scritto: > > > Hi, just trying to understand how our oVirt deployment compare to others. > > - 75 cluster (and 75 Data centers as we map them 1 to 1 due to security > > requirements) spanning +3 data centers > > - 328 oVirt servers > > - +1900 VMs running > > > > Majority are still on 4.3 (CentOS 7) but our engines (4 of them) runs > > RedHat 8/oVirt 4.4. Working to upgrade all hypervisors to RedHat 8 (or > > Rocky 8)/oVirt 4.4. > > When that is done we will start to upgrade to latest oVirt version but it > > takes time due to the size of our environment and we move slowly to ensure > > stability. > > _______________________________________________ > > Users mailing list -- users(a)ovirt.org > > To unsubscribe send an email to users-leave(a)ovirt.org > > Privacy Statement: https://www.ovirt.org/privacy-policy.html > > oVirt Code of Conduct: > > https://www.ovirt.org/community/about/community-guidelines/ > > List Archives: > > https://lists.ovirt.org/archives/list/users@ovirt.org/message/LM3GRRJDXVMJI… > > > > > -- > > Sandro Bonazzola > > MANAGER, SOFTWARE ENGINEERING, EMEA R&D PERFORMANCE & SCALE > > Red Hat EMEA > > sbonazzo(a)redhat.com > > > *Red Hat respects your work life balance. Therefore there is no need to > answer this email out of your office hours.* >

1 0

How many oVirt cluster, hosts and VMs do you have running?
by jlhm＠usa.net 02 Sep '22

02 Sep '22

Hi, just trying to understand how our oVirt deployment compare to others. - 75 cluster (and 75 Data centers as we map them 1 to 1 due to security requirements) spanning +3 data centers - 328 oVirt servers - +1900 VMs running Majority are still on 4.3 (CentOS 7) but our engines (4 of them) runs RedHat 8/oVirt 4.4. Working to upgrade all hypervisors to RedHat 8 (or Rocky 8)/oVirt 4.4. When that is done we will start to upgrade to latest oVirt version but it takes time due to the size of our environment and we move slowly to ensure stability.

2 1

Async release for ovirt-engine (4.5.2.5) is now available
by Lev Veyde 02 Sep '22

02 Sep '22

The oVirt Team has just released a new version of ovirt-engine package (4.5.2.5) that fixes an important bug: - Bug 2122174 <https://bugzilla.redhat.com/2122174> - engine-setup on separate DWH machine fails: Failed to execute stage 'Closing up': 'NoneType' object has no attribute 'open_sftp_client' The update is already available on resources.ovirt.org and should land on oVirt mirrors within 24 hours. Thanks in advance, -- Lev Veyde Senior Software Engineer, RHCE | RHCVA | MCITP Red Hat Israel <https://www.redhat.com> lev(a)redhat.com | lveyde(a)redhat.com <https://red.ht/sig> TRIED. TESTED. TRUSTED. <https://redhat.com/trusted>

1 0

Ubuntu NFS
by thilburn＠generalpacific.com 01 Sep '22

01 Sep '22

Hello, I was having trouble with getting an Ubuntu 22.04 NFS share working and after searching for hours I was able to figure out what was needed. Below is what I found if anyone else runs into this. My error was engine.log "...Unexpected return value: Status [code=701, message=Could not initialize cluster lock: ()]" Host supervdsm.log -open error -13 EACCES: no permission to open /ThePath/ids -check that daemon user sanlock *** group sanlock *** has access to disk or file. The fix was changing /etc/nfs.conf manage-gids=y ( Which is the default ) to # manage-gids=y ( Commenting this sets the default which is no ) It looks like in the past the fix was to change /etc/default/nfs-kernel-server Line RPCMOUNTDOPTS="--manage-gids" which I didn't need to change.

2 5

how kill backup operation
by Diego Ercolani 01 Sep '22

01 Sep '22

Hello I saw there are other thread asking how to delete disk snapshots from backup operation. We definitively need a tool to kill pending backup operations and locked snapshots. I Think this is very frustrating ovirt is a good piece of software but it's very immature in a dirty asyncronous world. We need a unified toolbox to clean manually and do database housekeeping.

3 12

Interested in contributing with Spanish translation.
by Luis Pereida 01 Sep '22

01 Sep '22

Hello, My name is Luis Pereida, I am Mexican, from Guadalajara and I am currently an application security specialist. Some time ago I met the O-Virt project and it helped me a lot to solve many situations where virtualization was the perfect option. Since some time ago I have been thinking about how to contribute to the project, and talking with some friends, they would like to have documentation in Spanish. Although English is basic for us, many times the context or expressions are hard to understand. I would like to help with that. How can I do it? I see that it is necessary to use a Zanata account. How can I get an account? Regards and thanks for being so supportive of the community.

3 2

Q: Engione 4.4.10 and New Host on CentOS Stream 9
by Andrei Verovski 31 Aug '22

31 Aug '22

Hi, I have engine version 4.4.10.7-1.el8, is it possible to set up new node host on CentOS Stream 9, or I need to upgrade engine to version 4.5 first (which is not right now possible because of some quite old nodes) ? Thanks Andrei

4 11

Moving SelfHosted Engine
by murat.celebi＠dbpro.com.tr 31 Aug '22

31 Aug '22

Hello We have got an Production ovirt installation with 1 datacenter and 2 clusters. Default Cluster has got 2 hosts in it. kvm01 and kvm02. PrimarySite cluster has got 2 hosts in it. kvm03 and kvm04 Our Self hosted engine is running on kvm01. We need to migrate the Selfhosted-engine to kvm03 or kvm04 since the hosts kvm01 and kvm02 hosts are going to be retired. Anyone has any idea how this can be accomplished?

3 2

Failed to delete snapshot (again)
by Giulio Casella 30 Aug '22

30 Aug '22

Hi folks, since some month I have an issue with snapshot removals (I have storware vprotect backup system, heavily using snapshots). After some time spent on a bugzilla (https://bugzilla.redhat.com/show_bug.cgi?id=1948599) we discovered that my issue is not depending on that bug :-( So they pointed me here again. Briefly: sometime snapshot removal fails, leaving snapshot in illegal state. Trying to remove again (via ovirt UI) keeps failing and doesn't solve. The only way to rebuild a consistent situation is live migrating affected disk to another storage domain; after moving the disk, snapshot is no more marked illegal and then I can remove it. You can imagine this is a bit tricky, specially for large disks. In my logs I can find: 2022-08-29 09:17:11,890+02 ERROR [org.ovirt.engine.core.bll.MergeStatusCommand] (EE-ManagedExecutorService-commandCoordinator-Thread-1) [0eced56f-689d-422b-b15c-20b824377b08] Failed to live merge. Top volume f8f84b1c-53ab-4c99-a01d-743ed3d7859b is still in qemu chain [0ea89fbc-d39a-48ff-aa2b-0381d79d7714, 55bb387f-01a6-41b6-b585-4bcaf2ea5e32, f8f84b1c-53ab-4c99-a01d-743ed3d7859b] My setup is ovirt-engine-4.5.2.4-1.el8.noarch, with hypervisors based on oVirt Node 4.5.2 (vdsm-4.50.2.2-1.el8). Thank you in advance. Regards, gc

3 4

Ovirt 4.4.7, can't renew certificate of ovirt engine (certificates expired)
by vk＠itiviti.com 30 Aug '22

30 Aug '22

Hi Team, I'm looking for your help since I didn't find any clear documentation. Is there somewhere in ovirt website a clear documentation about how to renew the engine certificates located in /etc/pki/ovirt-engine/certs/ We have an engine GUI not working, showing error message "PKIX path validation failed: java.security.cert.CertPathValidatorException: validity check failed". After checking, all the cert in /etc/pki/ovirt-engine/certs/ are expired. I didn't find a clear documentation on ovirt website, or even on redhat website (it was always about host but not the engine) Anyway I've read that the renew process can be done via "engine-setup --offline", but when I try it, it generates this error: --== PKI CONFIGURATION ==-- [ ERROR ] Failed to execute stage 'Environment customization': Unable to load certificate. See https://cryptography.io/en/latest/faq/#why-can-t-i-import-my-pem-file for more details. and in log file: File "/usr/lib64/python3.6/site-packages/cryptography/hazmat/backends/openssl/backend.py", line 1371, in load_pem_x509_certificate "Unable to load certificate. See https://cryptography.io/en/la" ValueError: Unable to load certificate. See https://cryptography.io/en/latest/faq/#why-can-t-i-import-my-pem-file for more details. 2022-08-29 19:16:29,502+0200 ERROR otopi.context context._executeMethod:154 Failed to execute stage 'Environment customization': Unable to load certificate. See https://cryptography.io/en/latest/faq/#why-can-t-i-import-my-pem-file for more details. I've also tried the manual procedure (using /usr/share/ovirt-engine/bin/pki-enroll-pkcs12.sh) mentioned in https://users.ovirt.narkive.com/4ugjgicE/ovirt-regenerating-new-ssl-certifi… (message from Alon Bar-Lev), but the 4th command always says I enter a wrogn apssword, but it's not. we are blocked here and we can't use our ovirt cluster, so it's pretty blocking. Thx a lot in advance

2 3

unable to bring up gluster bricks after 4.5 upgrade
by Jayme 29 Aug '22

29 Aug '22

Hello All, I've been struggling with a few issues upgrading my 3-node HCI custer from 4.4 to 4.5. At present the self hosted engine VM is properly running oVirt 4.5 on CentOS 8x stream. First host node, I set in maintenance and installed new node-ng image. I ran into issue with rescue mode on boot which appears to have been related to LVM devices bug. I was able to work past that and get the node to boot. The node running 4.5.2 image is booting properly and gluster/lvm mounts etc all look good. I am able to activate the host and run VMs on it etc. however, oVirt cli is showing that all bricks on host are DOWN. I was unable to get the bricks back up even after doing a force start of the volumes. Here is the glusterd log from the host in question when I try force start on the engine volume (other volumes are similar: ==> glusterd.log <== The message "I [MSGID: 106568] [glusterd-svc-mgmt.c:266:glusterd_svc_stop] 0-management: bitd service is stopped" repeated 2 times between [2022-08-29 18:09:56.027147 +0000] and [2022-08-29 18:10:34.694144 +0000] [2022-08-29 18:10:34.695348 +0000] I [MSGID: 106618] [glusterd-svc-helper.c:909:glusterd_attach_svc] 0-glusterd: adding svc glustershd (volume=engine) to existing process with pid 2473 [2022-08-29 18:10:34.695669 +0000] I [MSGID: 106131] [glusterd-proc-mgmt.c:84:glusterd_proc_stop] 0-management: scrub already stopped [2022-08-29 18:10:34.695691 +0000] I [MSGID: 106568] [glusterd-svc-mgmt.c:266:glusterd_svc_stop] 0-management: scrub service is stopped [2022-08-29 18:10:34.695832 +0000] I [MSGID: 106617] [glusterd-svc-helper.c:698:glusterd_svc_attach_cbk] 0-management: svc glustershd of volume engine attached successfully to pid 2473 [2022-08-29 18:10:34.703718 +0000] E [MSGID: 106115] [glusterd-mgmt.c:119:gd_mgmt_v3_collate_errors] 0-management: Post commit failed on gluster2.xxxxx. Please check log file for details. [2022-08-29 18:10:34.703774 +0000] E [MSGID: 106115] [glusterd-mgmt.c:119:gd_mgmt_v3_collate_errors] 0-management: Post commit failed on gluster1.xxxxx. Please check log file for details. [2022-08-29 18:10:34.703797 +0000] E [MSGID: 106664] [glusterd-mgmt.c:1969:glusterd_mgmt_v3_post_commit] 0-management: Post commit failed on peers [2022-08-29 18:10:34.703800 +0000] E [MSGID: 106664] [glusterd-mgmt.c:2664:glusterd_mgmt_v3_initiate_all_phases] 0-management: Post commit Op Failed If I run start command manually on host cli: gluster volume start engine force volume start: engine: failed: Post commit failed on gluster1.xxxx. Please check log file for details. Post commit failed on gluster2.xxxx. Please check log file for details. I feel like this may be some issue with the difference in major versions of GlusterFS on the nodes but I am unsure. The other nodes are running ovirt-node-ng-4.4.6.3 At this point I am afraid to bring down any other node to attempt upgrading it without the bricks in UP status on the first host. I do not want to lose quorum and potentially disrupt running VMs. Any idea why I can't seem to start the volumes on the upgraded host? Thanks!

1 2

(no subject)
by parallax 29 Aug '22

29 Aug '22

oVirt version:4.4.4.7-1.el8 I have several servers in cluster and I got this error: Data Center is being initialized, please wait for initialization to complete. VDSM command GetStoragePoolInfoVDS failed: PKIX path validation failed: java.security.cert.CertPathValidatorException: validity check failed the SPM role is constantly being transferred to the servers and I can't do anything in StorageDomains storages are in inactive status but virtual machines are runnig how ti fix it?

3 6

Commissioning hew host on centos9
by David Johnson 29 Aug '22

29 Aug '22

Good evening all, I am trying to commission an ovirt host on new hardware using Centos 9. I followed the pre-install instructions at https://www.ovirt.org/download/install_on_rhel.html, and eventually figured out that this line was missing from the preparation scripts: dnf install rdo-openvswitch-2.15 With that, I was able to continue installation. After completing the instructions, DNF update reports this: [root@localhost administrator]# dnf update --nobest Last metadata expiration check: 0:01:06 ago on Fri 26 Aug 2022 08:20:27 PM CDT. Dependencies resolved. Problem 1: package ovirt-openvswitch-2.15-4.el9.noarch requires openvswitch2.15, but none of the providers can be installed - package rdo-openvswitch-2:2.17-2.el9s.noarch obsoletes openvswitch2.15 < 2.17 provided by openvswitch2.15-2.15.0-99.el9s.x86_64 - package rdo-openvswitch-2:2.17-2.el9s.noarch obsoletes openvswitch2.15 < 2.17 provided by openvswitch2.15-2.15.0-51.el9s.x86_64 - package rdo-openvswitch-2:2.17-2.el9s.noarch obsoletes openvswitch2.15 < 2.17 provided by openvswitch2.15-2.15.0-56.el9s.x86_64 - package rdo-openvswitch-2:2.17-2.el9s.noarch obsoletes openvswitch2.15 < 2.17 provided by openvswitch2.15-2.15.0-81.el9s.x86_64 - cannot install the best update candidate for package ovirt-openvswitch-2.15-4.el9.noarch - cannot install the best update candidate for package openvswitch2.15-2.15.0-99.el9s.x86_64 Problem 2: problem with installed package ovirt-openvswitch-2.15-4.el9.noarch - package ovirt-openvswitch-2.15-4.el9.noarch requires openvswitch2.15, but none of the providers can be installed - package rdo-openvswitch-2:2.17-2.el9s.noarch obsoletes openvswitch2.15 < 2.17 provided by openvswitch2.15-2.15.0-99.el9s.x86_64 - package rdo-openvswitch-2:2.17-2.el9s.noarch obsoletes openvswitch2.15 < 2.17 provided by openvswitch2.15-2.15.0-51.el9s.x86_64 - package rdo-openvswitch-2:2.17-2.el9s.noarch obsoletes openvswitch2.15 < 2.17 provided by openvswitch2.15-2.15.0-56.el9s.x86_64 - package rdo-openvswitch-2:2.17-2.el9s.noarch obsoletes openvswitch2.15 < 2.17 provided by openvswitch2.15-2.15.0-81.el9s.x86_64 - cannot install the best update candidate for package rdo-openvswitch-1:2.15-2.el9s.noarch DNF info lists these [root@localhost administrator]# dnf list installed |grep openvswitch centos-release-nfv-openvswitch.noarch 1-4.el9s @c9s-extras-common network-scripts-openvswitch2.15.x86_64 2.15.0-99.el9s @centos-nfv-openvswitch openvswitch-selinux-extra-policy.noarch 1.0-31.el9s @centos-nfv-openvswitch *openvswitch2.15.x86_64 2.15.0-99.el9s @centos-nfv-openvswitch*ovirt-openvswitch.noarch 2.15-4.el9 @centos-ovirt45 python3-openvswitch2.15.x86_64 2.15.0-99.el9s @centos-nfv-openvswitch rdo-openvswitch.noarch 1:2.15-2.el9s @centos-openstack-yoga *Questions:* 1. Is this a problem? 2. If this is a problem, is there an FAQ sheet or known steps to work around/fix this? 3. Does there need to be an update to the pre-installation instructions? *David Johnson* *Director of Development, Maxis Technology* 844.696.2947 ext 702 (o) | 479.531.3590 (c) <https://www.linkedin.com/in/pojoguy/> <https://maxistechnology.com/wp-content/uploads/vcards/vcard-David_Johnson.v…> <https://maxistechnology.com/> *Follow us:* <https://www.linkedin.com/company/maxis-tech-inc/>

2 1

oVirt 4.5 self-hosted network not working
by Paul-Erik Törrönen 25 Aug '22

25 Aug '22

Specifically, the installation (ovirt-hosted-engine-setup) completes successfully, and the engine VM is up and running, but it can not be accessed from any other machine in the same subnet but the host machine. I can ssh (from the host) to the engine VM, and confirm that initiating the connection from the engine VM to any other machine, except the host, fails. The routing looks correct and stopping the firewalld on the engine VM does not make any difference. Nor does stopping the firewalld on the host either make any difference. This seems like a failure in correctly setting up the host routing for the engine VM. How do I fix this? Poltsi

1 0

Ovirt 4.5.2 problem with change CD during VM installation
by Facundo Badaracco 25 Aug '22

25 Aug '22

HI everyone i have a GlusterGS replica 3 node. All is working fine. Except when i want to install a win10 x64 test VM. The windows ISO loads fine, when it comes the time to put de virio win so the VM recongnizes the disk, the change CD feature doesnt change anything. I have tried with several ISOs, but in the windows installer it never changes the CD, always seems empty. Any ideas? thx in advance.

2 2

oVirt Engine Dashboard and information from Guest Agents
by markeczzz＠gmail.com 25 Aug '22

25 Aug '22

Hi! Recently I had a situation where single node oVirt setup was without network connection to shared storage. After shared storage got reconnected, everything got up as it was supposed to, but I can't get information from guest agents shown in ovirt Dashboard. I tried restarting ovirt-engine and then 2 vm-s started sending info from guest-agent, but rest of them not. Guest agent is running on those vm-s. I also tried restarting those vm-s. Hosted engine is up and healty, and ovirt-engine service is running and it looks ok also. I have this error in vdsm.log 2022-08-25 10:41:39,913+0200 INFO (periodic/1) [vdsm.api] FINISH repoStats return={'dff59a93-4dca-40cd-9c44-c0a74ebaa656': {'code': 0, 'lastCheck': '2.2', 'delay': '0.000840686', 'valid': True, 'version': 5, 'acquired': True, 'actual': True}, '30c0c9e4-b634-4a05-b921-7a30b69f6dd2': {'code': 0, 'lastCheck': '3.9', 'delay': '0.000830941', 'valid': True, 'version': 5, 'acquired': True, 'actual': True}, '6cb12d96-3a0a-4902-acb0-deb907ef945c': {'code': 0, 'lastCheck': '5.5', 'delay': '0.000857244', 'valid': True, 'version': 5, 'acquired': True, 'actual': True}, '559569b9-c33e-4fb0-be6a-51a178a228a9': {'code': 0, 'lastCheck': '5.3', 'delay': '0.000463871', 'valid': True, 'version': 5, 'acquired': True, 'actual': True}} from=internal, task_id=8c638713-f010-4692-ba0b-1ac94052fe7c (api:54) 2022-08-25 10:41:40,778+0200 ERROR (qgapoller/2) [virt.periodic.Operation] <bound method QemuGuestAgentPoller._poller of <vdsm.virt.qemuguestagent.QemuGuestAgentPoller object at 0x7f9efc08e6d8>> operation failed (periodic:204) Traceback (most recent call last): File "/usr/lib/python3.6/site-packages/vdsm/virt/periodic.py", line 202, in __call__ self._func() File "/usr/lib/python3.6/site-packages/vdsm/virt/qemuguestagent.py", line 493, in _poller vm_id, self._qga_call_get_vcpus(vm_obj)) File "/usr/lib/python3.6/site-packages/vdsm/virt/qemuguestagent.py", line 814, in _qga_call_get_vcpus if 'online' in vcpus: TypeError: argument of type 'NoneType' is not iterable 2022-08-25 10:41:41,551+0200 INFO (jsonrpc/2) [api.virt] START getStats() from=::1,50396, vmId=a6e745c4-2e4e-4672-8625-08b787832020 (api:48)

1 1