- Users - oVirt List Archives

Multiple hosts stuck in Connecting state waiting for storage pool to go up.
by ivan.lezhnjov.iv＠gmail.com 06 Nov '23

06 Nov '23

Hi! We have a problem with multiple hosts stuck in Connecting state, which I hoped somebody here could help us wrap our heads around. All hosts, except one, seem to have very similar symptoms but I'll focus on one host that represents the rest. So, the host is stuck in Connecting state and this what we see in oVirt log files. /var/log/ovirt-engine/engine.log: 2023-04-20 09:51:53,021+03 ERROR [org.ovirt.engine.core.vdsbroker.vdsbroker.GetCapabilitiesAsyncVDSCommand] (EE-ManagedScheduledExecutorService-engineScheduledThreadPool-Thread-37) [] Command 'GetCapabilitiesAsyncVDSCommand(HostName = ABC010-176-XYZ, VdsIdAndVdsVDSCommandParametersBase:{hostId='2c458562-3d4d-4408-afc9-9a9484984a91', vds='Host[ABC010-176-XYZ,2c458562-3d4d-4408-afc9-9a9484984a91]'})' execution failed: org.ovirt.vdsm.jsonrpc.client.ClientConnectionException: SSL session is invalid 2023-04-20 09:55:16,556+03 ERROR [org.ovirt.engine.core.dal.dbbroker.auditloghandling.AuditLogDirector] (EE-ManagedScheduledExecutorService-engineScheduledThreadPool-Thread-67) [] EVENT_ID: VDS_BROKER_COMMAND_FAILURE(10,802), VDSM ABC010-176-XYZ command Get Host Capabilities failed: Message timeout which can be caused by communication issues /var/log/vdsm/vdsm.log: 2023-04-20 17:48:51,977+0300 INFO (vmrecovery) [vdsm.api] START getConnectedStoragePoolsList() from=internal, task_id=ebce7c8c-6ded-454e-9aee-86edf72764ef (api:31) 2023-04-20 17:48:51,977+0300 INFO (vmrecovery) [vdsm.api] FINISH getConnectedStoragePoolsList return={'poollist': []} from=internal, task_id=ebce7c8c-6ded-454e-9aee-86edf72764ef (api:37) 2023-04-20 17:48:51,978+0300 INFO (vmrecovery) [vds] recovery: waiting for storage pool to go up (clientIF:723) Both engine.log and vdsm.log are flooded with these messages. They are repeated at regular intervals ad infinitum. This is one common symptom shared by multiple hosts in our deployment. They all have these message loops in engine.log and vdsm.log files. On all Running vdsm-client Host getConnectedStoragePools also returns an empty list represented by [] on all hosts (but interestingly there is one that showed Storage Pool UUID and yet it was still stuck in Connecting state). This particular host (ABC010-176-XYZ) is connected to 3 CEPH iSCSI Storage Domains and lsblk shows 3 block devices with matching UUIDs in their device components. So, the storage seems to be connected but the Storage Pool is not? How is that even possible? Now, what's even more weird is that we tried rebooting the host (via Administrator Portal) and it didn't help. We even tried removing and re-adding the host in Administrator Portal but to no avail. Additionally, the host refused to go into Maintenance mode so we had to enforce it by manually updating Engine DB. We also tried reinstalling the host via Administrator Portal and ran into another weird problem, which I'm not sure if it's a related one or a problem that deserves a dedicated discussion thread but, basically, the underlying Ansible playbook exited with the following error message: "stdout" : "fatal: [10.10.10.176]: UNREACHABLE! => {\"changed\": false, \"msg\": \"Data could not be sent to remote host \\\"10.10.10.176\\\". Make sure this host can be reached over ssh: \", \"unreachable\": true}", Counterintuitively, just before running Reinstall via Administrator Portal we had been able to reboot the same host (which as you know oVirt does via Ansible as well). So, no changes on the host in between just different Ansible playbooks. To confirm that we actually had access to the host over ssh we successfully ran ssh -p $PORT root(a)10.10.10.176 -i /etc/pki/ovirt-engine/keys/engine_id_rsa and it worked. That made us scratch our heads for a while but what seems to had fixed Ansible's ssh access problems was manual full stop of all VDSM-related systemd services on the host. It was just a wild guess but as soon as we stopped all VDSM services Ansible stopped complaining about not being able to reach the target host and successfully did its job. I'm sure you'd like to see more logs but I'm not certain what exactly is relevant. There are a ton of logs as this deployment is comprised of nearly 80 hosts. So, I guess it's best if you just request to see specific logs, messages or configuration details and I'll cherry-pick what's relevant. We don't really understand what's going on and would appreciate any help. We tried just about anything we could think of to resolve this issue and are running out of ideas what to do next. If you have any questions just ask and I'll do my best to answer them.

2 4

Is this still an active mailing list?
by vince seavello 06 Nov '23

06 Nov '23

I am attempting to install oVirt Node 4.5.4. I could use a little help with a few setup issues. I'm wondering if this mailing list is still active. I ask because there are several links on the ovirt.org site that are not functional. If this list is still active, I'll give some of the specifics of my system and the issues I'm running into. Vince Seavello Seavello Web Design c: 425.478.9682 f: 425.337.8712 e: vseavello(a)gmail.com

1 0

Re: Extend apache.cer and websocket-proxy.cer
by LS CHENG 04 Nov '23

04 Nov '23

Hi Yes it is generated with engine-setup. How do you extend the certificate validation value in engine-setup? (I am aware that browser can have problems with long duration certificates as explained in https://techbeacon.com/security/google-apple-mozilla-enforce-1-year-max-sec… ) Thanks On Sat, Nov 4, 2023 at 6:39 PM Matej Dujava <ovirt(a)kocurkovo.cz> wrote: > Hi, > > By self signed cert, you mean managed cert generated by ovirt itself > (engine-setup)? > > I found an issue https://bugzilla.redhat.com/show_bug.cgi?id=1824103 where > it's mentioned that safari (maybe other browsers too) have problem with > long self signed CA. Of it's not affecting your clients you can change > values and regenerate cert by engine-setup. > > You can always generate SSL cert by hand (openssl or cfssl ...) and > replace it with following > https://www.ovirt.org/documentation/administration_guide/#Replacing_the_Man… > . > > > On 4 November 2023 14:18:26 CET, LS CHENG <lsc.oraes(a)gmail.com> wrote: > >> Hi again >> >> Forgot to mention that I am using self signed certificates >> >> Thank you >> >> >> >> On Sat, Nov 4, 2023 at 2:07 PM LS CHENG <lsc.oraes(a)gmail.com> wrote: >> >>> Hi all >>> >>> I am running Oracle Linux Virtualization Manager 4.4. >>> >>> The default expiration length for apache.cer and websocket-proxy.cer is >>> 1 year, is there a way to extend them to 10 years? >>> >>> Thank you >>> >>> >>>

2 2

Extend apache.cer and websocket-proxy.cer
by LS CHENG 04 Nov '23

04 Nov '23

Hi all I am running Oracle Linux Virtualization Manager 4.4. The default expiration length for apache.cer and websocket-proxy.cer is 1 year, is there a way to extend them to 10 years? Thank you

1 1

Direct LUN I/O errors with SCSI Pass-through enabled
by mgs＠ordix.de 03 Nov '23

03 Nov '23

Hi, in our environment (Version 4.4.10.7) we use fibre channel LUNs, which we attach directly to the VMs (as Direct LUN) with VirtIO-SCSI and SCSI pass-through enabled. The virtual machines run an application that requires 4096 as physical_block_size and 512 as logical_block_size. For this reason, we had to enable SCSI pass-through. Only with SCSI pass-through the correct physical_block_size is passed through to the VM. Now we have the following problem on just about every VM: Error messages of the following form occur in the VMs (in /var/log/messages): kernel: blk_update_request: I/O error, dev sdd, sector 352194592 op 0x1:(WRITE) flags 0xc800 phys_seg 16 prio class 0 This error message coincides with a crash of the application. The error message seems to belong to SCSI. We are currently trying to find an alternative to SCSI pass-through. We want to use VirtIO and somehow pass the physical_block_size. Since the XML files of the VMs are transient, we cannot make any changes there. Does anyone have an idea what the error could be or how to pass the correct physical_block_size? Could VDSM hooks help with this? Thank you and regards Miguel

2 1

Unable to access Ovirt-4.5 engine and adding node4.5
by ankit＠eurus.net 03 Nov '23

03 Nov '23

Hi Saviour's, I am a very newbie and trying to install oVirt 4.5 on RHEL 8.7 and Node 4.5 and after succeeding will go in production. 1. Unable to access oVirt 4.5 if installed Keylock. 2. Unable to add Host when installed Keylock. setup details: VMware ovirt engine: 4GB RAM,4CPU 100 GB Node1: 4GB RAM,4CPU 100 GB Installation source: https://www.ovirt.org/documentation/installing_ovirt_as_a_standalone_manage… Let me know if you need more information about this. Could someone help me with this?

1 0

Affinity Labels in 4.4
by Alan G 02 Nov '23

02 Nov '23

Can anyone explain how affinity labels work in 4.4? I created a label containing a host and a VM. I had assumed that would require the VM to run on that host, but the VM continues to run on any host in the cluster. I then checked the 4.4 documentation and it says that I need the filters module section of scheduling policy to contain Label, but there's doesn't appear to be a filter called Label. Am I missing something?

1 0

Failed to login: PKIX path validation failed: java.security.cert.CertPathValidatorException: validity check failed
by p.olivera＠telfy.com 02 Nov '23

02 Nov '23

Hi community, We're encountering the following error when attempting to log in: Warning alert: PKIX path validation failed: java.security.cert.CertPathValidatorException: validity check failed The certificate is valid until 2027: [root@engine9 certs]# openssl x509 -subject -noout -dates -in engine.cer subject=C = US, O = telfy.com, CN = engine9.telfy.com notBefore=Sep 25 10:18:03 2022 GMT notAfter=Sep 27 10:18:03 2027 GMT It's worth noting that our time zone recently switched to GMT+1. Could this change be related to the issue? Has anyone else experienced this problem, and if so, how was it resolved? Thank you.

2 3

cannot reinstall hosts after moving self hosted engine to standalone VM and renaming ovirt UI FQDN
by vishal.merrit＠gmail.com 01 Nov '23

01 Nov '23

Getting error in engine.log on the engine VM: 2023-10-31 12:11:12,818Z ERROR [org.ovirt.engine.core.bll.hostdeploy.InstallVdsCommand] (default task-44) [9deb07c5-f87a-42fb-9c07-33f4d30b197d] Error during ValidateFailure.: java.lang.NullPointerException

1 0

Migrated VM has snapshots with same id
by mihailmataras＠gmail.com 31 Oct '23

31 Oct '23

Hi community, We've encountered problem with virtual machines migrated from RHV(4.4.9.5-0.1.el8ev) to oVirt (4.5.4-1.el8) using data domain. When we create snapshot on migrated VM (in oVirt), we see that snapshot has same id as "Active VM" snapshot and disk in illegal status. (we see that in web ui). Meanwhile in database "engine" id's are different. And vdsm-tool shows that there's only one snapshot in status "LEGAL". engine=# select image_guid,parentid,imagestatus,vm_snapshot_id,volume_type,volume_format,active from images where image_group_id='12a8999f-3f6e-4377-8378-40370aa68d3f'; image_guid | parentid | imagestatus | vm_snapshot_id | volume_type | volume_format | active --------------------------------------+--------------------------------------+-------------+--------------------------------------+-------------+---------------+-------- 5cf5a41d-7236-4c41-ae9c-7c6359210db6 | 6d9cc470-d66f-4e68-a0aa-6e8985f392b0 | 1 | 88d91f15-26ad-48c1-91ec-422c1c04e79b | 2 | 4 | t 6d9cc470-d66f-4e68-a0aa-6e8985f392b0 | 00000000-0000-0000-0000-000000000000 | 4 | a47261da-4ebb-42e3-a74a-0882f8e27c73 | 2 | 4 | f (2 rows) vdsm-tool dump-volume-chains 5e88e116-fb90-41ce-b9c3-01ccb9302b16 | grep 12a8999f-3f6e-4377-8378-40370aa68d3f -A 10 image: 12a8999f-3f6e-4377-8378-40370aa68d3f - 6d9cc470-d66f-4e68-a0aa-6e8985f392b0 status: OK, voltype: LEAF, format: COW, legality: LEGAL, type: SPARSE, capacity: 214748364800, truesize: 121198608384 We know how to fix this using database commands. But when we create snapshot , it happens again. Question is how to prevent that? Thank you.

1 0