- Users - oVirt List Archives

Questions and Suggestions for oVirt
by Daniel Gurgel 30 Jan '25

30 Jan '25

With the new leadership of the project, most users who admire and support oVirt would like to know the roadmap and future plans. From my perspective, I understand that Red Hat discontinued its support purely for financial reasons. Therefore, this is probably one of the key areas where oVirt needs to strengthen. Wouldn't it be worth seeking support and collaboration with Veeam and Oracle, given its "OLVM"? Especially considering that, after the VMware/Broadcom licensing changes, Oracle has been aggressively offering OLVM to customers as a migration alternative. A positive development was Veeam Backup & Replication recently adding support for OLVM. This has increased interest in the solution, but there are still limitations, such as migration/backup/restore between different hypervisors (interestingly, this feature is available for Proxmox/KVM, which uses similar technology). These are some of the main questions from customers: 1. When will it be updated to Enterprise Linux 9 or 10? 2. What support will be available for new technologies? 3. How about a change in the Engine/Manager, allowing VMs to be managed even when the Engine is unavailable! (Sorry, but this is one of oVirt's biggest bottlenecks, and inevitably, comparisons with VMware/VCenter arise.) 4. Support for VM migration between Datacenters should also be carefully considered. **Note**: The oVirt website needs to be more user-friendly for general users. I also suggest a fundraising campaign—I would be happy to contribute, and I believe the entire user base would as well!

1 0

Re: [EXT]Re: Re: [SOLVED] Re: Re: How to re-enroll (or renew) host certificates for a single-host hosted-engine deployment?
by Cosmin Cioranu 29 Jan '25

29 Jan '25

I have checked the solution and it works, apart from the console feature of the virtual machine. In my case, once a got control over the host, migrated all the vms and then Maintenance and then Enroll Certificate. So: an updated version ``` set -x ## Make the script echo everything out, so if it fails you know where set -e ## Make the script STOP on any error set my_date="$(date +"%Y%m%d%H%M%S")" # Backup the existing CA files /bin/cp -p /etc/pki/ovirt-engine/private/ca.pem /etc/pki/ovirt-engine/private/ca.pem.$my_date /bin/cp -p /etc/pki/ovirt-engine/ca.pem{,.$my_date} /bin/mv /etc/pki/ovirt-engine/certs/ca.der{,.$my_date} # Sign the key openssl x509 -signkey /etc/pki/ovirt-engine/private/ca.pem -in /etc/pki/ovirt-engine/ca.pem -out /etc/pki/ovirt-engine/ca.pem.new -days 3650 -sha256 openssl x509 -in /etc/pki/ovirt-engine/ca.pem.new -text > /etc/pki/ovirt-engine/ca.pem.new.full # Put the files into place /bin/mv -f /etc/pki/ovirt-engine/ca.pem.new.full /etc/pki/ovirt-engine/ca.pem /bin/cp -p /etc/pki/ovirt-engine/ca.pem.new /etc/pki/ovirt-engine/certs/ca.der ``` Now you need to copy the new CA file over to the host: Source: ENGINE `/etc/pki/ovirt-engine/ca.pem` Dest: HOST `/tmp/new-ca.pem` ########### On the oVirt Host ########### # Create a CSR using the information from the existing certificate and the existing key: openssl x509 -x509toreq -in /etc/pki/libvirt/clientcert.pem -out /tmp/HOST.csr -signkey /etc/pki/libvirt/private/clientkey.pem Now you need to copy the new CA file over to the host: Source: HOST `/tmp/HOST.csr` Dest: ENGINE `/etc/pki/ovirt-engine/requests/full.hostname.com.req` ########### On the ENGINE HOST ########### # Now sign it: /usr/share/ovirt-engine/bin/pki-enroll-request.sh --name=full.hostname.com # NB -- adding --san results in an error: --san=host.na.me (So no Subject Alternate Names) Now you need to copy the new Certificate file over to the host: Source: ENGINE /etc/pki/ovirt-engine/certs/full.hostname.com.cer Dest: HOST /tmp/new-cert.pem ########### On the oVirt Host ########### (see bellow the written script) #!/bin/bash #set -x set -e mydate="$(date +"%Y%m%d%H%M%S")" echo "[.] reference ${mydate}" for x in /tmp/new-ca.pem /tmp/new-cert.pem ; do echo -n "[.] checking $x" if [ ! -e "$x" ]; then echo "[not found]" exit else echo "[found]" fi done for x in /etc/pki/libvirt/clientcert.pem /etc/pki/vdsm/certs/vdsmcert.pem /etc/pki/vdsm/libvirt-migrate/server-cert.pem /etc/pki/vdsm/libvirt-spice/server-cert.pem /etc/pki/vdsm/libvirt-vnc/server-cert.pem; do echo -n "copying $x to ${x}.${mydate}" /bin/mv -n $x ${x}.${mydate} echo "[done]"; echo -n "[.] copying /tmp/new-cert.pem to $x" /bin/cp /tmp/new-cert.pem ${x} echo "[done]" echo -n "[.] setting permissions to $x 644 root:kvm" chmod 644 ${x} chown root:kvm ${x} echo "[done]" done for x in /etc/pki/vdsm/libvirt-migrate/ca-cert.pem /etc/pki/vdsm/certs/cacert.pem /etc/pki/vdsm/libvirt-vnc/ca-cert.pem /etc/pki/vdsm/libvirt-spice/ca-cert.pem /etc/pki/CA/cacert.pem; do echo -n "copying $x to ${x}.${mydate}" /bin/mv -n $x ${x}.${mydate} echo "[done]" echo -n "[.] copying /tmp/new-ca.pem to $x" /bin/cp /tmp/new-ca.pem ${x} echo "[done]"; echo -n "[.] setting permissions to $x 644 root:kvm" chmod 644 ${x} chown root:kvm ${x} echo "[done]" done for x in /etc/pki/vdsm/keys/vdsmkey.pem; do echo -n "copying $x to ${x}.${mydate}" /bin/mv -n $x ${x}.${mydate} echo "[done]" echo -n "[.] copying /etc/pki/libvirt/private/clientkey.pem to $x" /bin/cp /etc/pki/libvirt/private/clientkey.pem ${x} echo "[done]"; echo -n "[.] setting permissions to $x 644 root:kvm" chmod 644 ${x} chown root:kvm ${x} echo "[done]" done echo "[.] Checks " openssl x509 -in /etc/pki/libvirt/clientcert.pem -noout -dates openssl x509 -in /etc/pki/vdsm/certs/cacert.pem -noout -dates openssl x509 -in /etc/pki/vdsm/certs/vdsmcert.pem -noout -dates openssl x509 -in /etc/pki/vdsm/libvirt-migrate/ca-cert.pem -noout -dates openssl x509 -in /etc/pki/vdsm/libvirt-migrate/client-cert.pem -noout -dates openssl x509 -in /etc/pki/vdsm/libvirt-migrate/server-cert.pem -noout -dates openssl x509 -in /etc/pki/vdsm/libvirt-spice/ca-cert.pem -noout -dates openssl x509 -in /etc/pki/vdsm/libvirt-spice/server-cert.pem -noout -dates ```` Note: I whould backup the entire pki folder on both node and engine.

1 0

snapshot solution: Existing snapshots that were taken after this one will be erased.
by dhanaraj.ramesh＠yahoo.com 29 Jan '25

29 Jan '25

Hi Team, when I want to commit the older snapshots I'm getting warning stating " Existing snapshots that were taken after this one will be erased.". is there any way we can retain the latest snapshots as is in the chain? I knew cloning and template export options are there to secure that latest snapshot data but these are will consume additional space in storage and take time.

4 3

Re: [External] : Re: Unable to migrate vm with 4 (500g) disks from vmware to ovirt.
by dushyantk.sun＠gmail.com 24 Jan '25

24 Jan '25

Hi Marcos and Simon, Provided document was helpful. Thank you.

1 0

Re: [External] : Re: Unable to migrate vm with 4 (500g) disks from vmware to ovirt.
by dushyantk.sun＠gmail.com 23 Jan '25

23 Jan '25

Currently linux vm which i am trying to migrate has 4 (500GB) disk. All disks on same datastore. It is iscsi datastore. Do we have any configuration where we can enabled verbose mode while migration is going on to see what is happening as currently unable to identify failure. Also is there any limitation if vms with multiple disk is not possible to migrate?

2 2

Unable to migrate vm with 4 (500g) disks from vmware to ovirt.
by dushyantk.sun＠gmail.com 23 Jan '25

23 Jan '25

I have vm with multiple disk attached to it. While migrating vm from vmware to ovirt, it started but gets failed after 5-6 hours. However vms with single disk migrated successfully. Following ovirt UI to migrate vm. Please let me know if any one come across such scenario

4 6

oVirt Translations & Future
by Jean-Louis Dupond 23 Jan '25

23 Jan '25

Hi, We've setup some alternative for the (dead) Zanata translation project for oVirt. You can find the Weblate setup on the following location: https://translate.ovirt.tech/projects/ovirt-engine/ So if somebody wants to add translations, feel free! :) Will try to add translations for the other projects the coming days. Next to that, we are working hard to get oVirt better! If you want to help, feel free to open PR's or send me an mail. A lot can be done, so you won't get bored. Some things that we are looking into: - Java updates - Wildfly update - AlmaLinux 10 / CentOS Stream 10 support Feel free to contact me if you have further questions. Jean-Louis

6 7

Re: [External] : OLVM/ovirt 4.5 support for 5th Gen Intel Xeon Gold (Emerald Rapids)
by Simon Coter 23 Jan '25

23 Jan '25

Hi Hassan, we internally fixed it by reverting our patch and use the oVirt community one: <https://github.com/oVirt/ovirt-engine/pull/978> [978.png] core: Introduce new Intel, AMD and IBM CPU's by JasperB-TeamBlue · Pull Request #978 · oVirt/ovirt-engine<https://github.com/oVirt/ovirt-engine/pull/978> github.com<https://github.com/oVirt/ovirt-engine/pull/978> This will be released as part of the next Errata for OLVM. Simon On Jan 23, 2025, at 9:44 AM, Hassan Shabbir <hshabbir315(a)gmail.com> wrote: Hello Simon, Thank you for the response, what about 5th Gen Emerald Rapids? correct me if i'm wrong: support for 4th Gen Sapphire Rapids is already in place in OLVM. Regards, On Thu, Jan 23, 2025, 1:32 PM Simon Coter <simon.coter(a)oracle.com<mailto:simon.coter@oracle.com>> wrote: We introduced a patch to address the Sapphire Rapids family support on OLVM 4.5.5 We recently synced our patch with the one had been pushed by the oVirt community and should be out quite soon. Simon > On Jan 19, 2025, at 9:28 PM, hshabbir315(a)gmail.com<mailto:hshabbir315@gmail.com> wrote: > > Hi Guys, > > Can someone please confirm if 5th Gen Intel Gold (Emerald Rapids) CPUs are compatible with OLVM/ovirt 4.5.5.x? We have a OLVM/ovirt 4.5.5.x test lab setup cluster running on older hardware (Intel(R) Xeon(R) CPU E5-2660 v3 @ 2.60GHz - Haswell Family) with compatibility version 4.7. While going through the engine db backup we observed that there are no entries in the vdc_options table for Emerald Rapids family of Intel CPUs. Does this mean that these CPUs are not supported by ovirt 4.5.5.x? or these CPUs are supported but the extra features/instruction sets offered by these CPUs over Sapphire Rapids family of Intel CPUs will not be available to the VMs? > > Regards, > _______________________________________________ > Users mailing list -- users(a)ovirt.org<mailto:users@ovirt.org> > To unsubscribe send an email to users-leave(a)ovirt.org<mailto:users-leave@ovirt.org> > Privacy Statement: https://urldefense.com/v3/__https://www.ovirt.org/privacy-policy.html__;!!A… > oVirt Code of Conduct: https://urldefense.com/v3/__https://www.ovirt.org/community/about/community… > List Archives: https://urldefense.com/v3/__https://lists.ovirt.org/archives/list/users@ovi…

1 0

OLVM/ovirt 4.5 support for 5th Gen Intel Xeon Gold (Emerald Rapids)
by hshabbir315＠gmail.com 23 Jan '25

23 Jan '25

Hi Guys, Can someone please confirm if 5th Gen Intel Gold (Emerald Rapids) CPUs are compatible with OLVM/ovirt 4.5.5.x? We have a OLVM/ovirt 4.5.5.x test lab setup cluster running on older hardware (Intel(R) Xeon(R) CPU E5-2660 v3 @ 2.60GHz - Haswell Family) with compatibility version 4.7. While going through the engine db backup we observed that there are no entries in the vdc_options table for Emerald Rapids family of Intel CPUs. Does this mean that these CPUs are not supported by ovirt 4.5.5.x? or these CPUs are supported but the extra features/instruction sets offered by these CPUs over Sapphire Rapids family of Intel CPUs will not be available to the VMs? Regards,

2 1

Re: [External] : The KVM Host Status Stucking in Connecting
by natchawi28＠gmail.com 22 Jan '25

22 Jan '25

Hi, What about the version differences between the Engine and the KVM host? Does it matter? Engine: Oracle Linux 8.10 Kernel 5.15.0-302.167.6.el8uek.x86_64 KVM Host: Oracle Linux 8.6 Kernel 5.4.17-2136.311.6.1.el8uek.x86_64 I checked the Cluster Compatibility Version, and it appears to be compatible. Thank you,

2 1

ovirtmgmt claims no defroute after RHEL upgrade from 8.10 to 9.5
by Austin Coppock 22 Jan '25

22 Jan '25

First off apologies, I am unable to post log files as my oVirt servers have no access to the Internet. I have several oVirt clusters, all were running oVirt 4.5 on RHEL 8.10. I am in the process of upgrading them to RHEL 9.5. with a few minor updates of oVirt packages. The steps I used to do this are: • Backup up the standalone engine DB • Rebuild the engine to RHEL 9.5, reinstall oVirt, restore the DB • Log back in to the engine, place first hypervisor into maintenance and rebuild to RHEL 9.5. • Remove old hypervisor via the engine GUI and add back in to the cluster • Repeat for each hypervisor until all are upgraded. This has worked perfectly, everything seems to be normal, but I then noticed that if I select the “Network Interfaces” tab on a hypervisor it shows my bonded interface as “Out-of-sync”, clicking the “Sync All Networks” does nothing. If I click on “Setup Host Networks” and hover the mouse over the ovirtmgmt interface I see the message “Default Route. Host: false DC: true” If I run the command “vdsm-client Host getNetworkCapabilities” on the hypervisor, it shows that the {bridges: ovirtmgmt: ipv4defaultroute false} and {networks: ovirtmgmt: ipv4defaultroute false } on my clusters that are still on RHEL 8.10 the above command shows ipv4defaultroute true. Running the command “ip route show” shows that my default route IS on device ovirtmgmt I am not experiencing any network issues, but getting rid of this out-of-sync message would be nice. Is this a bug or a bad status issue? Or do I have some obscure setting not correct? For both RHEL 8.10 and RHEL 9.5 builds we have used Ansible to perform the tasks, (with minor tweaks). So, the same procedure has been used for both.

1 1

The KVM Host Status Stucking in Connecting
by natchawi28＠gmail.com 21 Jan '25

21 Jan '25

Please help me, I found that my Cluster is in Non-responsive status. When I look at the Host menu page, I see that the status is Connecting and it has been stuck for a long time and I can't do anything. Has anyone experienced this and how do I fix it? https://img5.pic.in.th/file/secure-sv1/Screenshot-2025-01-21-151647.png

2 1

additional language
by ebugaev＠itkey.com 20 Jan '25

20 Jan '25

Are there instructions on how to add an additional language to ovirt-engine? What needs to be done so that an additional language appears in the language selection list on the main page of the engine?

1 0

Renewal of VDSM certificate in one node cluster
by ikbal.charen＠kepler-technologies.com 20 Jan '25

20 Jan '25

Hello, I have a single-node cluster running Hosted Engine, version 4.4.10. While I'm able to renew the engine certificate, I'm unable to renew the VDSM certificate because I cannot put the host into maintenance mode (since it is running the Hosted Engine VM). What steps can I take to resolve this? Best regards,

1 0

Host reboot take an awful lot of time
by duparchy＠esrf.fr 20 Jan '25

20 Jan '25

KVM host is upgraded then rebooted. It seems OK (ssh available, no particular activity) but it takes an awful lot of time for the manager to acknowledge it and let us activating the host. This makes the upgrade process of a whole infrastructure (several hosts) very very long. Is it just us ? (oVirt 4.5 "Oracle" flavor)

3 2

Help With CPU Type
by Daniel Martins 20 Jan '25

20 Jan '25

Hi There, Please may you assist me with the below? I need to remove an oVirt host within a cluster that is using CPU Type Intel Westmere IBRS SSBD Family. Once the host is removed i need to change the CPU type to Intel Broadwell which the remaining hosts Use. So my question is will there need to be down time on the Virtual machines to change this? Regards, [cid:image001.png@01DB6B29.AA668C00]

1 0

Re: [External] : Really weird ! Image IO / browser upload only working on 1 cluster (out of many) - anyone got any advice ? (network access, certs seem fine.)
by morgan cox 17 Jan '25

17 Jan '25

Hi Marcos. Thanks for taking the time to respond... In reference to your questions : Are you using a single Engine deployment to manage all DCs/Clusters : Yes Which browser are you using to upload the image : I have tried 2 Firefox and Chrome (note it works on one cluster - the one that hosts1/2 are connected to) Can you try the upload process from a different browser or desktop/laptop?: Tired a different browser, cannot try different laptop - I have network access to the engine . Also - everything else seems ok - in NoVNC, Serial , etc just image io Any suggestions where to look into this ? Thanks

3 2

Intel CPUs supported by OLVM/ovirt 4.5
by Syed Hassan Shabbir 17 Jan '25

17 Jan '25

Hi, Guys, can someone plz guide me regarding ovirt support for the following CPUs: 1- Intel® Xeon® Gold 5520+ Processor 52.5M Cache, 2.20 GHz (Emerald Rapids) 2- Intel® Xeon® Gold 6526Y Processor 37.5M Cache, 2.80 GHz (Emerald Rapids) 3- Intel® Xeon® Gold 6426Y Processor 37.5M Cache, 2.50 GHz (Sapphire Rapids) 4- Intel® Xeon® Gold 5420+ Processor 52.5M Cache, 2.00 GHz (Sapphire Rapids) 5- Intel® Xeon® Silver 4510 Processor 30M Cache, 2.40 GHz (Sapphire Rapids) I want to deploy OLVM on these servers but couldn't find any entries in the vdc_options table for 5th Gen 'Emerald Rapids'. I am testing with ovirt version 4.5. Regards, Hassan

1 0

Welcome Jean-Louis Dupond to the oVirt Board
by Sandro Bonazzola 17 Jan '25

17 Jan '25

On behalf of the oVirt Board I’m thrilled to announce an exciting new addition: Jean-Louis Dupond. Jean-Louis brings a wealth of experience, expertise, and passion for open-source virtualization, making him an exceptional fit for the oVirt community. Read the full announce on the oVirt blog at https://blogs.ovirt.org/2025/01/welcome-jean-louis-dupond-to-the-ovirt-boar… -- Sandro Bonazzola

4 3

Managing Users with Keycloak WebUI
by Vladislav Solovei 16 Jan '25

16 Jan '25

Hello! I have a question about managing users through the Keycloak WebUI. With oVirt 4.5. Can I do this? Is it the right approach? Or should I avoid doing it? For example, if I want to create a user with 2FA (OTP)

1 0

Really weird ! Image IO / browser upload only working on 1 cluster (out of many) - anyone got any advice ? (network access, certs seem fine.)
by morgan cox 16 Jan '25

16 Jan '25

One for the mystery van .. I am running OLVM/ Ovirt 4.5 (engine) - hosts presently mised 4.4.x and 4.5.x - however version doesn't seen to be the issue We have muliple DC's/Clusters each cluster has 2 hosts and then have access to different fc SAN volumes - each san vol is a sperate storaqe domain I,e host 1 + host 2 --> San 1 host 3 + host 4 --> San 2 , etc SAN connection for the hosts/clusters generally is working fine and I can also upload to any SAN volume using the upload script - /usr/share/doc/python3-ovirt-engine-sdk4/examples/upload_disk.py However imageio via browser ... We have a major issue. I am not able to upload (via browser) any files to any storage domain apart from 1... (say San 1) Yes, I have imported the CA to browser, and All storage domains pass the 'test connection' -> "Connection to ovirt-imageio was successful." However I can see what the issue is (but not why there is this issue..) In the upload image section of all domains the available host to use (to piggyback connection) as the same - host 1 + host 2 But those hosts only have access to one SAN not all of them i.e for San 2 I should be able to choose host 3 + 4 ... (but only host 1/2 are showning in all domains..) If I use the upload script it uses correct host and successfully upload - but not via browser.. I have checked and /etc/ovirt-imageio/conf.d/60-vdsm.conf is the same for ALL hosts. Certs seem ok all a I can telnet to all hosts from engine on port 54322 Can anyone give me any ideas how to troubleshoot this ? Thanks

2 1

EOL glusterFS
by Olivier 16 Jan '25

16 Jan '25

Hello, GlusterFS seems to be EOL. At least no more dev from Red Hat, the github doesn't record any move from more than one year. Is there a plan to use another FS on oVirt instead of glusterFS ?

9 9

zanata.ovirt.org
by ebugaev＠itkey.com 15 Jan '25

15 Jan '25

hello, I would like to translate into Russian help me get an account https://zanata.ovirt.org/ to translate the interface, thank you)

3 3

Changing Disk Alocation Policy
by suporte＠logicworks.pt 13 Jan '25

13 Jan '25

Hi, It's possible to change a thin-provision disk to a preallocated disk? Storages are ISCSI Thanks -- Jose Ferradeira http://www.logicworks.pt

4 4

Re: [External] : Changing Disk Alocation Policy
by change_jeeringly679＠dralias.com 13 Jan '25

13 Jan '25

Hey, Jose is right, you can do this via the api. The server needs to be shutdown. I usually use the below curl command on the engine itself (login with ssh). You need 3 pieces of information: - You need the ce.pem of the engine (which you can download from the engine login page) and make it available. In the command below I assume it is in the local directory. - You need an administrator user in engine, in the below example I used admin. Replace "yourpassword" behind the : with your own. - You need the disk-id as it is know in oVirt. You need to replace "yourdiskid", with your own. You can find the id in Engine. Select Storage, then Disks, then find the disk you want to convert and click on it. The number behind ID is the one you want and has the following format: db7b649f-76b6-4fd1-9b90-aa6becc2a283 The command: curl --cacert './ca.pem' --user 'admin@internal:yourpassword!' --request POST --header 'Version: 4' --header 'Content-Type: application/xml' --header 'Accept: application/xml' --data ' <action> <disk> <sparse>true</sparse> <format>cow</format> <backup_mode>incremental</backup_mode> </disk> </action> ' https://engine.blueit.dk/ovirt-engine/api/disks/yourdiskid/convert The command will return immediately. In engine you will see it is working on the disk image. You get the ballon messages in the right bottom corner and you can see it working in the taskview on the webinterface. Once it is done, you can start the VM again, and the disk is now thin provisioned and incremental backups are allowed. Good luck

1 0

CentOS Stream 9 Ovirt 4.5
by eevans＠digitaldatatechs.com 13 Jan '25

13 Jan '25

I have a small lab set up with 3 ovirt Centos 9 nodes and one Centos 9 Ovirt "controller node" that manages. 2 problems: It will no sync networks on hosts. It will no launch a vm unless you go to system and choose an instance size. It will boot and the network card has to be hot added. I'm not sure what I'm missing, I can give log entries showing several bad lines in diggerent modules.' Engine log while updating hosts: tail -n 100 -f engine.log ' 2025-01-09 02:43:15,722-05 INFO [org.ovirt.engine.core.vdsbroker.vdsbroker.HSMGetAllTasksStatusesVDSCommand] (EE-ManagedScheduledExecutorService-engineScheduledThreadPool-Thread-25) [] HostName = kvm02 2025-01-09 02:43:15,722-05 ERROR [org.ovirt.engine.core.vdsbroker.vdsbroker.HSMGetAllTasksStatusesVDSCommand] (EE-ManagedScheduledExecutorService-engineScheduledThreadPool-Thread-25) [] Command 'HSMGetAllTasksStatusesVDSCommand(HostName = kvm02, VdsIdVDSCommandParametersBase:{hostId='cf4a626e-ad6e-4cc4-95f0-4d7c1329f0f6'})' execution failed: IRSGenericException: IRSErrorException: IRSNonOperationalException: Not SPM: () 2025-01-09 02:43:15,729-05 INFO [org.ovirt.engine.core.vdsbroker.vdsbroker.ConnectStoragePoolVDSCommand] (EE-ManagedScheduledExecutorService-engineScheduledThreadPool-Thread-25) [] START, ConnectStoragePoolVDSCommand(HostName = kvm04, ConnectStoragePoolVDSCommandParameters:{hostId='1408650c-aa50-452a-b2cc-24d36ae3cc20', vdsId='1408650c-aa50-452a-b2cc-24d36ae3cc20', storagePoolId='39acc250-bfba-11ef-adb1-70e2840574e3', masterVersion='765'}), log id: 5e145d4b 2025-01-09 02:43:15,729-05 INFO [org.ovirt.engine.core.vdsbroker.vdsbroker.ConnectStoragePoolVDSCommand] (EE-ManagedScheduledExecutorService-engineScheduledThreadPool-Thread-25) [] Executing with domain map: {bd71c70a-7d9b-4cb4-ae5d-4c75a96a1d94=active, ca1b2f4f-26cf-48cc-b219-7295a763b1ae=active, 028f24af-01a2-4c0d-9374-7d8be6e12469=active} 2025-01-09 02:43:18,057-05 INFO [org.ovirt.engine.core.vdsbroker.vdsbroker.ConnectStoragePoolVDSCommand] (EE-ManagedScheduledExecutorService-engineScheduledThreadPool-Thread-25) [] Command 'org.ovirt.engine.core.vdsbroker.vdsbroker.ConnectStoragePoolVDSCommand' return value 'StatusOnlyReturn [status=Status [code=304, message=Cannot find master domain: 'spUUID=39acc250-bfba-11ef-adb1-70e2840574e3, msdUUID=bd71c70a-7d9b-4cb4-ae5d-4c75a96a1d94']]' 2025-01-09 02:43:18,057-05 INFO [org.ovirt.engine.core.vdsbroker.vdsbroker.ConnectStoragePoolVDSCommand] (EE-ManagedScheduledExecutorService-engineScheduledThreadPool-Thread-25) [] HostName = kvm04 2025-01-09 02:43:18,057-05 ERROR [org.ovirt.engine.core.vdsbroker.vdsbroker.ConnectStoragePoolVDSCommand] (EE-ManagedScheduledExecutorService-engineScheduledThreadPool-Thread-25) [] Command 'ConnectStoragePoolVDSCommand(HostName = kvm04, ConnectStoragePoolVDSCommandParameters:{hostId='1408650c-aa50-452a-b2cc-24d36ae3cc20', vdsId='1408650c-aa50-452a-b2cc-24d36ae3cc20', storagePoolId='39acc250-bfba-11ef-adb1-70e2840574e3', masterVersion='765'})' execution failed: IRSGenericException: IRSErrorException: IRSNoMasterDomainException: Cannot find master domain: 'spUUID=39acc250-bfba-11ef-adb1-70e2840574e3, msdUUID=bd71c70a-7d9b-4cb4-ae5d-4c75a96a1d94' 2025-01-09 02:43:18,057-05 INFO [org.ovirt.engine.core.vdsbroker.vdsbroker.ConnectStoragePoolVDSCommand] (EE-ManagedScheduledExecutorService-engineScheduledThreadPool-Thread-25) [] FINISH, ConnectStoragePoolVDSCommand, return: , log id: 5e145d4b 2025-01-09 02:43:18,057-05 ERROR [org.ovirt.engine.core.vdsbroker.irsbroker.IrsBrokerCommand] (EE-ManagedScheduledExecutorService-engineScheduledThreadPool-Thread-25) [] IrsBroker::Failed::GetStoragePoolInfoVDS: IRSGenericException: IRSErrorException: IRSNoMasterDomainException: Cannot find master domain: 'spUUID=39acc250-bfba-11ef-adb1-70e2840574e3, msdUUID=bd71c70a-7d9b-4cb4-ae5d-4c75a96a1d94' 2025-01-09 02:43:22,344-05 ERROR [org.ovirt.engine.core.vdsbroker.vdsbroker.SpmStatusVDSCommand] (EE-ManagedScheduledExecutorService-engineScheduledThreadPool-Thread-25) [] Command 'SpmStatusVDSCommand(HostName = kvm04, SpmStatusVDSCommandParameters:{hostId='1408650c-aa50-452a-b2cc-24d36ae3cc20', storagePoolId='39acc250-bfba-11ef-adb1-70e2840574e3'})' execution failed: IRSGenericException: IRSErrorException: IRSNoMasterDomainException: Error validating master storage domain: ('MD read error',) 2025-01-09 02:43:22,344-05 INFO [org.ovirt.engine.core.vdsbroker.irsbroker.IrsProxyImpl] (EE-ManagedScheduledExecutorService-engineScheduledThreadPool-Thread-25) [] Irs placed on server '1408650c-aa50-452a-b2cc-24d36ae3cc20' failed. Proceed Failover 2025-01-09 02:43:22,349-05 INFO [org.ovirt.engine.core.vdsbroker.vdsbroker.ConnectStoragePoolVDSCommand] (EE-ManagedScheduledExecutorService-engineScheduledThreadPool-Thread-25) [] START, ConnectStoragePoolVDSCommand(HostName = kvm01, ConnectStoragePoolVDSCommandParameters:{hostId='acf66bf6-e159-4705-be47-484c99e746dd', vdsId='acf66bf6-e159-4705-be47-484c99e746dd', storagePoolId='39acc250-bfba-11ef-adb1-70e2840574e3', masterVersion='765'}), log id: 28df8c85 2025-01-09 02:43:22,349-05 INFO [org.ovirt.engine.core.vdsbroker.vdsbroker.ConnectStoragePoolVDSCommand] (EE-ManagedScheduledExecutorService-engineScheduledThreadPool-Thread-25) [] Executing with domain map: {bd71c70a-7d9b-4cb4-ae5d-4c75a96a1d94=active, ca1b2f4f-26cf-48cc-b219-7295a763b1ae=active, 028f24af-01a2-4c0d-9374-7d8be6e12469=active} 2025-01-09 02:43:25,267-05 ERROR [org.ovirt.engine.core.dal.dbbroker.auditloghandling.AuditLogDirector] (EE-ManagedScheduledExecutorService-engineScheduledThreadPool-Thread-25) [] EVENT_ID: VDS_BROKER_COMMAND_FAILURE(10,802), VDSM kvm01 command ConnectStoragePoolVDS failed: Cannot find master domain: 'spUUID=39acc250-bfba-11ef-adb1-70e2840574e3, msdUUID=bd71c70a-7d9b-4cb4-ae5d-4c75a96a1d94' 2025-01-09 02:43:25,267-05 INFO [org.ovirt.engine.core.vdsbroker.vdsbroker.ConnectStoragePoolVDSCommand] (EE-ManagedScheduledExecutorService-engineScheduledThreadPool-Thread-25) [] Command 'org.ovirt.engine.core.vdsbroker.vdsbroker.ConnectStoragePoolVDSCommand' return value 'StatusOnlyReturn [status=Status [code=304, message=Cannot find master domain: 'spUUID=39acc250-bfba-11ef-adb1-70e2840574e3, msdUUID=bd71c70a-7d9b-4cb4-ae5d-4c75a96a1d94']]' 2025-01-09 02:43:25,267-05 INFO [org.ovirt.engine.core.vdsbroker.vdsbroker.ConnectStoragePoolVDSCommand] (EE-ManagedScheduledExecutorService-engineScheduledThreadPool-Thread-25) [] HostName = kvm01 2025-01-09 02:43:25,267-05 ERROR [org.ovirt.engine.core.vdsbroker.vdsbroker.ConnectStoragePoolVDSCommand] (EE-ManagedScheduledExecutorService-engineScheduledThreadPool-Thread-25) [] Command 'ConnectStoragePoolVDSCommand(HostName = kvm01, ConnectStoragePoolVDSCommandParameters:{hostId='acf66bf6-e159-4705-be47-484c99e746dd', vdsId='acf66bf6-e159-4705-be47-484c99e746dd', storagePoolId='39acc250-bfba-11ef-adb1-70e2840574e3', masterVersion='765'})' execution failed: IRSGenericException: IRSErrorException: IRSNoMasterDomainException: Cannot find master domain: 'spUUID=39acc250-bfba-11ef-adb1-70e2840574e3, msdUUID=bd71c70a-7d9b-4cb4-ae5d-4c75a96a1d94' 2025-01-09 02:43:25,267-05 INFO [org.ovirt.engine.core.vdsbroker.vdsbroker.ConnectStoragePoolVDSCommand] (EE-ManagedScheduledExecutorService-engineScheduledThreadPool-Thread-25) [] FINISH, ConnectStoragePoolVDSCommand, return: , log id: 28df8c85 2025-01-09 02:43:25,267-05 ERROR [org.ovirt.engine.core.vdsbroker.irsbroker.IrsBrokerCommand] (EE-ManagedScheduledExecutorService-engineScheduledThreadPool-Thread-25) [] IrsBroker::Failed::GetStoragePoolInfoVDS: IRSGenericException: IRSErrorException: IRSNoMasterDomainException: Cannot find master domain: 'spUUID=39acc250-bfba-11ef-adb1-70e2840574e3, msdUUID=bd71c70a-7d9b-4cb4-ae5d-4c75a96a1d94' 2025-01-09 02:43:25,275-05 INFO [org.ovirt.engine.core.vdsbroker.irsbroker.IrsProxyImpl] (EE-ManagedScheduledExecutorService-engineScheduledThreadPool-Thread-25) [] Irs placed on server 'acf66bf6-e159-4705-be47-484c99e746dd' failed. Proceed Failover 2025-01-09 02:43:25,279-05 INFO [org.ovirt.engine.core.vdsbroker.vdsbroker.ConnectStoragePoolVDSCommand] (EE-ManagedScheduledExecutorService-engineScheduledThreadPool-Thread-25) [] START, ConnectStoragePoolVDSCommand(HostName = kvm02, ConnectStoragePoolVDSCommandParameters:{hostId='cf4a626e-ad6e-4cc4-95f0-4d7c1329f0f6', vdsId='cf4a626e-ad6e-4cc4-95f0-4d7c1329f0f6', storagePoolId='39acc250-bfba-11ef-adb1-70e2840574e3', masterVersion='765'}), log id: 20ec709f 2025-01-09 02:43:25,279-05 INFO [org.ovirt.engine.core.vdsbroker.vdsbroker.ConnectStoragePoolVDSCommand] (EE-ManagedScheduledExecutorService-engineScheduledThreadPool-Thread-25) [] Executing with domain map: {bd71c70a-7d9b-4cb4-ae5d-4c75a96a1d94=active, ca1b2f4f-26cf-48cc-b219-7295a763b1ae=active, 028f24af-01a2-4c0d-9374-7d8be6e12469=active} 2025-01-09 02:43:27,675-05 ERROR [org.ovirt.engine.core.dal.dbbroker.auditloghandling.AuditLogDirector] (EE-ManagedScheduledExecutorService-engineScheduledThreadPool-Thread-25) [] EVENT_ID: VDS_BROKER_COMMAND_FAILURE(10,802), VDSM kvm02 command ConnectStoragePoolVDS failed: Cannot find master domain: 'spUUID=39acc250-bfba-11ef-adb1-70e2840574e3, msdUUID=bd71c70a-7d9b-4cb4-ae5d-4c75a96a1d94' 2025-01-09 02:43:27,675-05 INFO [org.ovirt.engine.core.vdsbroker.vdsbroker.ConnectStoragePoolVDSCommand] (EE-ManagedScheduledExecutorService-engineScheduledThreadPool-Thread-25) [] Command 'org.ovirt.engine.core.vdsbroker.vdsbroker.ConnectStoragePoolVDSCommand' return value 'StatusOnlyReturn [status=Status [code=304, message=Cannot find master domain: 'spUUID=39acc250-bfba-11ef-adb1-70e2840574e3, msdUUID=bd71c70a-7d9b-4cb4-ae5d-4c75a96a1d94']]' 2025-01-09 02:43:27,675-05 INFO [org.ovirt.engine.core.vdsbroker.vdsbroker.ConnectStoragePoolVDSCommand] (EE-ManagedScheduledExecutorService-engineScheduledThreadPool-Thread-25) [] HostName = kvm02 2025-01-09 02:43:27,676-05 ERROR [org.ovirt.engine.core.vdsbroker.vdsbroker.ConnectStoragePoolVDSCommand] (EE-ManagedScheduledExecutorService-engineScheduledThreadPool-Thread-25) [] Command 'ConnectStoragePoolVDSCommand(HostName = kvm02, ConnectStoragePoolVDSCommandParameters:{hostId='cf4a626e-ad6e-4cc4-95f0-4d7c1329f0f6', vdsId='cf4a626e-ad6e-4cc4-95f0-4d7c1329f0f6', storagePoolId='39acc250-bfba-11ef-adb1-70e2840574e3', masterVersion='765'})' execution failed: IRSGenericException: IRSErrorException: IRSNoMasterDomainException: Cannot find master domain: 'spUUID=39acc250-bfba-11ef-adb1-70e2840574e3, msdUUID=bd71c70a-7d9b-4cb4-ae5d-4c75a96a1d94' 2025-01-09 02:43:27,676-05 INFO [org.ovirt.engine.core.vdsbroker.vdsbroker.ConnectStoragePoolVDSCommand] (EE-ManagedScheduledExecutorService-engineScheduledThreadPool-Thread-25) [] FINISH, ConnectStoragePoolVDSCommand, return: , log id: 20ec709f 2025-01-09 02:43:27,676-05 ERROR [org.ovirt.engine.core.vdsbroker.irsbroker.IrsBrokerCommand] (EE-ManagedScheduledExecutorService-engineScheduledThreadPool-Thread-25) [] IrsBroker::Failed::GetStoragePoolInfoVDS: IRSGenericException: IRSErrorException: IRSNoMasterDomainException: Cannot find master domain: 'spUUID=39acc250-bfba-11ef-adb1-70e2840574e3, msdUUID=bd71c70a-7d9b-4cb4-ae5d-4c75a96a1d94' 2025-01-09 02:43:27,741-05 INFO [org.ovirt.engine.core.bll.storage.pool.ReconstructMasterDomainCommand] (EE-ManagedThreadFactory-engine-Thread-572) [6727848e] Running command: ReconstructMasterDomainCommand internal: true. Entities affected : ID: bd71c70a-7d9b-4cb4-ae5d-4c75a96a1d94 Type: Storage 2025-01-09 02:43:27,765-05 WARN [org.ovirt.engine.core.bll.storage.pool.ReconstructMasterDomainCommand] (EE-ManagedThreadFactory-engine-Thread-572) [6727848e] skipping connect and refresh for all hosts, last master 'true', command status 'true' 2025-01-09 02:43:27,789-05 INFO [org.ovirt.engine.core.bll.eventqueue.EventQueueMonitor] (EE-ManagedThreadFactory-engine-Thread-571) [] Finished reconstruct for pool '39acc250-bfba-11ef-adb1-70e2840574e3'. Clearing event queue 2025-01-09 02:43:30,154-05 INFO [org.ovirt.engine.core.vdsbroker.gluster.GlusterServersListVDSCommand] (EE-ManagedScheduledExecutorService-engineScheduledThreadPool-Thread-41) [] START, GlusterServersListVDSCommand(HostName = kvm04, VdsIdVDSCommandParametersBase:{hostId='1408650c-aa50-452a-b2cc-24d36ae3cc20'}), log id: 770c0b37 2025-01-09 02:43:30,362-05 INFO [org.ovirt.engine.core.vdsbroker.gluster.GlusterServersListVDSCommand] (EE-ManagedScheduledExecutorService-engineScheduledThreadPool-Thread-41) [] FINISH, GlusterServersListVDSCommand, return: [192.168.254.108/24:CONNECTED, ovirt.digitaldatatechs.com:CONNECTED, kvm02.digitaldatatechs.com:CONNECTED, kvm01.digitaldatatechs.com:CONNECTED], log id: 770c0b37 2025-01-09 02:43:30,381-05 INFO [org.ovirt.engine.core.vdsbroker.gluster.AddGlusterServerVDSCommand] (EE-ManagedScheduledExecutorService-engineScheduledThreadPool-Thread-41) [] START, AddGlusterServerVDSCommand(HostName = kvm04, AddGlusterServerVDSParameters:{hostId='1408650c-aa50-452a-b2cc-24d36ae3cc20'}), log id: 3b2ce207 2025-01-09 02:43:33,630-05 INFO [org.ovirt.engine.core.vdsbroker.gluster.AddGlusterServerVDSCommand] (EE-ManagedScheduledExecutorService-engineScheduledThreadPool-Thread-41) [] FINISH, AddGlusterServerVDSCommand, return: , log id: 3b2ce207 2025-01-09 02:43:33,640-05 ERROR [org.ovirt.engine.core.dal.dbbroker.auditloghandling.AuditLogDirector] (EE-ManagedScheduledExecutorService-engineScheduledThreadPool-Thread-41) [] EVENT_ID: GLUSTER_SERVER_ADD_FAILED(4,436), Failed to add host kvm04 into Cluster dds_cluster. Add host failed: rc=107 out=() err=['Probe returned with Transport endpoint is not connected'] 2025-01-09 02:43:33,641-05 INFO [org.ovirt.engine.core.vdsbroker.gluster.GlusterVolumesListVDSCommand] (EE-ManagedScheduledExecutorService-engineScheduledThreadPool-Thread-41) [] START, GlusterVolumesListVDSCommand(HostName = kvm04, GlusterVolumesListVDSParameters:{hostId='1408650c-aa50-452a-b2cc-24d36ae3cc20'}), log id: 6a99b1ab 2025-01-09 02:43:33,748-05 INFO [org.ovirt.engine.core.vdsbroker.gluster.GlusterVolumesListVDSCommand] (EE-ManagedScheduledExecutorService-engineScheduledThreadPool-Thread-41) [] FINISH, GlusterVolumesListVDSCommand, return: {b79a00d7-cc2a-48d8-b881-699991f3d930=org.ovirt.engine.core.common.businessentities.gluster.GlusterVolumeEntity@205714d2}, log id: 6a99b1ab 2025-01-09 02:43:37,689-05 ERROR [org.ovirt.engine.core.vdsbroker.vdsbroker.SpmStatusVDSCommand] (EE-ManagedScheduledExecutorService-engineScheduledThreadPool-Thread-32) [] Command 'SpmStatusVDSCommand(HostName = kvm02, SpmStatusVDSCommandParameters:{hostId='cf4a626e-ad6e-4cc4-95f0-4d7c1329f0f6', storagePoolId='39acc250-bfba-11ef-adb1-70e2840574e3'})' execution failed: IRSGenericException: IRSErrorException: IRSNoMasterDomainException: Error validating master storage domain: ('MD read error',) 2025-01-09 02:43:37,691-05 INFO [org.ovirt.engine.core.vdsbroker.vdsbroker.HSMGetAllTasksStatusesVDSCommand] (EE-ManagedScheduledExecutorService-engineScheduledThreadPool-Thread-32) [] Command 'org.ovirt.engine.core.vdsbroker.vdsbroker.HSMGetAllTasksStatusesVDSCommand' return value ' TaskStatusListReturn:{status='Status [code=654, message=Not SPM: ()]'} ' 2025-01-09 02:43:37,691-05 INFO [org.ovirt.engine.core.vdsbroker.vdsbroker.HSMGetAllTasksStatusesVDSCommand] (EE-ManagedScheduledExecutorService-engineScheduledThreadPool-Thread-32) [] HostName = kvm02 2025-01-09 02:43:37,691-05 ERROR [org.ovirt.engine.core.vdsbroker.vdsbroker.HSMGetAllTasksStatusesVDSCommand] (EE-ManagedScheduledExecutorService-engineScheduledThreadPool-Thread-32) [] Command 'HSMGetAllTasksStatusesVDSCommand(HostName = kvm02, VdsIdVDSCommandParametersBase:{hostId='cf4a626e-ad6e-4cc4-95f0-4d7c1329f0f6'})' execution failed: IRSGenericException: IRSErrorException: IRSNonOperationalException: Not SPM: () 2025-01-09 02:43:37,698-05 INFO [org.ovirt.engine.core.vdsbroker.vdsbroker.ConnectStoragePoolVDSCommand] (EE-ManagedScheduledExecutorService-engineScheduledThreadPool-Thread-32) [] START, ConnectStoragePoolVDSCommand(HostName = kvm01, ConnectStoragePoolVDSCommandParameters:{hostId='acf66bf6-e159-4705-be47-484c99e746dd', vdsId='acf66bf6-e159-4705-be47-484c99e746dd', storagePoolId='39acc250-bfba-11ef-adb1-70e2840574e3', masterVersion='766'}), log id: 201774c9 2025-01-09 02:43:37,698-05 INFO [org.ovirt.engine.core.vdsbroker.vdsbroker.ConnectStoragePoolVDSCommand] (EE-ManagedScheduledExecutorService-engineScheduledThreadPool-Thread-32) [] Executing with domain map: {bd71c70a-7d9b-4cb4-ae5d-4c75a96a1d94=active, ca1b2f4f-26cf-48cc-b219-7295a763b1ae=active, 028f24af-01a2-4c0d-9374-7d8be6e12469=active} 2025-01-09 02:43:38,381-05 INFO [org.ovirt.engine.core.bll.scheduling.policyunits.EvenGuestDistributionBalancePolicyUnit] (EE-ManagedScheduledExecutorService-engineScheduledThreadPool-Thread-10) [] There is no host with more than 10 running guests, no balancing is needed 2025-01-09 02:43:38,400-05 INFO [org.ovirt.engine.core.vdsbroker.gluster.GlusterTasksListVDSCommand] (EE-ManagedScheduledExecutorService-engineScheduledThreadPool-Thread-59) [] START, GlusterTasksListVDSCommand(HostName = kvm02, VdsIdVDSCommandParametersBase:{hostId='cf4a626e-ad6e-4cc4-95f0-4d7c1329f0f6'}), log id: 702ab66f 2025-01-09 02:43:38,515-05 INFO [org.ovirt.engine.core.vdsbroker.gluster.GlusterTasksListVDSCommand] (EE-ManagedScheduledExecutorService-engineScheduledThreadPool-Thread-59) [] FINISH, GlusterTasksListVDSCommand, return: [], log id: 702ab66f 2025-01-09 02:43:40,598-05 INFO [org.ovirt.engine.core.vdsbroker.vdsbroker.ConnectStoragePoolVDSCommand] (EE-ManagedScheduledExecutorService-engineScheduledThreadPool-Thread-32) [] Command 'org.ovirt.engine.core.vdsbroker.vdsbroker.ConnectStoragePoolVDSCommand' return value 'StatusOnlyReturn [status=Status [code=304, message=Cannot find master domain: 'spUUID=39acc250-bfba-11ef-adb1-70e2840574e3, msdUUID=bd71c70a-7d9b-4cb4-ae5d-4c75a96a1d94']]' 2025-01-09 02:43:40,598-05 INFO [org.ovirt.engine.core.vdsbroker.vdsbroker.ConnectStoragePoolVDSCommand] (EE-ManagedScheduledExecutorService-engineScheduledThreadPool-Thread-32) [] HostName = kvm01 2025-01-09 02:43:40,598-05 ERROR [org.ovirt.engine.core.vdsbroker.vdsbroker.ConnectStoragePoolVDSCommand] (EE-ManagedScheduledExecutorService-engineScheduledThreadPool-Thread-32) [] Command 'ConnectStoragePoolVDSCommand(HostName = kvm01, ConnectStoragePoolVDSCommandParameters:{hostId='acf66bf6-e159-4705-be47-484c99e746dd', vdsId='acf66bf6-e159-4705-be47-484c99e746dd', storagePoolId='39acc250-bfba-11ef-adb1-70e2840574e3', masterVersion='766'})' execution failed: IRSGenericException: IRSErrorException: IRSNoMasterDomainException: Cannot find master domain: 'spUUID=39acc250-bfba-11ef-adb1-70e2840574e3, msdUUID=bd71c70a-7d9b-4cb4-ae5d-4c75a96a1d94' 2025-01-09 02:43:40,598-05 INFO [org.ovirt.engine.core.vdsbroker.vdsbroker.ConnectStoragePoolVDSCommand] (EE-ManagedScheduledExecutorService-engineScheduledThreadPool-Thread-32) [] FINISH, ConnectStoragePoolVDSCommand, return: , log id: 201774c9 2025-01-09 02:43:40,598-05 ERROR [org.ovirt.engine.core.vdsbroker.irsbroker.IrsBrokerCommand] (EE-ManagedScheduledExecutorService-engineScheduledThreadPool-Thread-32) [] IrsBroker::Failed::GetStoragePoolInfoVDS: IRSGenericException: IRSErrorException: IRSNoMasterDomainException: Cannot find master domain: 'spUUID=39acc250-bfba-11ef-adb1-70e2840574e3, msdUUID=bd71c70a-7d9b-4cb4-ae5d-4c75a96a1d94' 2025-01-09 02:43:40,606-05 INFO [org.ovirt.engine.core.vdsbroker.irsbroker.IrsProxyImpl] (EE-ManagedScheduledExecutorService-engineScheduledThreadPool-Thread-32) [] Irs placed on server 'acf66bf6-e159-4705-be47-484c99e746dd' failed. Proceed Failover 2025-01-09 02:43:40,611-05 INFO [org.ovirt.engine.core.vdsbroker.vdsbroker.ConnectStoragePoolVDSCommand] (EE-ManagedScheduledExecutorService-engineScheduledThreadPool-Thread-32) [] START, ConnectStoragePoolVDSCommand(HostName = kvm04, ConnectStoragePoolVDSCommandParameters:{hostId='1408650c-aa50-452a-b2cc-24d36ae3cc20', vdsId='1408650c-aa50-452a-b2cc-24d36ae3cc20', storagePoolId='39acc250-bfba-11ef-adb1-70e2840574e3', masterVersion='766'}), log id: 164d36cb 2025-01-09 02:43:40,611-05 INFO [org.ovirt.engine.core.vdsbroker.vdsbroker.ConnectStoragePoolVDSCommand] (EE-ManagedScheduledExecutorService-engineScheduledThreadPool-Thread-32) [] Executing with domain map: {bd71c70a-7d9b-4cb4-ae5d-4c75a96a1d94=active, ca1b2f4f-26cf-48cc-b219-7295a763b1ae=active, 028f24af-01a2-4c0d-9374-7d8be6e12469=active} 2025-01-09 02:43:42,950-05 ERROR [org.ovirt.engine.core.dal.dbbroker.auditloghandling.AuditLogDirector] (EE-ManagedScheduledExecutorService-engineScheduledThreadPool-Thread-32) [] EVENT_ID: VDS_BROKER_COMMAND_FAILURE(10,802), VDSM kvm04 command ConnectStoragePoolVDS failed: Cannot find master domain: 'spUUID=39acc250-bfba-11ef-adb1-70e2840574e3, msdUUID=bd71c70a-7d9b-4cb4-ae5d-4c75a96a1d94' 2025-01-09 02:43:42,950-05 INFO [org.ovirt.engine.core.vdsbroker.vdsbroker.ConnectStoragePoolVDSCommand] (EE-ManagedScheduledExecutorService-engineScheduledThreadPool-Thread-32) [] Command 'org.ovirt.engine.core.vdsbroker.vdsbroker.ConnectStoragePoolVDSCommand' return value 'StatusOnlyReturn [status=Status [code=304, message=Cannot find master domain: 'spUUID=39acc250-bfba-11ef-adb1-70e2840574e3, msdUUID=bd71c70a-7d9b-4cb4-ae5d-4c75a96a1d94']]' 2025-01-09 02:43:42,950-05 INFO [org.ovirt.engine.core.vdsbroker.vdsbroker.ConnectStoragePoolVDSCommand] (EE-ManagedScheduledExecutorService-engineScheduledThreadPool-Thread-32) [] HostName = kvm04 2025-01-09 02:43:42,950-05 ERROR [org.ovirt.engine.core.vdsbroker.vdsbroker.ConnectStoragePoolVDSCommand] (EE-ManagedScheduledExecutorService-engineScheduledThreadPool-Thread-32) [] Command 'ConnectStoragePoolVDSCommand(HostName = kvm04, ConnectStoragePoolVDSCommandParameters:{hostId='1408650c-aa50-452a-b2cc-24d36ae3cc20', vdsId='1408650c-aa50-452a-b2cc-24d36ae3cc20', storagePoolId='39acc250-bfba-11ef-adb1-70e2840574e3', masterVersion='766'})' execution failed: IRSGenericException: IRSErrorException: IRSNoMasterDomainException: Cannot find master domain: 'spUUID=39acc250-bfba-11ef-adb1-70e2840574e3, msdUUID=bd71c70a-7d9b-4cb4-ae5d-4c75a96a1d94' 2025-01-09 02:43:42,950-05 INFO [org.ovirt.engine.core.vdsbroker.vdsbroker.ConnectStoragePoolVDSCommand] (EE-ManagedScheduledExecutorService-engineScheduledThreadPool-Thread-32) [] FINISH, ConnectStoragePoolVDSCommand, return: , log id: 164d36cb 2025-01-09 02:43:42,950-05 ERROR [org.ovirt.engine.core.vdsbroker.irsbroker.IrsBrokerCommand] (EE-ManagedScheduledExecutorService-engineScheduledThreadPool-Thread-32) [] IrsBroker::Failed::GetStoragePoolInfoVDS: IRSGenericException: IRSErrorException: IRSNoMasterDomainException: Cannot find master domain: 'spUUID=39acc250-bfba-11ef-adb1-70e2840574e3, msdUUID=bd71c70a-7d9b-4cb4-ae5d-4c75a96a1d94' 2025-01-09 02:43:42,960-05 ERROR [org.ovirt.engine.core.vdsbroker.vdsbroker.SpmStatusVDSCommand] (EE-ManagedScheduledExecutorService-engineScheduledThreadPool-Thread-32) [] Command 'SpmStatusVDSCommand(HostName = kvm04, SpmStatusVDSCommandParameters:{hostId='1408650c-aa50-452a-b2cc-24d36ae3cc20', storagePoolId='39acc250-bfba-11ef-adb1-70e2840574e3'})' execution failed: IRSGenericException: IRSErrorException: IRSNoMasterDomainException: Error validating master storage domain: ('MD read error',) 2025-01-09 02:43:42,960-05 INFO [org.ovirt.engine.core.vdsbroker.irsbroker.IrsProxyImpl] (EE-ManagedScheduledExecutorService-engineScheduledThreadPool-Thread-32) [] Irs placed on server '1408650c-aa50-452a-b2cc-24d36ae3cc20' failed. Proceed Failover 2025-01-09 02:43:42,964-05 INFO [org.ovirt.engine.core.vdsbroker.vdsbroker.ConnectStoragePoolVDSCommand] (EE-ManagedScheduledExecutorService-engineScheduledThreadPool-Thread-32) [] START, ConnectStoragePoolVDSCommand(HostName = kvm02, ConnectStoragePoolVDSCommandParameters:{hostId='cf4a626e-ad6e-4cc4-95f0-4d7c1329f0f6', vdsId='cf4a626e-ad6e-4cc4-95f0-4d7c1329f0f6', storagePoolId='39acc250-bfba-11ef-adb1-70e2840574e3', masterVersion='766'}), log id: 863c329 2025-01-09 02:43:42,965-05 INFO [org.ovirt.engine.core.vdsbroker.vdsbroker.ConnectStoragePoolVDSCommand] (EE-ManagedScheduledExecutorService-engineScheduledThreadPool-Thread-32) [] Executing with domain map: {bd71c70a-7d9b-4cb4-ae5d-4c75a96a1d94=active, ca1b2f4f-26cf-48cc-b219-7295a763b1ae=active, 028f24af-01a2-4c0d-9374-7d8be6e12469=active} 2025-01-09 02:43:45,328-05 INFO [org.ovirt.engine.core.vdsbroker.vdsbroker.ConnectStoragePoolVDSCommand] (EE-ManagedScheduledExecutorService-engineScheduledThreadPool-Thread-32) [] Command 'org.ovirt.engine.core.vdsbroker.vdsbroker.ConnectStoragePoolVDSCommand' return value 'StatusOnlyReturn [status=Status [code=304, message=Cannot find master domain: 'spUUID=39acc250-bfba-11ef-adb1-70e2840574e3, msdUUID=bd71c70a-7d9b-4cb4-ae5d-4c75a96a1d94']]' 2025-01-09 02:43:45,328-05 INFO [org.ovirt.engine.core.vdsbroker.vdsbroker.ConnectStoragePoolVDSCommand] (EE-ManagedScheduledExecutorService-engineScheduledThreadPool-Thread-32) [] HostName = kvm02 2025-01-09 02:43:45,328-05 ERROR [org.ovirt.engine.core.vdsbroker.vdsbroker.ConnectStoragePoolVDSCommand] (EE-ManagedScheduledExecutorService-engineScheduledThreadPool-Thread-32) [] Command 'ConnectStoragePoolVDSCommand(HostName = kvm02, ConnectStoragePoolVDSCommandParameters:{hostId='cf4a626e-ad6e-4cc4-95f0-4d7c1329f0f6', vdsId='cf4a626e-ad6e-4cc4-95f0-4d7c1329f0f6', storagePoolId='39acc250-bfba-11ef-adb1-70e2840574e3', masterVersion='766'})' execution failed: IRSGenericException: IRSErrorException: IRSNoMasterDomainException: Cannot find master domain: 'spUUID=39acc250-bfba-11ef-adb1-70e2840574e3, msdUUID=bd71c70a-7d9b-4cb4-ae5d-4c75a96a1d94' 2025-01-09 02:43:45,328-05 INFO [org.ovirt.engine.core.vdsbroker.vdsbroker.ConnectStoragePoolVDSCommand] (EE-ManagedScheduledExecutorService-engineScheduledThreadPool-Thread-32) [] FINISH, ConnectStoragePoolVDSCommand, return: , log id: 863c329 2025-01-09 02:43:45,328-05 ERROR [org.ovirt.engine.core.vdsbroker.irsbroker.IrsBrokerCommand] (EE-ManagedScheduledExecutorService-engineScheduledThreadPool-Thread-32) [] IrsBroker::Failed::GetStoragePoolInfoVDS: IRSGenericException: IRSErrorException: IRSNoMasterDomainException: Cannot find master domain: 'spUUID=39acc250-bfba-11ef-adb1-70e2840574e3, msdUUID=bd71c70a-7d9b-4cb4-ae5d-4c75a96a1d94' 2025-01-09 02:43:45,388-05 INFO [org.ovirt.engine.core.bll.storage.pool.ReconstructMasterDomainCommand] (EE-ManagedThreadFactory-engine-Thread-580) [2e23e34] Running command: ReconstructMasterDomainCommand internal: true. Entities affected : ID: bd71c70a-7d9b-4cb4-ae5d-4c75a96a1d94 Type: Storage 2025-01-09 02:43:45,412-05 WARN [org.ovirt.engine.core.bll.storage.pool.ReconstructMasterDomainCommand] (EE-ManagedThreadFactory-engine-Thread-580) [2e23e34] skipping connect and refresh for all hosts, last master 'true', command status 'true' 2025-01-09 02:43:45,436-05 INFO [org.ovirt.engine.core.bll.eventqueue.EventQueueMonitor] (EE-ManagedThreadFactory-engine-Thread-579) [] Finished reconstruct for pool '39acc250-bfba-11ef-adb1-70e2840574e3'. Clearing event queue 2025-01-09 02:43:48,763-05 INFO [org.ovirt.engine.core.vdsbroker.gluster.GlusterServersListVDSCommand] (EE-ManagedScheduledExecutorService-engineScheduledThreadPool-Thread-96) [] START, GlusterServersListVDSCommand(HostName = kvm04, VdsIdVDSCommandParametersBase:{hostId='1408650c-aa50-452a-b2cc-24d36ae3cc20'}), log id: db1640e 2025-01-09 02:43:48,970-05 INFO [org.ovirt.engine.core.vdsbroker.gluster.GlusterServersListVDSCommand] (EE-ManagedScheduledExecutorService-engineScheduledThreadPool-Thread-96) [] FINISH, GlusterServersListVDSCommand, return: [192.168.254.108/24:CONNECTED, ovirt.digitaldatatechs.com:CONNECTED, kvm02.digitaldatatechs.com:CONNECTED, kvm01.digitaldatatechs.com:CONNECTED], log id: db1640e 2025-01-09 02:43:48,983-05 INFO [org.ovirt.engine.core.vdsbroker.gluster.AddGlusterServerVDSCommand] (EE-ManagedScheduledExecutorService-engineScheduledThreadPool-Thread-96) [] START, AddGlusterServerVDSCommand(HostName = kvm04, AddGlusterServerVDSParameters:{hostId='1408650c-aa50-452a-b2cc-24d36ae3cc20'}), log id: 717f84ae 2025-01-09 02:43:52,255-05 INFO [org.ovirt.engine.core.vdsbroker.gluster.AddGlusterServerVDSCommand] (EE-ManagedScheduledExecutorService-engineScheduledThreadPool-Thread-96) [] FINISH, AddGlusterServerVDSCommand, return: , log id: 717f84ae 2025-01-09 02:43:52,277-05 ERROR [org.ovirt.engine.core.dal.dbbroker.auditloghandling.AuditLogDirector] (EE-ManagedScheduledExecutorService-engineScheduledThreadPool-Thread-96) [] EVENT_ID: GLUSTER_SERVER_ADD_FAILED(4,436), Failed to add host kvm04 into Cluster dds_cluster. Add host failed: rc=107 out=() err=['Probe returned with Transport endpoint is not connected'] 2025-01-09 02:43:52,278-05 INFO [org.ovirt.engine.core.vdsbroker.gluster.GlusterVolumesListVDSCommand] (EE-ManagedScheduledExecutorService-engineScheduledThreadPool-Thread-96) [] START, GlusterVolumesListVDSCommand(HostName = kvm04, GlusterVolumesListVDSParameters:{hostId='1408650c-aa50-452a-b2cc-24d36ae3cc20'}), log id: 6a43f9e6 2025-01-09 02:43:52,384-05 INFO [org.ovirt.engine.core.vdsbroker.gluster.GlusterVolumesListVDSCommand] (EE-ManagedScheduledExecutorService-engineScheduledThreadPool-Thread-96) [] FINISH, GlusterVolumesListVDSCommand, return: {b79a00d7-cc2a-48d8-b881-699991f3d930=org.ovirt.engine.core.common.businessentities.gluster.GlusterVolumeEntity@205714d2}, log id: 6a43f9e6 2025-01-09 02:43:55,341-05 ERROR [org.ovirt.engine.core.vdsbroker.vdsbroker.SpmStatusVDSCommand] (EE-ManagedScheduledExecutorService-engineScheduledThreadPool-Thread-40) [] Command 'SpmStatusVDSCommand(HostName = kvm02, SpmStatusVDSCommandParameters:{hostId='cf4a626e-ad6e-4cc4-95f0-4d7c1329f0f6', storagePoolId='39acc250-bfba-11ef-adb1-70e2840574e3'})' execution failed: IRSGenericException: IRSErrorException: IRSNoMasterDomainException: Error validating master storage domain: ('MD read error',) 2025-01-09 02:43:55,343-05 INFO [org.ovirt.engine.core.vdsbroker.vdsbroker.HSMGetAllTasksStatusesVDSCommand] (EE-ManagedScheduledExecutorService-engineScheduledThreadPool-Thread-40) [] Command 'org.ovirt.engine.core.vdsbroker.vdsbroker.HSMGetAllTasksStatusesVDSCommand' return value ' TaskStatusListReturn:{status='Status [code=654, message=Not SPM: ()]'} ' 2025-01-09 02:43:55,343-05 INFO [org.ovirt.engine.core.vdsbroker.vdsbroker.HSMGetAllTasksStatusesVDSCommand] (EE-ManagedScheduledExecutorService-engineScheduledThreadPool-Thread-40) [] HostName = kvm02 2025-01-09 02:43:55,343-05 ERROR [org.ovirt.engine.core.vdsbroker.vdsbroker.HSMGetAllTasksStatusesVDSCommand] (EE-ManagedScheduledExecutorService-engineScheduledThreadPool-Thread-40) [] Command 'HSMGetAllTasksStatusesVDSCommand(HostName = kvm02, VdsIdVDSCommandParametersBase:{hostId='cf4a626e-ad6e-4cc4-95f0-4d7c1329f0f6'})' execution failed: IRSGenericException: IRSErrorException: IRSNonOperationalException: Not SPM: () 2025-01-09 02:43:55,350-05 INFO [org.ovirt.engine.core.vdsbroker.vdsbroker.ConnectStoragePoolVDSCommand] (EE-ManagedScheduledExecutorService-engineScheduledThreadPool-Thread-40) [] START, ConnectStoragePoolVDSCommand(HostName = kvm04, ConnectStoragePoolVDSCommandParameters:{hostId='1408650c-aa50-452a-b2cc-24d36ae3cc20', vdsId='1408650c-aa50-452a-b2cc-24d36ae3cc20', storagePoolId='39acc250-bfba-11ef-adb1-70e2840574e3', masterVersion='767'}), log id: 35cf220b 2025-01-09 02:43:55,350-05 INFO [org.ovirt.engine.core.vdsbroker.vdsbroker.ConnectStoragePoolVDSCommand] (EE-ManagedScheduledExecutorService-engineScheduledThreadPool-Thread-40) [] Executing with domain map: {bd71c70a-7d9b-4cb4-ae5d-4c75a96a1d94=active, ca1b2f4f-26cf-48cc-b219-7295a763b1ae=active, 028f24af-01a2-4c0d-9374-7d8be6e12469=active} 2025-01-09 02:43:57,661-05 INFO [org.ovirt.engine.core.vdsbroker.vdsbroker.ConnectStoragePoolVDSCommand] (EE-ManagedScheduledExecutorService-engineScheduledThreadPool-Thread-40) [] Command 'org.ovirt.engine.core.vdsbroker.vdsbroker.ConnectStoragePoolVDSCommand' return value 'StatusOnlyReturn [status=Status [code=304, message=Cannot find master domain: 'spUUID=39acc250-bfba-11ef-adb1-70e2840574e3, msdUUID=bd71c70a-7d9b-4cb4-ae5d-4c75a96a1d94']]' 2025-01-09 02:43:57,661-05 INFO [org.ovirt.engine.core.vdsbroker.vdsbroker.ConnectStoragePoolVDSCommand] (EE-ManagedScheduledExecutorService-engineScheduledThreadPool-Thread-40) [] HostName = kvm04 2025-01-09 02:43:57,661-05 ERROR [org.ovirt.engine.core.vdsbroker.vdsbroker.ConnectStoragePoolVDSCommand] (EE-ManagedScheduledExecutorService-engineScheduledThreadPool-Thread-40) [] Command 'ConnectStoragePoolVDSCommand(HostName = kvm04, ConnectStoragePoolVDSCommandParameters:{hostId='1408650c-aa50-452a-b2cc-24d36ae3cc20', vdsId='1408650c-aa50-452a-b2cc-24d36ae3cc20', storagePoolId='39acc250-bfba-11ef-adb1-70e2840574e3', masterVersion='767'})' execution failed: IRSGenericException: IRSErrorException: IRSNoMasterDomainException: Cannot find master domain: 'spUUID=39acc250-bfba-11ef-adb1-70e2840574e3, msdUUID=bd71c70a-7d9b-4cb4-ae5d-4c75a96a1d94' 2025-01-09 02:43:57,661-05 INFO [org.ovirt.engine.core.vdsbroker.vdsbroker.ConnectStoragePoolVDSCommand] (EE-ManagedScheduledExecutorService-engineScheduledThreadPool-Thread-40) [] FINISH, ConnectStoragePoolVDSCommand, return: , log id: 35cf220b 2025-01-09 02:43:57,661-05 ERROR [org.ovirt.engine.core.vdsbroker.irsbroker.IrsBrokerCommand] (EE-ManagedScheduledExecutorService-engineScheduledThreadPool-Thread-40) [] IrsBroker::Failed::GetStoragePoolInfoVDS: IRSGenericException: IRSErrorException: IRSNoMasterDomainException: Cannot find master domain: 'spUUID=39acc250-bfba-11ef-adb1-70e2840574e3, msdUUID=bd71c70a-7d9b-4cb4-ae5d-4c75a96a1d94' 2025-01-09 02:43:57,671-05 ERROR [org.ovirt.engine.core.vdsbroker.vdsbroker.SpmStatusVDSCommand] (EE-ManagedScheduledExecutorService-engineScheduledThreadPool-Thread-40) [] Command 'SpmStatusVDSCommand(HostName = kvm04, SpmStatusVDSCommandParameters:{hostId='1408650c-aa50-452a-b2cc-24d36ae3cc20', storagePoolId='39acc250-bfba-11ef-adb1-70e2840574e3'})' execution failed: IRSGenericException: IRSErrorException: IRSNoMasterDomainException: Error validating master storage domain: ('MD read error',) 2025-01-09 02:43:57,671-05 INFO [org.ovirt.engine.core.vdsbroker.irsbroker.IrsProxyImpl] (EE-ManagedScheduledExecutorService-engineScheduledThreadPool-Thread-40) [] Irs placed on server '1408650c-aa50-452a-b2cc-24d36ae3cc20' failed. Proceed Failover 2025-01-09 02:43:57,675-05 INFO [org.ovirt.engine.core.vdsbroker.vdsbroker.ConnectStoragePoolVDSCommand] (EE-ManagedScheduledExecutorService-engineScheduledThreadPool-Thread-40) [] START, ConnectStoragePoolVDSCommand(HostName = kvm02, ConnectStoragePoolVDSCommandParameters:{hostId='cf4a626e-ad6e-4cc4-95f0-4d7c1329f0f6', vdsId='cf4a626e-ad6e-4cc4-95f0-4d7c1329f0f6', storagePoolId='39acc250-bfba-11ef-adb1-70e2840574e3', masterVersion='767'}), log id: 2ff24c19 2025-01-09 02:43:57,676-05 INFO [org.ovirt.engine.core.vdsbroker.vdsbroker.ConnectStoragePoolVDSCommand] (EE-ManagedScheduledExecutorService-engineScheduledThreadPool-Thread-40) [] Executing with domain map: {bd71c70a-7d9b-4cb4-ae5d-4c75a96a1d94=active, ca1b2f4f-26cf-48cc-b219-7295a763b1ae=active, 028f24af-01a2-4c0d-9374-7d8be6e12469=active} 2025-01-09 02:44:00,065-05 ERROR [org.ovirt.engine.core.dal.dbbroker.auditloghandling.AuditLogDirector] (EE-ManagedScheduledExecutorService-engineScheduledThreadPool-Thread-40) [] EVENT_ID: VDS_BROKER_COMMAND_FAILURE(10,802), VDSM kvm02 command ConnectStoragePoolVDS failed: Cannot find master domain: 'spUUID=39acc250-bfba-11ef-adb1-70e2840574e3, msdUUID=bd71c70a-7d9b-4cb4-ae5d-4c75a96a1d94' 2025-01-09 02:44:00,065-05 INFO [org.ovirt.engine.core.vdsbroker.vdsbroker.ConnectStoragePoolVDSCommand] (EE-ManagedScheduledExecutorService-engineScheduledThreadPool-Thread-40) [] Command 'org.ovirt.engine.core.vdsbroker.vdsbroker.ConnectStoragePoolVDSCommand' return value 'StatusOnlyReturn [status=Status [code=304, message=Cannot find master domain: 'spUUID=39acc250-bfba-11ef-adb1-70e2840574e3, msdUUID=bd71c70a-7d9b-4cb4-ae5d-4c75a96a1d94']]' 2025-01-09 02:44:00,065-05 INFO [org.ovirt.engine.core.vdsbroker.vdsbroker.ConnectStoragePoolVDSCommand] (EE-ManagedScheduledExecutorService-engineScheduledThreadPool-Thread-40) [] HostName = kvm02 2025-01-09 02:44:00,065-05 ERROR [org.ovirt.engine.core.vdsbroker.vdsbroker.ConnectStoragePoolVDSCommand] (EE-ManagedScheduledExecutorService-engineScheduledThreadPool-Thread-40) [] Command 'ConnectStoragePoolVDSCommand(HostName = kvm02, ConnectStoragePoolVDSCommandParameters:{hostId='cf4a626e-ad6e-4cc4-95f0-4d7c1329f0f6', vdsId='cf4a626e-ad6e-4cc4-95f0-4d7c1329f0f6', storagePoolId='39acc250-bfba-11ef-adb1-70e2840574e3', masterVersion='767'})' execution failed: IRSGenericException: IRSErrorException: IRSNoMasterDomainException: Cannot find master domain: 'spUUID=39acc250-bfba-11ef-adb1-70e2840574e3, msdUUID=bd71c70a-7d9b-4cb4-ae5d-4c75a96a1d94' 2025-01-09 02:44:00,065-05 INFO [org.ovirt.engine.core.vdsbroker.vdsbroker.ConnectStoragePoolVDSCommand] (EE-ManagedScheduledExecutorService-engineScheduledThreadPool-Thread-40) [] FINISH, ConnectStoragePoolVDSCommand, return: , log id: 2ff24c19 2025-01-09 02:44:00,065-05 ERROR [org.ovirt.engine.core.vdsbroker.irsbroker.IrsBrokerCommand] (EE-ManagedScheduledExecutorService-engineScheduledThreadPool-Thread-40) [] IrsBroker::Failed::GetStoragePoolInfoVDS: IRSGenericException: IRSErrorException: IRSNoMasterDomainException: Cannot find master domain: 'spUUID=39acc250-bfba-11ef-adb1-70e2840574e3, msdUUID=bd71c70a-7d9b-4cb4-ae5d-4c75a96a1d94' 2025-01-09 02:44:04,367-05 ERROR [org.ovirt.engine.core.vdsbroker.vdsbroker.SpmStatusVDSCommand] (EE-ManagedScheduledExecutorService-engineScheduledThreadPool-Thread-40) [] Command 'SpmStatusVDSCommand(HostName = kvm02, SpmStatusVDSCommandParameters:{hostId='cf4a626e-ad6e-4cc4-95f0-4d7c1329f0f6', storagePoolId='39acc250-bfba-11ef-adb1-70e2840574e3'})' execution failed: IRSGenericException: IRSErrorException: IRSNoMasterDomainException: Error validating master storage domain: ('MD read error',) 2025-01-09 02:44:04,367-05 INFO [org.ovirt.engine.core.vdsbroker.irsbroker.IrsProxyImpl] (EE-ManagedScheduledExecutorService-engineScheduledThreadPool-Thread-40) [] Irs placed on server 'cf4a626e-ad6e-4cc4-95f0-4d7c1329f0f6' failed. Proceed Failover 2025-01-09 02:44:04,372-05 INFO [org.ovirt.engine.core.vdsbroker.vdsbroker.ConnectStoragePoolVDSCommand] (EE-ManagedScheduledExecutorService-engineScheduledThreadPool-Thread-40) [] START, ConnectStoragePoolVDSCommand(HostName = kvm01, ConnectStoragePoolVDSCommandParameters:{hostId='acf66bf6-e159-4705-be47-484c99e746dd', vdsId='acf66bf6-e159-4705-be47-484c99e746dd', storagePoolId='39acc250-bfba-11ef-adb1-70e2840574e3', masterVersion='767'}), log id: 23ed7788 2025-01-09 02:44:04,372-05 INFO [org.ovirt.engine.core.vdsbroker.vdsbroker.ConnectStoragePoolVDSCommand] (EE-ManagedScheduledExecutorService-engineScheduledThreadPool-Thread-40) [] Executing with domain map: {bd71c70a-7d9b-4cb4-ae5d-4c75a96a1d94=active, ca1b2f4f-26cf-48cc-b219-7295a763b1ae=active, 028f24af-01a2-4c0d-9374-7d8be6e12469=active} 2025-01-09 02:44:07,292-05 ERROR [org.ovirt.engine.core.dal.dbbroker.auditloghandling.AuditLogDirector] (EE-ManagedScheduledExecutorService-engineScheduledThreadPool-Thread-40) [] EVENT_ID: VDS_BROKER_COMMAND_FAILURE(10,802), VDSM kvm01 command ConnectStoragePoolVDS failed: Cannot find master domain: 'spUUID=39acc250-bfba-11ef-adb1-70e2840574e3, msdUUID=bd71c70a-7d9b-4cb4-ae5d-4c75a96a1d94' 2025-01-09 02:44:07,292-05 INFO [org.ovirt.engine.core.vdsbroker.vdsbroker.ConnectStoragePoolVDSCommand] (EE-ManagedScheduledExecutorService-engineScheduledThreadPool-Thread-40) [] Command 'org.ovirt.engine.core.vdsbroker.vdsbroker.ConnectStoragePoolVDSCommand' return value 'StatusOnlyReturn [status=Status [code=304, message=Cannot find master domain: 'spUUID=39acc250-bfba-11ef-adb1-70e2840574e3, msdUUID=bd71c70a-7d9b-4cb4-ae5d-4c75a96a1d94']]' 2025-01-09 02:44:07,292-05 INFO [org.ovirt.engine.core.vdsbroker.vdsbroker.ConnectStoragePoolVDSCommand] (EE-ManagedScheduledExecutorService-engineScheduledThreadPool-Thread-40) [] HostName = kvm01 2025-01-09 02:44:07,292-05 ERROR [org.ovirt.engine.core.vdsbroker.vdsbroker.ConnectStoragePoolVDSCommand] (EE-ManagedScheduledExecutorService-engineScheduledThreadPool-Thread-40) [] Command 'ConnectStoragePoolVDSCommand(HostName = kvm01, ConnectStoragePoolVDSCommandParameters:{hostId='acf66bf6-e159-4705-be47-484c99e746dd', vdsId='acf66bf6-e159-4705-be47-484c99e746dd', storagePoolId='39acc250-bfba-11ef-adb1-70e2840574e3', masterVersion='767'})' execution failed: IRSGenericException: IRSErrorException: IRSNoMasterDomainException: Cannot find master domain: 'spUUID=39acc250-bfba-11ef-adb1-70e2840574e3, msdUUID=bd71c70a-7d9b-4cb4-ae5d-4c75a96a1d94' 2025-01-09 02:44:07,292-05 INFO [org.ovirt.engine.core.vdsbroker.vdsbroker.ConnectStoragePoolVDSCommand] (EE-ManagedScheduledExecutorService-engineScheduledThreadPool-Thread-40) [] FINISH, ConnectStoragePoolVDSCommand, return: , log id: 23ed7788 2025-01-09 02:44:07,292-05 ERROR [org.ovirt.engine.core.vdsbroker.irsbroker.IrsBrokerCommand] (EE-ManagedScheduledExecutorService-engineScheduledThreadPool-Thread-40) [] IrsBroker::Failed::GetStoragePoolInfoVDS: IRSGenericException: IRSErrorException: IRSNoMasterDomainException: Cannot find master domain: 'spUUID=39acc250-bfba-11ef-adb1-70e2840574e3, msdUUID=bd71c70a-7d9b-4cb4-ae5d-4c75a96a1d94' 2025-01-09 02:44:07,375-05 INFO [org.ovirt.engine.core.bll.storage.pool.ReconstructMasterDomainCommand] (EE-ManagedThreadFactory-engine-Thread-594) [6e7f60bb] Running command: ReconstructMasterDomainCommand internal: true. Entities affected : ID: bd71c70a-7d9b-4cb4-ae5d-4c75a96a1d94 Type: Storage 2025-01-09 02:44:07,412-05 INFO [org.ovirt.engine.core.bll.network.host.SyncAllHostNetworksCommand] (default task-7) [9913b205-9b2e-4f8a-86e3-048a9cdff2ba] Running command: SyncAllHostNetworksCommand internal: false. Entities affected : ID: acf66bf6-e159-4705-be47-484c99e746dd Type: VDSAction group CONFIGURE_HOST_NETWORK with role type ADMIN 2025-01-09 02:44:07,421-05 INFO [org.ovirt.engine.core.vdsbroker.gluster.GlusterServersListVDSCommand] (EE-ManagedScheduledExecutorService-engineScheduledThreadPool-Thread-25) [] START, GlusterServersListVDSCommand(HostName = kvm04, VdsIdVDSCommandParametersBase:{hostId='1408650c-aa50-452a-b2cc-24d36ae3cc20'}), log id: 340c8f9d 2025-01-09 02:44:07,423-05 WARN [org.ovirt.engine.core.bll.storage.pool.ReconstructMasterDomainCommand] (EE-ManagedThreadFactory-engine-Thread-594) [6e7f60bb] skipping connect and refresh for all hosts, last master 'true', command status 'true' 2025-01-09 02:44:07,423-05 INFO [org.ovirt.engine.core.bll.network.host.PersistentHostSetupNetworksCommand] (default task-7) [7814c7ae] Running command: PersistentHostSetupNetworksCommand internal: true. Entities affected : ID: acf66bf6-e159-4705-be47-484c99e746dd Type: VDS 2025-01-09 02:44:07,448-05 INFO [org.ovirt.engine.core.bll.eventqueue.EventQueueMonitor] (EE-ManagedThreadFactory-engine-Thread-593) [] Finished reconstruct for pool '39acc250-bfba-11ef-adb1-70e2840574e3'. Clearing event queue 2025-01-09 02:44:07,460-05 INFO [org.ovirt.engine.core.dal.dbbroker.auditloghandling.AuditLogDirector] (default task-7) [7814c7ae] EVENT_ID: PERSIST_HOST_SETUP_NETWORK_ON_HOST(10,902), (1/1): Applying network's changes on host kvm01. (User: admin@ovirt@internalkeycloak-authz) 2025-01-09 02:44:07,461-05 INFO [org.ovirt.engine.core.bll.network.host.HostSetupNetworksCommand] (default task-7) [7814c7ae] Before acquiring lock-timeout 'EngineLock:{exclusiveLocks='[HOST_NETWORKacf66bf6-e159-4705-be47-484c99e746dd=HOST_NETWORK]', sharedLocks=''}' 2025-01-09 02:44:07,461-05 INFO [org.ovirt.engine.core.bll.network.host.HostSetupNetworksCommand] (default task-7) [7814c7ae] Lock-timeout acquired to object 'EngineLock:{exclusiveLocks='[HOST_NETWORKacf66bf6-e159-4705-be47-484c99e746dd=HOST_NETWORK]', sharedLocks=''}' 2025-01-09 02:44:07,497-05 INFO [org.ovirt.engine.core.bll.network.host.HostSetupNetworksCommand] (default task-7) [7814c7ae] Running command: HostSetupNetworksCommand internal: true. Entities affected : ID: acf66bf6-e159-4705-be47-484c99e746dd Type: VDSAction group CONFIGURE_HOST_NETWORK with role type ADMIN 2025-01-09 02:44:07,497-05 INFO [org.ovirt.engine.core.bll.network.host.HostSetupNetworksCommand] (default task-7) [7814c7ae] Before acquiring lock in order to prevent monitoring for host 'kvm01' from data-center 'DDS_Data_Center' 2025-01-09 02:44:07,497-05 INFO [org.ovirt.engine.core.bll.network.host.HostSetupNetworksCommand] (default task-7) [7814c7ae] Lock acquired, from now a monitoring of host will be skipped for host 'kvm01' from data-center 'DDS_Data_Center' 2025-01-09 02:44:07,498-05 INFO [org.ovirt.engine.core.vdsbroker.vdsbroker.HostSetupNetworksVDSCommand] (default task-7) [7814c7ae] START, HostSetupNetworksVDSCommand(HostName = kvm01, HostSetupNetworksVdsCommandParameters:{hostId='acf66bf6-e159-4705-be47-484c99e746dd', vds='Host[kvm01,acf66bf6-e159-4705-be47-484c99e746dd]', rollbackOnFailure='true', commitOnSuccess='true', connectivityTimeout='120', networks='[HostNetwork:{defaultRoute='false', bonding='false', networkName='vlan3', vdsmName='vlan3', nicName='eno2', vlan='3', vmNetwork='true', stp='false', properties='[]', ipv4BootProtocol='STATIC_IP', ipv4Address='172.50.26.252', ipv4Netmask='255.255.255.0', ipv4Gateway='172.50.26.1', ipv6BootProtocol='NONE', ipv6Address='null', ipv6Prefix='null', ipv6Gateway='null', nameServers='null'}, HostNetwork:{defaultRoute='true', bonding='false', networkName='ovirtmgmt', vdsmName='ovirtmgmt', nicName='eno1', vlan='null', vmNetwork='true', stp='false', properties='[]', ipv4BootProtocol='STAT IC_IP', ipv4Address='192.168.254.252', ipv4Netmask='255.255.255.0', ipv4Gateway='192.168.254.1', ipv6BootProtocol='NONE', ipv6Address='null', ipv6Prefix='null', ipv6Gateway='null', nameServers='[NameServer:{address='192.168.254.222'}, NameServer:{address='192.168.254.232'}]'}]', removedNetworks='[]', bonds='[]', removedBonds='[]', clusterSwitchType='LEGACY', managementNetworkChanged='true'}), log id: 56a27c18 2025-01-09 02:44:07,499-05 INFO [org.ovirt.engine.core.vdsbroker.vdsbroker.HostSetupNetworksVDSCommand] (default task-7) [7814c7ae] FINISH, HostSetupNetworksVDSCommand, return: , log id: 56a27c18 2025-01-09 02:44:07,631-05 INFO [org.ovirt.engine.core.vdsbroker.gluster.GlusterServersListVDSCommand] (EE-ManagedScheduledExecutorService-engineScheduledThreadPool-Thread-25) [] FINISH, GlusterServersListVDSCommand, return: [192.168.254.108/24:CONNECTED, ovirt.digitaldatatechs.com:CONNECTED, kvm02.digitaldatatechs.com:CONNECTED, kvm01.digitaldatatechs.com:CONNECTED], log id: 340c8f9d 2025-01-09 02:44:07,642-05 INFO [org.ovirt.engine.core.vdsbroker.gluster.AddGlusterServerVDSCommand] (EE-ManagedScheduledExecutorService-engineScheduledThreadPool-Thread-25) [] START, AddGlusterServerVDSCommand(HostName = kvm04, AddGlusterServerVDSParameters:{hostId='1408650c-aa50-452a-b2cc-24d36ae3cc20'}), log id: 7c25cce3 2025-01-09 02:44:10,879-05 INFO [org.ovirt.engine.core.vdsbroker.gluster.AddGlusterServerVDSCommand] (EE-ManagedScheduledExecutorService-engineScheduledThreadPool-Thread-25) [] FINISH, AddGlusterServerVDSCommand, return: , log id: 7c25cce3 2025-01-09 02:44:10,923-05 ERROR [org.ovirt.engine.core.dal.dbbroker.auditloghandling.AuditLogDirector] (EE-ManagedScheduledExecutorService-engineScheduledThreadPool-Thread-25) [] EVENT_ID: GLUSTER_SERVER_ADD_FAILED(4,436), Failed to add host kvm04 into Cluster dds_cluster. Add host failed: rc=107 out=() err=['Probe returned with Transport endpoint is not connected'] 2025-01-09 02:44:10,923-05 INFO [org.ovirt.engine.core.vdsbroker.gluster.GlusterVolumesListVDSCommand] (EE-ManagedScheduledExecutorService-engineScheduledThreadPool-Thread-25) [] START, GlusterVolumesListVDSCommand(HostName = kvm04, GlusterVolumesListVDSParameters:{hostId='1408650c-aa50-452a-b2cc-24d36ae3cc20'}), log id: effd1e8 2025-01-09 02:44:11,033-05 INFO [org.ovirt.engine.core.vdsbroker.gluster.GlusterVolumesListVDSCommand] (EE-ManagedScheduledExecutorService-engineScheduledThreadPool-Thread-25) [] FINISH, GlusterVolumesListVDSCommand, return: {b79a00d7-cc2a-48d8-b881-699991f3d930=org.ovirt.engine.core.common.businessentities.gluster.GlusterVolumeEntity@205714d2}, log id: effd1e8 2025-01-09 02:44:17,314-05 INFO [org.ovirt.engine.core.vdsbroker.vdsbroker.HSMGetAllTasksStatusesVDSCommand] (EE-ManagedScheduledExecutorService-engineScheduledThreadPool-Thread-52) [] Command 'org.ovirt.engine.core.vdsbroker.vdsbroker.HSMGetAllTasksStatusesVDSCommand' return value ' TaskStatusListReturn:{status='Status [code=654, message=Not SPM: ()]'} ' 2025-01-09 02:44:17,315-05 INFO [org.ovirt.engine.core.vdsbroker.vdsbroker.HSMGetAllTasksStatusesVDSCommand] (EE-ManagedScheduledExecutorService-engineScheduledThreadPool-Thread-52) [] HostName = kvm01 2025-01-09 02:44:17,315-05 ERROR [org.ovirt.engine.core.vdsbroker.vdsbroker.HSMGetAllTasksStatusesVDSCommand] (EE-ManagedScheduledExecutorService-engineScheduledThreadPool-Thread-52) [] Command 'HSMGetAllTasksStatusesVDSCommand(HostName = kvm01, VdsIdVDSCommandParametersBase:{hostId='acf66bf6-e159-4705-be47-484c99e746dd'})' execution failed: IRSGenericException: IRSErrorException: IRSNonOperationalException: Not SPM: () 2025-01-09 02:44:17,331-05 INFO [org.ovirt.engine.core.vdsbroker.vdsbroker.ConnectStoragePoolVDSCommand] (EE-ManagedScheduledExecutorService-engineScheduledThreadPool-Thread-52) [] START, ConnectStoragePoolVDSCommand(HostName = kvm04, ConnectStoragePoolVDSCommandParameters:{hostId='1408650c-aa50-452a-b2cc-24d36ae3cc20', vdsId='1408650c-aa50-452a-b2cc-24d36ae3cc20', storagePoolId='39acc250-bfba-11ef-adb1-70e2840574e3', masterVersion='768'}), log id: bcbf59e 2025-01-09 02:44:17,331-05 INFO [org.ovirt.engine.core.vdsbroker.vdsbroker.ConnectStoragePoolVDSCommand] (EE-ManagedScheduledExecutorService-engineScheduledThreadPool-Thread-52) [] Executing with domain map: {bd71c70a-7d9b-4cb4-ae5d-4c75a96a1d94=active, ca1b2f4f-26cf-48cc-b219-7295a763b1ae=active, 028f24af-01a2-4c0d-9374-7d8be6e12469=active} 2025-01-09 02:44:18,272-05 WARN [org.ovirt.engine.core.dal.dbbroker.auditloghandling.AuditLogDirector] (default task-7) [7814c7ae] EVENT_ID: VDS_NETWORKS_OUT_OF_SYNC(1,110), Host kvm01's following network(s) are not synchronized with their Logical Network configuration: ovirtmgmt,vlan3. 2025-01-09 02:44:18,283-05 INFO [org.ovirt.engine.core.bll.network.host.HostSetupNetworksCommand] (default task-7) [7814c7ae] Host setup networks finished. Lock released. Monitoring can run now for host 'kvm01' from data-center 'DDS_Data_Center' 2025-01-09 02:44:18,285-05 INFO [org.ovirt.engine.core.bll.network.host.HostSetupNetworksCommand] (default task-7) [7814c7ae] Lock freed to object 'EngineLock:{exclusiveLocks='[HOST_NETWORKacf66bf6-e159-4705-be47-484c99e746dd=HOST_NETWORK]', sharedLocks=''}' 2025-01-09 02:44:18,305-05 INFO [org.ovirt.engine.core.dal.dbbroker.auditloghandling.AuditLogDirector] (default task-7) [7814c7ae] EVENT_ID: PERSIST_SETUP_NETWORK_ON_HOST_FINISHED(10,903), (1/1): Successfully applied changes on host kvm01. (User: admin@ovirt@internalkeycloak-authz) 2025-01-09 02:44:18,316-05 INFO [org.ovirt.engine.core.dal.dbbroker.auditloghandling.AuditLogDirector] (default task-7) [7814c7ae] EVENT_ID: HOST_SYNC_ALL_NETWORKS_FINISHED(10,901), Managed to sync all host kvm01 networks. 2025-01-09 02:44:19,668-05 ERROR [org.ovirt.engine.core.dal.dbbroker.auditloghandling.AuditLogDirector] (EE-ManagedScheduledExecutorService-engineScheduledThreadPool-Thread-52) [] EVENT_ID: VDS_BROKER_COMMAND_FAILURE(10,802), VDSM kvm04 command ConnectStoragePoolVDS failed: Cannot find master domain: 'spUUID=39acc250-bfba-11ef-adb1-70e2840574e3, msdUUID=bd71c70a-7d9b-4cb4-ae5d-4c75a96a1d94' 2025-01-09 02:44:19,668-05 INFO [org.ovirt.engine.core.vdsbroker.vdsbroker.ConnectStoragePoolVDSCommand] (EE-ManagedScheduledExecutorService-engineScheduledThreadPool-Thread-52) [] Command 'org.ovirt.engine.core.vdsbroker.vdsbroker.ConnectStoragePoolVDSCommand' return value 'StatusOnlyReturn [status=Status [code=304, message=Cannot find master domain: 'spUUID=39acc250-bfba-11ef-adb1-70e2840574e3, msdUUID=bd71c70a-7d9b-4cb4-ae5d-4c75a96a1d94']]' 2025-01-09 02:44:19,668-05 INFO [org.ovirt.engine.core.vdsbroker.vdsbroker.ConnectStoragePoolVDSCommand] (EE-ManagedScheduledExecutorService-engineScheduledThreadPool-Thread-52) [] HostName = kvm04 2025-01-09 02:44:19,668-05 ERROR [org.ovirt.engine.core.vdsbroker.vdsbroker.ConnectStoragePoolVDSCommand] (EE-ManagedScheduledExecutorService-engineScheduledThreadPool-Thread-52) [] Command 'ConnectStoragePoolVDSCommand(HostName = kvm04, ConnectStoragePoolVDSCommandParameters:{hostId='1408650c-aa50-452a-b2cc-24d36ae3cc20', vdsId='1408650c-aa50-452a-b2cc-24d36ae3cc20', storagePoolId='39acc250-bfba-11ef-adb1-70e2840574e3', masterVersion='768'})' execution failed: IRSGenericException: IRSErrorException: IRSNoMasterDomainException: Cannot find master domain: 'spUUID=39acc250-bfba-11ef-adb1-70e2840574e3, msdUUID=bd71c70a-7d9b-4cb4-ae5d-4c75a96a1d94' 2025-01-09 02:44:19,668-05 INFO [org.ovirt.engine.core.vdsbroker.vdsbroker.ConnectStoragePoolVDSCommand] (EE-ManagedScheduledExecutorService-engineScheduledThreadPool-Thread-52) [] FINISH, ConnectStoragePoolVDSCommand, return: , log id: bcbf59e 2025-01-09 02:44:19,668-05 ERROR [org.ovirt.engine.core.vdsbroker.irsbroker.IrsBrokerCommand] (EE-ManagedScheduledExecutorService-engineScheduledThreadPool-Thread-52) [] IrsBroker::Failed::GetStoragePoolInfoVDS: IRSGenericException: IRSErrorException: IRSNoMasterDomainException: Cannot find master domain: 'spUUID=39acc250-bfba-11ef-adb1-70e2840574e3, msdUUID=bd71c70a-7d9b-4cb4-ae5d-4c75a96a1d94' 2025-01-09 02:44:19,679-05 ERROR [org.ovirt.engine.core.vdsbroker.vdsbroker.SpmStatusVDSCommand] (EE-ManagedScheduledExecutorService-engineScheduledThreadPool-Thread-52) [] Command 'SpmStatusVDSCommand(HostName = kvm04, SpmStatusVDSCommandParameters:{hostId='1408650c-aa50-452a-b2cc-24d36ae3cc20', storagePoolId='39acc250-bfba-11ef-adb1-70e2840574e3'})' execution failed: IRSGenericException: IRSErrorException: IRSNoMasterDomainException: Error validating master storage domain: ('MD read error',) 2025-01-09 02:44:19,679-05 INFO [org.ovirt.engine.core.vdsbroker.irsbroker.IrsProxyImpl] (EE-ManagedScheduledExecutorService-engineScheduledThreadPool-Thread-52) [] Irs placed on server '1408650c-aa50-452a-b2cc-24d36ae3cc20' failed. Proceed Failover 2025-01-09 02:44:19,684-05 INFO [org.ovirt.engine.core.vdsbroker.vdsbroker.ConnectStoragePoolVDSCommand] (EE-ManagedScheduledExecutorService-engineScheduledThreadPool-Thread-52) [] START, ConnectStoragePoolVDSCommand(HostName = kvm01, ConnectStoragePoolVDSCommandParameters:{hostId='acf66bf6-e159-4705-be47-484c99e746dd', vdsId='acf66bf6-e159-4705-be47-484c99e746dd', storagePoolId='39acc250-bfba-11ef-adb1-70e2840574e3', masterVersion='768'}), log id: 251dbd22 2025-01-09 02:44:19,684-05 INFO [org.ovirt.engine.core.vdsbroker.vdsbroker.ConnectStoragePoolVDSCommand] (EE-ManagedScheduledExecutorService-engineScheduledThreadPool-Thread-52) [] Executing with domain map: {bd71c70a-7d9b-4cb4-ae5d-4c75a96a1d94=active, ca1b2f4f-26cf-48cc-b219-7295a763b1ae=active, 028f24af-01a2-4c0d-9374-7d8be6e12469=active} 2025-01-09 02:44:22,181-05 INFO [org.ovirt.engine.core.vdsbroker.vdsbroker.ConnectStoragePoolVDSCommand] (EE-ManagedScheduledExecutorService-engineScheduledThreadPool-Thread-52) [] Command 'org.ovirt.engine.core.vdsbroker.vdsbroker.ConnectStoragePoolVDSCommand' return value 'StatusOnlyReturn [status=Status [code=304, message=Cannot find master domain: 'spUUID=39acc250-bfba-11ef-adb1-70e2840574e3, msdUUID=bd71c70a-7d9b-4cb4-ae5d-4c75a96a1d94']]' 2025-01-09 02:44:22,181-05 INFO [org.ovirt.engine.core.vdsbroker.vdsbroker.ConnectStoragePoolVDSCommand] (EE-ManagedScheduledExecutorService-engineScheduledThreadPool-Thread-52) [] HostName = kvm01 2025-01-09 02:44:22,181-05 ERROR [org.ovirt.engine.core.vdsbroker.vdsbroker.ConnectStoragePoolVDSCommand] (EE-ManagedScheduledExecutorService-engineScheduledThreadPool-Thread-52) [] Command 'ConnectStoragePoolVDSCommand(HostName = kvm01, ConnectStoragePoolVDSCommandParameters:{hostId='acf66bf6-e159-4705-be47-484c99e746dd', vdsId='acf66bf6-e159-4705-be47-484c99e746dd', storagePoolId='39acc250-bfba-11ef-adb1-70e2840574e3', masterVersion='768'})' execution failed: IRSGenericException: IRSErrorException: IRSNoMasterDomainException: Cannot find master domain: 'spUUID=39acc250-bfba-11ef-adb1-70e2840574e3, msdUUID=bd71c70a-7d9b-4cb4-ae5d-4c75a96a1d94' 2025-01-09 02:44:22,181-05 INFO [org.ovirt.engine.core.vdsbroker.vdsbroker.ConnectStoragePoolVDSCommand] (EE-ManagedScheduledExecutorService-engineScheduledThreadPool-Thread-52) [] FINISH, ConnectStoragePoolVDSCommand, return: , log id: 251dbd22 2025-01-09 02:44:22,181-05 ERROR [org.ovirt.engine.core.vdsbroker.irsbroker.IrsBrokerCommand] (EE-ManagedScheduledExecutorService-engineScheduledThreadPool-Thread-52) [] IrsBroker::Failed::GetStoragePoolInfoVDS: IRSGenericException: IRSErrorException: IRSNoMasterDomainException: Cannot find master domain: 'spUUID=39acc250-bfba-11ef-adb1-70e2840574e3, msdUUID=bd71c70a-7d9b-4cb4-ae5d-4c75a96a1d94' 2025-01-09 02:44:22,189-05 INFO [org.ovirt.engine.core.vdsbroker.irsbroker.IrsProxyImpl] (EE-ManagedScheduledExecutorService-engineScheduledThreadPool-Thread-52) [] Irs placed on server 'acf66bf6-e159-4705-be47-484c99e746dd' failed. Proceed Failover 2025-01-09 02:44:22,198-05 INFO [org.ovirt.engine.core.vdsbroker.vdsbroker.ConnectStoragePoolVDSCommand] (EE-ManagedScheduledExecutorService-engineScheduledThreadPool-Thread-52) [] START, ConnectStoragePoolVDSCommand(HostName = kvm02, ConnectStoragePoolVDSCommandParameters:{hostId='cf4a626e-ad6e-4cc4-95f0-4d7c1329f0f6', vdsId='cf4a626e-ad6e-4cc4-95f0-4d7c1329f0f6', storagePoolId='39acc250-bfba-11ef-adb1-70e2840574e3', masterVersion='768'}), log id: 1f60ebb1 2025-01-09 02:44:22,198-05 INFO [org.ovirt.engine.core.vdsbroker.vdsbroker.ConnectStoragePoolVDSCommand] (EE-ManagedScheduledExecutorService-engineScheduledThreadPool-Thread-52) [] Executing with domain map: {bd71c70a-7d9b-4cb4-ae5d-4c75a96a1d94=active, ca1b2f4f-26cf-48cc-b219-7295a763b1ae=active, 028f24af-01a2-4c0d-9374-7d8be6e12469=active} 2025-01-09 02:44:24,581-05 INFO [org.ovirt.engine.core.vdsbroker.vdsbroker.ConnectStoragePoolVDSCommand] (EE-ManagedScheduledExecutorService-engineScheduledThreadPool-Thread-52) [] Command 'org.ovirt.engine.core.vdsbroker.vdsbroker.ConnectStoragePoolVDSCommand' return value 'StatusOnlyReturn [status=Status [code=304, message=Cannot find master domain: 'spUUID=39acc250-bfba-11ef-adb1-70e2840574e3, msdUUID=bd71c70a-7d9b-4cb4-ae5d-4c75a96a1d94']]' 2025-01-09 02:44:24,581-05 INFO [org.ovirt.engine.core.vdsbroker.vdsbroker.ConnectStoragePoolVDSCommand] (EE-ManagedScheduledExecutorService-engineScheduledThreadPool-Thread-52) [] HostName = kvm02 2025-01-09 02:44:24,581-05 ERROR [org.ovirt.engine.core.vdsbroker.vdsbroker.ConnectStoragePoolVDSCommand] (EE-ManagedScheduledExecutorService-engineScheduledThreadPool-Thread-52) [] Command 'ConnectStoragePoolVDSCommand(HostName = kvm02, ConnectStoragePoolVDSCommandParameters:{hostId='cf4a626e-ad6e-4cc4-95f0-4d7c1329f0f6', vdsId='cf4a626e-ad6e-4cc4-95f0-4d7c1329f0f6', storagePoolId='39acc250-bfba-11ef-adb1-70e2840574e3', masterVersion='768'})' execution failed: IRSGenericException: IRSErrorException: IRSNoMasterDomainException: Cannot find master domain: 'spUUID=39acc250-bfba-11ef-adb1-70e2840574e3, msdUUID=bd71c70a-7d9b-4cb4-ae5d-4c75a96a1d94' 2025-01-09 02:44:24,582-05 INFO [org.ovirt.engine.core.vdsbroker.vdsbroker.ConnectStoragePoolVDSCommand] (EE-ManagedScheduledExecutorService-engineScheduledThreadPool-Thread-52) [] FINISH, ConnectStoragePoolVDSCommand, return: , log id: 1f60ebb1 2025-01-09 02:44:24,582-05 ERROR [org.ovirt.engine.core.vdsbroker.irsbroker.IrsBrokerCommand] (EE-ManagedScheduledExecutorService-engineScheduledThreadPool-Thread-52) [] IrsBroker::Failed::GetStoragePoolInfoVDS: IRSGenericException: IRSErrorException: IRSNoMasterDomainException: Cannot find master domain: 'spUUID=39acc250-bfba-11ef-adb1-70e2840574e3, msdUUID=bd71c70a-7d9b-4cb4-ae5d-4c75a96a1d94' 2025-01-09 02:44:24,658-05 INFO [org.ovirt.engine.core.bll.storage.pool.ReconstructMasterDomainCommand] (EE-ManagedThreadFactory-engine-Thread-610) [2af75e62] Running command: ReconstructMasterDomainCommand internal: true. Entities affected : ID: bd71c70a-7d9b-4cb4-ae5d-4c75a96a1d94 Type: Storage 2025-01-09 02:44:24,691-05 WARN [org.ovirt.engine.core.bll.storage.pool.ReconstructMasterDomainCommand] (EE-ManagedThreadFactory-engine-Thread-610) [2af75e62] skipping connect and refresh for all hosts, last master 'true', command status 'true' 2025-01-09 02:44:24,715-05 INFO [org.ovirt.engine.core.bll.eventqueue.EventQueueMonitor] (EE-ManagedThreadFactory-engine-Thread-609) [] Finished reconstruct for pool '39acc250-bfba-11ef-adb1-70e2840574e3'. Clearing event queue While attempting to start vm will be sent after but I think the net allocation issue with hosts is the center of the vm issues,,

2 1

Restart oVirt-Engine
by Jeremey Wise 09 Jan '25

09 Jan '25

How ,without reboot of hosting system, do I restart the oVirt engine? # I tried below but do not seem to effect the virtual machine [root@thor iso]# systemctl restart ov ovirt-ha-agent.service ovirt-imageio.service ovn-controller.service ovs-delete-transient-ports.service ovirt-ha-broker.service ovirt-vmconsole-host-sshd.service ovsdb-server.service ovs-vswitchd.service [root@thor iso]# # You cannot restart the VM " HostedEngine " as it responses: Error while executing action: HostedEngine: - Cannot restart VM. This VM is not managed by the engine. Reason is I had to do some work on a node. Reboot it.. it is back up.. network is all fine.. Cockpit working fine... and gluster fine.. But oVirt-Engine refuses to accept the node is up. -- p <jeremey.wise(a)gmail.com>enguinpages

4 4

Host deploy failure: Configure OVN for oVirt
by stephan.badenhorst＠fnb.co.za 05 Jan '25

05 Jan '25

Good day, I am running into a problem during host deploy via the oVirt engine GUI since upgrading to ovirt-engine-4.5.5-1.el8. The "Configure OVN for oVirt" task seem to fail when trying to run the vdsm-tool ovn-config command. Host deploy used to work fine when the engine was on version 4.5.4. Anyone that can guide me on the right path to get past this issue? Does not seem to be a new problem - https://lists.ovirt.org/archives/list/users@ovirt.org/thread/IDLGSBQFX35EHH… Log extract: 2024-02-01 14:48:19 SAST - TASK [ovirt-provider-ovn-driver : Configure OVN for oVirt] ********************* . . . "stdout" : "fatal: [mob-r1-l-ovirt-aa-1-23.x.fnb.co.za]: FAILED! => {\"changed\": true, \"cmd\": [\"vdsm-tool\", \"ovn-config\", \"192.168.2.100\", \"host23.mydomain.com\"], \"delta\": \"0:00:00.538143\", \"end \": \"2024-02-01 14:48:20.596823\", \"msg\": \"non-zero return code\", \"rc\": 1, \"start\": \"2024-02-01 14:48:20.058680\", \"stderr\": \"Traceback (most recent call last):\\n File \\\"/usr/lib/python3.6/site-packages/vdsm/t ool/ovn_config.py\\\", line 117, in get_network\\n return networks[net_name]\\nKeyError: 'host23.mydomain.com'\\n\\nDuring handling of the above exception, another exception occurred:\\n\\nTraceback (most rec ent call last):\\n File \\\"/usr/bin/vdsm-tool\\\", line 195, in main\\n return tool_command[cmd][\\\"command\\\"](*args)\\n File \\\"/usr/lib/python3.6/site-packages/vdsm/tool/ovn_config.py\\\", line 63, in ovn_config\\n ip_address = get_ip_addr(get_network(network_caps(), net_name))\\n File \\\"/usr/lib/python3.6/site-packages/vdsm/tool/ovn_config.py\\\", line 119, in get_network\\n raise NetworkNotFoundError(net_name)\\nvdsm.tool.ovn _config.NetworkNotFoundError: host23.mydomain.com\", \"stderr_lines\": [\"Traceback (most recent call last):\", \" File \\\"/usr/lib/python3.6/site-packages/vdsm/tool/ovn_config.py\\\", line 117, in get_network \", \" return networks[net_name]\", \"KeyError: 'host23.mydomain.com'\", \"\", \"During handling of the above exception, another exception occurred:\", \"\", \"Traceback (most recent call last):\", \" File \ \\"/usr/bin/vdsm-tool\\\", line 195, in main\", \" return tool_command[cmd][\\\"command\\\"](*args)\", \" File \\\"/usr/lib/python3.6/site-packages/vdsm/tool/ovn_config.py\\\", line 63, in ovn_config\", \" ip_address = get_ip_addr(get_network(network_caps(), net_name))\", \" File \\\"/usr/lib/python3.6/site-packages/vdsm/tool/ovn_config.py\\\", line 119, in get_network\", \" raise NetworkNotFoundError(net_name)\", \"vdsm.tool.ovn_config. NetworkNotFoundError: host23.mydomain.com\"], \"stdout\": \"\", \"stdout_lines\": []}", Thanks in advance!! Stephan

5 5

Re: Host deploy failure: Configure OVN for oVirt
by Andrea Cavegn 03 Jan '25

03 Jan '25

We use ovirt-engine 4.5.5-1.el8 and for us the following did the trick: 1. First you have to decide, if you need the ovn/ovs functionality. This is a cluster level decision! * If needed, you need to have a functional (You can test it) external network provider configured (Administration -> Providers -> External Network Provider) ?? As far as I know, a default "ovirt-provider-ovn" is already present in each installation * Then in the cluster option you choose the "Default Network Provider". If you don't need OVS/OVN, then select "No Default Provider", else select the one you need * If you change here anything, ovirt want you to reinstall all your hosts in this cluster! 2. Then, replace the ansible file in your ovirt-engine: * /usr/share/ovirt-engine/ansible-runner-service-project/project/roles/ovirt-provider-ovn-driver/tasks/configure.yml * The new version can be found here: https://github.com/oVirt/ovirt-engine/blob/master/packaging/ansible-runner-… as far as I can tell, only the last line (when statement) changes Probably, only ppl who don't need ovn/ovs had (Setting: "No Default Provider") had this problem. 1. Reinstall the hosts As always, know what you do, before you anything.

1 0

Enroll Certificate
by Денис Панкратьев 03 Jan '25

03 Jan '25

Hello! My ovirt host (4.5.5) certificate is about to expire (in 46 days, Jan 20, 2025). d.pankratiev@zabbix ~]$ echo | openssl s_client -servername ovirt1.gtc.local -connect XXX.XXX.XXX.XXX.XXX:9090 2>/dev/null | openssl x509 -noout -dates 2>/dev/null notBefore=Dec 22 05:27:04 2023 GMT notAfter=Jan 20 05:27:04 2025 GMT 1. I have migrated all VMs to another host 2. I put the host in maintenance mode (Managеment -> Maintenance) 3. Do enroll certificate (Installation -> Enroll Certificate) 4. Everything went successfully, without errors (Enrolling certificate for host ovirt1.gtc.local was completed successfully). But the certificate date did not change, still expire (after 46 days, Jan 20 2025). d.pankratiev@zabbix ~]$ echo | openssl s_client -servername ovirt1.gtc.local -connect XXX.XXX.XXX.XXX.XXX:9090 2>/dev/null | openssl x509 -noout -dates 2>/dev/null notBefore=Dec 22 05:27:04 2023 GMT notAfter=Jan 20 05:27:04 2025 GMT What did I do wrong? Is there anything else I need to do?

5 5

oVirt and TPM
by Fabrice Bacchella 30 Dec '24

30 Dec '24

Did some one use TPM in oVirt ? I tried to active it but failed to use. The VM is configured to use an EFI secure boot. The VM (a RHEL 9.5 OS) see it, but I can’t communicate with it: $ dmesg | grep -i tpm [ 0.000000] efi: SMBIOS=0x7ddd5000 TPMFinalLog=0x7dfe7000 ACPI=0x7df7e000 ACPI 2.0=0x7df7e014 MEMATTR=0x7cbd9018 MOKvar=0x7dd82000 RNG=0x7df71018 TPMEventLog=0x7c544018 [ 0.018626] ACPI: TPM2 0x000000007DF76000 00004C (v04 BOCHS BXPC 00000001 BXPC 00000001) [ 0.018657] ACPI: Reserving TPM2 table memory at [mem 0x7df76000-0x7df7604b] [ 5.312318] systemd[1]: TPM2 PCR Machine ID Measurement was skipped because of an unmet condition check (ConditionPathExists=/sys/firmware/efi/efivars/StubPcrKernelImage-4a67b082-0a4c-41cf-b6c7-440b29bb8c4f). pcsc_scan fails to read it: $ sudo pcsc_scan -s -v PC/SC device scanner V 1.6.2 (c) 2001-2022, Ludovic Rousseau <ludovic.rousseau(a)free.fr> SCardEstablishContext: RPC transport error.

2 2

Curl error (7): Couldn't connect to server for https://mirrorlist.ovirt.org/mirrorlist-ovirt-4.5-el8
by eyup ustaoglu 29 Dec '24

29 Dec '24

Hello. I keep getting this error during the ovirt PREPARE VM phase. Is there a site where I can get the current repositories under /etc/yum.repos.d? Also, have ovirt updates stopped? Why are repo addresses that have been unreachable for a long time still in the ovirt node?

1 0

Cannot prepare internal mirrorlist: Curl error (6): Couldn't resolve host name for http://mirrorlist.centos.org/?release=8-stream&arch=x86_64&repo=virt-ovirt-45 [Could not resolve host: mirrorlist.centos.org]
by onr onr 26 Dec '24

26 Dec '24

I am in the Prepare VM step in ovirt installation. and I keep getting the above error. Why is the unused repo address defined even though I downloaded the latest ovirt-node? Where can I find repo addresses for el8 that can be accessed for installation? Is ovirt a discontinued system?

1 0

How to stop backup operation on vm
by marco.pegoraro＠univr.it 23 Dec '24

23 Dec '24

Hi, I have the following problem. In the db engine I have the following backups: engine=# select * from vm_backups; backup_id | from_checkpoint_id | to_checkpoint_id | vm_id | phase | _create_date | host_id | description | _update_date | backup_type | snapshot_id | is_stopped --------------------------------------+--------------------+--------------------------------------+------------------------------------ --+-------+----------------------------+---------+------------------------------------------------------------------------------------- -----------------------------------+----------------------------+-------------+--------------------------------------+------------ 864beee4-c87d-447c-a249-83a38fd1d4e7 | | 0ecf0a56-e7bb-4133-9ac7-441aa0d84545 | e1465242-3724-405b-9750-250d1eeb974 2 | Ready | 2024-12-18 16:42:54.717+01 | | {"type":"veeam","job_uuid":"0c3a6b0a-b46e-4ca2-8ed1-3d29c4b221ab","session_id":"4706 6400-4f1f-456e-ab94-6c3af7043809"} | 2024-12-18 16:43:06.736+01 | hybrid | 234de52d-8739-499a-8031-b8d6f6db0422 | f ace9cdf3-ad51-45b9-af6f-1728b3941187 | | e5b1ed24-4781-43dc-a1fd-b71173fd1602 | f2e2a274-71f7-4e4a-87ae-942ce0b78dd 2 | Ready | 2024-12-19 11:35:16.67+01 | | {"type":"veeam","job_uuid":"0c3a6b0a-b46e-4ca2-8ed1-3d29c4b221ab","session_id":"8d89 018d-ad81-4857-a4e3-42cf0b38a41d"} | 2024-12-19 11:35:40.907+01 | hybrid | c9108e74-8bdc-401a-b27d-adab99f28682 | f (2 rows) I want to stop the backups and delete them. I ran the following commands on the KVM nodes to stop the backups: #vdsm-client VM stop_backup vmID=e1465242-3724-405b-9750-250d1eeb9742 backup_id=864beee4-c87d-447c-a249-83a38fd1d4e7 { "code": 0, "message": "Done" } #vdsm-client VM stop_backup vmID=e1465242-3724-405b-9750-250d1eeb9742 backup_id=864beee4-c87d-447c-a249-83a38fd1d4e7 { "code": 0, "message": "Done" } but in the db there are still: engine=# select * from vm_backups; backup_id | from_checkpoint_id | to_checkpoint_id | vm_id | phase | _create_date | host_id | description | _update_date | backup_type | snapshot_id | is_stopped --------------------------------------+--------------------+--------------------------------------+------------------------------------ --+-------+----------------------------+---------+------------------------------------------------------------------------------------- -----------------------------------+----------------------------+-------------+--------------------------------------+------------ 864beee4-c87d-447c-a249-83a38fd1d4e7 | | 0ecf0a56-e7bb-4133-9ac7-441aa0d84545 | e1465242-3724-405b-9750-250d1eeb974 2 | Ready | 2024-12-18 16:42:54.717+01 | | {"type":"veeam","job_uuid":"0c3a6b0a-b46e-4ca2-8ed1-3d29c4b221ab","session_id":"4706 6400-4f1f-456e-ab94-6c3af7043809"} | 2024-12-18 16:43:06.736+01 | hybrid | 234de52d-8739-499a-8031-b8d6f6db0422 | f ace9cdf3-ad51-45b9-af6f-1728b3941187 | | e5b1ed24-4781-43dc-a1fd-b71173fd1602 | f2e2a274-71f7-4e4a-87ae-942ce0b78dd 2 | Ready | 2024-12-19 11:35:16.67+01 | | {"type":"veeam","job_uuid":"0c3a6b0a-b46e-4ca2-8ed1-3d29c4b221ab","session_id":"8d89 018d-ad81-4857-a4e3-42cf0b38a41d"} | 2024-12-19 11:35:40.907+01 | hybrid | c9108e74-8bdc-401a-b27d-adab99f28682 | f (2 rows) Do you have any idea how I can accomplish this? Thank you very much.

1 0

"Master" Storage Domain
by duparchy＠esrf.fr 21 Dec '24

21 Dec '24

Hi What is the role of the "Master Storage Domain" ? (and what consequences if it is unresponsive for a while)

2 1

Re: Problems when add storagedomain "master"
by Erik＠deerlickgroup.com 21 Dec '24

21 Dec '24

I think you can onoy have one master. Have you tried just promoting a domain vs. trying to add a new one? Erik Sent from my iPad

1 0

Migrate from Centos8 to Centos9
by suporte＠logicworks.pt 19 Dec '24

19 Dec '24

Hi, I need to migrate from Centos8 to Centos9, engine+ 2 nodes, iSCSI storage, My idea is backup Engine on Centos8: engine-backup --scope=all --mode=backup --file=/root/backup Then install a new engine machine with Centos9 and restore: engine-backup --mode=restore --file=backup --log=backup_restore.log engine-setup Now Engine on Centos9 is controlling the 2 nodes on Centos8 Than, I migrate all VMs from node2 to node1. Install node2 with Centos9 Migrate all VMs from node1 to node2 Install node1 with Centos9 Do you think it will work? Thanks -- Jose Ferradeira http://www.logicworks.pt

4 8

Re: [External] : Adding ISO doesn't work
by Stanley Ang 18 Dec '24

18 Dec '24

Hi Marcos Is the problem fixed on 4.5.5-1.28 already? Is the repLacement of olvm45-security-fixes supposed to be a workaround?? I have already changed olvm45-security-fixes.conf to the one that you attached in the mail but the problem still persisted. Stan

2 1

CentOS 9 Stream Ovirt 4;5
by eevans＠digitaldatatechs.com 16 Dec '24

16 Dec '24

My setup: 1 Seperate physical ovirt engine with 3 Centos 9 nodes; I have imported ova's from Ovirt 4.3 and that will not launch. They stay in waiting to launch for bout 15 seconds then shutdown. ALso, the node networks will not sync. I turned off firewalls to make sure it wasn't blocked with the same result. I also disabled selinux with the same result. Any help would be appereciated.

1 0

Uploading .iso image with REST API creates <id>.meta with wrong DISKTYPE value
by markul11＠protonmail.com 12 Dec '24

12 Dec '24

When creating file (.iso image) with the web GUI the .meta file has DISKTYPE=ISOF, but when using REST, DISKTYPE=DATA and the file is not listed on the "Attach CD" drop down when creating a new vm. Using API is the only way for me cos I can't find a fix to a connection problem from Firefox/Fedora 39 to the web GUI, so all transfers end up as "suspended by the system". I am using the following POST to crate the file: curl -k -u "admin@internal:secretpw" -X POST \ -H "Content-Type: application/xml" \ -d '<disk><name>Rocky-9-latest-x86_64-boot.iso</name><alias>Rocky-9-latest-x86_64-boot.iso</alias>\ <description>Rocky-9-latest-x86_64-boot.iso</description>\ <provisioned_size>1068498944</provisioned_size><format>raw</format>\ <storage_domains><storage_domain><name>isos</name>\ </storage_domain></storage_domains></disk>' \ https://ovirtsvc.adm.cnt.uksw.edu.pl/ovirt-engine/api/disks The resulting <id>.meta is: CAP=1068498944 CTIME=1732282300 DESCRIPTION={"DiskAlias":"Rocky-9-latest-x86_64-boot.iso","DiskDescription":"Rocky-9-latest-x86_64-boot.iso"} DISKTYPE=DATA [...] FORMAT=RAW [...]TYPE=SPARSE VOLTYPE=LEAF EOF

2 2

oVirt New version?
by ecsi＠ecsi.hu 11 Dec '24

11 Dec '24

There hasn't been a new version for over a year, and not long ago, you asked for help with the release of a new version. What is the current status of the project?

1 0

Problems when add storagedomain "master"
by brancomrt＠gmail.com 09 Dec '24

09 Dec '24

Hi, I have a problem when i add iscsi storagedomain with ansible playbook. It's the sample of my task in playbook: - name: Add Storage master ovirt.ovirt.ovirt_storage_domain: auth: "{{ ovirt_auth }}" name: data_iscsi host: xxx.xxxx.xxx data_center: "{{ data_center_name }}" domain_function: data iscsi: target: iqn.2007-11.com.nimblestorage:olvm-infra-dc-v66459751dbd365d4.00000026.bd96f285 lun_id: 2c32e833f545870cf6c9ce90085f296bd address: 192.168.3.252 port: 3260 username: '' password: '' timeout: 240 discard_after_delete: True backup: False critical_space_action_blocker: 5 warning_low_space: 10 This is the "ERROR" when executing the palybook: TASK [olvm_addstorage : Add Storage master] ****************************************************************************** An exception occurred during task execution. To see the full traceback, use -vvv. The error was: ovirtsdk4.Error: Fault reason is "Operation Failed". Fault detail is "[Network error during communication with the Host.]". HTTP response code is 400. fatal: [infradcatm-r6e1l10.atech.com.br]: FAILED! => {"changed": false, "msg": "Fault reason is \"Operation Failed\". Fault detail is \"[Network error during communication with the Host.]\". HTTP response code is 400."} Logs of engine: # tail -f /var/log/ovirt-engine/engine.log 2024-12-09 05:54:21,154Z ERROR [org.ovirt.vdsm.jsonrpc.client.reactors.ReactorClient] (SSL Stomp Reactor) [] Connection timeout for host 'xxxxx.xxxxxx.xxx.xxx', last response arrived 1501 ms ago. 2024-12-09 05:54:22,218Z INFO [org.ovirt.engine.core.sso.service.AuthenticationService] (default task-22) [] User admin@internal-authz with profile [internal] successfully logged in with scopes: ovirt-app-api ovirt-ext=token-info:authz-search ovirt-ext=token-info:public-authz-search ovirt-ext=token-info:validate ovirt-ext=token:password-access 2024-12-09 05:54:22,738Z INFO [org.ovirt.engine.core.bll.aaa.CreateUserSessionCommand] (default task-22) [46e35dea] Running command: CreateUserSessionCommand internal: false. 2024-12-09 05:54:22,794Z INFO [org.ovirt.engine.core.dal.dbbroker.auditloghandling.AuditLogDirector] (default task-22) [46e35dea] EVENT_ID: USER_VDC_LOGIN(30), User admin@internal-authz connecting from '192.168.29.20' using session 'n1uuzlRLnIXfFAua0AOD+2sk0g+RKku9vB6baJ4UoYdoxZ5JuNtySPblCXQkHG9Yz9RPBZiOMjowN+gy6gJyqg==' logged in. 2024-12-09 05:54:22,913Z INFO [org.ovirt.engine.core.bll.storage.connection.ConnectStorageToVdsCommand] (default task-22) [c8f05ab3-d362-4e8c-bfac-0ed5d6abee98] Running command: ConnectStorageToVdsCommand internal: false. Entities affected : ID: aaa00000-0000-0000-0000-123456789aaa Type: SystemAction group CREATE_STORAGE_DOMAIN with role type ADMIN 2024-12-09 05:54:22,916Z INFO [org.ovirt.engine.core.vdsbroker.vdsbroker.ConnectStorageServerVDSCommand] (default task-22) [c8f05ab3-d362-4e8c-bfac-0ed5d6abee98] START, ConnectStorageServerVDSCommand(HostName = xxxxx.xxxxxx.xxx.xxx, StorageServerConnectionManagementVDSParameters:{hostId='260fd31d-b8a3-4ac7-a919-da2964b05b2b', storagePoolId='00000000-0000-0000-0000-000000000000', storageType='ISCSI', connectionList='[StorageServerConnections:{id='null', connection='192.168.3.252', iqn='iqn.2007-11.com.nimblestorage:olvm-infra-dc-v66459751dbd365d4.00000026.bd96f285', vfsType='null', mountOptions='null', nfsVersion='null', nfsRetrans='null', nfsTimeo='null', iface='null', netIfaceName='null'}]', sendNetworkEventOnFailure='true'}), log id: 5842a368 2024-12-09 05:54:22,918Z INFO [org.ovirt.vdsm.jsonrpc.client.reactors.ReactorClient] (SSL Stomp Reactor) [] Connecting to xxxxx.xxxxxx.xxx.xxx/192.168.29.20 2024-12-09 05:54:22,918Z INFO [org.ovirt.vdsm.jsonrpc.client.reactors.ReactorClient] (SSL Stomp Reactor) [] Connected to xxxxx.xxxxxx.xxx.xxx/192.168.29.20:54321 2024-12-09 05:54:22,920Z ERROR [org.ovirt.engine.core.vdsbroker.vdsbroker.ConnectStorageServerVDSCommand] (default task-22) [c8f05ab3-d362-4e8c-bfac-0ed5d6abee98] Command 'ConnectStorageServerVDSCommand(HostName = xxxxx.xxxxxx.xxx.xxx, StorageServerConnectionManagementVDSParameters:{hostId='260fd31d-b8a3-4ac7-a919-da2964b05b2b', storagePoolId='00000000-0000-0000-0000-000000000000', storageType='ISCSI', connectionList='[StorageServerConnections:{id='null', connection='192.168.3.252', iqn='iqn.2007-11.com.nimblestorage:olvm-infra-dc-v66459751dbd365d4.00000026.bd96f285', vfsType='null', mountOptions='null', nfsVersion='null', nfsRetrans='null', nfsTimeo='null', iface='null', netIfaceName='null'}]', sendNetworkEventOnFailure='true'})' execution failed: java.net.ConnectException: Connection refused 2024-12-09 05:54:22,920Z INFO [org.ovirt.engine.core.vdsbroker.vdsbroker.ConnectStorageServerVDSCommand] (default task-22) [c8f05ab3-d362-4e8c-bfac-0ed5d6abee98] FINISH, ConnectStorageServerVDSCommand, return: , log id: 5842a368 2024-12-09 05:54:22,920Z ERROR [org.ovirt.engine.core.bll.storage.connection.ConnectStorageToVdsCommand] (default task-22) [c8f05ab3-d362-4e8c-bfac-0ed5d6abee98] Command 'org.ovirt.engine.core.bll.storage.connection.ConnectStorageToVdsCommand' failed: EngineException: org.ovirt.engine.core.vdsbroker.vdsbroker.VDSNetworkException: java.net.ConnectException: Connection refused (Failed with error VDS_NETWORK_ERROR and code 5022) 2024-12-09 05:54:22,920Z INFO [org.ovirt.engine.core.vdsbroker.VdsManager] (EE-ManagedThreadFactory-engine-Thread-51) [c8f05ab3-d362-4e8c-bfac-0ed5d6abee98] Clearing domains data for host xxxxx.xxxxxx.xxx.xxx 2024-12-09 05:54:22,920Z WARN [org.ovirt.engine.core.vdsbroker.VdsManager] (EE-ManagedThreadFactory-engine-Thread-51) [c8f05ab3-d362-4e8c-bfac-0ed5d6abee98] Host 'xxxxx.xxxxxx.xxx.xxx' is not responding. 2024-12-09 05:54:22,924Z ERROR [org.ovirt.engine.core.bll.storage.connection.ConnectStorageToVdsCommand] (default task-22) [c8f05ab3-d362-4e8c-bfac-0ed5d6abee98] Transaction rolled-back for command 'org.ovirt.engine.core.bll.storage.connection.ConnectStorageToVdsCommand'. 2024-12-09 05:54:22,962Z ERROR [org.ovirt.engine.api.restapi.resource.AbstractBackendResource] (default task-22) [] Operation Failed: [Network error during communication with the Host.] 2024-12-09 05:54:24,676Z INFO [org.ovirt.vdsm.jsonrpc.client.reactors.ReactorClient] (SSL Stomp Reactor) [] Connecting to xxxxx.xxxxxx.xxx.xxx/192.168.29.20 2024-12-09 05:54:24,677Z INFO [org.ovirt.vdsm.jsonrpc.client.reactors.ReactorClient] (SSL Stomp Reactor) [] Connected to xxxxx.xxxxxx.xxx.xxx/192.168.29.20:54321 2024-12-09 05:54:24,775Z ERROR [org.ovirt.engine.core.dal.dbbroker.auditloghandling.AuditLogDirector] (EE-ManagedScheduledExecutorService-engineScheduledThreadPool-Thread-97) [] EVENT_ID: VDS_BROKER_COMMAND_FAILURE(10,802), VDSM xxxxx.xxxxxx.xxx.xxx command Get Host Capabilities failed: Recovering from crash or Initializing 2024-12-09 05:54:24,776Z ERROR [org.ovirt.engine.core.vdsbroker.monitoring.HostMonitoring] (EE-ManagedScheduledExecutorService-engineScheduledThreadPool-Thread-97) [] Unable to RefreshCapabilities: VDSRecoveringException: Recovering from crash or Initializing 2024-12-09 05:54:30,275Z INFO [org.ovirt.engine.core.vdsbroker.vdsbroker.GetHardwareInfoAsyncVDSCommand] (EE-ManagedScheduledExecutorService-engineScheduledThreadPool-Thread-56) [] START, GetHardwareInfoAsyncVDSCommand(HostName = xxxxx.xxxxxx.xxx.xxx, VdsIdAndVdsVDSCommandParametersBase:{hostId='260fd31d-b8a3-4ac7-a919-da2964b05b2b', vds='Host[xxxxx.xxxxxx.xxx.xxx,260fd31d-b8a3-4ac7-a919-da2964b05b2b]'}), log id: 47684f5d 2024-12-09 05:54:30,275Z INFO [org.ovirt.engine.core.vdsbroker.vdsbroker.GetHardwareInfoAsyncVDSCommand] (EE-ManagedScheduledExecutorService-engineScheduledThreadPool-Thread-56) [] FINISH, GetHardwareInfoAsyncVDSCommand, return: , log id: 47684f5d 2024-12-09 05:54:30,275Z WARN [org.ovirt.engine.core.vdsbroker.VdsManager] (EE-ManagedScheduledExecutorService-engineScheduledThreadPool-Thread-56) [] Host 'xxxxx.xxxxxx.xxx.xxx' is running with SELinux in 'DISABLED' mode 2024-12-09 05:54:30,321Z WARN [org.ovirt.engine.core.dal.dbbroker.auditloghandling.AuditLogDirector] (EE-ManagedScheduledExecutorService-engineScheduledThreadPool-Thread-56) [] EVENT_ID: VDS_NETWORKS_OUT_OF_SYNC(1,110), Host xxxxx.xxxxxx.xxx.xxx's following network(s) are not synchronized with their Logical Network configuration: ovirtmgmt. 2024-12-09 05:54:30,364Z INFO [org.ovirt.engine.core.bll.HandleVdsCpuFlagsOrClusterChangedCommand] (EE-ManagedScheduledExecutorService-engineScheduledThreadPool-Thread-56) [6133e03b] Running command: HandleVdsCpuFlagsOrClusterChangedCommand internal: true. Entities affected : ID: 260fd31d-b8a3-4ac7-a919-da2964b05b2b Type: VDS 2024-12-09 05:54:30,366Z INFO [org.ovirt.engine.core.bll.UpdateClusterCommand] (EE-ManagedScheduledExecutorService-engineScheduledThreadPool-Thread-56) [6f8cbaaa] Lock Acquired to object 'EngineLock:{exclusiveLocks='[]', sharedLocks='[]'}' 2024-12-09 05:54:30,373Z INFO [org.ovirt.engine.core.bll.UpdateClusterCommand] (EE-ManagedScheduledExecutorService-engineScheduledThreadPool-Thread-56) [6f8cbaaa] Running command: UpdateClusterCommand internal: true. Entities affected : ID: 1ff65d4a-458d-49a9-a66d-9d1bfdfc5df7 Type: ClusterAction group EDIT_CLUSTER_CONFIGURATION with role type ADMIN 2024-12-09 05:54:30,435Z WARN [org.ovirt.engine.core.dal.dbbroker.auditloghandling.AuditLogDirector] (EE-ManagedScheduledExecutorService-engineScheduledThreadPool-Thread-56) [6f8cbaaa] EVENT_ID: FENCE_DISABLED_IN_CLUSTER_POLICY(9,016), Fencing is disabled in Fencing Policy of the Cluster infradc, so HA VMs running on a non-responsive host will not be restarted elsewhere. 2024-12-09 05:54:30,465Z INFO [org.ovirt.engine.core.dal.dbbroker.auditloghandling.AuditLogDirector] (EE-ManagedScheduledExecutorService-engineScheduledThreadPool-Thread-56) [6f8cbaaa] EVENT_ID: SYSTEM_UPDATE_CLUSTER(835), Host cluster infradc was updated by system 2024-12-09 05:54:30,465Z INFO [org.ovirt.engine.core.bll.UpdateClusterCommand] (EE-ManagedScheduledExecutorService-engineScheduledThreadPool-Thread-56) [6f8cbaaa] Lock freed to object 'EngineLock:{exclusiveLocks='[]', sharedLocks='[]'}' 2024-12-09 05:54:30,621Z INFO [org.ovirt.engine.core.bll.InitVdsOnUpCommand] (EE-ManagedScheduledExecutorService-engineScheduledThreadPool-Thread-48) [6ff01f24] Running command: InitVdsOnUpCommand internal: true. Entities affected : ID: d8c291be-6953-4185-8a51-080b5bf013a2 Type: StoragePool 2024-12-09 05:54:30,703Z ERROR [org.ovirt.engine.core.bll.pm.FenceProxyLocator] (EE-ManagedScheduledExecutorService-engineScheduledThreadPool-Thread-48) [6ff01f24] Can not run fence action on host 'xxxxx.xxxxxx.xxx.xxx', no suitable proxy host was found. 2024-12-09 05:54:30,705Z INFO [org.ovirt.engine.core.vdsbroker.vdsbroker.SetMOMPolicyParametersVDSCommand] (EE-ManagedScheduledExecutorService-engineScheduledThreadPool-Thread-48) [6ff01f24] START, SetMOMPolicyParametersVDSCommand(HostName = xxxxx.xxxxxx.xxx.xxx, MomPolicyVDSParameters:{hostId='260fd31d-b8a3-4ac7-a919-da2964b05b2b'}), log id: 2fc3f5c3 2024-12-09 05:54:30,730Z INFO [org.ovirt.engine.core.vdsbroker.vdsbroker.SetMOMPolicyParametersVDSCommand] (EE-ManagedScheduledExecutorService-engineScheduledThreadPool-Thread-48) [6ff01f24] FINISH, SetMOMPolicyParametersVDSCommand, return: , log id: 2fc3f5c3 2024-12-09 05:54:30,820Z INFO [org.ovirt.engine.core.bll.hostdev.RefreshHostDevicesCommand] (EE-ManagedScheduledExecutorService-engineScheduledThreadPool-Thread-48) [272ff804] Running command: RefreshHostDevicesCommand internal: true. Entities affected : ID: 260fd31d-b8a3-4ac7-a919-da2964b05b2b Type: VDSAction group MANIPULATE_HOST with role type ADMIN 2024-12-09 05:54:32,122Z INFO [org.ovirt.engine.core.dal.dbbroker.auditloghandling.AuditLogDirector] (EE-ManagedScheduledExecutorService-engineScheduledThreadPool-Thread-48) [272ff804] EVENT_ID: VDS_DETECTED(13), Status of host xxxxx.xxxxxx.xxx.xxx was set to Up. 2024-12-09 05:54:32,224Z INFO [org.ovirt.engine.core.bll.HandleVdsCpuFlagsOrClusterChangedCommand] (EE-ManagedScheduledExecutorService-engineScheduledThreadPool-Thread-48) [1887e967] Running command: HandleVdsCpuFlagsOrClusterChangedCommand internal: true. Entities affected : ID: 260fd31d-b8a3-4ac7-a919-da2964b05b2b Type: VDS 2024-12-09 05:54:32,227Z INFO [org.ovirt.engine.core.bll.UpdateClusterCommand] (EE-ManagedScheduledExecutorService-engineScheduledThreadPool-Thread-48) [3932330e] Lock Acquired to object 'EngineLock:{exclusiveLocks='[]', sharedLocks='[]'}' 2024-12-09 05:54:32,233Z INFO [org.ovirt.engine.core.bll.UpdateClusterCommand] (EE-ManagedScheduledExecutorService-engineScheduledThreadPool-Thread-48) [3932330e] Running command: UpdateClusterCommand internal: true. Entities affected : ID: 1ff65d4a-458d-49a9-a66d-9d1bfdfc5df7 Type: ClusterAction group EDIT_CLUSTER_CONFIGURATION with role type ADMIN 2024-12-09 05:54:32,282Z WARN [org.ovirt.engine.core.dal.dbbroker.auditloghandling.AuditLogDirector] (EE-ManagedScheduledExecutorService-engineScheduledThreadPool-Thread-48) [3932330e] EVENT_ID: FENCE_DISABLED_IN_CLUSTER_POLICY(9,016), Fencing is disabled in Fencing Policy of the Cluster infradc, so HA VMs running on a non-responsive host will not be restarted elsewhere. 2024-12-09 05:54:32,283Z INFO [org.ovirt.engine.core.bll.UpdateClusterCommand] (EE-ManagedScheduledExecutorService-engineScheduledThreadPool-Thread-48) [3932330e] Lock freed to object 'EngineLock:{exclusiveLocks='[]', sharedLocks='[]'}' 2024-12-09 05:54:32,286Z INFO [org.ovirt.engine.core.bll.HandleVdsVersionCommand] (EE-ManagedScheduledExecutorService-engineScheduledThreadPool-Thread-48) [5bd74a4b] Running command: HandleVdsVersionCommand internal: true. Entities affected : ID: 260fd31d-b8a3-4ac7-a919-da2964b05b2b Type: VDS 2024-12-09 05:54:32,288Z INFO [org.ovirt.engine.core.vdsbroker.VdsManager] (EE-ManagedScheduledExecutorService-engineScheduledThreadPool-Thread-48) [5bd74a4b] Received first domain report for host xxxxx.xxxxxx.xxx.xxx 2024-12-09 05:55:39,126Z INFO [org.ovirt.engine.core.sso.servlets.OAuthRevokeServlet] (default task-22) [] User admin@internal-authz with profile [internal] successfully logged out 2024-12-09 05:55:39,131Z INFO [org.ovirt.engine.core.bll.aaa.TerminateSessionsForTokenCommand] (default task-26) [736ce9c] Running command: TerminateSessionsForTokenCommand internal: true. 2024-12-09 05:59:41,230Z INFO [org.ovirt.engine.core.bll.utils.ThreadPoolMonitoringService] (EE-ManagedScheduledExecutorService-engineThreadMonitoringThreadPool-Thread-1) [] Thread pool 'commandCoordinator' is using 0 threads out of 10, 2 threads waiting for tasks. 2024-12-09 05:59:41,230Z INFO [org.ovirt.engine.core.bll.utils.ThreadPoolMonitoringService] (EE-ManagedScheduledExecutorService-engineThreadMonitoringThreadPool-Thread-1) [] Thread pool 'default' is using 0 threads out of 1, 5 threads waiting for tasks. 2024-12-09 05:59:41,230Z INFO [org.ovirt.engine.core.bll.utils.ThreadPoolMonitoringService] (EE-ManagedScheduledExecutorService-engineThreadMonitoringThreadPool-Thread-1) [] Thread pool 'engineScheduledThreadPool' is using 0 threads out of 1, 100 threads waiting for tasks. 2024-12-09 05:59:41,230Z INFO [org.ovirt.engine.core.bll.utils.ThreadPoolMonitoringService] (EE-ManagedScheduledExecutorService-engineThreadMonitoringThreadPool-Thread-1) [] Thread pool 'engineThreadMonitoringThreadPool' is using 1 threads out of 1, 0 threads waiting for tasks. 2024-12-09 05:59:41,230Z INFO [org.ovirt.engine.core.bll.utils.ThreadPoolMonitoringService] (EE-ManagedScheduledExecutorService-engineThreadMonitoringThreadPool-Thread-1) [] Thread pool 'hostUpdatesChecker' is using 0 threads out of 5, 1 threads waiting for tasks. Versions: ansible-core: 2.16.3-2.el8 Pacotes instalados ovirt-ansible-collection.noarch 3.2.0-1.17.el8 @local_ovirt-4.5 ovirt-host.x86_64 4.5.0-3.el8 @local_ovirt-4.5 ovirt-host-dependencies.x86_64 4.5.0-3.el8 @local_ovirt-4.5 ovirt-hosted-engine-ha.noarch 2.5.1-1.el8 @local_ovirt-4.5 ovirt-hosted-engine-setup.noarch 2.7.1-1.3.el8 @local_ovirt-4.5 ovirt-imageio-client.x86_64 2.5.0-1.el8 @local_ovirt-4.5 ovirt-imageio-common.x86_64 2.5.0-1.el8 @local_ovirt-4.5 ovirt-imageio-daemon.x86_64 2.5.0-1.el8 @local_ovirt-4.5 ovirt-openvswitch.noarch 2.15-4.el8 @local_ovirt-4.5 ovirt-openvswitch-ipsec.noarch 2.15-4.el8 @local_ovirt-4.5 ovirt-openvswitch-ovn.noarch 2.15-4.el8 @local_ovirt-4.5 ovirt-openvswitch-ovn-common.noarch 2.15-4.el8 @local_ovirt-4.5 ovirt-openvswitch-ovn-host.noarch 2.15-4.el8 @local_ovirt-4.5 ovirt-provider-ovn-driver.noarch 1.2.36-1.el8 @local_ovirt-4.5 ovirt-python-openvswitch.noarch 2.15-4.el8 @local_ovirt-4.5 ovirt-vmconsole.noarch 1.0.9-4.el8 @local_ovirt-4.5 ovirt-vmconsole-host.noarch 1.0.9-4.el8 @local_ovirt-4.5 Regards, Alex Martins

1 1

Unable to access ovirt Admin Screen from ovirt Host
by louisb＠ameritech.net 07 Dec '24

07 Dec '24

I've reinstalled ovirt 4.4 on my server remotely via cockpit terminal. I'm able to access the ovirt admin screen remotely from the laptop that I used for the install. However, using the same URL I'm unable to gain access to the admin screen. Following the instruction in the documentation I've modified the file: /etc/ovirt-engine/engine.conf.d/99-custom-sso-setup.conf, to reflect the DNS name and I enter in the IP address. But I'm still unable to access the screen from the server console. What else needs to change in order to gain access from the server console? Thanks

5 8

Re: [External] : Mouse cursor disappears in "noVNC" browser console in fullscreen mode
by Alex 05 Dec '24

05 Dec '24

Thank you!!! OL9.4 and Ubuntu24. In both guest OS. Em qui., 5 de dez. de 2024 às 16:42, John Priest <john.priest(a)oracle.com> escreveu: > Best to upgrade to the latest release of the engine which is 4.5.5-1.28 > There were some noVNC console issues with previous releases and the latest > version was released two weeks ago: > https://linux.oracle.com/errata/ELBA-2024-12817.html > While the issues were more access vs. mouse cursor it is always > recommended to run with the latest. > > What is the guest OS in the VM? Is this a new problem or a new > installation? > > John Priest > Director OLVM/OVM Product Management > 508.202.2534 > > -----Original Message----- > From: brancomrt(a)gmail.com <brancomrt(a)gmail.com> > Sent: Thursday, December 5, 2024 1:55 PM > To: users(a)ovirt.org > Subject: [External] : [ovirt-users] Mouse cursor disappears in "noVNC" > browser console in fullscreen mode > > I'm having a problem using the "noVNC" console for virtual machines in > OLVM Version 4.5.5-1.20.el8. The mouse cursor disappears in fullscreen mode > in "noVNC" browser console. > _______________________________________________ > Users mailing list -- users(a)ovirt.org > To unsubscribe send an email to users-leave(a)ovirt.org Privacy Statement: > https://urldefense.com/v3/__https://www.ovirt.org/privacy-policy.html__;!!A… > oVirt Code of Conduct: > https://urldefense.com/v3/__https://www.ovirt.org/community/about/community… > List Archives: > https://urldefense.com/v3/__https://lists.ovirt.org/archives/list/users@ovi… >

1 0

Mouse cursor disappears in "noVNC" browser console in fullscreen mode
by brancomrt＠gmail.com 05 Dec '24

05 Dec '24

I'm having a problem using the "noVNC" console for virtual machines in OLVM Version 4.5.5-1.20.el8. The mouse cursor disappears in fullscreen mode in "noVNC" browser console.

2 1

VM has been paused due to no Storage space error.
by suporte＠logicworks.pt 05 Dec '24

05 Dec '24

Hello, running ovirt Version 4.5.4-1.el8 on Centos 8, randomly we have this error: VM has been paused due to no Storage space error. We have plenty of space on the iSCSI storage. This is a preallocated disk, VirtIO-SCSi. No user interaction. It happens, so far, with 3 VM, Windows and Ubuntu. This service was stopped: dnf-makecache.service This is what I found on the engine log: 2024-08-19 01:04:35,522+01 INFO [org.ovirt.engine.core.vdsbroker.monitoring.VmAnalyzer] (ForkJoinPool-1-worker-25) [eb7e5f1] VM 'ccc65521-934d-4f77-adf3-9f9eeb83a4f8'(Bravo) moved from 'Up' --> 'Paused' 2024-08-19 01:04:35,665+01 ERROR [org.ovirt.engine.core.dal.dbbroker.auditloghandling.AuditLogDirector] (ForkJoinPool-1-worker-25) [eb7e5f1] EVENT_ID: VM_PAUSED_ENOSPC(138), VM Bravo has been paused due to no Storage space error. 2024-08-19 09:26:35,855+01 INFO [org.ovirt.engine.core.vdsbroker.monitoring.VmAnalyzer] (ForkJoinPool-1-worker-29) [72482216] VM 'ccc65521-934d-4f77-adf3-9f9eeb83a4f8'(Bravo) moved from 'Paused' --> 'Down' 2024-08-19 09:26:48,114+01 INFO [org.ovirt.engine.core.vdsbroker.monitoring.VmAnalyzer] (ForkJoinPool-1-worker-15) [72482216] VM 'ccc65521-934d-4f77-adf3-9f9eeb83a4f8'(Bravo) moved from 'WaitForLaunch' --> 'PoweringUp' 2024-08-19 09:27:50,062+01 INFO [org.ovirt.engine.core.vdsbroker.monitoring.VmAnalyzer] (EE-ManagedScheduledExecutorService-engineScheduledThreadPool-Thread-6) [] VM 'ccc65521-934d-4f77-adf3-9f9eeb83a4f8'(Bravo) moved from 'PoweringUp' --> 'Up' 2024-08-19 09:29:25,145+01 INFO [org.ovirt.engine.core.vdsbroker.monitoring.VmAnalyzer] (ForkJoinPool-1-worker-15) [72482216] VM 'ccc65521-934d-4f77-adf3-9f9eeb83a4f8'(Bravo) moved from 'Up' --> 'Paused' 2024-08-19 09:29:25,273+01 ERROR [org.ovirt.engine.core.dal.dbbroker.auditloghandling.AuditLogDirector] (ForkJoinPool-1-worker-15) [72482216] EVENT_ID: VM_PAUSED_ENOSPC(138), VM Bravo has been paused due to no Storage space error. 2024-08-19 09:37:26,128+01 INFO [org.ovirt.engine.core.vdsbroker.monitoring.VmAnalyzer] (ForkJoinPool-1-worker-15) [6d88f065] VM 'ccc65521-934d-4f77-adf3-9f9eeb83a4f8'(Bravo) moved from 'Paused' --> 'Down' 2024-08-19 09:41:43,300+01 INFO [org.ovirt.engine.core.vdsbroker.monitoring.VmAnalyzer] (ForkJoinPool-1-worker-15) [6d88f065] VM 'ccc65521-934d-4f77-adf3-9f9eeb83a4f8'(Bravo) moved from 'WaitForLaunch' --> 'PoweringUp' 2024-08-19 09:42:14,882+01 INFO [org.ovirt.engine.core.vdsbroker.monitoring.VmAnalyzer] (ForkJoinPool-1-worker-23) [6d88f065] VM 'ccc65521-934d-4f77-adf3-9f9eeb83a4f8'(Bravo) moved from 'PoweringUp' --> 'Up' 2024-08-19 09:42:59,792+01 INFO [org.ovirt.engine.core.vdsbroker.monitoring.VmAnalyzer] (ForkJoinPool-1-worker-15) [6d88f065] VM 'ccc65521-934d-4f77-adf3-9f9eeb83a4f8'(Bravo) moved from 'Up' --> 'Paused' 2024-08-19 09:42:59,894+01 ERROR [org.ovirt.engine.core.dal.dbbroker.auditloghandling.AuditLogDirector] (ForkJoinPool-1-worker-15) [6d88f065] EVENT_ID: VM_PAUSED_ENOSPC(138), VM Bravo has been paused due to no Storage space error. 2024-08-19 09:45:30,334+01 INFO [org.ovirt.engine.core.vdsbroker.monitoring.VmAnalyzer] (ForkJoinPool-1-worker-15) [6b3d8ee] VM 'ccc65521-934d-4f77-adf3-9f9eeb83a4f8'(Bravo) moved from 'Paused' --> 'Down' 2024-08-19 09:47:51,068+01 INFO [org.ovirt.engine.core.vdsbroker.monitoring.VmAnalyzer] (ForkJoinPool-1-worker-15) [6b3d8ee] VM 'ccc65521-934d-4f77-adf3-9f9eeb83a4f8'(Bravo) moved from 'WaitForLaunch' --> 'PoweringUp' 2024-08-19 09:48:50,710+01 INFO [org.ovirt.engine.core.vdsbroker.monitoring.VmAnalyzer] (EE-ManagedScheduledExecutorService-engineScheduledThreadPool-Thread-80) [] VM 'ccc65521-934d-4f77-adf3-9f9eeb83a4f8'(Bravo) moved from 'PoweringUp' --> 'Up' 2024-08-19 10:06:38,810+01 INFO [org.ovirt.engine.core.vdsbroker.monitoring.VmAnalyzer] (ForkJoinPool-1-worker-15) [1dd98021] VM 'ccc65521-934d-4f77-adf3-9f9eeb83a4f8'(Bravo) moved from 'PoweringDown' --> 'Down' 2024-08-19 10:08:11,606+01 INFO [org.ovirt.engine.core.vdsbroker.monitoring.VmAnalyzer] (ForkJoinPool-1-worker-15) [1dd98021] VM 'ccc65521-934d-4f77-adf3-9f9eeb83a4f8'(Bravo) moved from 'WaitForLaunch' --> 'PoweringUp' 2024-08-19 10:09:12,507+01 INFO [org.ovirt.engine.core.vdsbroker.monitoring.VmAnalyzer] (EE-ManagedScheduledExecutorService-engineScheduledThreadPool-Thread-25) [] VM 'ccc65521-934d-4f77-adf3-9f9eeb83a4f8'(Bravo) moved from 'PoweringUp' --> 'Up' 2024-08-19 10:21:13,835+01 INFO [org.ovirt.engine.core.vdsbroker.monitoring.VmAnalyzer] (ForkJoinPool-1-worker-15) [63fa2421] VM 'ccc65521-934d-4f77-adf3-9f9eeb83a4f8'(Bravo) moved from 'Up' --> 'Down' 2024-08-19 10:25:19,302+01 INFO [org.ovirt.engine.core.vdsbroker.monitoring.VmAnalyzer] (ForkJoinPool-1-worker-15) [63fa2421] VM 'ccc65521-934d-4f77-adf3-9f9eeb83a4f8'(Bravo) moved from 'WaitForLaunch' --> 'PoweringUp' 2024-08-19 10:26:05,456+01 INFO [org.ovirt.engine.core.vdsbroker.monitoring.VmAnalyzer] (ForkJoinPool-1-worker-3) [63fa2421] VM 'ccc65521-934d-4f77-adf3-9f9eeb83a4f8'(Bravo) moved from 'PoweringUp' --> 'Up' And we cannot start the VM anymore. Any idea? Thanks -- Jose Ferradeira http://www.logicworks.pt

4 16

Ovirt-host no conection
by victordanco＠hotmail.com 03 Dec '24

03 Dec '24

Good morning ovirt community. We have a problem and is that our host fails to connect to the manager. Checking the logs of engine.log I find the following: 2024-12-03 08:49:47,581-05 ERROR [org.ovirt.engine.core.bll.provider.network.SyncNetworkProviderCommand] (EE-ManagedScheduledExecutorService-engineScheduledThreadPool-Thread-96) [794ec20a] Command 'org.ovirt.engine.core.bll.provider.network.SyncNetworkProviderCommand' failed: EngineException: (Failed with error unable to find valid certification path to requested target and code 5050) 2024-12-03 08:49:47,582-05 INFO [org.ovirt.engine.core.bll.provider.network.SyncNetworkProviderCommand] (EE-ManagedScheduledExecutorService-engineScheduledThreadPool-Thread-96) [794ec20a] Lock freed to object 'EngineLock:{exclusiveLocks='[970b9a31-6101-4bff-8503-289e6252832d=PROVIDER]', sharedLocks=''}' 2024-12-03 08:50:10,581-05 ERROR [org.ovirt.engine.core.dal.dbbroker.auditloghandling.AuditLogDirector] (EE-ManagedScheduledExecutorService-engineScheduledThreadPool-Thread-36) [] EVENT_ID: VDS_BROKER_COMMAND_FAILURE(10,802), VDSM my-host.com command Get Host Capabilities failed: Message timeout which can be caused by communication issues 2024-12-03 08:50:10,582-05 ERROR [org.ovirt.engine.core.vdsbroker.monitoring.HostMonitoring] (EE-ManagedScheduledExecutorService-engineScheduledThreadPool-Thread-36) [] Unable to RefreshCapabilities: VDSNetworkException: VDSGenericException: VDSNetworkException: Message timeout which can be caused by communication issues Any idea how to solve it? We have played a lot the certificates lately looking for solutions to the problem.

1 0

Re: [Security] Re: PKIX path validation failed: java.security.cert.CertPathValidatorException: validity check failed
by Sandro Bonazzola 03 Dec '24

03 Dec '24

Hi, dropping security as this is not an oVirt security topic. I'm sorry you have troubles with your oVirt installation but please remind oVirt project is not a commercial product supported with an help desk. It's a community project where users and developers collaborate to develop a software in the hope it helps others. If you need a supported product you should look for an oVirt downstream or a support service. There are some listed on https://www.ovirt.org/community/user-stories/users-and-providers.html Il giorno lun 2 dic 2024 alle ore 11:10 Madhan D <Madhan.D(a)brilyant.com> ha scritto: > Dear Team, > > We sent mail to users(a)ovirt.org but no response yet. Can you please check > and revert. > > It’s critical for us. We were unable to get the console access. > > > > *Thanks and Regards* > > *Madhan D* > > +91 7418999761 > madhan.d(a)brilyant.com > www.brilyant.com > > -- Sandro Bonazzola

3 5

Long term certificates
by eshwayri＠gmail.com 02 Dec '24

02 Dec '24

Before no one is left to answer this question, what long term certificates do we need to worry about with oVirt? I am not talking about the ones engine-setup can re-create, and I am not talking about the Apache certificate that is easy to tie to an external CA; I am talking about any internal 10 year CA type certificates that will eventually expire. This would be especially true if they depend on any RedHat CA authority's signature to be valid. How easy would it be to replace them? Just thinking ahead.

1 0

oVirt slow backups from ISCSI storage domain to nfs target
by Don Dupuis 29 Nov '24

29 Nov '24

Hi I am using oVirt 4.4.10.7 with vms on iscsi Lenovo DE-6600F(nvme interfaced ssds). Backing up thin vms performance is fine, but with preallocated disks, backups to nfs storage with 200Gb interface is only transferring qcow2 image to nfs at 140MiB per sec. Why is qemu-nbd so slow with preallocated images? Does anyone have any issues related to this? Any help would be appreciated. I have made iscsid.conf changes, linux kernel boot changes, and still the same performance. Thanks Don

3 2