April 2020 - Users - oVirt List Archives

How to load VMs after importing a domain
by Shareef Jalloq 16 Apr '20

16 Apr '20

I've followed the online instructions on importing a pre-configured domain into a new data centre but I can't see how to import the VMs. The documentation just says, "You can now import virtual machines and templates from the storage domain to the data center." with no other info. What do I need to do in order to get my VMs up and running? Cheers, Shareef.

4 4

Single SPM Host orphaned from ovirt-engine, cannot confirm host rebooted or maintenance host.
by Terry 16 Apr '20

16 Apr '20

We have number of clusters connected to ovirt-engine. Some of these are single host clusters (running ovirt-release43-4.3.5.2-1 on CentOS7) with local storage. Recently, ovirt-engine started reporting one of these hosts NonResponsive, VMs were still running on the host but ovirt seems unable to communicate with it, testing shows no issues connecting engine -> host:vdsm and likewise the host can communicate with the engine on port 80 and 443. The host in question cannot be managed via IPMI for power management but we are able to perform an SSH reboot via the engine interface. We opted to login to the running virtual machines, shut them down and issue the SSH reboot from the engine. The server changes to rebooting status for some time and then reports NonResponsive state. We are unable to maintenance the host or confirm host has been rebooted manually as we are presented with the following "Error while executing action: Cannot perform confirm 'Host has been rebooted'. Another power management action is already in progress." The VDSM logs on the host in question are continually showing: 2020-04-16 08:23:51,478+0000 INFO (vmrecovery) [vds] recovery: waiting for storage pool to go up (clientIF:711) 2020-04-16 08:23:52,332+0000 INFO (jsonrpc/7) [vdsm.api] FINISH getStoragePoolInfo error=Unknown pool id, pool not connected: (u'6baea5dc-b049-47c2-a94f-5229c37c62d0',) from=::ffff:10.10.1.252,33680, task_id=420249a4-55c0-436d-92c7-ea1286a0e287 (api:52) 2020-04-16 08:23:52,332+0000 ERROR (jsonrpc/7) [storage.TaskManager.Task] (Task='420249a4-55c0-436d-92c7-ea1286a0e287') Unexpected error (task:875) Traceback (most recent call last): File "/usr/lib/python2.7/site-packages/vdsm/storage/task.py", line 882, in _run return fn(*args, **kargs) File "<string>", line 2, in getStoragePoolInfo File "/usr/lib/python2.7/site-packages/vdsm/common/api.py", line 50, in method ret = func(*args, **kwargs) File "/usr/lib/python2.7/site-packages/vdsm/storage/hsm.py", line 2550, in getStoragePoolInfo pool = self.getPool(spUUID) File "/usr/lib/python2.7/site-packages/vdsm/storage/hsm.py", line 351, in getPool raise se.StoragePoolUnknown(spUUID) StoragePoolUnknown: Unknown pool id, pool not connected: (u'6baea5dc-b049-47c2-a94f-5229c37c62d0',) 2020-04-16 08:23:52,333+0000 INFO (jsonrpc/7) [storage.TaskManager.Task] (Task='420249a4-55c0-436d-92c7-ea1286a0e287') aborting: Task is aborted: "Unknown pool id, pool not connected: (u'6baea5dc-b049-47c2-a94f-5229c37c62d0',)" - code 309 (task:1181) During this period, the following is observed in the engine logs: 2020-04-16 08:23:52,307Z ERROR [org.ovirt.engine.core.dal.dbbroker.auditloghandling.AuditLogDirector] (EE-ManagedThreadFactory-engineScheduled-Thread-31) [] EVENT_ID: VDS_BROKER_COMMAND_FAILURE(10,802), VDSM compute01.ovirt.local command SpmStatusVDS failed: Message timeout which can be caused by communication issues 2020-04-16 08:23:52,307Z ERROR [org.ovirt.engine.core.vdsbroker.vdsbroker.SpmStatusVDSCommand] (EE-ManagedThreadFactory-engineScheduled-Thread-31) [] Command 'SpmStatusVDSCommand(HostName = compute01.ovirt.local, SpmStatusVDSCommandParameters:{hostId='67dc53da-d5ee-461e-87de-2ca6dd78637f', storagePoolId='6baea5dc-b049-47c2-a94f-5229c37c62d0'})' execution failed: VDSGenericException: VDSNetworkException: Message timeout which can be caused by communication issues 2020-04-16 08:23:52,346Z ERROR [org.ovirt.engine.core.vdsbroker.irsbroker.GetStoragePoolInfoVDSCommand] (EE-ManagedThreadFactory-engineScheduled-Thread-31) [] Failed in 'GetStoragePoolInfoVDS' method 2020-04-16 08:23:52,355Z ERROR [org.ovirt.engine.core.dal.dbbroker.auditloghandling.AuditLogDirector] (EE-ManagedThreadFactory-engineScheduled-Thread-31) [] EVENT_ID: IRS_BROKER_COMMAND_FAILURE(10,803), VDSM command GetStoragePoolInfoVDS failed: Unknown pool id, pool not connected: (u'6baea5dc-b049-47c2-a94f-5229c37c62d0',) 2020-04-16 08:23:52,356Z ERROR [org.ovirt.engine.core.vdsbroker.irsbroker.IrsBrokerCommand] (EE-ManagedThreadFactory-engineScheduled-Thread-31) [] IrsBroker::Failed::GetStoragePoolInfoVDS: IRSGenericException: IRSErrorException: Failed to GetStoragePoolInfoVDS, error = Unknown pool id, pool not connected: (u'6baea5dc-b049-47c2-a94f-5229c37c62d0',), code = 309 The metadata file for the local storage domain looks fine? ALIGNMENT=1048576 BLOCK_SIZE=512 CLASS=Data DESCRIPTION=compute01_local_storage IOOPTIMEOUTSEC=10 LEASERETRIES=3 LEASETIMESEC=60 LOCKPOLICY= LOCKRENEWALINTERVALSEC=5 MASTER_VERSION=1 POOL_DESCRIPTION=compute01_local POOL_DOMAINS=1cc26dea-688c-40cc-bda6-38b00054001e:Active POOL_SPM_ID=-1 POOL_SPM_LVER=-1 POOL_UUID=6baea5dc-b049-47c2-a94f-5229c37c62d0 REMOTE_PATH=/mnt/ovirt_datastore ROLE=Master SDUUID=1cc26dea-688c-40cc-bda6-38b00054001e TYPE=LOCALFS VERSION=5 _SHA_CKSUM=24c85256b889d0b3384e7975c660f4a5cbb58d33 I would assume this has happened because ovirt was unable to power cycle the machine and now can't confirm the SPM state? Normally in a case like this we would confirm the host has been manually rebooted but we're unable to do that. How can I clear the power management action that ovirt-engine thinks is in progress?

1 1

disk snapshot in illegal status
by David David 16 Apr '20

16 Apr '20

hello after removing "auto generated" snapshot, the vm snapshot became to illegal status and in the "snapshot" tab also marked in illegal status now my vm is not booting but "vdsm-client Volume getInfo" shows the snapshot status is legal vdsm-client Volume getInfo \ storagepoolID=00000000-0000-0000-0000-000000000000 \ storagedomainID=d17aa967-f7f7-4d9f-98e4-7022854da4f5 \ imageID=1844873e-45ef-4a2b-a991-2df6e92a3e2d \ volumeID=f447cd4f-9ab3-489d-8c2a-b2e69456496f { "status": "OK", "lease": { "path": "/dev/d17aa967-f7f7-4d9f-98e4-7022854da4f5/leases", "owners": [], "version": null, "offset": 132120576 }, "domain": "d17aa967-f7f7-4d9f-98e4-7022854da4f5", "capacity": "21474836480", "voltype": "LEAF", "description": "None", "parent": "00000000-0000-0000-0000-000000000000", "format": "RAW", "generation": 1, "image": "1844873e-45ef-4a2b-a991-2df6e92a3e2d", "uuid": "f447cd4f-9ab3-489d-8c2a-b2e69456496f", "disktype": "DATA", "legality": "LEGAL", "mtime": "0", "apparentsize": "21474836480", "truesize": "21474836480", "type": "PREALLOCATED", "children": [], "pool": "", "ctime": "1530472945" } how to fix the problem?

2 4

ovirt-engine unresponsive - how to rescue?
by Shareef Jalloq 16 Apr '20

16 Apr '20

So my engine has gone down and I can't ssh into it either. If I try to log into the web-ui of the node it is running on, I get redirected because the node can't reach the engine. What are my next steps? Shareef.

4 35

yum mirrors 403 forbidden
by Shareef Jalloq 16 Apr '20

16 Apr '20

I seem to be taking 2 steps back and 1 forward at the moment. What could I have done on a node to start getting 403 forbidden errors when using yum? [root@ovirt-node-00 ~]# yum repolist Loaded plugins: enabled_repos_upload, fastestmirror, imgbased-persist, package_upload, product-id, search-disabled-repos, vdsmupgrade, versionlock Loading mirror speeds from cached hostfile ovirt-4.3-epel/x86_64/metalink | 13 kB 00:00:00 * ovirt-4.3-epel: mirror.de.leaseweb.net http://mirror.centos.org/centos/7/sclo/x86_64/rh/repodata/repomd.xml: [Errno 14] HTTP Error 403 - Forbidden Trying other mirror. To address this issue please refer to the below wiki article https://wiki.centos.org/yum-errors If above article doesn't help to resolve this issue please use https://bugs.centos.org/. http://mirror.centos.org/centos/7/sclo/x86_64/rh/repodata/repomd.xml: [Errno 14] HTTP Error 403 - Forbidden Trying other mirror. ovirt-4.3/7 | 3.0 kB 00:00:00 http://mirror.centos.org/centos/7/storage/x86_64/gluster-6/repodata/repomd.…: [Errno 14] HTTP Error 403 - Forbidden Trying other mirror. http://mirror.centos.org/centos/7/opstools/x86_64/repodata/repomd.xml: [Errno 14] HTTP Error 403 - Forbidden Trying other mirror. http://mirror.centos.org/centos/7/virt/x86_64/ovirt-4.3/repodata/repomd.xml: [Errno 14] HTTP Error 403 - Forbidden Trying other mirror. http://mirror.centos.org/centos/7/virt/x86_64/kvm-common/repodata/repomd.xml: [Errno 14] HTTP Error 403 - Forbidden Trying other mirror. http://fedorapeople.org/groups/virt/virtio-win/repo/latest/repodata/repomd.…: [Errno 14] HTTP Error 403 - Forbidden Trying other mirror. sac-gluster-ansible/x86_64 | 3.3 kB 00:00:00 repo id repo name status centos-sclo-rh-release/x86_64 CentOS-7 - SCLo rh 0 ovirt-4.3/7 Latest oVirt 4.3 Release 2,831 ovirt-4.3-centos-gluster6/x86_64 CentOS-7 - Gluster 6 0 ovirt-4.3-centos-opstools/x86_64 CentOS-7 - OpsTools - release 0 ovirt-4.3-centos-ovirt43/x86_64 CentOS-7 - oVirt 4.3 0 ovirt-4.3-centos-qemu-ev/x86_64 CentOS-7 - QEMU EV 0 ovirt-4.3-epel/x86_64 Extra Packages for Enterprise Linux 7 - x86_64 13,244 ovirt-4.3-virtio-win-latest virtio-win builds roughly matching what will be shipped in upcoming RHEL 0 sac-gluster-ansible/x86_64 Copr repo for gluster-ansible owned by sac 18 repolist: 16,093 Uploading Enabled Repositories Report Cannot upload enabled repos report, is this client registered?

2 3

cloud-init ovirt
by PunkGeek 16 Apr '20

16 Apr '20

Hello, How can I use ovirt tools that configure cloud-init vms by python? https://red.ht/2K1gBkn Actually, I want just to use this tool in my python libvirt script. Best regards

2 1

Can't migrate over migration network
by briandumont＠gmail.com 15 Apr '20

15 Apr '20

Hello, I am trying to configure a dedicated migration network and am unable to get it to work. It is failing with a no route to host error in the cockpit logs, specifically a libvirtd error virNetSocketNewConnectTCP:591 : unable to connect to server at '192.168.1.50:49152': No route to host. Trying to migrate from Host 1 to Host 2. 192.168.1.50 is the ip address on the source host for the migration. If I try to migrate in the other direction I get the same error with tthat hosts migration network ip. My migration network is setup on a different VLAN than my ovirtmgmt network. Here is my configuration: - 2 Hosts - Host 1 - nic 1 - ovirtmgmt - 192.168.1.24/24 - VLAN 1 - nic 2 - migration network - 192.168.1.50/24 VLAN 25 - Host 2 - nic 1 - ovirtmgmnt - 192.168.1.24/24 - VLAN 1 - nic 2 - migration network - 192.168.1.51/24 - VLAN 25 Migration network is setup as follows: - Assign - Yes - Required - Yes - VM Network - No (tried it with VM Nework = Yes with same results) - Migration Network - Yes I think I've got something fundamentally wrong with my host networking but I can't figure it out. Appreciate the help.

4 11

NUMA Pinning bug with Hugepages?
by Alan G 15 Apr '20

15 Apr '20

Hi, I seem to have found an issue when trying to setup a high performance VM utilising hugepages and NUMA pinning. The VM is configured for 32GB RAM and uses hugepages of size 1G. The host has two NUMA nodes each having 64GB RAM (for 128GB total system RAM). No other VMs are running or otherwise pinned to the host. The VM is pinned to node 0. And the required hugepages are allocated with echo 32 > /sys/devices/system/node/node0/hugepages/hugepages-1048576kB/nr_hugepages I then attempt to start the VM and get a pop-up error saying "cannot accommodate memory of VM's pinned virtual NUMA nodes within host's physical NUMA nodes". However, if I remove the hugepages from node 0 echo 0 > /sys/devices/system/node/node0/hugepages/hugepages-1048576kB/nr_hugepages Start the VM and then immediately re-create the hugepqages then everything works as expected. It seems to me that oVirt is considering the hugepages as allocated memory even if they are not in use. Is this correct? Thanks, Alan

3 3

upgrade VM cluster compatibility version
by Bill James 15 Apr '20

15 Apr '20

I'm trying to upgrade my cluster to 4.3. Currently its running 4.2.8.2-1.el7. But the cluster compatibility version is 3.6. I try to change that and it says I need to edit the VM. https://www.ovirt.org/documentation/upgrade-guide/chap-Post-Upgrade_Tasks.h… says: *Important:* An error message may warn that some virtual machines and templates are incorrectly configured. To fix this error, edit each virtual machine manually. The *Edit Virtual Machine* window provides additional validations and warnings that show what to correct. Sometimes the issue is automatically corrected and the virtual machine’s configuration just needs to be saved again. After editing each virtual machine, you will be able to change the cluster compatibility version. I edit the VM and there are no errors or warnings. I see no way to change the cluster compatibility level on a VM. How do I change the Custom Compatibility Version?? -- This email, its contents and attachments contain information from J2 Global, Inc. and/or its affiliates which may be privileged, confidential or otherwise protected from disclosure. The information is intended to be for the addressee(s) only. If you are not an addressee, any disclosure, copy, distribution or use of the contents of this message is prohibited. If you have received this email in error, please notify the sender by reply email and delete the original message and any copies.

2 2

Get OAuth token locally without password?
by Chris Adams 14 Apr '20

14 Apr '20

I'm working on a cron job to run on the engine to do some tasks via the API. Is there a way I can get an API OAuth token created without actually storing a password? For example, some script that can be run to directly create a session in the database and return the token. Obviously this _can_ be done, it's a matter of knowing the right bits to do (so basically wondering if that has already been written). I only have internal users, so no external authentication store available. -- Chris Adams <cma(a)cmadams.net>

2 1

Migrate engine to another storage domain?
by Simon Nylund 14 Apr '20

14 Apr '20

Is there a successful way of migrating an up and running engine to another storage domain? I've been reading through the archive and tested a few various ways without any success. The only way i found working without errors was deploying a new engine attached to the new storage domain, and import the old data domain for the vms, but then you lose all the settings for the previous engine (ofc). And have to redo all of that.

1 0

export vm from 3.4 to 4.3
by Budur Nagaraju 14 Apr '20

14 Apr '20

Hi Can someone help in exporting a vm from 3.5 to 4.3 version , any help or links for exporting a vm ? Thanks, Nagaraju

4 5

Total LUN Limitations
by Ryan Chewning 14 Apr '20

14 Apr '20

I'm hoping that someone might be able to clarify a limitation found in the RedHat documentation. The documentation indicates that there is a limitation of 300 LUNS per block based storage device. Does this include LUNs which are directly mapped to a VM? Or does this referrer to LUNs which serve as a storage repository for ISOs / VM disks?

1 0

Host fails to enter in maintenance due to migration failure
by Vinícius Ferrão 14 Apr '20

14 Apr '20

Hello, I’ve a host that’s preparing to maintenance for almost 20 hours now. There’s a huge VM on it, with 32 gigs of RAM and this VM is failing migration. So, there’s a way to cancel prepare for maintenance, so the host can stop trying this migration that’s end up failing? I can just shutdown the VM to do the maintenance on the host... Thanks,

2 3

Re: oVirt 4.4.0 Beta release refresh is now available for testing
by Joop 13 Apr '20

13 Apr '20

On 13-4-2020 09:45, Satheesaran Sundaramoorthi wrote: > > > On Thu, Apr 9, 2020 at 3:37 PM Sandro Bonazzola <sbonazzo(a)redhat.com > <mailto:sbonazzo@redhat.com>> wrote: > > > > Does the hyperconverged installation now work with glusterfs? > I tested alpha/beta1/2 and the latter won't get past the > storage step. > > > Not yet. There was the issue with glusterfs storage domain and storage > domain blocksize probe check. > This is addressed with a new ioprocess-1.4.1 package for the bug[1]. > > We are testing currently with this fix and it works well. But hitting > yet another issue, where Hosted Engine deployment > fails at the very last stage in the deployment. Gluster developer is > looking in to it, as I respond here. > Soon we will have a working build and I will let you once these issues > are settled. > > [1] - https://bugzilla.redhat.com/show_bug.cgi?id=1820283 > > Thanks, > sas > > Thanks for the message. Looking through the logs I saw references to the blocksize but since that is contained in the ansible scripts which get downloaded to the host I gave up finding the source versions to try to see it I could get it going beyond that failing step. Anyway, thanks for informing me, Regards, Joop

3 3

engine certificate problems in MacOS Catalina
by Jayme 13 Apr '20

13 Apr '20

I recently setup a new ovirt environment using latest 4.3.9 installer. I can't seem to get the novnc client to work for the life of me in safari or chrome on MacOS catalina. I have downloaded the CA from the login page and imported it into keychain and made sure it was fully trusted. In both system and login keychains. Looking at this: https://support.apple.com/en-us/HT210176 seems to suggest that the certificate may be invalid if the issue date is longer than 825 days. The certificate generated by ovirt installer seems to be valid for 5 years. I'm not sure if this is the issue or something else is wrong but if cert length is a problem is there a way for me to re-regenerate a new certificate with a shorter issue period?

1 0

oVirt LUN Limitations
by Ryan Chewning 13 Apr '20

13 Apr '20

I'm hoping that someone might be able to clarify a limitation found in the RedHat documentation. The documentation indicates that there is a limitation of 300 LUNS per block based storage device. Does this include LUNs which are directly mapped to a VM? Or does this referrer to LUNs which serve as a storage repository for ISOs / VM disks? Thanks!

1 0

One more time
by eevans＠digitaldatatechs.com 13 Apr '20

13 Apr '20

I have thought about the issue with Windows vm's reporting the timezone incorrectly. I reaqlized that just last month, before I created my vm's that the Daylight Savings Time took affect moving our time up 1 hour. SO, based on that, instead of setting my time zone to US Eastern (GMT -5), I set it to SA Western (GMT -4) and the nasty exclamation mark disappeared. This tells me that DST (Daylight Savings Time) is not an option or configurable in Ovirt. I think this should be an easy fix for those of use that are stuck with it. I don't think the rest of the world participates in DST but most of the United States does. It would be nice to have those zones added as a configurable option that would switch on the dates it is nationally indicated to change. I hope the developers read this. Thanks.

1 0

DR on hyperconverged deployment
by wodel youchi 13 Apr '20

13 Apr '20

Hi, Need to understand somethings about DR on oVirt-HI - What does mean : Scheduling regular backups using geo-replication (point 3.3.4 RHHI 1.7 Doc Maintaining RHHI) : - Does this mean creating a check-point? - If yes, does this mean that the geo-replication process will sync data up to that check-point and then stops the synchronization, then repeat the same cycle the day after? does this mean that the minimum RPO is one day? - I created a snapshot of a VM on the source Manager, I synced the volume then I executed a DR, The VM was started on the Target Manager but the VM didn't have its snapshot, any idea??? Regards, be safe.

2 6

multiple subnets on hosts for vm's
by eevans＠digitaldatatechs.com 12 Apr '20

12 Apr '20

This is not critical but some functionality I think should be in Ovirt and maybe I am going about it the wrong way. I have several subnets on my physical network. My engine host is separate from the clustered hosts that house the vm's. In order for the vm's on the other subnets (vlans) to get Internet access, I have to add the extra ip addrs on the ovirtmgmt network interface. I have read and tried every solution I could find trying to add teh subnets without using the extra ip addrs option but the machine either becomes isolated or will not start at all. Can someone tell me how to add the vlans so the engine host sees and utilizes them? The engine host can get to resources on the vlans because of the router config but will not let any subnet except ovirtmgmt outside the cluster. This is not critical but something I would like to get working for future deployments. Thank you for your help. Eric

2 1

unsubscribe me please
by Raz Tamir 12 Apr '20

12 Apr '20

EOM -- Raz Tamir

1 0

Storage domain in maintenance
by Strahil Nikolov 11 Apr '20

11 Apr '20

Hello All, I have several storage domains in maintenance mode and I can't neither detach, nor activate. Can someone explain to me what is actually doing the engine whwn I try to activate - so I can debug it? The error from vdsm-client is that the storage domain does not exist, yet it is in the DB . I assume that there is somw storage issue, but in order to debug it - I need to know what is gooing on under the hood. Thanks in advance Best Regards, Strahil Nikolov

3 7

Windows 10 Pro 64 (1909) crashes when migrating
by Maton, Brett 11 Apr '20

11 Apr '20

I recently added a Windows 10 Pro 64 bit (release 1909) VM, and I'm seeing a lot of failures when oVirt tries to move the VM to another host (triggered by load balancing), These errors are showing up in the UI event log Migration failed (VM: <vm label>, Source: <host 1>, Destination: <host 2>). Followed by: VM <host> is down with error. Exit message: Lost connection with qemu process. Google returned some references to 'options kvm ignore_msrs=1' which I've added to /etc/modprobe/d/kvm.conf and restarted the hosts but that doesn't appear to have made a difference. Is this a known issue with Windows 10 guests?

3 7

Firewall GARP not reachable to VM
by k.betsis＠gmail.com 10 Apr '20

10 Apr '20

Hi all Does anyone know how i can allow my Firewall VM cluster act as the default gateway to VMs within the same network? I've configured the GARP functionality on the OPNSENSE firewalls (PFSENSE fork). VMs within the same network can ping the firewall IP addresses successfully but not the GARP IP. The ovirt network has been configured with the MAC Address Anti-spoofing to false. One firewall has been configured with virtio network drivers and the with e1000 both exhibiting the same behavior. Currently all VMs have been configured with a default gateway the primary firewall. Network workarounds using BGP and attributes can work, but are way to complicate to streamline for all VMs when a simple VRRP can do the job. Any ideas what i am missing?

2 1

Data domain down due to space crunch
by Crazy Ayansh 10 Apr '20

10 Apr '20

Hi Guys, I am using Ovirt 4.3.7 in my environment.I am using servers local storage for data domains. Today one of the servers data domain got full and due to this my VMs paused. [image: image.png] Now i am unable to export my VM as the storage in ovirt is down.....What should i do now.....any suggestions.....? Thanks

2 2