- Users - oVirt List Archives

Multitenant scenario in oVirt
by Michal Gutowski 17 Apr '20

17 Apr '20

Hi oVirt community, I'm playing with a multitenant use-case in oVirt 3.4.6... My setup is as follows: - I have two working Data Centers (DC1 and DC2) - I created two additional users DC1-admin and DC2-admin - In DC1 permission settings I've added DC1-admin as a user with a builtin DataCenterAdmin Role. - In DC2 permission settings I've added DC2-admin as a user with a builtin DataCenterAdmin Role. Now in terms of permissions all is good: DC1-admin is not able to modify anything in DC2 and DC2-admin is not able to modify anything in DC1. However in both the Admin Portal and the VM Portal DC1-admin and DC2-admin can still see all other datacenter resources. My expectation was that if I login to the Admin Portal as e.g. DC2-admin I will only see DC2 datacenter in the GUI and nothing else. Same with VM Portal. I played with different user settings but I couldn't make it work... I think the problem is that whatever user you create it will always belong to the build-in "everyone" group and inherit permission to see everything in the portal. Is it possible to achieve a scenario where e.g. DC2-admin will login to the Admin Portal and only see resources that belong to DC2 and nothing else? Thanks, Michal

4 4

Sometimes paused due to unknown storage error on gluster
by Gianluca Cecchi 16 Apr '20

16 Apr '20

Hello, having deployed oVirt 4.3.9 single host HCI with Gluster, I see some times VM going into paused state for the error above and needing to manually run it (sometimes this resumal operation fails). Actually it only happened with empty disk (thin provisioned) and sudden high I/O during the initial phase of install of the OS; it didn't happened then during normal operaton (even with 600MB/s of throughput). I suspect something related to metadata extension not able to be in pair with the speed of the physical disk growing.... similar to what happens for block based storage domains where the LVM layer has to extend the logical volume representing the virtual disk My real world reproduction of the error is during install of OCP 4.3.8 master node, when Red Hat Cores OS boots from network and wipes the disk and I think then transfer an image, so doing high immediate I/O. The VM used as master node has been created with a 120Gb thin provisioned disk (virtio-scsi type) and starts with disk just initialized and empty, going through PXE install. I get this line inside events for the VM Mar 27, 2020, 12:35:23 AM VM master01 has been paused due to unknown storage error. Here logs around the time frame above: - engine.log https://drive.google.com/file/d/1zpNo5IgFVTAlKXHiAMTL-uvaoXSNMVRO/view?usp=… - vdsm.log https://drive.google.com/file/d/1v8kR0N6PdHBJ5hYzEYKl4-m7v1Lb_cYX/view?usp=… Any suggestions? The disk of the VM is on vmstore storage domain and its gluster volume settings are: [root@ovirt tmp]# gluster volume info vmstore Volume Name: vmstore Type: Distribute Volume ID: a6203d77-3b9d-49f9-94c5-9e30562959c4 Status: Started Snapshot Count: 0 Number of Bricks: 1 Transport-type: tcp Bricks: Brick1: ovirtst.mydomain.storage:/gluster_bricks/vmstore/vmstore Options Reconfigured: performance.low-prio-threads: 32 storage.owner-gid: 36 performance.read-ahead: off user.cifs: off storage.owner-uid: 36 performance.io-cache: off performance.quick-read: off network.ping-timeout: 30 features.shard: on network.remote-dio: off cluster.eager-lock: enable performance.strict-o-direct: on transport.address-family: inet nfs.disable: on [root@ovirt tmp]# What about config above, related to eventual optimizations to be done based on having single host? And comparing with the virt group of options: [root@ovirt tmp]# cat /var/lib/glusterd/groups/virt performance.quick-read=off performance.read-ahead=off performance.io-cache=off performance.low-prio-threads=32 network.remote-dio=enable cluster.eager-lock=enable cluster.quorum-type=auto cluster.server-quorum-type=server cluster.data-self-heal-algorithm=full cluster.locking-scheme=granular cluster.shd-max-threads=8 cluster.shd-wait-qlength=10000 features.shard=on user.cifs=off cluster.choose-local=off client.event-threads=4 server.event-threads=4 performance.client-io-threads=on [root@ovirt tmp]# ? Thanks Gianluca

5 23

Broken doco link for New Host->Kernel boot parameters
by Shareef Jalloq 16 Apr '20

16 Apr '20

Hi, I'm trying to set up one of our nodes with PCI passthrough but there is a dead lnk in the documentation for the kernel boot parameter section of creating a new host. https://www.ovirt.org/documentation/admin-guide/sect-Explanation_of_Setting… Anyone know where this went or can help me with the options? Shareef.

3 8

How to load VMs after importing a domain
by Shareef Jalloq 16 Apr '20

16 Apr '20

I've followed the online instructions on importing a pre-configured domain into a new data centre but I can't see how to import the VMs. The documentation just says, "You can now import virtual machines and templates from the storage domain to the data center." with no other info. What do I need to do in order to get my VMs up and running? Cheers, Shareef.

4 4

Single SPM Host orphaned from ovirt-engine, cannot confirm host rebooted or maintenance host.
by Terry 16 Apr '20

16 Apr '20

We have number of clusters connected to ovirt-engine. Some of these are single host clusters (running ovirt-release43-4.3.5.2-1 on CentOS7) with local storage. Recently, ovirt-engine started reporting one of these hosts NonResponsive, VMs were still running on the host but ovirt seems unable to communicate with it, testing shows no issues connecting engine -> host:vdsm and likewise the host can communicate with the engine on port 80 and 443. The host in question cannot be managed via IPMI for power management but we are able to perform an SSH reboot via the engine interface. We opted to login to the running virtual machines, shut them down and issue the SSH reboot from the engine. The server changes to rebooting status for some time and then reports NonResponsive state. We are unable to maintenance the host or confirm host has been rebooted manually as we are presented with the following "Error while executing action: Cannot perform confirm 'Host has been rebooted'. Another power management action is already in progress." The VDSM logs on the host in question are continually showing: 2020-04-16 08:23:51,478+0000 INFO (vmrecovery) [vds] recovery: waiting for storage pool to go up (clientIF:711) 2020-04-16 08:23:52,332+0000 INFO (jsonrpc/7) [vdsm.api] FINISH getStoragePoolInfo error=Unknown pool id, pool not connected: (u'6baea5dc-b049-47c2-a94f-5229c37c62d0',) from=::ffff:10.10.1.252,33680, task_id=420249a4-55c0-436d-92c7-ea1286a0e287 (api:52) 2020-04-16 08:23:52,332+0000 ERROR (jsonrpc/7) [storage.TaskManager.Task] (Task='420249a4-55c0-436d-92c7-ea1286a0e287') Unexpected error (task:875) Traceback (most recent call last): File "/usr/lib/python2.7/site-packages/vdsm/storage/task.py", line 882, in _run return fn(*args, **kargs) File "<string>", line 2, in getStoragePoolInfo File "/usr/lib/python2.7/site-packages/vdsm/common/api.py", line 50, in method ret = func(*args, **kwargs) File "/usr/lib/python2.7/site-packages/vdsm/storage/hsm.py", line 2550, in getStoragePoolInfo pool = self.getPool(spUUID) File "/usr/lib/python2.7/site-packages/vdsm/storage/hsm.py", line 351, in getPool raise se.StoragePoolUnknown(spUUID) StoragePoolUnknown: Unknown pool id, pool not connected: (u'6baea5dc-b049-47c2-a94f-5229c37c62d0',) 2020-04-16 08:23:52,333+0000 INFO (jsonrpc/7) [storage.TaskManager.Task] (Task='420249a4-55c0-436d-92c7-ea1286a0e287') aborting: Task is aborted: "Unknown pool id, pool not connected: (u'6baea5dc-b049-47c2-a94f-5229c37c62d0',)" - code 309 (task:1181) During this period, the following is observed in the engine logs: 2020-04-16 08:23:52,307Z ERROR [org.ovirt.engine.core.dal.dbbroker.auditloghandling.AuditLogDirector] (EE-ManagedThreadFactory-engineScheduled-Thread-31) [] EVENT_ID: VDS_BROKER_COMMAND_FAILURE(10,802), VDSM compute01.ovirt.local command SpmStatusVDS failed: Message timeout which can be caused by communication issues 2020-04-16 08:23:52,307Z ERROR [org.ovirt.engine.core.vdsbroker.vdsbroker.SpmStatusVDSCommand] (EE-ManagedThreadFactory-engineScheduled-Thread-31) [] Command 'SpmStatusVDSCommand(HostName = compute01.ovirt.local, SpmStatusVDSCommandParameters:{hostId='67dc53da-d5ee-461e-87de-2ca6dd78637f', storagePoolId='6baea5dc-b049-47c2-a94f-5229c37c62d0'})' execution failed: VDSGenericException: VDSNetworkException: Message timeout which can be caused by communication issues 2020-04-16 08:23:52,346Z ERROR [org.ovirt.engine.core.vdsbroker.irsbroker.GetStoragePoolInfoVDSCommand] (EE-ManagedThreadFactory-engineScheduled-Thread-31) [] Failed in 'GetStoragePoolInfoVDS' method 2020-04-16 08:23:52,355Z ERROR [org.ovirt.engine.core.dal.dbbroker.auditloghandling.AuditLogDirector] (EE-ManagedThreadFactory-engineScheduled-Thread-31) [] EVENT_ID: IRS_BROKER_COMMAND_FAILURE(10,803), VDSM command GetStoragePoolInfoVDS failed: Unknown pool id, pool not connected: (u'6baea5dc-b049-47c2-a94f-5229c37c62d0',) 2020-04-16 08:23:52,356Z ERROR [org.ovirt.engine.core.vdsbroker.irsbroker.IrsBrokerCommand] (EE-ManagedThreadFactory-engineScheduled-Thread-31) [] IrsBroker::Failed::GetStoragePoolInfoVDS: IRSGenericException: IRSErrorException: Failed to GetStoragePoolInfoVDS, error = Unknown pool id, pool not connected: (u'6baea5dc-b049-47c2-a94f-5229c37c62d0',), code = 309 The metadata file for the local storage domain looks fine? ALIGNMENT=1048576 BLOCK_SIZE=512 CLASS=Data DESCRIPTION=compute01_local_storage IOOPTIMEOUTSEC=10 LEASERETRIES=3 LEASETIMESEC=60 LOCKPOLICY= LOCKRENEWALINTERVALSEC=5 MASTER_VERSION=1 POOL_DESCRIPTION=compute01_local POOL_DOMAINS=1cc26dea-688c-40cc-bda6-38b00054001e:Active POOL_SPM_ID=-1 POOL_SPM_LVER=-1 POOL_UUID=6baea5dc-b049-47c2-a94f-5229c37c62d0 REMOTE_PATH=/mnt/ovirt_datastore ROLE=Master SDUUID=1cc26dea-688c-40cc-bda6-38b00054001e TYPE=LOCALFS VERSION=5 _SHA_CKSUM=24c85256b889d0b3384e7975c660f4a5cbb58d33 I would assume this has happened because ovirt was unable to power cycle the machine and now can't confirm the SPM state? Normally in a case like this we would confirm the host has been manually rebooted but we're unable to do that. How can I clear the power management action that ovirt-engine thinks is in progress?

1 1

disk snapshot in illegal status
by David David 16 Apr '20

16 Apr '20

hello after removing "auto generated" snapshot, the vm snapshot became to illegal status and in the "snapshot" tab also marked in illegal status now my vm is not booting but "vdsm-client Volume getInfo" shows the snapshot status is legal vdsm-client Volume getInfo \ storagepoolID=00000000-0000-0000-0000-000000000000 \ storagedomainID=d17aa967-f7f7-4d9f-98e4-7022854da4f5 \ imageID=1844873e-45ef-4a2b-a991-2df6e92a3e2d \ volumeID=f447cd4f-9ab3-489d-8c2a-b2e69456496f { "status": "OK", "lease": { "path": "/dev/d17aa967-f7f7-4d9f-98e4-7022854da4f5/leases", "owners": [], "version": null, "offset": 132120576 }, "domain": "d17aa967-f7f7-4d9f-98e4-7022854da4f5", "capacity": "21474836480", "voltype": "LEAF", "description": "None", "parent": "00000000-0000-0000-0000-000000000000", "format": "RAW", "generation": 1, "image": "1844873e-45ef-4a2b-a991-2df6e92a3e2d", "uuid": "f447cd4f-9ab3-489d-8c2a-b2e69456496f", "disktype": "DATA", "legality": "LEGAL", "mtime": "0", "apparentsize": "21474836480", "truesize": "21474836480", "type": "PREALLOCATED", "children": [], "pool": "", "ctime": "1530472945" } how to fix the problem?

2 4

ovirt-engine unresponsive - how to rescue?
by Shareef Jalloq 16 Apr '20

16 Apr '20

So my engine has gone down and I can't ssh into it either. If I try to log into the web-ui of the node it is running on, I get redirected because the node can't reach the engine. What are my next steps? Shareef.

4 35

yum mirrors 403 forbidden
by Shareef Jalloq 16 Apr '20

16 Apr '20

I seem to be taking 2 steps back and 1 forward at the moment. What could I have done on a node to start getting 403 forbidden errors when using yum? [root@ovirt-node-00 ~]# yum repolist Loaded plugins: enabled_repos_upload, fastestmirror, imgbased-persist, package_upload, product-id, search-disabled-repos, vdsmupgrade, versionlock Loading mirror speeds from cached hostfile ovirt-4.3-epel/x86_64/metalink | 13 kB 00:00:00 * ovirt-4.3-epel: mirror.de.leaseweb.net http://mirror.centos.org/centos/7/sclo/x86_64/rh/repodata/repomd.xml: [Errno 14] HTTP Error 403 - Forbidden Trying other mirror. To address this issue please refer to the below wiki article https://wiki.centos.org/yum-errors If above article doesn't help to resolve this issue please use https://bugs.centos.org/. http://mirror.centos.org/centos/7/sclo/x86_64/rh/repodata/repomd.xml: [Errno 14] HTTP Error 403 - Forbidden Trying other mirror. ovirt-4.3/7 | 3.0 kB 00:00:00 http://mirror.centos.org/centos/7/storage/x86_64/gluster-6/repodata/repomd.…: [Errno 14] HTTP Error 403 - Forbidden Trying other mirror. http://mirror.centos.org/centos/7/opstools/x86_64/repodata/repomd.xml: [Errno 14] HTTP Error 403 - Forbidden Trying other mirror. http://mirror.centos.org/centos/7/virt/x86_64/ovirt-4.3/repodata/repomd.xml: [Errno 14] HTTP Error 403 - Forbidden Trying other mirror. http://mirror.centos.org/centos/7/virt/x86_64/kvm-common/repodata/repomd.xml: [Errno 14] HTTP Error 403 - Forbidden Trying other mirror. http://fedorapeople.org/groups/virt/virtio-win/repo/latest/repodata/repomd.…: [Errno 14] HTTP Error 403 - Forbidden Trying other mirror. sac-gluster-ansible/x86_64 | 3.3 kB 00:00:00 repo id repo name status centos-sclo-rh-release/x86_64 CentOS-7 - SCLo rh 0 ovirt-4.3/7 Latest oVirt 4.3 Release 2,831 ovirt-4.3-centos-gluster6/x86_64 CentOS-7 - Gluster 6 0 ovirt-4.3-centos-opstools/x86_64 CentOS-7 - OpsTools - release 0 ovirt-4.3-centos-ovirt43/x86_64 CentOS-7 - oVirt 4.3 0 ovirt-4.3-centos-qemu-ev/x86_64 CentOS-7 - QEMU EV 0 ovirt-4.3-epel/x86_64 Extra Packages for Enterprise Linux 7 - x86_64 13,244 ovirt-4.3-virtio-win-latest virtio-win builds roughly matching what will be shipped in upcoming RHEL 0 sac-gluster-ansible/x86_64 Copr repo for gluster-ansible owned by sac 18 repolist: 16,093 Uploading Enabled Repositories Report Cannot upload enabled repos report, is this client registered?

2 3

cloud-init ovirt
by PunkGeek 16 Apr '20

16 Apr '20

Hello, How can I use ovirt tools that configure cloud-init vms by python? https://red.ht/2K1gBkn Actually, I want just to use this tool in my python libvirt script. Best regards

2 1

Can't migrate over migration network
by briandumont＠gmail.com 15 Apr '20

15 Apr '20

Hello, I am trying to configure a dedicated migration network and am unable to get it to work. It is failing with a no route to host error in the cockpit logs, specifically a libvirtd error virNetSocketNewConnectTCP:591 : unable to connect to server at '192.168.1.50:49152': No route to host. Trying to migrate from Host 1 to Host 2. 192.168.1.50 is the ip address on the source host for the migration. If I try to migrate in the other direction I get the same error with tthat hosts migration network ip. My migration network is setup on a different VLAN than my ovirtmgmt network. Here is my configuration: - 2 Hosts - Host 1 - nic 1 - ovirtmgmt - 192.168.1.24/24 - VLAN 1 - nic 2 - migration network - 192.168.1.50/24 VLAN 25 - Host 2 - nic 1 - ovirtmgmnt - 192.168.1.24/24 - VLAN 1 - nic 2 - migration network - 192.168.1.51/24 - VLAN 25 Migration network is setup as follows: - Assign - Yes - Required - Yes - VM Network - No (tried it with VM Nework = Yes with same results) - Migration Network - Yes I think I've got something fundamentally wrong with my host networking but I can't figure it out. Appreciate the help.

4 11