- Users - oVirt List Archives

Install test lab single host HCI with plain CentOS as OS
by Gianluca Cecchi 30 Oct '20

30 Oct '20

Hello, if I want to install a test lab environment of type single host HCI and want to use CentOS 8.2 as the OS for the host, can I still use the graphical wizard from cockpit? I see it accessing the server... Or is it thought only to be run from ovirt-node-ng system? In case it is not possible is there a command line workflow I can use to get the same as the wizard playbook run? BTW: I want to use CentOS for this test lab because I need to install custom packages (for performance counters and other things) so I'd prefer the flexibility to have a plain CentOS and not the NG node. Thanks in advance, Gianluca

3 9

Add Node to a single node installation with self hosted engine.
by Marcel d'Heureuse 29 Oct '20

29 Oct '20

Hi, I got a problem with my Ovirt installation. Normally we deliver Ovirt as single node installation and we told our service guys if the internal client will have more redundancy we need two more server and add this to the single node installation. i thought that no one would order two new servers. Now I have the problem to get the system running. First item is, that this environment has no internet access. So I can't install software by update with yum. The Ovirt installation is running on Ovirt node 4.3.9 boot USB Stick. All three servers have the same software installed. On the single node I have installed the hosted Engine package 1,1 GB to deploy the self-hosted engine without internet. That works. Gluster, Ovirt, Self-Hosted engine are running on the server 01. What should I do first? Deploy the Glusterfs first and then add the two new hosts to the single node installation? Or should I deploy a new Ovirt System to the two new hosts and add later the cleaned host to the new Ovirt System? I have not found any items in this mailing list which gives me an idea what I should do now. Br Marcel

5 8

Re: vdsm with NFS storage reboot or shutdown more than 15 minutes. with error failed to unmount /rhev/data-center/mnt/172.18.81.14:_home_nfs_data: Device or resource busy
by Strahil Nikolov 29 Oct '20

29 Oct '20

That's not expected. You definately need to check the Engine's logs (on the Hosted or Dedicated Engine system) and the vdsm logs on the host. Usually , the first step is to "evacuate" (live migrate) all VMs from the Host and if it fails to do that in a reasonable timeframe - the maintenance is cancelled.Next it will set the host into maintenance and most probably (not sure about this one) the engine will assign a new host as SPM. Best Regards, Strahil Nikolov В сряда, 28 октомври 2020 г., 05:04:44 Гринуич+2, lifuqiong(a)sunyainfo.com <lifuqiong(a)sunyainfo.com> написа: Hi, Strahil, Thank you for your reply. I've try setting host to maintenance and the host reboot immediately, What does vdsm do when setting host to maintenance? Thank you Best Regards Mark Lee > From: Strahil Nikolov via Users > Date: 2020-10-27 23:44 > To: users; lifuqiong(a)sunyainfo.com > Subject: [ovirt-users] Re: vdsm with NFS storage reboot or shutdown more than 15 minutes. with error failed to unmount /rhev/data-center/mnt/172.18.81.14:_home_nfs_data: Device or resource busy > When you set a host to maintenance from oVirt API/UI, one of the tasks is to umount any shared storage (incluing the NFS you got). Then rebooting should work like a charm. > > > > Why did you reboot without putting the node in maintenance ? > > > > P.S.: Do not confuse rebooting with fencing - the latter kills the node ungracefully in order to safely start HA VMs on another node. > > > > Best Regards, > > Strahil Nikolov > > > > > > > > > > > > > > В вторник, 27 октомври 2020 г., 10:27:01 Гринуич+2, lifuqiong(a)sunyainfo.com <lifuqiong(a)sunyainfo.com> написа: > > > > > > > > > > > > > > > > Hi everyone: > > Description of problem: > > > > When exec "reboot" or "shutdown -h 0" cmd on vdsm server, the vdsm server will reboot or shutdown more than 30 minutes. the screen shows '[FAILED] Failed unmouting /rhev/data-center/mnt/172.18.81.41:_home_nfs_data'. > > other messages may be useful: [] watchdog: watchdog0: watchdog did not stop! []systemd-shutdown[5594]: Failed to unmount /rhev/data-center/mnt/172.18.81.14:_home_nfs_data: Device or resource busy > > []systemd-shutdown[1]: Failed to wait for process: Protocol error > > []systemd-shutdown[5595]: Failed to remount '/' read-only: Device or resource busy > > []systemd-shutdown[1]: Failed to wait for process: Protocol error > > dracut Warning: Killing all remaining processes > > dracut Warning: Killing all remaining processes > > > > Version-Release number of selected component (if applicable): > > Software Version:4.2.8.2-1.el7 > > OS: CentOS Linux release 7.5.1804 (Core) > > How reproducible: > > 100% > > Steps to Reproduce: > > 1. my test enviroment is one Ovirt engine(172.17.81.17) with 4 vdsm servers, exec "reboot" cmd in one of the vdsm servers(172.17.99.105), the server will reboot more than 30 minutes.ovirt-engine : 172.17.81.17/16 > > vdsm: 172.17.99.105/16 > > nfs server: 172.17.81.14/16Actual results: > > As above. the server will reboot more than 30 minutes > > Expected results: > > the server will reboot in a short time. > > What I have done: > > I have capture packet in nfs server while vdsm is rebooting, I found vdsm is always sending nfs packet to nfs server circularly as follows:this is some log files while I reboot vdsm 172.17.99.105 in 2020-10-26 22:12:34. Some conclusion is: > > 1. the vdsm.log said the vdsm 2020-10-26 22:12:34,461+0800 ERROR (check/loop) [storage.Monitor] Error checking path /rhev/data-center/mnt/172.18.81.14:_home_nfs_data/02c4c6ea-7ca9-40f1-a1d0-f1636bc1824e/dom_md/metadata > > 2. the sanlock.log said 2020-10-26 22:13:05 1454 [3301]: s1 delta_renew read timeout 10 sec offset 0 /rhev/data-center/mnt/172.18.81.14:_home_nfs_data/02c4c6ea-7ca9-40f1-a1d0-f1636bc1824e/dom_md/ids > > 3. there is nothing message import to this issue.The logs is in the attachment.I'm very appreciate if anyone can help me. Thank you. > > _______________________________________________ > > Users mailing list -- users(a)ovirt.org > > To unsubscribe send an email to users-leave(a)ovirt.org > > Privacy Statement: https://www.ovirt.org/privacy-policy.html > > oVirt Code of Conduct: https://www.ovirt.org/community/about/community-guidelines/ > > List Archives: https://lists.ovirt.org/archives/list/users@ovirt.org/message/C2GATAD35SUVW… > > _______________________________________________ > > Users mailing list -- users(a)ovirt.org > > To unsubscribe send an email to users-leave(a)ovirt.org > > Privacy Statement: https://www.ovirt.org/privacy-policy.html > > oVirt Code of Conduct: https://www.ovirt.org/community/about/community-guidelines/ > > List Archives: https://lists.ovirt.org/archives/list/users@ovirt.org/message/T3ETYUH2QDB7Z… >

1 0

[ANN] oVirt 4.4.3 Seventh Release Candidate is now available for testing
by Lev Veyde 29 Oct '20

29 Oct '20

oVirt 4.4.3 Seventh Release Candidate is now available for testing The oVirt Project is pleased to announce the availability of oVirt 4.4.3 Seventh Release Candidate for testing, as of October 29th, 2020. This update is the third in a series of stabilization updates to the 4.4 series. How to prevent hosts entering emergency mode after upgrade from oVirt 4.4.1 Note: Upgrading from 4.4.2 GA should not require re-doing these steps, if already performed while upgrading from 4.4.1 to 4.4.2 GA. These are only required to be done once. Due to Bug 1837864 <https://bugzilla.redhat.com/show_bug.cgi?id=1837864> - Host enter emergency mode after upgrading to latest build If you have your root file system on a multipath device on your hosts you should be aware that after upgrading from 4.4.1 to 4.4.3 you may get your host entering emergency mode. In order to prevent this be sure to upgrade oVirt Engine first, then on your hosts: 1. Remove the current lvm filter while still on 4.4.1, or in emergency mode (if rebooted). 2. Reboot. 3. Upgrade to 4.4.3 (redeploy in case of already being on 4.4.3). 4. Run vdsm-tool config-lvm-filter to confirm there is a new filter in place. 5. Only if not using oVirt Node: - run "dracut --force --add multipath” to rebuild initramfs with the correct filter configuration 6. Reboot. Documentation - If you want to try oVirt as quickly as possible, follow the instructions on the Download <https://ovirt.org/download/> page. - For complete installation, administration, and usage instructions, see the oVirt Documentation <https://ovirt.org/documentation/>. - For upgrading from a previous version, see the oVirt Upgrade Guide <https://ovirt.org/documentation/upgrade_guide/>. - For a general overview of oVirt, see About oVirt <https://ovirt.org/community/about.html>. Important notes before you try it Please note this is a pre-release build. The oVirt Project makes no guarantees as to its suitability or usefulness. This pre-release must not be used in production. You can hit: Problem: cannot install the best update candidate for package ovirt-engine-metrics-1.4.1.1-1.el8.noarch - nothing provides rhel-system-roles >= 1.0-19 needed by ovirt-engine-metrics-1.4.2-1.el8.noarch in order to get rhel-system-roles >= 1.0-19 you need https://buildlogs.centos.org/centos/8/virt/x86_64/ovirt-44/ repo since that package can be promoted to release only at 4.4.3 GA. Installation instructions For installation instructions and additional information please refer to: https://ovirt.org/documentation/ This release is available now on x86_64 architecture for: * Red Hat Enterprise Linux 8.2 or newer * CentOS Linux (or similar) 8.2 or newer This release supports Hypervisor Hosts on x86_64 and ppc64le architectures for: * Red Hat Enterprise Linux 8.2 or newer * CentOS Linux (or similar) 8.2 or newer * oVirt Node 4.4 based on CentOS Linux 8.2 (available for x86_64 only) See the release notes [1] for installation instructions and a list of new features and bugs fixed. Notes: - oVirt Appliance is already available for CentOS Linux 8 - oVirt Node NG is already available for CentOS Linux 8 Additional Resources: * Read more about the oVirt 4.4.3 release highlights: http://www.ovirt.org/release/4.4.3/ * Get more oVirt project updates on Twitter: https://twitter.com/ovirt * Check out the latest project news on the oVirt blog: http://www.ovirt.org/blog/ [1] http://www.ovirt.org/release/4.4.3/ [2] http://resources.ovirt.org/pub/ovirt-4.4-pre/iso/ -- Lev Veyde Senior Software Engineer, RHCE | RHCVA | MCITP Red Hat Israel <https://www.redhat.com> lev(a)redhat.com | lveyde(a)redhat.com <https://red.ht/sig> TRIED. TESTED. TRUSTED. <https://redhat.com/trusted>

1 0

Hosted Engine install via cockpit - proxy issue
by simon＠justconnect.ie 29 Oct '20

29 Oct '20

I am installing oVirt in a closed environment where internet access is controlled by proxies. This works until the hosted engine install via cockpit where it fails to complete as it appears to require internet access to the repository. The only workaround I have found is to ssh onto the engine ‘mid install’ and add the proxy address to /etc/dnf/dnf.conf. After doing this the install is successful. Am I missing something or does this type of install require unfettered internet access?

3 3

when configuring multi path logical network selection area is empty, hence not able to configure multipathing
by dhanaraj.ramesh＠yahoo.com 29 Oct '20

29 Oct '20

Hi team, I have 4 node cluster where in each node I configured 2 dedicated 10 gig NIC with dedicated subnet each ( NIC 1 = 10.10.10.0/24, NIC 2 = 10.10.20.0/24 ) and on the array side I configured 2 targets with 10.10.10.0 /24 & another 2 target with 10.10.20.0/24 subnet. without any errors I could check in all four paths and able to mount iscsi luns in all 4 nodes. However when i try to configure mutipathing at Data center level I could see all the paths but not the logical network, it stays empty although I configured logical network label for both NICs with dedicated names as ISCSI1 & ISCI2. these logical names visible and green at the host network level, no errors, they are just L2 IP config Am I missing something here? what else I should do to enable multiplathing

1 0

Improve glusterfs performance
by suporte＠logicworks.pt 28 Oct '20

28 Oct '20

Hello, Can anyone help me in how can I improve the performance of glusterfs to work with oVirt? Thanks -- Jose Ferradeira http://www.logicworks.pt

5 23

SPICE proxy behind nginx reverse proxy
by Colin Coe 28 Oct '20

28 Oct '20

Hi all As per $SUBJECT, I have a SPICE proxy behind a reverse proxy which all external VDI users are forced to use. We've only started doing this in the last week or so but I'm not getting heaps of reports SPICE sessions "freezing". The testing that I've done shows that a SPICE session that is unattended for 10-15 minutes hangs or freezes. By this I mean that you can't interact with the VM (using SPICE) via mouse or Keyboard. Restarting the SPICE session fixes the problem. Is anyone else doing this? If so, have you noticed the SPICE session freezes? Thanks in advance

4 4

Re: vdsm with NFS storage reboot or shutdown more than 15 minutes. with error failed to unmount /rhev/data-center/mnt/172.18.81.14:_home_nfs_data: Device or resource busy
by Strahil Nikolov 28 Oct '20

28 Oct '20

When you set a host to maintenance from oVirt API/UI, one of the tasks is to umount any shared storage (incluing the NFS you got). Then rebooting should work like a charm. Why did you reboot without putting the node in maintenance ? P.S.: Do not confuse rebooting with fencing - the latter kills the node ungracefully in order to safely start HA VMs on another node. Best Regards, Strahil Nikolov В вторник, 27 октомври 2020 г., 10:27:01 Гринуич+2, lifuqiong(a)sunyainfo.com <lifuqiong(a)sunyainfo.com> написа: Hi everyone: Description of problem: When exec "reboot" or "shutdown -h 0" cmd on vdsm server, the vdsm server will reboot or shutdown more than 30 minutes. the screen shows '[FAILED] Failed unmouting /rhev/data-center/mnt/172.18.81.41:_home_nfs_data'. other messages may be useful: [] watchdog: watchdog0: watchdog did not stop! []systemd-shutdown[5594]: Failed to unmount /rhev/data-center/mnt/172.18.81.14:_home_nfs_data: Device or resource busy []systemd-shutdown[1]: Failed to wait for process: Protocol error []systemd-shutdown[5595]: Failed to remount '/' read-only: Device or resource busy []systemd-shutdown[1]: Failed to wait for process: Protocol error dracut Warning: Killing all remaining processes dracut Warning: Killing all remaining processes Version-Release number of selected component (if applicable): Software Version:4.2.8.2-1.el7 OS: CentOS Linux release 7.5.1804 (Core) How reproducible: 100% Steps to Reproduce: 1. my test enviroment is one Ovirt engine(172.17.81.17) with 4 vdsm servers, exec "reboot" cmd in one of the vdsm servers(172.17.99.105), the server will reboot more than 30 minutes.ovirt-engine : 172.17.81.17/16 vdsm: 172.17.99.105/16 nfs server: 172.17.81.14/16Actual results: As above. the server will reboot more than 30 minutes Expected results: the server will reboot in a short time. What I have done: I have capture packet in nfs server while vdsm is rebooting, I found vdsm is always sending nfs packet to nfs server circularly as follows:this is some log files while I reboot vdsm 172.17.99.105 in 2020-10-26 22:12:34. Some conclusion is: 1. the vdsm.log said the vdsm 2020-10-26 22:12:34,461+0800 ERROR (check/loop) [storage.Monitor] Error checking path /rhev/data-center/mnt/172.18.81.14:_home_nfs_data/02c4c6ea-7ca9-40f1-a1d0-f1636bc1824e/dom_md/metadata 2. the sanlock.log said 2020-10-26 22:13:05 1454 [3301]: s1 delta_renew read timeout 10 sec offset 0 /rhev/data-center/mnt/172.18.81.14:_home_nfs_data/02c4c6ea-7ca9-40f1-a1d0-f1636bc1824e/dom_md/ids 3. there is nothing message import to this issue.The logs is in the attachment.I'm very appreciate if anyone can help me. Thank you. _______________________________________________ Users mailing list -- users(a)ovirt.org To unsubscribe send an email to users-leave(a)ovirt.org Privacy Statement: https://www.ovirt.org/privacy-policy.html oVirt Code of Conduct: https://www.ovirt.org/community/about/community-guidelines/ List Archives: https://lists.ovirt.org/archives/list/users@ovirt.org/message/C2GATAD35SUVW…

2 1

Migrated disk from NFS to iSCSI - Unable to Boot
by Wesley Stewart 28 Oct '20

28 Oct '20

This is a new one. I migrated from an NFS share to an iSCSI share on a small single node oVirt system (Currently running 4.3.10). After migrating a disk (Virtual Machine -> Disk -> Move), I was unable to boot to it. The console tells me "No bootable device". This is a Centos7 guest. I booted into a CentOS7 ISO and tried a few things... fdisk -l shows me a 40GB disk (/dev/sda). fsck -f tells me "bad magic number in superblock" lvdisplay and pvdisplay show nothing. Even if I can't boot to the drive I would love to recover a couple of documents from here if possible. Does anyone have any suggestions? I am running out of ideas.

1 1

oVirt 4.4 Upgrade issue with pki for libvirt-vnc
by lee.hanel＠gmail.com 27 Oct '20

27 Oct '20

Greetings, After reverting the ovirt_disk module to the ovirt_disk_28, I'm able to get passed that step, however now I'm running into a new issue. When it tries to start the vm after moving it from local storage to the hosted storage, I get the following errors: 2020-10-27 21:42:17,334+0000 ERROR (vm/9562a74e) [virt.vm] (vmId='9562a74e-2e6c-433b-ac0a-75a2acc7398d') The vm start process failed (vm:872) Traceback (most recent call last): File "/usr/lib/python3.6/site-packages/vdsm/virt/vm.py", line 802, in _startUnderlyingVm self._run() File "/usr/lib/python3.6/site-packages/vdsm/virt/vm.py", line 2615, in _run dom.createWithFlags(flags) File "/usr/lib/python3.6/site-packages/vdsm/common/libvirtconnection.py", line 131, in wrapper ret = f(*args, **kwargs) File "/usr/lib/python3.6/site-packages/vdsm/common/function.py", line 94, in wrapper return func(inst, *args, **kwargs) File "/usr/lib64/python3.6/site-packages/libvirt.py", line 1265, in createWithFlags if ret == -1: raise libvirtError ('virDomainCreateWithFlags() failed', dom=self) libvirt.libvirtError: internal error: process exited while connecting to monitor: 2020-10-27T21:42:16.133517Z qemu-kvm: -object tls-creds-x509,id=vnc-tls-creds0,dir=/etc/pki/vdsm/libvirt-vnc,endpoint=server,verify-peer=no: Cannot load certificate '/etc/pki/vdsm/libvirt-vnc/server-cert.pem' & key '/etc/pki/vdsm/libvirt-vnc/server-key.pem': Error while reading file. 2020-10-27 21:42:17,335+0000 INFO (vm/9562a74e) [virt.vm] (vmId='9562a74e-2e6c-433b-ac0a-75a2acc7398d') Changed state to Down: internal error: process exited while connecting to monitor: 2020-10-27T21:42:16.133517Z qemu-kvm: -object tls-creds-x509,id=vnc-tls-creds0,dir=/etc/pki/vdsm/libvirt-vnc,endpoint=server,verify-peer=no: Cannot load certificate '/etc/pki/vdsm/libvirt-vnc/server-cert.pem' & key '/etc/pki/vdsm/libvirt-vnc/server-key.pem': Error while reading file. (code=1) (vm:1636) the permissions on the files appears to be correct. https://bugzilla.redhat.com/show_bug.cgi?id=1634742 appears similar, but i took the added precaution of completely removing the vdsm packages and /etc/pki/vdsm and /etc/libvirt. Anyone have any additional troubleshooting steps?

1 0

Gluster Domain Storage full
by suporte＠logicworks.pt 27 Oct '20

27 Oct '20

Hello, I'm running oVirt Version 4.3.4.3-1.el7. I have a small GlusterFS Domain storage brick on a dedicated filesystem serving only one VM. The VM filled all the Domain storage. The Linux filesystem has 4.1G available and 100% used, the mounted brick has 0GB available and 100% used I can not do anything with this disk, for example, if I try to move it to another Gluster Domain Storage get the message: Error while executing action: Cannot move Virtual Disk. Low disk space on Storage Domain Any idea? Thanks -- Jose Ferradeira http://www.logicworks.pt

2 12

oVirt 4.4 Upgrade issue?
by lee.hanel＠gmail.com 27 Oct '20

27 Oct '20

Greetings, I'm trying to preform an upgrade from 4.3 to 4.4 using the hosted engine option of https://github.com/ovirt/ovirt-ansible-collection. Unfortunately, when it goes to create the hosted engine disk images I get the following: [Cannot move Virtual Disk. The operation is not supported for HOSTED_ENGINE_METADATA disks.] This appears to be related to https://bugzilla.redhat.com/show_bug.cgi?id=1883817, BUT I've manually applied that patch. Shouldn't it be creating a NEW disk image instead of trying to move the existing one? Any help would be appriciated. Thanks, Lee

2 2

vdsm with NFS storage reboot or shutdown more than 15 minutes. with error failed to unmount /rhev/data-center/mnt/172.18.81.14:_home_nfs_data: Device or resource busy
by lifuqiong＠sunyainfo.com 27 Oct '20

27 Oct '20

Hi everyone: I met problem as follows: Description of problem: When exec "reboot" or "shutdown -h 0" cmd on vdsm server, the vdsm server will reboot or shutdown more than 30 minutes. the screen shows '[FAILED] Failed unmouting /rhev/data-center/mnt/172.18.81.41:_home_nfs_data'. other messages may be useful: This message was shown in the screen. [] watchdog: watchdog0: watchdog did not stop! []systemd-shutdown[5594]: Failed to unmount /rhev/data-center/mnt/172.18.81.14:_home_nfs_data: Device or resource busy []systemd-shutdown[1]: Failed to wait for process: Protocol error []systemd-shutdown[5595]: Failed to remount '/' read-only: Device or resource busy []systemd-shutdown[1]: Failed to wait for process: Protocol error dracut Warning: Killing all remaining processes dracut Warning: Killing all remaining processes Version-Release number of selected component (if applicable): Software Version:4.2.8.2-1.el7 OS: CentOS Linux release 7.5.1804 (Core)How reproducible: 100% Steps to Reproduce: 1. my test enviroment is one Ovirt engine(172.17.81.17) with 4 vdsm servers, exec "reboot" cmd in one of the vdsm servers(172.17.99.105), the server will reboot more than 30 minutes.ovirt-engine : 172.17.81.17/16 vdsm: 172.17.99.105/16 nfs server: 172.17.81.14/16Actual results: As above. the server will reboot more than 30 minutes Expected results: the server will reboot in a short time. What I have done: I have capture packet in nfs server while vdsm is rebooting, I found vdsm server keeps sending nfs packet to nfs server circularly ;there are some log files while I reboot vdsm 172.17.99.105 in 2020-10-26 22:12:34. Some conclusion is: 1. the vdsm.log said the vdsm 2020-10-26 22:12:34,461+0800 ERROR (check/loop) [storage.Monitor] Error checking path /rhev/data-center/mnt/172.18.81.14:_home_nfs_data/02c4c6ea-7ca9-40f1-a1d0-f1636bc1824e/dom_md/metadata 2. the sanlock.log said 2020-10-26 22:13:05 1454 [3301]: s1 delta_renew read timeout 10 sec offset 0 /rhev/data-center/mnt/172.18.81.14:_home_nfs_data/02c4c6ea-7ca9-40f1-a1d0-f1636bc1824e/dom_md/ids 3. there is nothing message import to this issue.The logs is in the attachment.I'm very appreciate if anyone can help me. Thank you.Your Sincerely,Mark Lee

1 0

vdsm with NFS storage reboot or shutdown more than 15 minutes. with error failed to unmount /rhev/data-center/mnt/172.18.81.14:_home_nfs_data: Device or resource busy
by lifuqiong＠sunyainfo.com 27 Oct '20

27 Oct '20

Hi everyone: I met a problem as follows: Description of problem: When exec "reboot" or "shutdown -h 0" cmd on vdsm server, the vdsm server will reboot or shutdown more than 30 minutes. the screen shows '[FAILED] Failed unmouting /rhev/data-center/mnt/172.18.81.41:_home_nfs_data'. other messages may be useful: This message was shown in the screen. [] watchdog: watchdog0: watchdog did not stop! []systemd-shutdown[5594]: Failed to unmount /rhev/data-center/mnt/172.18.81.14:_home_nfs_data: Device or resource busy []systemd-shutdown[1]: Failed to wait for process: Protocol error []systemd-shutdown[5595]: Failed to remount '/' read-only: Device or resource busy []systemd-shutdown[1]: Failed to wait for process: Protocol error dracut Warning: Killing all remaining processes dracut Warning: Killing all remaining processes Version-Release number of selected component (if applicable): Software Version:4.2.8.2-1.el7 OS: CentOS Linux release 7.5.1804 (Core)How reproducible: 100% Steps to Reproduce: 1. my test enviroment is one Ovirt engine(172.17.81.17) with 4 vdsm servers, exec "reboot" cmd in one of the vdsm servers(172.17.99.105), the server will reboot more than 30 minutes.ovirt-engine : 172.17.81.17/16 vdsm: 172.17.99.105/16 nfs server: 172.17.81.14/16Actual results: As above. the server will reboot more than 30 minutes Expected results: the server will reboot in a short time. What I have done: I have capture packet in nfs server while vdsm is rebooting, I found vdsm is always sending nfs packet to nfs server circularly as follows:this is some log files while I reboot vdsm 172.17.99.105 in 2020-10-26 22:12:34. Some conclusion is: 1. the vdsm.log said the vdsm 2020-10-26 22:12:34,461+0800 ERROR (check/loop) [storage.Monitor] Error checking path /rhev/data-center/mnt/172.18.81.14:_home_nfs_data/02c4c6ea-7ca9-40f1-a1d0-f1636bc1824e/dom_md/metadata 2. the sanlock.log said 2020-10-26 22:13:05 1454 [3301]: s1 delta_renew read timeout 10 sec offset 0 /rhev/data-center/mnt/172.18.81.14:_home_nfs_data/02c4c6ea-7ca9-40f1-a1d0-f1636bc1824e/dom_md/ids 3. there is nothing message import to this issue.The logs is in the attachment.I'm very appreciate if anyone can help me. Thank you.

1 0

update host to 4.4: manual ovn config necessary?
by Gianluca Cecchi 27 Oct '20

27 Oct '20

Hello, I have updated an external engine from 4.3 to 4.4 and the OVN configuration seems to have been retained: [root@ovmgr1 ovirt-engine]# ovn-nbctl show switch fc2fc4e8-ff71-4ec3-ba03-536a870cd483 (ovirt-ovn192-1e252228-ade7-47c8-acda-5209be358fcf) switch 101d686d-7930-4176-b41a-b306d7c30a1a (ovirt-ovn17217-4bb1d1a7-020d-4843-9ac7-dc4204b528e5) port c1ec60a4-b4f3-4cb5-8985-43c086156e83 addresses: ["00:1a:4a:19:01:89 dynamic"] port 174b69f8-00ed-4e25-96fc-7db11ea8a8b9 addresses: ["00:1a:4a:19:01:59 dynamic"] port ccbd6188-78eb-437b-9df9-9929e272974b addresses: ["00:1a:4a:19:01:88 dynamic"] port 7e96ca70-c9e3-4efe-9ac5-e56c18476437 addresses: ["00:1a:4a:19:01:83 dynamic"] port d2c2d9f1-8fc3-4f17-9ada-76fe3a168e65 addresses: ["00:1a:4a:19:01:5e dynamic"] port 4d13d63e-5ff3-41c1-9b6b-feac343b514b addresses: ["00:1a:4a:19:01:60 dynamic"] port 66359e79-56c4-47e0-8196-2241706329f6 addresses: ["00:1a:4a:19:01:68 dynamic"] switch 87012fa6-ffaa-4fb0-bd91-b3eb7c0a2fc1 (ovirt-ovn193-d43a7928-0dc8-49d3-8755-5d766dff821a) port 2ae7391b-4297-4247-a315-99312f6392e6 addresses: ["00:1a:4a:19:01:51 dynamic"] switch 9e77163a-c4e4-4abf-a554-0388e6b5e4ce (ovirt-ovn172-4ac7ba24-aad5-432d-b1d2-672eaeea7d63) [root@ovmgr1 ovirt-engine]# Then I updated one of the 3 Linux hosts (not node ng) through remove from web admin gui, install from scratch of CentOS 8.2 OS, configure repos and then add new host (with the same name) in engine and I was able to connect to storage (iSCSI) and start VMs in general on the host. Coming to OVN part it seems it has not been configured on the upgraded host. Is it expected? Eg on engine I only see chassis for the 2 hosts still in 4.3: [root@ovmgr1 ovirt-engine]# ovn-sbctl show Chassis "b8872ab5-4606-4a79-b77d-9d956a18d349" hostname: "ov301.mydomain" Encap geneve ip: "10.4.192.34" options: {csum="true"} Port_Binding "174b69f8-00ed-4e25-96fc-7db11ea8a8b9" Port_Binding "66359e79-56c4-47e0-8196-2241706329f6" Chassis "ddecf0da-4708-4f93-958b-6af365a5eeca" hostname: "ov300.mydomain" Encap geneve ip: "10.4.192.33" options: {csum="true"} Port_Binding "ccbd6188-78eb-437b-9df9-9929e272974b" [root@ovmgr1 ovirt-engine]# What to do to add the upgraded 4.4 host? Can they live together for the OVN part? Thanks, Gianluca

1 0

when configuring multi path logical network selection area is empty, hence not able to configure multipathing
by dhanaraj.ramesh＠yahoo.com 27 Oct '20

27 Oct '20

Hi team, I have 4 node cluster where in each node I configured 2 dedicated 10 gig NIC with dedicated subnet each ( NIC 1 = 10.10.10.0/24, NIC 2 = 10.10.20.0/24 ) and on the array side I configured 2 targets with 10.10.10.0 /24 & another 2 target with 10.10.20.0/24 subnet. without any errors I could check in all four paths and able to mount iscsi luns in all 4 nodes. However when i try to configure mutipathing at Data center level I could see all the paths but not the logical network, it stays empty although I configured logical network label for both NICs with dedicated names as ISCSI1 & ISCI2. these logical names visible and green at the host network level, no errors, they are just L2 IP config Am I missing something here? what else I should do to enable multiplathing

1 0

vdsm with NFS storage reboot or shutdown more than 15 minutes. with error failed to unmount /rhev/data-center/mnt/172.18.81.14:_home_nfs_data: Device or resource busy
by lifuqiong＠sunyainfo.com 27 Oct '20

27 Oct '20

Hi everyone: Description of problem: When exec "reboot" or "shutdown -h 0" cmd on vdsm server, the vdsm server will reboot or shutdown more than 30 minutes. the screen shows '[FAILED] Failed unmouting /rhev/data-center/mnt/172.18.81.41:_home_nfs_data'. other messages may be useful: [] watchdog: watchdog0: watchdog did not stop! []systemd-shutdown[5594]: Failed to unmount /rhev/data-center/mnt/172.18.81.14:_home_nfs_data: Device or resource busy []systemd-shutdown[1]: Failed to wait for process: Protocol error []systemd-shutdown[5595]: Failed to remount '/' read-only: Device or resource busy []systemd-shutdown[1]: Failed to wait for process: Protocol error dracut Warning: Killing all remaining processes dracut Warning: Killing all remaining processes Version-Release number of selected component (if applicable): Software Version:4.2.8.2-1.el7 OS: CentOS Linux release 7.5.1804 (Core) How reproducible: 100% Steps to Reproduce: 1. my test enviroment is one Ovirt engine(172.17.81.17) with 4 vdsm servers, exec "reboot" cmd in one of the vdsm servers(172.17.99.105), the server will reboot more than 30 minutes.ovirt-engine : 172.17.81.17/16 vdsm: 172.17.99.105/16 nfs server: 172.17.81.14/16Actual results: As above. the server will reboot more than 30 minutes Expected results: the server will reboot in a short time. What I have done: I have capture packet in nfs server while vdsm is rebooting, I found vdsm is always sending nfs packet to nfs server circularly as follows:this is some log files while I reboot vdsm 172.17.99.105 in 2020-10-26 22:12:34. Some conclusion is: 1. the vdsm.log said the vdsm 2020-10-26 22:12:34,461+0800 ERROR (check/loop) [storage.Monitor] Error checking path /rhev/data-center/mnt/172.18.81.14:_home_nfs_data/02c4c6ea-7ca9-40f1-a1d0-f1636bc1824e/dom_md/metadata 2. the sanlock.log said 2020-10-26 22:13:05 1454 [3301]: s1 delta_renew read timeout 10 sec offset 0 /rhev/data-center/mnt/172.18.81.14:_home_nfs_data/02c4c6ea-7ca9-40f1-a1d0-f1636bc1824e/dom_md/ids 3. there is nothing message import to this issue.The logs is in the attachment.I'm very appreciate if anyone can help me. Thank you.

1 0

problems installing standard Linux as nodes in 4.4
by Gianluca Cecchi 27 Oct '20

27 Oct '20

Hello, due to missing megaraid_sas kernel module in RH EL 8 and CentOS 8, I'm deploying a new oVirt host using CentOS 8 and elrepo kernel driver and not ovirt node ng. Based on installation guide: - install CentOS 8.2 ("Server" chosen as base environment) - yum install https://resources.ovirt.org/pub/yum-repo/ovirt-release44.rpm - yum install cockpit-ovirt-dashboard - yum update - reboot Try to add host from engine web admin gui, I get: Host ov200 installation failed. Failed to execute Ansible host-deploy role: Failed to execute call to start playbook. . Please check logs for more details: /var/log/ovirt-engine/ansible-runner-service.log. Inside the log file above on engine: 2020-10-08 11:58:43,389 - runner_service.controllers.hosts - DEBUG - Request received, content-type :None 2020-10-08 11:58:43,390 - runner_service.controllers.hosts - INFO - 127.0.0.1 - GET /api/v1/hosts/ov200 2020-10-08 11:58:43,398 - runner_service.controllers.playbooks - DEBUG - Request received, content-type :application/json; charset=UTF-8 2020-10-08 11:58:43,398 - runner_service.controllers.playbooks - INFO - 127.0.0.1 - POST /api/v1/playbooks/ovirt-host-deploy.yml Do I have to enable any module or pre-install anything else before adding it? BTW: on host [root@ov200 ~]# rpm -q ansible ansible-2.9.13-2.el8.noarch [root@ov200 ~]# Thanks, Gianluca

4 18

Left over hibernation disks that we can't delete
by james＠deanimaconsulting.com 27 Oct '20

27 Oct '20

After putting the VMs in our environment into hibernation, one of the VMs that came out of hibernation still has the metadata and memory dumb hanging around as 2 disks with status ok. They are not attached to any VM but we are unable to delete them. The VM in question came out of the hibernation without any issues. If they are no longer required how can we tidy them up? Thanks, James

2 2

adminstration portal wont complete load, looping
by Philip Brown 27 Oct '20

27 Oct '20

I have an odd situation: When I go to https://ovengine/ovirt-engine/webadmin/?locale=en_US after authentication passes... it shows the top banner of oVirt OPEN VIRTUALIZATION MANAGER and the Loading ... in the center. but never gets past that. Any suggestions on how I could investigate and fix this? background: I recently updated certs to be signed wildcard certs, but this broke consoles somehow. So I restored the original certs, and restarted things... but got stuck with this. Interestingly, the VM portal loads fine. But not the admin portal. -- Philip Brown| Sr. Linux System Administrator | Medata, Inc. 5 Peters Canyon Rd Suite 250 Irvine CA 92606 Office 714.918.1310| Fax 714.918.1325 pbrown(a)medata.com| www.medata.com

3 8

reinstall 4.3 host from 4.4 engine
by Gianluca Cecchi 26 Oct '20

26 Oct '20

Hello, supposing I'm with an already upgraded 4.4.2 engine and in my environment I still have some 4.3.10 hosts based on CentOS Linux 7.x (or 4.3.10 ng nodes), is it supported to reinstall such host in case of any modification in configuration? Or will the engine try to install them in a "4.4" way? Thanks in advance, Gianluca

1 0

Manual VM Migration fails
by Anton Louw 26 Oct '20

26 Oct '20

Hello Everybody, I am having a strange issue. When I try and manually migrate a VM from one host to another, I get an error stating: "Migration failed (VM: VM1, Source: node6.example.com, Destination: node3.example.com)" I have tried with a few different machines, and it pops up with the same error. I have attached the VDSM logs of both source and destination nodes. The time frame is 16:01 I see the below in that time frame, but not quite sure what I need to change: 2020-10-23 16:01:14,419+0200 ERROR (migsrc/42186f82) [virt.vm] (vmId='42186f82-b84c-7e65-e736-e6331acd04ed') Failed to migrate (migration:450) Traceback (most recent call last): File "/usr/lib/python2.7/site-packages/vdsm/virt/migration.py", line 431, in _regular_run time.time(), migrationParams, machineParams File "/usr/lib/python2.7/site-packages/vdsm/virt/migration.py", line 505, in _startUnderlyingMigration self._perform_with_conv_schedule(duri, muri) File "/usr/lib/python2.7/site-packages/vdsm/virt/migration.py", line 591, in _perform_with_conv_schedule self._perform_migration(duri, muri) File "/usr/lib/python2.7/site-packages/vdsm/virt/migration.py", line 525, in _perform_migration self._migration_flags) File "/usr/lib/python2.7/site-packages/vdsm/virt/virdomain.py", line 100, in f ret = attr(*args, **kwargs) File "/usr/lib/python2.7/site-packages/vdsm/common/libvirtconnection.py", line 131, in wrapper ret = f(*args, **kwargs) File "/usr/lib/python2.7/site-packages/vdsm/common/function.py", line 94, in wrapper return func(inst, *args, **kwargs) File "/usr/lib64/python2.7/site-packages/libvirt.py", line 1781, in migrateToURI3 if ret == -1: raise libvirtError ('virDomainMigrateToURI3() failed', dom=self) libvirtError: operation failed: guest CPU doesn't match specification: missing features: spec-ctrl,ssbd Anton Louw Cloud Engineer: Storage and Virtualization ______________________________________ D: 087 805 1572 | M: N/A A: Rutherford Estate, 1 Scott Street, Waverley, Johannesburg anton.louw(a)voxtelecom.co.za www.vox.co.za

4 7

oVirt test clustered DOA after upgrade to latest 4.4.2 (AFAIK). Network issue?
by Gilboa Davara 26 Oct '20

26 Oct '20

Hello all, I just upgraded one of my test oVirt setups to latest. Post reboot, the machine lost network and hosted engine didn't start. When I connected to the machine, I noticed all the /etc/sysconfig/network-scripts/ifcfg-* files disappeared. No idea why / how. (Possibly I did something wrong and forgot about it.) Long story story short, I copied the two missing files (ifcfg-onb0 ethernet device configuration and the ifcfg-ovirtmgmt bridge configuration) from another ovirt host, changed the ovirtmgmt IP address and UUID to match the UUID reported in logs as missing and restarted NetworkManager and restarted all the oVirt related services (vdsmd, supervdsmd, ovirt-*, etc). Sadly enough, even with both onb0 and ovirtmgmt up, vdsm still complains about the missing network (ovirtmgmt) and refuses to start the hosted engine. Reboot doesn't seem to change anything. In the main log I see the following errors: Oct 22 16:33:31 office-wx-otest vdsm[2634]: WARN Attempting to remove a non existing network: ovirtmgmt/1da8c5b7-999c-4ada-8287-1f35de6ce21d Oct 22 16:33:31 office-wx-otest vdsm[2634]: WARN Attempting to remove a non existing net user: ovirtmgmt/1da8c5b7-999c-4ada-8287-1f35de6ce21d Oct 22 16:33:31 office-wx-otest vdsm[2634]: WARN Attempting to remove a non existing network: ovirtmgmt/1da8c5b7-999c-4ada-8287-1f35de6ce21d Oct 22 16:33:31 office-wx-otest vdsm[2634]: WARN Attempting to remove a non existing net user: ovirtmgmt/1da8c5b7-999c-4ada-8287-1f35de6ce21d As it is one of my oVirt test setup, I can simply redeploy the host and continue from there, but I rather use this experience to learn how to fix oVirt such issues in the future. Logs attached. https://drive.google.com/file/d/12ugy6CuaFaMvXYt6uGT4D_EHIW6nXttb/view?usp=… $ PAGER= nmcli connection show NAME UUID TYPE DEVICE ovirtmgmt 1da8c5b7-999c-4ada-8287-1f35de6ce21d bridge ovirtmgmt onb0 48332db3-8939-bff3-6b71-772a28c9e7b8 ethernet onb0 $ PAGER= nmcli device show GENERAL.DEVICE: ovirtmgmt GENERAL.TYPE: bridge GENERAL.HWADDR: FC:AA:14:6B:A8:E0 GENERAL.MTU: 1500 GENERAL.STATE: 100 (connected) GENERAL.CONNECTION: ovirtmgmt GENERAL.CON-PATH: /org/freedesktop/NetworkManager/ActiveConnection/2 IP4.ADDRESS[1]: 192.168.2.117/24 IP4.GATEWAY: 192.168.2.100 IP4.ROUTE[1]: dst = 192.168.2.0/24, nh = 0.0.0.0, mt = 425 IP4.ROUTE[2]: dst = 0.0.0.0/0, nh = 192.168.2.100, mt = 425 IP4.DNS[1]: 192.168.2.100 IP4.DNS[2]: 8.8.8.8 IP6.GATEWAY: -- GENERAL.DEVICE: onb0 GENERAL.TYPE: ethernet GENERAL.HWADDR: FC:AA:14:6B:A8:E0 GENERAL.MTU: 1500 GENERAL.STATE: 100 (connected) GENERAL.CONNECTION: onb0 GENERAL.CON-PATH: /org/freedesktop/NetworkManager/ActiveConnection/3 WIRED-PROPERTIES.CARRIER: on IP4.GATEWAY: -- GENERAL.DEVICE: ;vdsmdummy; GENERAL.TYPE: bridge GENERAL.HWADDR: 92:8B:9A:5E:C1:3E GENERAL.MTU: 1500 GENERAL.STATE: 10 (unmanaged) GENERAL.CONNECTION: -- GENERAL.CON-PATH: -- IP4.GATEWAY: -- IP6.GATEWAY: -- GENERAL.DEVICE: lo GENERAL.TYPE: loopback GENERAL.HWADDR: 00:00:00:00:00:00 GENERAL.MTU: 65536 GENERAL.STATE: 10 (unmanaged) GENERAL.CONNECTION: -- GENERAL.CON-PATH: -- IP4.ADDRESS[1]: 127.0.0.1/8 IP4.GATEWAY: -- IP6.GATEWAY: -- GENERAL.DEVICE: br-int GENERAL.TYPE: openvswitch GENERAL.HWADDR: 8E:15:6A:F8:3C:45 GENERAL.MTU: 1500 GENERAL.STATE: 10 (unmanaged) GENERAL.CONNECTION: -- GENERAL.CON-PATH: -- IP4.GATEWAY: -- IP6.GATEWAY: -- GENERAL.DEVICE: ovs-system GENERAL.TYPE: openvswitch GENERAL.HWADDR: E2:09:EA:A2:BD:70 GENERAL.MTU: 1500 GENERAL.STATE: 10 (unmanaged) GENERAL.CONNECTION: -- GENERAL.CON-PATH: -- IP4.GATEWAY: -- IP6.GATEWAY: - Gilboa office-wx-otest-vdsm.bz2 <https://drive.google.com/file/d/12ugy6CuaFaMvXYt6uGT4D_EHIW6nXttb/view?usp=…>

2 4

sr-iov network config assistance IOMMU group error
by Edward Berger 26 Oct '20

26 Oct '20

I'm' trying to bring up a test ovirt 4.4 cluster with sr-iov virtual infiniband interfaces on Mellanox ConnectX-6 cards in some older Opteron servers while we await delivery of new Epyc servers. I have the hypervisors configured for sr-iov and showing the interfaces, but when I try to bring up a VM with the host device selected it fails to boot. "Failed Launching ..." shows up on the engine UI. [image: sriov-hostdev.JPG] The /var/log/libvirt/qemu log for the VM seems to think something is wrong with the IOMMU group. If I scroll over to the right it says N/A under the IOMMU column. Is there some step I need to 'create IOMMU groups' or any other suggestions? [root@n001 qemu]# more C*log 2020-10-21 20:26:39.266+0000: starting up libvirt version: 6.0.0, package: 25.2. el8 (CBS <cbs(a)centos.org>, 2020-09-10-13:20:10, ), qemu version: 4.2.0qemu-kvm-4 .2.0-29.el8.3, kernel: 4.18.0-193.19.1.el8_2.x86_64, hostname: n001.bil.psc.edu LC_ALL=C \ PATH=/usr/local/sbin:/usr/local/bin:/usr/sbin:/usr/bin \ HOME=/var/lib/libvirt/qemu/domain-1-C8.2test \ XDG_DATA_HOME=/var/lib/libvirt/qemu/domain-1-C8.2test/.local/share \ XDG_CACHE_HOME=/var/lib/libvirt/qemu/domain-1-C8.2test/.cache \ XDG_CONFIG_HOME=/var/lib/libvirt/qemu/domain-1-C8.2test/.config \ QEMU_AUDIO_DRV=spice \ /usr/libexec/qemu-kvm \ -name guest=C8.2test,debug-threads=on \ -S \ -object secret,id=masterKey0,format=raw,file=/var/lib/libvirt/qemu/domain-1-C8.2 test/master-key.aes \ -machine pc-q35-rhel8.1.0,accel=kvm,usb=off,dump-guest-core=off \ -cpu Opteron_G5 \ -m size=16777216k,slots=16,maxmem=67108864k \ -overcommit mem-lock=off \ -smp 4,maxcpus=16,sockets=16,dies=1,cores=1,threads=1 \ -numa node,nodeid=0,cpus=0-15,mem=16384 \ -uuid 07eb98af-0d9f-4f9c-967a-8af302c68703 \ -smbios type=1,manufacturer=oVirt,product=RHEL,version=8.2-2.2004.0.2.el8,serial =47513848-0036-2500-905b-0025905bff22,uuid=07eb98af-0d9f-4f9c-967a-8af302c68703, family=oVirt \ -no-user-config \ -nodefaults \ -chardev socket,id=charmonitor,fd=39,server,nowait \ -mon chardev=charmonitor,id=monitor,mode=control \ -rtc base=2020-10-21T20:26:38,driftfix=slew \ -global kvm-pit.lost_tick_policy=delay \ -no-hpet \ -no-shutdown \ -global ICH9-LPC.disable_s3=1 \ -global ICH9-LPC.disable_s4=1 \ -boot strict=on \ -device pcie-root-port,port=0x10,chassis=1,id=pci.1,bus=pcie.0,multifunction=on, addr=0x2 \ -device pcie-root-port,port=0x11,chassis=2,id=pci.2,bus=pcie.0,addr=0x2.0x1 \ -device pcie-root-port,port=0x12,chassis=3,id=pci.3,bus=pcie.0,addr=0x2.0x2 \ -device pcie-root-port,port=0x13,chassis=4,id=pci.4,bus=pcie.0,addr=0x2.0x3 \ -device pcie-root-port,port=0x14,chassis=5,id=pci.5,bus=pcie.0,addr=0x2.0x4 \ -device pcie-root-port,port=0x15,chassis=6,id=pci.6,bus=pcie.0,addr=0x2.0x5 \ -device pcie-root-port,port=0x16,chassis=7,id=pci.7,bus=pcie.0,addr=0x2.0x6 \ -device pcie-root-port,port=0x17,chassis=8,id=pci.8,bus=pcie.0,addr=0x2.0x7 \ -device pcie-root-port,port=0x18,chassis=9,id=pci.9,bus=pcie.0,multifunction=on, addr=0x3 \ -device pcie-root-port,port=0x19,chassis=10,id=pci.10,bus=pcie.0,addr=0x3.0x1 \ -device pcie-root-port,port=0x1a,chassis=11,id=pci.11,bus=pcie.0,addr=0x3.0x2 \ -device pcie-root-port,port=0x1b,chassis=12,id=pci.12,bus=pcie.0,addr=0x3.0x3 \ -device pcie-root-port,port=0x1c,chassis=13,id=pci.13,bus=pcie.0,addr=0x3.0x4 \ -device pcie-root-port,port=0x1d,chassis=14,id=pci.14,bus=pcie.0,addr=0x3.0x5 \ -device pcie-root-port,port=0x1e,chassis=15,id=pci.15,bus=pcie.0,addr=0x3.0x6 \ -device pcie-root-port,port=0x1f,chassis=16,id=pci.16,bus=pcie.0,addr=0x3.0x7 \ -device qemu-xhci,p2=8,p3=8,id=ua-cd3a4abd-57e9-4f2e-a61f-01128ead8702,bus=pci.2 ,addr=0x0 \ -device virtio-scsi-pci,id=ua-64b19c77-98d3-4946-a0f8-c43c1347a5dc,bus=pci.1,add r=0x0 \ -device virtio-serial-pci,id=virtio-serial0,max_ports=16,bus=pci.4,addr=0x0 \ -device ide-cd,bus=ide.2,id=ua-271c6d06-1f3c-4e45-99cc-a39f39a6dd1b,werror=repor t,rerror=report \ -blockdev '{"driver":"file","filename":"/rhev/data-center/mnt/10.200.16.152: _var _nfs__shares_data/662d0028-ceaf-4641-87fa-839254e15209/images/4c332ece-f6db-4b22 -879c-b6912c5b27c0/2918a5f1-c801-4a34-8cf7-89045265f6f3","aio":"threads","node-n ame":"libvirt-1-storage","cache":{"direct":true,"no-flush":false},"auto-read-onl y":true,"discard":"unmap"}' \ -blockdev '{"node-name":"libvirt-1-format","read-only":false,"cache":{"direct":t rue,"no-flush":false},"driver":"raw","file":"libvirt-1-storage"}' \ -device scsi-hd,bus=ua-64b19c77-98d3-4946-a0f8-c43c1347a5dc.0,channel=0,scsi-id= 0,lun=0,device_id=4c332ece-f6db-4b22-879c-b6912c5b27c0,drive=libvirt-1-format,id =ua-4c332ece-f6db-4b22-879c-b6912c5b27c0,bootindex=1,write-cache=on,serial=4c332 ece-f6db-4b22-879c-b6912c5b27c0,werror=stop,rerror=stop \ -chardev socket,id=charchannel0,fd=41,server,nowait \ -device virtserialport,bus=virtio-serial0.0,nr=1,chardev=charchannel0,id=channel 0,name=ovirt-guest-agent.0 \ -chardev socket,id=charchannel1,fd=42,server,nowait \ -device virtserialport,bus=virtio-serial0.0,nr=2,chardev=charchannel1,id=channel 1,name=org.qemu.guest_agent.0 \ -chardev spicevmc,id=charchannel2,name=vdagent \ -device virtserialport,bus=virtio-serial0.0,nr=3,chardev=charchannel2,id=channel 2,name=com.redhat.spice.0 \ -device usb-tablet,id=input0,bus=ua-cd3a4abd-57e9-4f2e-a61f-01128ead8702.0,port= 1 \ -vnc 128.182.82.72:0,password \ -k en-us \ -spice port=5901,tls-port=5902,addr=128.182.82.72,x509-dir=/etc/pki/vdsm/libvirt -spice,tls-channel=main,tls-channel=display,tls-channel=inputs,tls-channel=curso r,tls-channel=playback,tls-channel=record,tls-channel=smartcard,tls-channel=usbr edir,seamless-migration=on \ -device qxl-vga,id=ua-bd6ad9d1-2d02-40fa-b087-4fc4786a2f44,ram_size=67108864,vra m_size=33554432,vram64_size_mb=0,vgamem_mb=16,max_outputs=1,bus=pcie.0,addr=0x1 \ -device vfio-pci,host=0000:04:00.1,id=ua-7fce29ba-fcf7-4bd3-a680-21b200d44e42,bu s=pci.3,addr=0x0 \ -object rng-random,id=objua-e1fb82d8-2455-4763-a5d9-22d7a9903806,filename=/dev/u random \ -device virtio-rng-pci,rng=objua-e1fb82d8-2455-4763-a5d9-22d7a9903806,id=ua-e1fb 82d8-2455-4763-a5d9-22d7a9903806,bus=pci.5,addr=0x0 \ -sandbox on,obsolete=deny,elevateprivileges=deny,spawn=deny,resourcecontrol=deny \ -msg timestamp=on 2020-10-21 20:26:39.267+0000: Domain id=1 is tainted: custom-hypervisor-feature libvirt: error : internal error: Invalid device 0000:04:00.1 iommu_group file / sys/bus/pci/devices/0000:04:00.1/iommu_group is not a symlink 2020-10-21 20:26:39.298+0000: shutting down, reason=failed 2020-10-21 20:27:41.683+0000: starting up libvirt version: 6.0.0, package: 25.2. el8 (CBS <cbs(a)centos.org>, 2020-09-10-13:20:10, ), qemu version: 4.2.0qemu-kvm-4 .2.0-29.el8.3, kernel: 4.18.0-193.19.1.el8_2.x86_64, hostname: n001.bil.psc.edu LC_ALL=C \ PATH=/usr/local/sbin:/usr/local/bin:/usr/sbin:/usr/bin \ HOME=/var/lib/libvirt/qemu/domain-2-C8.2test \ XDG_DATA_HOME=/var/lib/libvirt/qemu/domain-2-C8.2test/.local/share \ XDG_CACHE_HOME=/var/lib/libvirt/qemu/domain-2-C8.2test/.cache \ XDG_CONFIG_HOME=/var/lib/libvirt/qemu/domain-2-C8.2test/.config \ QEMU_AUDIO_DRV=spice \ /usr/libexec/qemu-kvm \ -name guest=C8.2test,debug-threads=on \ -S \ -object secret,id=masterKey0,format=raw,file=/var/lib/libvirt/qemu/domain-2-C8.2 test/master-key.aes \ -machine pc-q35-rhel8.1.0,accel=kvm,usb=off,dump-guest-core=off \ -cpu Opteron_G5 \ -m size=16777216k,slots=16,maxmem=67108864k \ -overcommit mem-lock=off \ -smp 4,maxcpus=16,sockets=16,dies=1,cores=1,threads=1 \ -numa node,nodeid=0,cpus=0-15,mem=16384 \ -uuid 07eb98af-0d9f-4f9c-967a-8af302c68703 \ -smbios type=1,manufacturer=oVirt,product=RHEL,version=8.2-2.2004.0.2.el8,serial =47513848-0036-2500-905b-0025905bff22,uuid=07eb98af-0d9f-4f9c-967a-8af302c68703, family=oVirt \ -no-user-config \ -nodefaults \ -chardev socket,id=charmonitor,fd=38,server,nowait \ -mon chardev=charmonitor,id=monitor,mode=control \ -rtc base=2020-10-21T20:27:40,driftfix=slew \ -global kvm-pit.lost_tick_policy=delay \ -no-hpet \ -no-shutdown \ -global ICH9-LPC.disable_s3=1 \ -global ICH9-LPC.disable_s4=1 \ -boot strict=on \ -device pcie-root-port,port=0x10,chassis=1,id=pci.1,bus=pcie.0,multifunction=on, addr=0x2 \ -device pcie-root-port,port=0x11,chassis=2,id=pci.2,bus=pcie.0,addr=0x2.0x1 \ -device pcie-root-port,port=0x12,chassis=3,id=pci.3,bus=pcie.0,addr=0x2.0x2 \ -device pcie-root-port,port=0x13,chassis=4,id=pci.4,bus=pcie.0,addr=0x2.0x3 \ -device pcie-root-port,port=0x14,chassis=5,id=pci.5,bus=pcie.0,addr=0x2.0x4 \ -device pcie-root-port,port=0x15,chassis=6,id=pci.6,bus=pcie.0,addr=0x2.0x5 \ -device pcie-root-port,port=0x16,chassis=7,id=pci.7,bus=pcie.0,addr=0x2.0x6 \ -device pcie-root-port,port=0x17,chassis=8,id=pci.8,bus=pcie.0,addr=0x2.0x7 \ -device pcie-root-port,port=0x18,chassis=9,id=pci.9,bus=pcie.0,multifunction=on, addr=0x3 \ -device pcie-root-port,port=0x19,chassis=10,id=pci.10,bus=pcie.0,addr=0x3.0x1 \ -device pcie-root-port,port=0x1a,chassis=11,id=pci.11,bus=pcie.0,addr=0x3.0x2 \ -device pcie-root-port,port=0x1b,chassis=12,id=pci.12,bus=pcie.0,addr=0x3.0x3 \ -device pcie-root-port,port=0x1c,chassis=13,id=pci.13,bus=pcie.0,addr=0x3.0x4 \ -device pcie-root-port,port=0x1d,chassis=14,id=pci.14,bus=pcie.0,addr=0x3.0x5 \ -device pcie-root-port,port=0x1e,chassis=15,id=pci.15,bus=pcie.0,addr=0x3.0x6 \ -device pcie-root-port,port=0x1f,chassis=16,id=pci.16,bus=pcie.0,addr=0x3.0x7 \ -device qemu-xhci,p2=8,p3=8,id=ua-cd3a4abd-57e9-4f2e-a61f-01128ead8702,bus=pci.2 ,addr=0x0 \ -device virtio-scsi-pci,id=ua-64b19c77-98d3-4946-a0f8-c43c1347a5dc,bus=pci.1,add r=0x0 \ -device virtio-serial-pci,id=virtio-serial0,max_ports=16,bus=pci.4,addr=0x0 \ -device ide-cd,bus=ide.2,id=ua-271c6d06-1f3c-4e45-99cc-a39f39a6dd1b,werror=repor t,rerror=report \ -blockdev '{"driver":"file","filename":"/rhev/data-center/mnt/10.200.16.152: _var _nfs__shares_data/662d0028-ceaf-4641-87fa-839254e15209/images/4c332ece-f6db-4b22 -879c-b6912c5b27c0/2918a5f1-c801-4a34-8cf7-89045265f6f3","aio":"threads","node-n ame":"libvirt-1-storage","cache":{"direct":true,"no-flush":false},"auto-read-onl y":true,"discard":"unmap"}' \ -blockdev '{"node-name":"libvirt-1-format","read-only":false,"cache":{"direct":t rue,"no-flush":false},"driver":"raw","file":"libvirt-1-storage"}' \ -device scsi-hd,bus=ua-64b19c77-98d3-4946-a0f8-c43c1347a5dc.0,channel=0,scsi-id= 0,lun=0,device_id=4c332ece-f6db-4b22-879c-b6912c5b27c0,drive=libvirt-1-format,id =ua-4c332ece-f6db-4b22-879c-b6912c5b27c0,bootindex=1,write-cache=on,serial=4c332 ece-f6db-4b22-879c-b6912c5b27c0,werror=stop,rerror=stop \ -chardev socket,id=charchannel0,fd=40,server,nowait \ -device virtserialport,bus=virtio-serial0.0,nr=1,chardev=charchannel0,id=channel 0,name=ovirt-guest-agent.0 \ -chardev socket,id=charchannel1,fd=41,server,nowait \ -device virtserialport,bus=virtio-serial0.0,nr=2,chardev=charchannel1,id=channel 1,name=org.qemu.guest_agent.0 \ -chardev spicevmc,id=charchannel2,name=vdagent \ -device virtserialport,bus=virtio-serial0.0,nr=3,chardev=charchannel2,id=channel 2,name=com.redhat.spice.0 \ -device usb-tablet,id=input0,bus=ua-cd3a4abd-57e9-4f2e-a61f-01128ead8702.0,port= 1 \ -vnc 128.182.82.72:0,password \ -k en-us \ -spice port=5901,tls-port=5902,addr=128.182.82.72,x509-dir=/etc/pki/vdsm/libvirt -spice,tls-channel=main,tls-channel=display,tls-channel=inputs,tls-channel=curso r,tls-channel=playback,tls-channel=record,tls-channel=smartcard,tls-channel=usbr edir,seamless-migration=on \ -device qxl-vga,id=ua-bd6ad9d1-2d02-40fa-b087-4fc4786a2f44,ram_size=67108864,vra m_size=33554432,vram64_size_mb=0,vgamem_mb=16,max_outputs=1,bus=pcie.0,addr=0x1 \ -device vfio-pci,host=0000:04:00.1,id=ua-7fce29ba-fcf7-4bd3-a680-21b200d44e42,bu s=pci.3,addr=0x0 \ -object rng-random,id=objua-e1fb82d8-2455-4763-a5d9-22d7a9903806,filename=/dev/u random \ -device virtio-rng-pci,rng=objua-e1fb82d8-2455-4763-a5d9-22d7a9903806,id=ua-e1fb 82d8-2455-4763-a5d9-22d7a9903806,bus=pci.5,addr=0x0 \ -sandbox on,obsolete=deny,elevateprivileges=deny,spawn=deny,resourcecontrol=deny \ -msg timestamp=on 2020-10-21 20:27:41.684+0000: Domain id=2 is tainted: custom-hypervisor-feature libvirt: error : internal error: Invalid device 0000:04:00.1 iommu_group file / sys/bus/pci/devices/0000:04:00.1/iommu_group is not a symlink 2020-10-21 20:27:41.715+0000: shutting down, reason=failed

2 2