December 2020 - Users - oVirt List Archives

4.4.4 hosted engine deployment failure
by Diggy Mc 27 Dec '20

27 Dec '20

While trying to deploy a 4.4.4 hosted engine via the cockpit interface, I received the following error(s): [ INFO ] TASK [ovirt.ovirt.hosted_engine_setup : Obtain SSO token using username/password credentials] [ INFO ] ok: [localhost] [ INFO ] TASK [ovirt.ovirt.hosted_engine_setup : Wait for the host to be up] [ INFO ] TASK [ovirt.ovirt.hosted_engine_setup : Notify the user about a failure] [ ERROR ] fatal: [localhost]: FAILED! => {"changed": false, "msg": "Host is not up, please check logs, perhaps also on the engine machine"} ... [ INFO ] TASK [ovirt.ovirt.hosted_engine_setup : Notify the user about a failure] [ ERROR ] fatal: [localhost]: FAILED! => {"changed": false, "msg": "The system may not be provisioned according to the playbook results: please check the logs for the issue, fix accordingly or re-deploy from scratch.\n"} I am trying to build a new oVirt 4.4.4 environment using Oracle Linux 8.3 host servers. I created /var on a separate LV from / (root). I also configured adapter bond per the docs and the same way I did with my 4.3.7 environment. Did I miss a configuration requirement for the OL 8.3 installation? Where can I find more details about the error(s)?

2 1

engine storage fail after upgrade
by ozmen62＠hotmail.com 25 Dec '20

25 Dec '20

Hi, After upgrade from 4.3 to 4.4 pop-ups some error on engine. It becomes unavailable several times for a 2-3 minutes and cames back in a day after some research on the system, found some logs On THE hosted_storage there are 2 events 1- Failed to update VMs/Templates OVF data for Storage Domain hosted_storage in Data Center XXX 2- Failed to update OVF disks 9cbb34d0-06b0-4ce7-a3fa-7dfed689c442, OVF data isn't updated on those OVF stores (Data Center XXX, Storage Domain hosted_storage). Is there any idea how can i fix this?

2 1

what is OVF_STORE ?
by tommy 24 Dec '20

24 Dec '20

What's meaning of the word OVF ? What's OVF_STORE ? Thanks.

3 2

FW: oVirt 4.4 and Active directory
by Latchezar Filtchev 24 Dec '20

24 Dec '20

Hello , I think I resolved this issue. It is dig response when resolving the domain name! CentOS-7 - bind-utils-9.11.4-16.P2.el7_8.6.x86_64; Windows AD level 2008R2; in my case dig returns answer with ;; ANSWER SECTION: mb118.local. 600 IN A 192.168.1.7 IP address returned is address of DC CentOS-8 - bind-utils-9.11.20-5.el8.x86_64; Same Domain Controller; dig returns answer without ;;ANSWER SECTION e.g. IP address of DC cannot be identified. The solution is to add directive '+nocookie', after '+tcp' in the file /usr/share/ovirt-engine-extension-aaa-ldap/setup/plugins/ovirt-engine-extension-aaa-ldap/ldap/common.py The section starts at line 144: @staticmethod def _resolver(plugin, record, what): rc, stdout, stderr = plugin.execute( args=( ( plugin.command.get('dig'), '+noall', '+answer', '+tcp', '+nocookie', what, record ) ), ) return stdout With this change execution of ovirt-engine-extension-aaa-ldap-setup completes successfully and joins fresh install of oVirt 4.4 to Active Directory. If level of AD is 2016 '+nocookie' change is not needed. Happy holydays to all of you! Stay safe! Thank you! Best, Latcho From: Latchezar Filtchev Sent: Tuesday, November 24, 2020 10:31 AM To: users(a)ovirt.org Subject: oVirt 4.4 and Active directory Hello All, Fresh standalone installation of oVirt 4.3 (CentOS 7) . Execution of ovirt-engine-extension-aaa-ldap-setup completes normally and DC is connected to AD (Domain functional level: Windows Server 2008 ). On the same hardware fresh standalone installation of oVirt 4.4. Installation of engine completed with warning: 2020-11-23 14:50:46,159+0200 WARNING otopi.plugins.ovirt_engine_common.base.network.hostname hostname._validateFQDNresolvability:308 Failed to resolve 44-8.mb118.local using DNS, it can be resolved only locally Despite warning engine portal is resolvable after installation. Execution of ovirt-engine-extension-aaa-ldap-setup ends with: [ INFO ] Stage: Environment customization Welcome to LDAP extension configuration program Available LDAP implementations: 1 - 389ds 2 - 389ds RFC-2307 Schema 3 - Active Directory 4 - IBM Security Directory Server 5 - IBM Security Directory Server RFC-2307 Schema 6 - IPA 7 - Novell eDirectory RFC-2307 Schema 8 - OpenLDAP RFC-2307 Schema 9 - OpenLDAP Standard Schema 10 - Oracle Unified Directory RFC-2307 Schema 11 - RFC-2307 Schema (Generic) 12 - RHDS 13 - RHDS RFC-2307 Schema 14 - iPlanet Please select: 3 Please enter Active Directory Forest name: mb118.local [ INFO ] Resolving Global Catalog SRV record for mb118.local [WARNING] Cannot resolve Global Catalog SRV record for mb118.local. Please check you have entered correct Active Directory forest name and check that forest is resolvable by your system DNS servers [ ERROR ] Failed to execute stage 'Environment customization': Active Directory forest is not resolvable, please make sure you've entered correct forest name. If for some reason you can't use forest and you need some special configuration instead, please refer to examples directory provided by ovirt-engine-extension-aaa-ldap package. [ INFO ] Stage: Clean up Log file is available at /tmp/ovirt-engine-extension-aaa-ldap-setup-20201123113909-bj749k.log: [ INFO ] Stage: Pre-termination [ INFO ] Stage: Termination Can someone advise on this? Thank you! Best, Latcho

2 1

leave
by Cam Alliance 24 Dec '20

24 Dec '20

1 0

Ovirt VM import issue
by Deekshith 23 Dec '20

23 Dec '20

Hi Team , We are not able to import the Virtual machine from ova file into Ovirt .Kindly help us Regards Deekshith

3 2

Unable to live migrate a VM from 4.4.2 to 4.4.3 CentOS Linux host
by Gianluca Cecchi 23 Dec '20

23 Dec '20

Hello, I was able to update an external CentOS Linux 8.2 standalone engine from 4.4.2 to 4.4.3 (see dedicated thread). Then I was able to put into maintenance one 4.4.2 host (CentOS Linux 8.2 based, not ovirt node ng) and run: [root@ov301 ~]# dnf update Last metadata expiration check: 0:27:11 ago on Wed 11 Nov 2020 08:48:04 PM CET. Dependencies resolved. ====================================================================================================================== Package Arch Version Repository Size ====================================================================================================================== Installing: kernel x86_64 4.18.0-193.28.1.el8_2 BaseOS 2.8 M kernel-core x86_64 4.18.0-193.28.1.el8_2 BaseOS 28 M kernel-modules x86_64 4.18.0-193.28.1.el8_2 BaseOS 23 M ovirt-ansible-collection noarch 1.2.1-1.el8 ovirt-4.4 276 k replacing ovirt-ansible-engine-setup.noarch 1.2.4-1.el8 replacing ovirt-ansible-hosted-engine-setup.noarch 1.1.8-1.el8 Upgrading: ansible noarch 2.9.15-2.el8 ovirt-4.4-centos-ovirt44 17 M bpftool x86_64 4.18.0-193.28.1.el8_2 BaseOS 3.4 M cockpit-ovirt-dashboard noarch 0.14.13-1.el8 ovirt-4.4 3.5 M ioprocess x86_64 1.4.2-1.el8 ovirt-4.4 37 k kernel-tools x86_64 4.18.0-193.28.1.el8_2 BaseOS 3.0 M kernel-tools-libs x86_64 4.18.0-193.28.1.el8_2 BaseOS 2.8 M libiscsi x86_64 1.18.0-8.module_el8.2.0+524+f765f7e0 AppStream 89 k nftables x86_64 1:0.9.3-12.el8_2.1 BaseOS 311 k ovirt-hosted-engine-ha noarch 2.4.5-1.el8 ovirt-4.4 325 k ovirt-hosted-engine-setup noarch 2.4.8-1.el8 ovirt-4.4 227 k ovirt-imageio-client x86_64 2.1.1-1.el8 ovirt-4.4 21 k ovirt-imageio-common x86_64 2.1.1-1.el8 ovirt-4.4 155 k ovirt-imageio-daemon x86_64 2.1.1-1.el8 ovirt-4.4 15 k ovirt-provider-ovn-driver noarch 1.2.32-1.el8 ovirt-4.4 27 k ovirt-release44 noarch 4.4.3-1.el8 ovirt-4.4 17 k python3-ioprocess x86_64 1.4.2-1.el8 ovirt-4.4 33 k python3-nftables x86_64 1:0.9.3-12.el8_2.1 BaseOS 25 k python3-ovirt-engine-sdk4 x86_64 4.4.6-1.el8 ovirt-4.4 560 k python3-perf x86_64 4.18.0-193.28.1.el8_2 BaseOS 2.9 M python3-pyasn1 noarch 0.4.6-3.el8 ovirt-4.4-centos-opstools 140 k python3-pyasn1-modules noarch 0.4.6-3.el8 ovirt-4.4-centos-opstools 151 k qemu-img x86_64 15:4.2.0-29.el8.6 ovirt-4.4-advanced-virtualization 1.0 M qemu-kvm x86_64 15:4.2.0-29.el8.6 ovirt-4.4-advanced-virtualization 118 k qemu-kvm-block-curl x86_64 15:4.2.0-29.el8.6 ovirt-4.4-advanced-virtualization 129 k qemu-kvm-block-gluster x86_64 15:4.2.0-29.el8.6 ovirt-4.4-advanced-virtualization 131 k qemu-kvm-block-iscsi x86_64 15:4.2.0-29.el8.6 ovirt-4.4-advanced-virtualization 136 k qemu-kvm-block-rbd x86_64 15:4.2.0-29.el8.6 ovirt-4.4-advanced-virtualization 130 k qemu-kvm-block-ssh x86_64 15:4.2.0-29.el8.6 ovirt-4.4-advanced-virtualization 131 k qemu-kvm-common x86_64 15:4.2.0-29.el8.6 ovirt-4.4-advanced-virtualization 1.2 M qemu-kvm-core x86_64 15:4.2.0-29.el8.6 ovirt-4.4-advanced-virtualization 3.4 M selinux-policy noarch 3.14.3-41.el8_2.8 BaseOS 615 k selinux-policy-targeted noarch 3.14.3-41.el8_2.8 BaseOS 15 M spice-server x86_64 0.14.2-1.el8_2.1 AppStream 404 k tzdata noarch 2020d-1.el8 BaseOS 471 k vdsm x86_64 4.40.35.1-1.el8 ovirt-4.4 1.4 M vdsm-api noarch 4.40.35.1-1.el8 ovirt-4.4 106 k vdsm-client noarch 4.40.35.1-1.el8 ovirt-4.4 24 k vdsm-common noarch 4.40.35.1-1.el8 ovirt-4.4 136 k vdsm-hook-ethtool-options noarch 4.40.35.1-1.el8 ovirt-4.4 9.8 k vdsm-hook-fcoe noarch 4.40.35.1-1.el8 ovirt-4.4 10 k vdsm-hook-openstacknet noarch 4.40.35.1-1.el8 ovirt-4.4 18 k vdsm-hook-vhostmd noarch 4.40.35.1-1.el8 ovirt-4.4 17 k vdsm-hook-vmfex-dev noarch 4.40.35.1-1.el8 ovirt-4.4 11 k vdsm-http noarch 4.40.35.1-1.el8 ovirt-4.4 15 k vdsm-jsonrpc noarch 4.40.35.1-1.el8 ovirt-4.4 31 k vdsm-network x86_64 4.40.35.1-1.el8 ovirt-4.4 331 k vdsm-python noarch 4.40.35.1-1.el8 ovirt-4.4 1.3 M vdsm-yajsonrpc noarch 4.40.35.1-1.el8 ovirt-4.4 40 k Installing dependencies: NetworkManager-ovs x86_64 1:1.22.14-1.el8 ovirt-4.4-copr:copr.fedorainfracloud.org:networkmanager:NetworkManager-1.22 144 k Transaction Summary ====================================================================================================================== Install 5 Packages Upgrade 48 Packages Total download size: 116 M After reboot I can activate the host (strange that I see many pop up messages about "finished activating host") and the host is shown as OS Version: RHEL - 8.2 - 2.2004.0.2.el8 OS Description: CentOS Linux 8 (Core) Kernel Version: 4.18.0 - 193.28.1.el8_2.x86_64 KVM Version: 4.2.0 - 29.el8.6 LIBVIRT Version: libvirt-6.0.0-25.2.el8 VDSM Version: vdsm-4.40.35.1-1.el8 SPICE Version: 0.14.2 - 1.el8_2.1 GlusterFS Version: [N/A] CEPH Version: librbd1-12.2.7-9.el8 Open vSwitch Version: [N/A] Nmstate Version: nmstate-0.2.10-1.el8 Kernel Features: MDS: (Vulnerable: Clear CPU buffers attempted, no microcode; SMT vulnerable), L1TF: (Mitigation: PTE Inversion; VMX: conditional cache flushes, SMT vulnerable), SRBDS: (Not affected), MELTDOWN: (Mitigation: PTI), SPECTRE_V1: (Mitigation: usercopy/swapgs barriers and __user pointer sanitization), SPECTRE_V2: (Mitigation: Full generic retpoline, IBPB: conditional, IBRS_FW, STIBP: conditional, RSB filling), ITLB_MULTIHIT: (KVM: Mitigation: Split huge pages), TSX_ASYNC_ABORT: (Not affected), SPEC_STORE_BYPASS: (Mitigation: Speculative Store Bypass disabled via prctl and seccomp) VNC Encryption: Disabled FIPS mode enabled: Disabled while another host still in 4.4.2: OS Version: RHEL - 8.2 - 2.2004.0.2.el8 OS Description: CentOS Linux 8 (Core) Kernel Version: 4.18.0 - 193.19.1.el8_2.x86_64 KVM Version: 4.2.0 - 29.el8.3 LIBVIRT Version: libvirt-6.0.0-25.2.el8 VDSM Version: vdsm-4.40.26.3-1.el8 SPICE Version: 0.14.2 - 1.el8 GlusterFS Version: [N/A] CEPH Version: librbd1-12.2.7-9.el8 Open vSwitch Version: [N/A] Nmstate Version: nmstate-0.2.10-1.el8 Kernel Features: MDS: (Vulnerable: Clear CPU buffers attempted, no microcode; SMT vulnerable), L1TF: (Mitigation: PTE Inversion; VMX: conditional cache flushes, SMT vulnerable), SRBDS: (Not affected), MELTDOWN: (Mitigation: PTI), SPECTRE_V1: (Mitigation: usercopy/swapgs barriers and __user pointer sanitization), SPECTRE_V2: (Mitigation: Full generic retpoline, IBPB: conditional, IBRS_FW, STIBP: conditional, RSB filling), ITLB_MULTIHIT: (KVM: Mitigation: Split huge pages), TSX_ASYNC_ABORT: (Not affected), SPEC_STORE_BYPASS: (Mitigation: Speculative Store Bypass disabled via prctl and seccomp) VNC Encryption: Disabled FIPS mode enabled: Disabled But if I try to move away VMs from the 4.4.2 host to the 4.4.3 one I get error: Failed to migrate VM c8client to Host ov301 . Trying to migrate to another Host. (btw: there is no other active host; there is a ov300 host that is in maintenance) No available host was found to migrate VM c8client to. It seems the root error in engine.log is: 2020-11-11 21:44:42,487+01 ERROR [org.ovirt.engine.core.vdsbroker.monitoring.VmAnalyzer] (ForkJoinPool-1-worker-11) [] Migration of VM 'c8client' to host 'ov301' failed: VM destroyed during the startup. On target host in /var/log/libvirt/qemu/c8clinet.log I see: 2020-11-11 20:44:40.981+0000: shutting down, reason=failed In target vdsm.log 2020-11-11 21:44:39,958+0100 INFO (jsonrpc/1) [jsonrpc.JsonRpcServer] RPC call VM.migrationCreate took more than 1.00 seconds to succeed: 1.97 (__init__:316) 2020-11-11 21:44:40,230+0100 INFO (periodic/3) [vdsm.api] START repoStats(domains=()) from=internal, task_id=cb51fd4a-09d3-4d77-821b-391da2467487 (api:48) 2020-11-11 21:44:40,231+0100 INFO (periodic/3) [vdsm.api] FINISH repoStats return={'fa33df49-b09d-4f86-9719-ede649542c21': {'code': 0, 'lastCheck': '4.1', 'delay': '0.000836715', 'valid': True, 'version': 4, 'acquired': True, 'actual': True}} from=internal, task_id=cb51fd4a-09d3-4d77-821b-391da2467487 (api:54) 2020-11-11 21:44:41,929+0100 INFO (jsonrpc/5) [api.virt] START destroy(gracefulAttempts=1) from=::ffff:10.4.192.32,52266, vmId=c95da734-7ed1-4caa-bacb-3fa24f4efb56 (api:48) 2020-11-11 21:44:41,930+0100 INFO (jsonrpc/5) [virt.vm] (vmId='c95da734-7ed1-4caa-bacb-3fa24f4efb56') Release VM resources (vm:4666) 2020-11-11 21:44:41,930+0100 INFO (jsonrpc/5) [virt.vm] (vmId='c95da734-7ed1-4caa-bacb-3fa24f4efb56') Stopping connection (guestagent:444) 2020-11-11 21:44:41,930+0100 INFO (jsonrpc/5) [vdsm.api] START teardownImage(sdUUID='fa33df49-b09d-4f86-9719-ede649542c21', spUUID='ef17cad6-7724-4cd8-96e3-9af6e529db51', imgUUID='ff10a405-cc61-4d00-a83f-3ee04b19f381', volUUID=None) from=::ffff:10.4.192.32,52266, task_id=177461c0-83d6-4c90-9c5c-3cc8ee9150c7 (api:48) It seems that during the host update the OVN configuration has not been maintained. Right now all my active VMs are with at least a vnic on OVN so I cannot test the scenario of migrating a VM without OVN based vnic. In fact on engine I see only the currently active host in 4.4.2 (ov200) and another host that is in maintenance (it is still in 4.3.10; I wanted to update to 4.4.2 but I realized that 4.4.3 has been out...): [root@ovmgr1 ovirt-engine]# ovn-sbctl show Chassis "6a46b802-5a50-4df5-b1af-e73f58a57164" hostname: "ov200.mydomain" Encap geneve ip: "10.4.192.32" options: {csum="true"} Port_Binding "2ae7391b-4297-4247-a315-99312f6392e6" Port_Binding "c1ec60a4-b4f3-4cb5-8985-43c086156e83" Port_Binding "174b69f8-00ed-4e25-96fc-7db11ea8a8b9" Port_Binding "66359e79-56c4-47e0-8196-2241706329f6" Port_Binding "ccbd6188-78eb-437b-9df9-9929e272974b" Chassis "ddecf0da-4708-4f93-958b-6af365a5eeca" hostname: "ov300.mydomain" Encap geneve ip: "10.4.192.33" options: {csum="true"} [root@ovmgr1 ovirt-engine]# Any hint about the reason of losing OVN config for ov301 and the correct procedure to get it again and persiste future updates? NOTE: this was a cluster in 4.3.10 and I updated it to 4.4.2 and I noticed that the OVN config was not retained and I had to run on hosts: [root@ov200 ~]# vdsm-tool ovn-config engine_ip ov200_ip_on_mgmt Using default PKI files Created symlink /etc/systemd/system/multi-user.target.wants/openvswitch.service → /usr/lib/systemd/system/openvswitch.service. Created symlink /etc/systemd/system/multi-user.target.wants/ovn-controller.service → /usr/lib/systemd/system/ovn-controller.service. [root@ov200 ~]# Now it seems the problem persists... Why do I have to run each time? Gianluca

1 3

Get Host Capabilities failed: Internal JSON-RPC error: {'reason': 'internal error: Duplicate key'}
by tommy 23 Dec '20

23 Dec '20

Hi,everyone: I got this error in my ovirt env: VDSM ooengh1.tltd.com command Get Host Capabilities failed: Internal JSON-RPC error: {'reason': 'internal error: Duplicate key'} Systemctl message is: Dec 23 20:48:48 ooengh1.tltd.com vdsm[2431]: ERROR Internal server error Traceback (most recent call last): File "/usr/lib/python2.7/site-packages/yajsonrpc/__init__.py", line 345, in _handle_request res = method(**params) File "/usr/lib/python2.7/site-packages/vdsm/rpc/Bridge.py", line 198, in _dynamicMethod result = fn(*methodArgs) File "<string>", line 2, in getCapabilities File "/usr/lib/python2.7/site-packages/vdsm/common/api.py", line 50, in method ret = func(*args, **kwargs) File "/usr/lib/python2.7/site-packages/vdsm/API.py", line 1371, in getCapabilities c = caps.get() File "/usr/lib/python2.7/site-packages/vdsm/host/caps.py", line 93, in get machinetype.compatible_cpu_models()) File "/usr/lib/python2.7/site-packages/vdsm/common/cache.py", line 43, in __call__ value = self.func(*args) File "/usr/lib/python2.7/site-packages/vdsm/machinetype.py", line 142, in compatible_cpu_models all_models = domain_cpu_models(c, arch, cpu_mode) File "/usr/lib/python2.7/site-packages/vdsm/machinetype.py", line 97, in domain_cpu_models domcaps = conn.getDomainCapabilities(None, arch, None, virt_type, 0) File "/usr/lib/python2.7/site-packages/vdsm/common/libvirtconnection.py", line 131, in wrapper ret = f(*args, **kwargs) File "/usr/lib/python2.7/site-packages/vdsm/common/function.py", line 94, in wrapper return func(inst, *args, **kwargs) File "/usr/lib64/python2.7/site-packages/libvirt.py", line 3844, in getDomainCapabilities if ret is None: raise libvirtError ('virConnectGetDomainCapabilities() failed', conn=self) libvirtError: internal error: Duplicate key AnyOne can help me ? Thanks!

1 0

Cannot ping from a nested VM
by wodel youchi 23 Dec '20

23 Dec '20

Hi, We have an HCI plateforme based upon 4.4.3 version. We activated the use of nested kvm and we reinstalled all the nodes to be able to use it. Then we created a new nested HCI on top of the physical HCI to test ansible deployment. The deployment went well until the phase which adds the two other nodes (the two other virtualized hypervisors), but the VM-Manager couldn't add them. After investigation, we found that the nested VM-Manager could not communicate except with it's first virtualized hypervisor. To make the nested VM-Manager capable of communicating with the outside world, we had to stop firewalld on the physical node of the HCI. Any ideas? Regards. <https://www.avast.com/sig-email?utm_medium=email&utm_source=link&utm_campai…> Virus-free. www.avast.com <https://www.avast.com/sig-email?utm_medium=email&utm_source=link&utm_campai…> <#DAB4FAD8-2DD7-40BB-A1B8-4E2AA1F9FDF2>

3 2

Upgrade to 4.4.4
by Jonathan Baecker 23 Dec '20

23 Dec '20

Hello, I'm running here a upgrade from 4.4.3 to latest 4.4.4, on a 3 node self hosted cluster. The engine upgrade went fine and now I'm on host upgrades. When I check there the updates it shows only *ovirt-node-ng-image-update-4.4.4-1.el8.noarch.rpm*. For that I have run manual updates on each host, with maintenance mode -> yum update -> reboot. When I run now on the engine *cat /etc/redhat-release *it show: *CentOS Linux release 8.3.2011* But on my nodes it shows still: *CentOS Linux release 8.2.2004 (Core)* How can this be? Best regards Jonathan

1 2