- Users - oVirt List Archives

Testing ovirt 4.4.1 Nested KVM on Skylake-client (core i5) does not work
by wodel youchi 14 Sep '20

14 Sep '20

Hi, I've been using my core i5 6500 (skylake-client) for some time now to test oVirt on my machine. However this is no longer the case. I am using Fedora 32 as my base system with nested-kvm enabled, when I try to install oVirt 4.4 as HCI single node, I get an error in the last phase which consists of copying the VM-Manager to the engine volume and boot it. It is the boot that causes the problem, I get an error about the CPU : *the CPU is incompatible with host CPU: Host CPU does not provide required features: mpx* *This is the CPU part from virsh domcapabilities on my physical machine* <cpu> <mode name='host-passthrough' supported='yes'/> <mode name='host-model' supported='yes'> *<model fallback='forbid'>Skylake-Client-IBRS</model> * <vendor>Intel</vendor> <feature policy='require' name='ss'/> <feature policy='require' name='vmx'/> <feature policy='require' name='pdcm'/> <feature policy='require' name='hypervisor'/> <feature policy='require' name='tsc_adjust'/> <feature policy='require' name='clflushopt'/> <feature policy='require' name='umip'/> <feature policy='require' name='md-clear'/> <feature policy='require' name='stibp'/> <feature policy='require' name='arch-capabilities'/> <feature policy='require' name='ssbd'/> <feature policy='require' name='xsaves'/> <feature policy='require' name='pdpe1gb'/> <feature policy='require' name='invtsc'/> <feature policy='require' name='ibpb'/> <feature policy='require' name='amd-ssbd'/> <feature policy='require' name='skip-l1dfl-vmentry'/> </mode> <mode name='custom' supported='yes'> <model usable='yes'>qemu64</model> <model usable='yes'>qemu32</model> <model usable='no'>phenom</model> <model usable='yes'>pentium3</model> <model usable='yes'>pentium2</model> <model usable='yes'>pentium</model> <model usable='yes'>n270</model> <model usable='yes'>kvm64</model> <model usable='yes'>kvm32</model> <model usable='yes'>coreduo</model> <model usable='yes'>core2duo</model> <model usable='no'>athlon</model> <model usable='yes'>Westmere-IBRS</model> <model usable='yes'>Westmere</model> <model usable='no'>Skylake-Server-IBRS</model> <model usable='no'>Skylake-Server</model> <model usable='yes'>Skylake-Client-IBRS</model> <model usable='yes'>Skylake-Client</model> <model usable='yes'>SandyBridge-IBRS</model> <model usable='yes'>SandyBridge</model> <model usable='yes'>Penryn</model> <model usable='no'>Opteron_G5</model> <model usable='no'>Opteron_G4</model> <model usable='no'>Opteron_G3</model> <model usable='yes'>Opteron_G2</model> <model usable='yes'>Opteron_G1</model> <model usable='yes'>Nehalem-IBRS</model> <model usable='yes'>Nehalem</model> <model usable='yes'>IvyBridge-IBRS</model> <model usable='yes'>IvyBridge</model> <model usable='no'>Icelake-Server</model> <model usable='no'>Icelake-Client</model> <model usable='yes'>Haswell-noTSX-IBRS</model> <model usable='yes'>Haswell-noTSX</model> <model usable='yes'>Haswell-IBRS</model> <model usable='yes'>Haswell</model> <model usable='no'>EPYC-IBPB</model> <model usable='no'>EPYC</model> <model usable='no'>Dhyana</model> <model usable='yes'>Conroe</model> <model usable='no'>Cascadelake-Server</model> <model usable='yes'>Broadwell-noTSX-IBRS</model> <model usable='yes'>Broadwell-noTSX</model> <model usable='yes'>Broadwell-IBRS</model> <model usable='yes'>Broadwell</model> <model usable='yes'>486</model> </mode> </cpu> *Here is the lscpu of my physical machine* # lscpu Architecture: x86_64 CPU op-mode(s): 32-bit, 64-bit Byte Order: Little Endian Address sizes: 39 bits physical, 48 bits virtual CPU(s): 4 On-line CPU(s) list: 0-3 Thread(s) per core: 1 Core(s) per socket: 4 Socket(s): 1 NUMA node(s): 1 Vendor ID: GenuineIntel CPU family: 6 Model: 94 Model name: Intel(R) Core(TM) i5-6500 CPU @ 3.20GHz Stepping: 3 CPU MHz: 954.588 CPU max MHz: 3600.0000 CPU min MHz: 800.0000 BogoMIPS: 6399.96 Virtualization: VT-x L1d cache: 128 KiB L1i cache: 128 KiB L2 cache: 1 MiB L3 cache: 6 MiB NUMA node0 CPU(s): 0-3 Vulnerability Itlb multihit: KVM: Mitigation: Split huge pages Vulnerability L1tf: Mitigation; PTE Inversion; VMX conditional cache flushes, SMT disabled Vulnerability Mds: Mitigation; Clear CPU buffers; SMT disabled Vulnerability Meltdown: Mitigation; PTI Vulnerability Spec store bypass: Mitigation; Speculative Store Bypass disabled via prctl and seccomp Vulnerability Spectre v1: Mitigation; usercopy/swapgs barriers and __user pointer sanitization Vulnerability Spectre v2: Mitigation; Full generic retpoline, IBPB conditional, IBRS_FW, STIBP disabled, RSB filling Vulnerability Srbds: Vulnerable: No microcode Vulnerability Tsx async abort: Mitigation; Clear CPU buffers; SMT disabled Flags: fpu vme de pse tsc msr pae mce cx8 apic sep mtrr pge mca cmov pat pse36 clflush dts acpi mmx fxsr sse sse2 ss ht tm pbe syscall nx pdpe1gb rdtscp lm constan t_tsc art arch_perfmon pebs bts rep_good nopl xtopology nonstop_tsc cpuid aperfmperf pni pclmulqdq dtes64 monitor ds_cpl vmx smx est tm2 ssse3 sdbg fma cx16 xtpr pdcm pcid sse4_1 sse4_2 x2apic movbe popcnt tsc_deadline_timer aes xsave avx f16c rdrand lahf_lm abm 3dnowprefetch cpuid_fault invpcid_single pti ssbd ibrs ibpb stibp tpr_shadow vnmi flexpriority ept vpid ept_ad fsgsbase tsc_adjust bmi1 hle avx2 smep bmi2 erms invpcid rtm *mpx* rdseed adx smap clflushopt in tel_pt xsaveopt xsavec xgetbv1 xsaves dtherm ida arat pln pts hwp hwp_notify hwp_act_window hwp_epp md_clear flush_l1d *Here is the CPU part from virsh dumpxml of my ovirt hypervisor* <cpu mode='custom' match='exact' check='full'> <model fallback='forbid'>Skylake-Client-IBRS</model> <vendor>Intel</vendor> <feature policy='require' name='ss'/> <feature policy='require' name='vmx'/> <feature policy='require' name='pdcm'/> <feature policy='require' name='hypervisor'/> <feature policy='require' name='tsc_adjust'/> <feature policy='require' name='clflushopt'/> <feature policy='require' name='umip'/> <feature policy='require' name='md-clear'/> <feature policy='require' name='stibp'/> <feature policy='require' name='arch-capabilities'/> <feature policy='require' name='ssbd'/> <feature policy='require' name='xsaves'/> <feature policy='require' name='pdpe1gb'/> <feature policy='require' name='ibpb'/> <feature policy='require' name='amd-ssbd'/> <feature policy='require' name='skip-l1dfl-vmentry'/> <feature policy='disable' name='mpx'/> </cpu> *Here is the lcpu of my ovirt hypervisor* [root@node1 ~]# lscpu Architecture : x86_64 Mode(s) opératoire(s) des processeurs : 32-bit, 64-bit Boutisme : Little Endian Processeur(s) : 4 Liste de processeur(s) en ligne : 0-3 Thread(s) par cœur : 1 Cœur(s) par socket : 1 Socket(s) : 4 Nœud(s) NUMA : 1 Identifiant constructeur : GenuineIntel Famille de processeur : 6 Modèle : 94 Nom de modèle : Intel Core Processor (Skylake, IBRS) Révision : 3 Vitesse du processeur en MHz : 3191.998 BogoMIPS : 6383.99 Virtualisation : VT-x Constructeur d'hyperviseur : KVM Type de virtualisation : complet Cache L1d : 32K Cache L1i : 32K Cache L2 : 4096K Cache L3 : 16384K Nœud NUMA 0 de processeur(s) : 0-3 Drapaux : fpu vme de pse tsc msr pae mce cx8 apic sep mtrr pge mca cmov pat pse36 clflush mmx fxsr sse sse2 ss syscall nx pdpe1gb rdtscp lm constant_tsc rep_go od nopl xtopology cpuid tsc_known_freq pni pclmulqdq vmx ssse3 fma cx16 pdcm pcid sse4_1 sse4_2 x2apic movbe popcnt tsc_deadline_timer aes xsave avx f16c rdrand hypervisor lahf_lm abm 3dnow prefetch cpuid_fault invpcid_single pti ssbd ibrs ibpb stibp tpr_shadow vnmi flexpriority ept vpid fsgsbase tsc_adjust bmi1 hle avx2 smep bmi2 erms invpcid rtm rdseed adx smap clflushopt xs aveopt xsavec xgetbv1 xsaves arat umip md_clear arch_capabilities it seems not all the flags are presented to the hypervisor especially the mpx which causes the error Is there a workaround for this? Regards.

4 6

oVirt HCI issue: GlusterD refusing to start due to quorum issues for volumes where it isn’t contributing any brick
by thomas＠hoberg.net 14 Sep '20

14 Sep '20

Sorry if it's a duplicate: I go an error on post... And yes I posted it on the Gluster slack first, but I am using Gluster only because the marketing on oVirt HCI worked so well... I got 3 recycled servers for an oVirt test environment first and set those up as 3-node HCI using defaults mostly, 2 replica + 1 arbiter, 'engine', 'vmstore' and 'data' volumes with single bricks for each node. I call these group A. Then I got another set of five machines, let's call them group B, with somewhat different hardware characteristics than group A, but nicely similar between themselves. I wanted to add these to the farm as compute nodes but also use their storage as general GlusterFS storage for a wider use. Group B machines were added as hosts and set up to run hosted-engine, but they do not contribute bricks to the normal oVirt volumes 'engine', 'vmstore' or 'data'. With some Ansible trickery I managed to set up two dispersed volumes (4 data: 1 redundancy) on group B 'scratch' and 'tape', mostly for external GlusterFS use. oVirt picked them up automagically, so I guess they could also be used with VMs. I expect to get more machines and adding them one-by-one to dispersed volumes with a fine balance between capacity and redundancy made me so enthusiastic about oVirt HCI in the first place... After some weeks of fine operation I had to restart a machine from group B for maintenance. When it came back up, GlusterD refuses to come online, because it doesn't have "quorum for volumes 'engine', 'vmstore' and 'data'".... It's a small surprise it doesn't *have* quorum, what's a bigger surprise is that it *asks* for quorum in a volume where it's not contributing any bricks. What's worse is that it then refuses to start serving its bricks for 'scratch' and 'tape', which are now growing apart without any chance of healing. How do I fix this? Is this a bug (my interpretation) or do I fundamentlly misunderstand how Gluster as a hyper scale out file system is supposed to work with potentially thousands of hosts contributing each dozens of bricks to each of hundreds of volumes in a single name space?

1 0

Probably dns problem: Internal JSON-RPC error
by jan.kleefeld＠jan-home.de 14 Sep '20

14 Sep '20

I have a clean installed CentOS 8.2 2004 on my server. The self hosted engine deploy (ovirt-4.4) aborts with the following message: [ ERROR ] fatal: [localhost]: FAILED! => {"changed": false, "msg": "The host has been set in non_operational status, deployment errors: code 505: Host red.colors.ovirt.local installation failed. Failed to configure management network on the host., code 1120: Failed to configure management network on host red.colors.ovirt.local due to setup networks failure., code 9000: Failed to verify Power Management configuration for Host red.colors.ovirt.local., code 10802: VDSM red.colors.ovirt.local command HostSetupNetworksVDS failed: Internal JSON-RPC error: {'reason': ' desired ======= --- dns-resolver: search: [] server: - 192.168.2.150 - fe80::1%eno1 current ======= --- dns-resolver: search: [] server: - 192.168.2.150 difference ========== --- desired +++ current @@ -3,4 +3,3 @@ search: [] server: - 192.168.2.150 - - fe80::1%eno1 '}, fix accordingly and re-deploy."} ~# cat /etc/resolv.conf # Generated by NetworkManager search colors.ovirt.local nameserver 192.168.2.150 nameserver fe80::1%eno1 I am confused, because the probalby missing line is present. Is there maybe another config file, where this last line could be missing? Maybe I can force somehow the installer, to reload the resolv conf, so it can fetch ne new line? I appreciate your help. Jan

3 3

OVN Geneve tunnels not been established
by Konstantinos B 11 Sep '20

11 Sep '20

Hi all We have a small installation based on OVIRT 4.3. 1 Cluster is based on Centos 7 and the other on OVIRT NG Node image. The environment was stable till an upgrade took place a couple of months ago. As such we had to re-install one of the Centos 7 node and start from scratch. Even though the installation completed successfully and VMs are created, the following are not working as expected: 1. ovn geneve tunnels are not established with the other Centos 7 node in the cluster. 2. Centos 7 node is configured by ovirt engine however no geneve tunnel is established when "ovn-sbctl show" is issued on the engine. 3. no flows are shown on the engine on port 6642 for the ovs db. Does anyone have any experience on how to troubleshoot OVN on ovirt? Thank you

2 1

"Add NFS storage domain", "status": "FAILED",
by info＠worldhostess.com 10 Sep '20

10 Sep '20

I am new to oVirt and this is my first installation https://ovirt.org/documentation/installing_ovirt_as_a_self-hosted_engine_us ing_the_cockpit_web_interface/ Everything seems fine, but then when I tried step 5, "Hosted Engine Deployment" it keep giving me this error. This information might be too much, but I am not familiar to know what is important. 2020-09-08 06:53:58,991+0200 DEBUG var changed: host "localhost" var "datacenter_name" type "<class 'ansible.utils.unsafe_proxy.AnsibleUnsafeText'>" value: ""Default"" 2020-09-08 06:53:58,991+0200 INFO ansible ok {'status': 'OK', 'ansible_type': 'task', 'ansible_playbook': '/usr/share/ovirt-hosted-engine-setup/ansible/trigger_role.yml', 'ansible_host': 'localhost', 'ansible_task': 'Fetch Datacenter name', 'task_duration': 0} 2020-09-08 06:53:58,991+0200 DEBUG ansible on_any args <ansible.executor.task_result.TaskResult object at 0x7f825fb425c0> kwargs 2020-09-08 06:53:59,242+0200 INFO ansible task start {'status': 'OK', 'ansible_type': 'task', 'ansible_playbook': '/usr/share/ovirt-hosted-engine-setup/ansible/trigger_role.yml', 'ansible_task': 'ovirt.hosted_engine_setup : Add NFS storage domain'} 2020-09-08 06:53:59,242+0200 DEBUG ansible on_any args TASK: ovirt.hosted_engine_setup : Add NFS storage domain kwargs is_conditional:False 2020-09-08 06:53:59,242+0200 DEBUG ansible on_any args localhostTASK: ovirt.hosted_engine_setup : Add NFS storage domain kwargs 2020-09-08 06:56:04,962+0200 DEBUG var changed: host "localhost" var "ansible_play_hosts" type "<class 'list'>" value: "[]" 2020-09-08 06:56:04,963+0200 DEBUG var changed: host "localhost" var "ansible_play_batch" type "<class 'list'>" value: "[]" 2020-09-08 06:56:04,963+0200 DEBUG var changed: host "localhost" var "play_hosts" type "<class 'list'>" value: "[]" 2020-09-08 06:56:04,963+0200 ERROR ansible failed { "ansible_host": "localhost", "ansible_playbook": "/usr/share/ovirt-hosted-engine-setup/ansible/trigger_role.yml", "ansible_result": { "_ansible_no_log": false, "changed": false, "exception": "Traceback (most recent call last):\n File \"/tmp/ansible_ovirt_storage_domain_payload_ir_v98cz/ansible_ovirt_storage_d omain_payload.zip/ansible/modules/cloud/ovirt/ovirt_storage_domain.py\", line 792, in main\n File \"/tmp/ansible_ovirt_storage_domain_payload_ir_v98cz/ansible_ovirt_storage_d omain_payload.zip/ansible/module_utils/ovirt.py\", line 623, in create\n **kwargs\n File \"/usr/lib64/python3.6/site-packages/ovirtsdk4/services.py\", line 26097, in add\n return self._internal_add(storage_domain, headers, query, wait)\n File \"/usr/lib64/python3.6/site-packages/ovirtsdk4/service.py\", line 232, in _internal_add\n return future.wait() if wait else future\n File \"/usr/lib64/python3.6/site-packages/ovirtsdk4/service.py\", line 55, in wait\n return self._code(response)\n File \"/usr/lib64/python3.6/site-packages/ovirtsdk4/service.py\", line 229, in callback\n self._check_fault(response)\n File \"/usr/lib64/python3.6/site-packages/ovirtsdk4/service.py\", line 132, in _check_fault\n self._raise_error(response, body)\n File \"/usr/lib64/python3.6/site-packages/ovirtsdk4/service.py\", line 118, in _raise_error\n raise error\novirtsdk4.Error: Fault reason is \"Operation Failed\". Fault detail is \"[Problem while trying to mount target]\". HTTP response code is 400.\n", "invocation": { "module_args": { "backup": null, "comment": null, "critical_space_action_blocker": null, "data_center": "Default", "description": null, "destroy": null, "discard_after_delete": null, "domain_function": "data", "fcp": null, "fetch_nested": false, "format": null, "glusterfs": null, "host": "host.worldhostess.co.za", "id": null, "iscsi": null, "localfs": null, "managed_block_storage": null, "name": "hosted_storage", "nested_attributes": [], "nfs": { "address": "storage01.worldhostess.co.za", "mount_options": null, "path": "/data/nfs", "version": "auto" }, "poll_interval": 3, "posixfs": null, "state": "unattached", "timeout": 180, "wait": true, "warning_low_space": null, "wipe_after_delete": null } }, "msg": "Fault reason is \"Operation Failed\". Fault detail is \"[Problem while trying to mount target]\". HTTP response code is 400." }, "ansible_task": "Add NFS storage domain", "ansible_type": "task", "status": "FAILED", "task_duration": 125 } 2020-09-08 06:56:04,963+0200 DEBUG ansible on_any args <ansible.executor.task_result.TaskResult object at 0x7f825f9c3a58> kwargs ignore_errors:None 2020-09-08 06:56:04,964+0200 INFO ansible stats { "ansible_playbook": "/usr/share/ovirt-hosted-engine-setup/ansible/trigger_role.yml", "ansible_playbook_duration": "02:16 Minutes", "ansible_result": "type: <class 'dict'>\nstr: {'localhost': {'ok': 14, 'failures': 1, 'unreachable': 0, 'changed': 0, 'skipped': 2, 'rescued': 0, 'ignored': 0}}", "ansible_type": "finish", "status": "FAILED" } 2020-09-08 06:56:04,964+0200 INFO SUMMARY: Duration Task Name -------- -------- [ < 1 sec ] Execute just a specific set of steps [ 00:01 ] Force facts gathering [ < 1 sec ] Check local VM dir stat [ < 1 sec ] Obtain SSO token using username/password credentials [ < 1 sec ] Fetch host facts [ < 1 sec ] Fetch cluster ID [ < 1 sec ] Fetch cluster facts [ < 1 sec ] Fetch Datacenter facts [ < 1 sec ] Fetch Datacenter ID [ < 1 sec ] Fetch Datacenter name [ FAILED ] Add NFS storage domain 2020-09-08 06:56:04,965+0200 DEBUG ansible on_any args <ansible.executor.stats.AggregateStats object at 0x7f826252ef60> kwargs Yours Sincerely, Dr Henni Appelgryn PhD DBA MBA AEP MDP WorldHostess Skype: genie_za WeChat: HenniAppelgryn LinkedIn: https://www.linkedin.com/in/henni-appelgryn/ Hosting with a difference

4 7

Failed to create a new VM in ovirt due to "The host did not satisfy internal filter Memory" error
by KISHOR K 10 Sep '20

10 Sep '20

Hi All, I'm new to Ovirt and not having a perfect experience with Ovirt yet. I ran into a strange issue today when I tried to create a new VM with 32GB in ovirt. VM creation basically due to an error, pasted below. Cannot run VM. There is no host that satisfies current scheduling constraints. See below for details:, The host host-01 did not satisfy internal filter Memory., The host host-01 did not satisfy internal filter Memory. After some troubleshooting, I found that there was enough available memory (around 120 GB free ) in the host to host new VM and there were around 10 VMs already running on this host. But, I noticed that ovirt is actually scheduling and creating VM based on "Max free Memory for scheduling new VMs" value, which seems to be set/updated based on total allocated memory for all VMs running on the host and it's not really the then consumed memory by running VMs. Can anyone help to explain if it is some kind of bug in ovirt or it expected behavior? If it is expected behavior, is there any possibility to change it to make sure VM is created based on actual free memory? Thanks a lot in advance for your support. /Kishore /Kishore

4 8

Diskless oVirt Nodes
by harryo.dk＠gmail.com 10 Sep '20

10 Sep '20

Hi, why does my ovirt hypervisor node crash when i pull the hypervisor boot drive? The following thread claims that the oVirt node hypervisor should lodad into memory, so boot disk failure shouldn't have any impact on a running node. https://lists.ovirt.org/pipermail/users/2014-December/029771.html Only memory failure should have a impact, and if RAM fails there is a possibility that the hypervisor OS would have crashed regardless. If you load the hypervisor OS into memory there will be moved/combined a point of failure (the boot disk) with another(the RAM). RAM=memory

3 6

Hyperconverged set up with Gluster and 6 Nodes
by Holger Petrick 09 Sep '20

09 Sep '20

Dear All, in all documentation to set up oVirt in a Hyperconverged system with Gluster says that it must have 3 Enterprise nodes. In our test environment we set up successfully with oVirt wizard an oVirt cluster with gluster + Arbiter and hosted engine. Is it also possible to build a cluster with 6 nodes? The wizard gives only the option to add 3 nodes. Is this kind of design supported? My idea is, first to set up the Gluster, then install the oVirt nodes and deploy the engine. Would this work? Any documentation avaiable? Thanks Holger

3 5

Hosted engine install failure: ipv6.gateway: gateway cannot be set if there are no addresses configured
by Sverker Abrahamsson 08 Sep '20

08 Sep '20

Hi, I'm attempting to install hosted engine but getting this failure: [ ERROR ] fatal: [localhost]: FAILED! => {"changed": false, "msg": "The host has been set in non_operational status, deployment errors: code 505: Host h1-mgmt.limetransit.com installation failed. Failed to configure management network on the host., code 1120: Failed to configure management network on host h1-mgmt.limetransit.com due to setup networks failure., code 9000: Failed to verify Power Management configuration for Host h1-mgmt.limetransit.com., code 10802: VDSM h1-mgmt.limetransit.com command HostSetupNetworksVDS failed: Internal JSON-RPC error: {'reason': 'Unexpected failure of libnm when running the mainloop: run execution'}, fix accordingly and re-deploy."} Looking in vdsm.log I find this which I believe is the root cause: MainProcess|jsonrpc/0::DEBUG::2020-09-02 16:38:25,897::supervdsm_server::93::SuperVdsm.ServerCallback::(wrapper) call setupNetworks with ({'ovirtmgmt': {'vlan': '4000', 'netmask': '255.255.255.0', 'ipv6autoconf': False, 'nic': 'enp4s0', 'bridged': 'true', 'ipaddr': '172.27.1.1', 'defaultRoute': True, 'dhcpv6': False, 'STP': 'no', 'mtu': 1500, 'switch': 'legacy'}}, {}, {'connectivityTimeout': 120, 'commitOnSuccess': True, 'connectivityCh eck': 'true'}) {} MainProcess|jsonrpc/0::INFO::2020-09-02 16:38:25,897::api::220::root::(setupNetworks) Setting up network according to configuration: networks:{'ovirtmgmt': {'vlan': '4000', 'netmask': '255.255.255.0', 'ipv6autoconf': Fal se, 'nic': 'enp4s0', 'bridged': 'true', 'ipaddr': '172.27.1.1', 'defaultRoute': True, 'dhcpv6': False, 'STP': 'no', 'mtu': 1500, 'switch': 'legacy'}}, bondings:{}, options:{'connectivityTimeout': 120, 'commitOnSuccess': True, 'connectivityCheck': 'true'} MainProcess|jsonrpc/0::DEBUG::2020-09-02 16:38:25,902::cmdutils::130::root::(exec_cmd) /sbin/tc qdisc show (cwd None) MainProcess|jsonrpc/0::DEBUG::2020-09-02 16:38:25,906::cmdutils::138::root::(exec_cmd) SUCCESS: <err> = b''; <rc> = 0 MainProcess|jsonrpc/0::DEBUG::2020-09-02 16:38:25,945::vsctl::74::root::(commit) Executing commands: /usr/bin/ovs-vsctl --timeout=5 --oneline --format=json -- list Bridge -- list Port -- list Interface MainProcess|jsonrpc/0::DEBUG::2020-09-02 16:38:25,945::cmdutils::130::root::(exec_cmd) /usr/bin/ovs-vsctl --timeout=5 --oneline --format=json -- list Bridge -- list Port -- list Interface (cwd None) MainProcess|jsonrpc/0::DEBUG::2020-09-02 16:38:25,952::cmdutils::138::root::(exec_cmd) SUCCESS: <err> = b''; <rc> = 0 MainProcess|jsonrpc/0::INFO::2020-09-02 16:38:25,957::netconfpersistence::58::root::(setNetwork) Adding network ovirtmgmt({'vlan': 4000, 'netmask': '255.255.255.0', 'ipv6autoconf': False, 'nic': 'enp4s0', 'bridged': True , 'ipaddr': '172.27.1.1', 'defaultRoute': True, 'dhcpv6': False, 'mtu': 1500, 'switch': 'legacy', 'stp': False, 'bootproto': 'none', 'nameservers': ['213.133.98.98']}) MainProcess|jsonrpc/0::DEBUG::2020-09-02 16:38:25,958::commands::153::common.commands::(start) /usr/bin/taskset --cpu-list 0-7 /usr/libexec/vdsm/hooks/before_network_setup/50_fcoe (cwd None) MainProcess|jsonrpc/0::INFO::2020-09-02 16:38:26,154::hooks::122::root::(_runHooksDir) /usr/libexec/vdsm/hooks/before_network_setup/50_fcoe: rc=0 err=b'' MainProcess|jsonrpc/0::INFO::2020-09-02 16:38:26,155::configurator::195::root::(_setup_nmstate) Processing setup through nmstate MainProcess|jsonrpc/0::INFO::2020-09-02 16:38:26,175::configurator::197::root::(_setup_nmstate) Desired state: {'interfaces': [{'name': 'enp4s0', 'state': 'up', 'mtu': 1500}, {'vlan': {'id': 4000, 'base-iface': 'enp4s0'} , 'name': 'enp4s0.4000', 'type': 'vlan', 'state': 'up', 'mtu': 1500, 'ipv4': {'enabled': False}, 'ipv6': {'enabled': False}}, {'name': 'ovirtmgmt', 'type': 'linux-bridge', 'state': 'up', 'mtu': 1500, 'bridge': {'port': [ {'name': 'enp4s0.4000'}], 'options': {'stp': {'enabled': False}}}, 'ipv4': {'enabled': True, 'address': [{'ip': '172.27.1.1', 'prefix-length': 24}], 'dhcp': False}, 'ipv6': {'enabled': False}}], 'dns-resolver': {'config' : {'server': ['213.133.98.98']}}} MainProcess|jsonrpc/0::DEBUG::2020-09-02 16:38:26,217::checkpoint::121::root::(create) Checkpoint /org/freedesktop/NetworkManager/Checkpoint/1 created for all devices: 60 MainProcess|jsonrpc/0::DEBUG::2020-09-02 16:38:26,218::netapplier::239::root::(_add_interfaces) Adding new interfaces: ['ovirtmgmt'] MainProcess|jsonrpc/0::DEBUG::2020-09-02 16:38:26,221::netapplier::251::root::(_edit_interfaces) Editing interfaces: ['enp4s0.4000', 'enp4s0'] MainProcess|jsonrpc/0::DEBUG::2020-09-02 16:38:26,224::nmclient::136::root::(execute_next_action) Executing NM action: func=add_connection_async MainProcess|jsonrpc/0::DEBUG::2020-09-02 16:38:26,231::connection::329::root::(_add_connection_callback) Connection adding succeeded: dev=ovirtmgmt MainProcess|jsonrpc/0::DEBUG::2020-09-02 16:38:26,232::nmclient::136::root::(execute_next_action) Executing NM action: func=commit_changes_async MainProcess|jsonrpc/0::ERROR::2020-09-02 16:38:26,233::nmclient::200::root::(quit) NM main-loop aborted: Connection update failed: error=nm-connection-error-quark: ipv6.gateway: gateway cannot be set if there are no addresses configured (7), dev=enp4s0/<enum NM_DEVICE_STATE_ACTIVATED of type NM.DeviceState> MainProcess|jsonrpc/0::DEBUG::2020-09-02 16:38:26,246::checkpoint::164::root::(rollback) Checkpoint /org/freedesktop/NetworkManager/Checkpoint/1 rollback executed: dbus.Dictionary({dbus.String('/org/freedesktop/NetworkManager/Devices/6'): dbus.UInt32(0), dbus.String('/org/freedesktop/NetworkManager/Devices/7'): dbus.UInt32(0), dbus.String('/org/freedesktop/NetworkManager/Devices/8'): dbus.UInt32(0), dbus.String('/org/freedesktop/NetworkManager/Devices/9'): dbus.UInt32(0), dbus.String('/org/freedesktop/NetworkManager/Devices/3'): dbus.UInt32(0), dbus.String('/org/freedesktop/NetworkManager/Devices/2'): dbus.UInt32(0), dbus.String('/org/freedesktop/NetworkManager/Devices/4'): dbus.UInt32(0), dbus.String('/org/freedesktop/NetworkManager/Devices/1'): dbus.UInt32(0), dbus.String('/org/freedesktop/NetworkManager/Devices/5'): dbus.UInt32(0)}, signature=dbus.Signature('su')) MainProcess|jsonrpc/0::ERROR::2020-09-02 16:38:31,251::supervdsm_server::97::SuperVdsm.ServerCallback::(wrapper) Error in setupNetworks Traceback (most recent call last): File "/usr/lib/python3.6/site-packages/vdsm/supervdsm_server.py", line 95, in wrapper res = func(*args, **kwargs) File "/usr/lib/python3.6/site-packages/vdsm/network/api.py", line 241, in setupNetworks _setup_networks(networks, bondings, options, net_info) File "/usr/lib/python3.6/site-packages/vdsm/network/api.py", line 266, in _setup_networks networks, bondings, options, net_info, in_rollback File "/usr/lib/python3.6/site-packages/vdsm/network/netswitch/configurator.py", line 154, in setup _setup_nmstate(networks, bondings, options, in_rollback) File "/usr/lib/python3.6/site-packages/vdsm/network/netswitch/configurator.py", line 199, in _setup_nmstate nmstate.setup(desired_state, verify_change=not in_rollback) File "/usr/lib/python3.6/site-packages/vdsm/network/nmstate.py", line 63, in setup state_apply(desired_state, verify_change=verify_change) File "/usr/lib/python3.6/site-packages/libnmstate/deprecation.py", line 40, in wrapper return func(*args, **kwargs) File "/usr/lib/python3.6/site-packages/libnmstate/nm/nmclient.py", line 96, in wrapped ret = func(*args, **kwargs) File "/usr/lib/python3.6/site-packages/libnmstate/netapplier.py", line 73, in apply state.State(desired_state), verify_change, commit, rollback_timeout File "/usr/lib/python3.6/site-packages/libnmstate/netapplier.py", line 163, in _apply_ifaces_state con_profiles=ifaces_add_configs + ifaces_edit_configs, File "/usr/lib64/python3.6/contextlib.py", line 88, in __exit__ next(self.gen) File "/usr/lib/python3.6/site-packages/libnmstate/netapplier.py", line 232, in _setup_providers mainloop.run(timeout=MAINLOOP_TIMEOUT) File "/usr/lib/python3.6/site-packages/libnmstate/nm/nmclient.py", line 177, in run f"Unexpected failure of libnm when running the mainloop: {err}" libnmstate.error.NmstateLibnmError: Unexpected failure of libnm when running the mainloop: run execution My host have a main network interface and a vlan interface on the same physical interface which is an internal management network which should be used as management network. The vlan interface doesn't have any ipv6 interface, but nmclient fails on setting ipv6 gateway. This seems like a bug? /Sverker _______________________________________________ discuss mailing list discuss(a)openvswitch.org https://mail.openvswitch.org/mailman/listinfo/ovs-discuss

4 21

iSCSI multipath issue
by Nick Kas 08 Sep '20

08 Sep '20

Hello evryone, setup ovirt 4.4.1 on CentOS 8.2 as an experiment, and I am trying to get an iSCSI domain working but have issues. The little experimental cluster has 3 hosts. There is an ovirtmgmt network on the default vlan, and two iSCSI network (172.27.0/1.X) with vlans 20/21. ovirtmgmt has all the functions (Data, display, migration etc), and the iSCSI networks nothing yet, and they are not set as required. The SAN device is already serving a few iSCSI volumes to a vmware cluster, so I know things are fine on this end. It has two controllers, and four NICs per controller so a total of 8 NICs, half of the NICS per controller on 172.27.0.X and half on 172.27.1.X. When I create the iSCSI domain, I login to only one of the targets, and add the Volume, all is good and I can use the disc fine. However when I login to more than one of the targets, then I start having issues with the Volume. Even when I enabled multipath in the cluster, and I created a single multipath by selecting both of the 172.27.0/1.X networks, and all the targets, the end result was the same. The hosts have difficulty accessing the volume, they may even swing between 'non-operational' and 'up' if I transfer data to the volume. When I ssh to the hosts and i check things in the command line I also get inconsistent results between hosts, and blocks that appear with lsblk when I first setup iSCSI have dissapeared after I try to actively use the volume. I am new to iSCSI so I am not sure how to debug this. I am not sure if my multipath configuration is correct or not. The documentation on this part was not very detailed. I also tried to remove the domain, and try to experiment with mounting the iSCSI volume from the command line, but I cannot even discover the target from the command line, which is very bizarre. The command iscsiadm --mode discovery --target sendtargets --portal 172.27.0.55 --discover returns the message 'iscsiadm: cannot make connection to 172.27.0.55: No route to host'. Yet through ovirt, and if I select only one target, everything work fine! Any suggestions on how to start debugging this would really be appreciated.

2 3

New Installation OVIRT NODE 4.4.1
by henni＠africadragon.com 08 Sep '20

08 Sep '20

I am on step 5, but I am not sure what I am doing wrong. I get the following error. Any help will be appreciated. I have a subdomain, but I am not sure if it is needed. I am not sure if I have to add things in /etc/hosts [ INFO ] ok: [localhost] [ INFO ] TASK [ovirt.hosted_engine_setup : Obtain SSO token using username/password credentials] [ INFO ] ok: [localhost] [ INFO ] TASK [ovirt.hosted_engine_setup : Fetch host facts] [ INFO ] ok: [localhost] [ INFO ] TASK [ovirt.hosted_engine_setup : Fetch cluster ID] [ INFO ] ok: [localhost] [ INFO ] TASK [ovirt.hosted_engine_setup : Fetch cluster facts] [ INFO ] ok: [localhost] [ INFO ] TASK [ovirt.hosted_engine_setup : Fetch Datacenter facts] [ INFO ] ok: [localhost] [ INFO ] TASK [ovirt.hosted_engine_setup : Fetch Datacenter ID] [ INFO ] ok: [localhost] [ INFO ] TASK [ovirt.hosted_engine_setup : Fetch Datacenter name] [ INFO ] ok: [localhost] [ INFO ] TASK [ovirt.hosted_engine_setup : Add NFS storage domain] [ ERROR ] ovirtsdk4.Error: Fault reason is "Operation Failed". Fault detail is "[Problem while trying to mount target]". HTTP response code is 400. [ ERROR ] fatal: [localhost]: FAILED! => {"changed": false, "msg": "Fault reason is \"Operation Failed\". Fault detail is \"[Problem while trying to mount target]\". HTTP response code is 400."}

2 1

Engine certificate and HTTPS : broken on recent Firefox
by Michaël Couren 08 Sep '20

08 Sep '20

Hi, Is there a "verified"* way to add a valid certificate to an oVirt 4.3 engine ? (or suppress HTTP Strict Transport Security) It is no more possible to add a security exception (on latest Firefox 80.0.1 Linux) and I don't want to use 2 browsers (like sanp-chromium just for oVirt...) *"verified" : I mean we have already tried official doc (many months ago) but with no success Thanks -- Cordialement / Best regards, Michaël Couren, ABES, Montpellier, France.

2 1

Using Ansible to automate failed Drive replacement
by kushagra2agarwal＠gmail.com 07 Sep '20

07 Sep '20

Do we have any inbuild ansible modules/tasks which can help to automate replacing failed drive/volume in oVirt hypervisor

3 5

Re: KeyCloak Integration
by Artur Socha 07 Sep '20

07 Sep '20

Hi Anton, Just to let you know. I investigated this issue. If you want to use keycloak in version >=10 you would need to define all additional scopes as 'optional client scopes' in your client configuration. In my case, on my test environment, I only had to add 'ovirt-ext=auth:sequence-priority=~' but in your case you may need all listed in error_description: *{"error_description":"Cannot authenticate user Invalid scopes: ovirt-app-api ovirt-ext=revoke:revoke-all ovirt-ext=token-info:authz-search ovirt-ext=token-info:public-authz-search ovirt-ext=token-info:validate ovirt-ext=token:password-access.","error":"access_denied"}* This configuration change is required because it has been changed/fixed how 'unknown' scopes are handled in keycloak. Now keycloak must always be aware of all scopes and previously unknown ones were simply ignored. Here is BZ with details: https://bugzilla.redhat.com/show_bug.cgi?id=1849569 best, Artur On Tue, Jun 23, 2020 at 5:03 PM Artur Socha <asocha(a)redhat.com> wrote: > On Tue, 2020-06-23 at 14:41 +0000, Anton Louw wrote: > > > > Hi Artur, > > > > Apologies for the late response. So we have downgraded the version of > KeyCloak, and all seems to be working 100% again, I can obtain a token, and > do API calls. > > Hi Anton, > I'm glad it works now. This keycloak version (9.0.x) will stay for some > time the recommended & supported choice for oVirt because it is part of > 'Red Hat SSO' just like oVirt is part of 'Red Hat Virtualization'. > Artur > > > > Thank you very much for all the help > > > > *From:* Artur Socha <asocha(a)redhat.com> > *Sent:* 22 June 2020 16:52 > *To:* Anton Louw <Anton.Louw(a)voxtelecom.co.za>; users(a)ovirt.org > *Cc:* Stephen Hutchinson <Stephen.Hutchinson(a)voxtelecom.co.za> > *Subject:* Re: [ovirt-users] KeyCloak Integration > > > > On Mon, 2020-06-22 at 15:14 +0200, Artur Socha wrote: > > Anton, > > I managed to re-create the issue on my local environment. > > Previously I tested it against Keycloak 8.0.1 with users loaded from LDAP. > Currently I have users/groups created via Keycloak management panel. I need > to investigate it further which of the two changes is the root cause (it > works fine with the old setup) > > > > One more update: it seems the issue is keycloak version related. Trying to > figure out what was changed and how it affected engine sso integration. > > > > Latest keycloak version I tested and verified that works is 9.0.3. Perhaps > it could be possible for you to use it until we fully support 10.0.x ? > > Artur > > > > *Anton Louw* > *Cloud Engineer: Storage and Virtualization* at *Vox* > ------------------------------ > *T:* 087 805 0000 | *D:* 087 805 1572 > *M:* N/A > *E:* anton.louw(a)voxtelecom.co.za > *A:* Rutherford Estate, 1 Scott Street, Waverley, Johannesburg > www.vox.co.za > > [image: F] <https://www.facebook.com/voxtelecomZA> > [image: T] <https://www.twitter.com/voxtelecom> > [image: I] <https://www.instagram.com/voxtelecomza/> > [image: L] <https://www.linkedin.com/company/voxtelecom> > [image: Y] <https://www.youtube.com/user/VoxTelecom> > > Artur > > > > On Mon, 2020-06-22 at 11:05 +0000, Anton Louw wrote: > > > > Hi Artur, > > > > Great, thanks a lot! 😊 > > > > > > *Anton Louw* > > *Cloud Engineer: Storage and Virtualization* at *Vox* > ------------------------------ > > *T:* 087 805 0000 | *D:* 087 805 1572 > *M:* N/A > *E:* anton.louw(a)voxtelecom.co.za > *A:* Rutherford Estate, 1 Scott Street, Waverley, Johannesburg > www.vox.co.za > > > > [image: F] <https://www.facebook.com/voxtelecomZA> > > > > [image: T] <https://www.twitter.com/voxtelecom> > > > > [image: I] <https://www.instagram.com/voxtelecomza> > > > > [image: L] <https://www.linkedin.com/company/voxtelecom> > > > > [image: Y] <https://www.youtube.com/user/VoxTelecom> > > > > > > *From:* Artur Socha <asocha(a)redhat.com> > *Sent:* 22 June 2020 11:23 > *To:* Anton Louw <Anton.Louw(a)voxtelecom.co.za>; users(a)ovirt.org > *Cc:* Stephen Hutchinson <Stephen.Hutchinson(a)voxtelecom.co.za> > *Subject:* Re: [ovirt-users] KeyCloak Integration > > > > Hi Anton, > > Thanks for the specs. I have create BZ issue for tracking: > > https://bugzilla.redhat.com/show_bug.cgi?id=1849569 > > Feel free to add comments/change it when needed. > > > > Artur > > > > On Fri, 2020-06-19 at 10:57 +0000, Anton Louw wrote: > > > > Hi Artur, > > > > Please see below: > > > > ovirt-engine.noarch 4.3.10.4-1.el7 @ovirt-4.3 > > ovirt-engine-extension-aaa-misc.noarch 1.0.4-1.el7 @ovirt-4.3 > > mod_auth_openidc.x86_64 1.8.8-5.el7 @base > > > > [root@virt ~]# cat /etc/*elease > > CentOS Linux release 7.7.1908 (Core) > > NAME="CentOS Linux" > > VERSION="7 (Core)" > > ID="centos" > > ID_LIKE="rhel fedora" > > VERSION_ID="7" > > PRETTY_NAME="CentOS Linux 7 (Core)" > > ANSI_COLOR="0;31" > > CPE_NAME="cpe:/o:centos:centos:7" > > HOME_URL="https://www.centos.org/" > > BUG_REPORT_URL="https://bugs.centos.org/" > > > > CENTOS_MANTISBT_PROJECT="CentOS-7" > > CENTOS_MANTISBT_PROJECT_VERSION="7" > > REDHAT_SUPPORT_PRODUCT="centos" > > REDHAT_SUPPORT_PRODUCT_VERSION="7" > > > > CentOS Linux release 7.7.1908 (Core) > > CentOS Linux release 7.7.1908 (Core) > > > > KeyCloak – > > > > Server Version > > 10.0.1 > > > > Thanks a lot for your help Artur. Please let me know if you need anything > else. > > > > *From:* Artur Socha <asocha(a)redhat.com> > *Sent:* 19 June 2020 12:39 > *To:* Anton Louw <Anton.Louw(a)voxtelecom.co.za>; users(a)ovirt.org > *Cc:* Stephen Hutchinson <Stephen.Hutchinson(a)voxtelecom.co.za> > *Subject:* Re: [ovirt-users] KeyCloak Integration > > > > On Fri, 2020-06-19 at 10:21 +0000, Anton Louw wrote: > > > > Yes I didn’t get to the OVN part yet, as I first wanted to test the if the > token can be obtained. > > > > This is the first time we are testing KeyCloak in any environment, so we > have never been able to obtain a token for API access. > > > > Please post the exact versions of: > > - ovirt-engine* : > > yum list --installed | grep ovirt-engine > > yum list --intalled | grep ovirt-engine-extension-aaa-misc > > yum list --installed | grep mod_auth_openidc > > - keycloak > > - OS > > cat /etc/*elease > > > > I'll submit a bug ... which, most likely, I will assign to myself anyway :) > > > > Artur > > > > > > *Anton Louw* > > *Cloud Engineer: Storage and Virtualization* at *Vox* > ------------------------------ > > *T:* 087 805 0000 | *D:* 087 805 1572 > *M:* N/A > *E:* anton.louw(a)voxtelecom.co.za > *A:* Rutherford Estate, 1 Scott Street, Waverley, Johannesburg > www.vox.co.za > > > > [image: F] <https://www.facebook.com/voxtelecomZA> > > > > [image: T] <https://www.twitter.com/voxtelecom> > > > > [image: I] <https://www.instagram.com/voxtelecomza> > > > > [image: L] <https://www.linkedin.com/company/voxtelecom> > > > > [image: Y] <https://www.youtube.com/user/VoxTelecom> > > > > > > Thanks > > > > *From:* Artur Socha <asocha(a)redhat.com> > *Sent:* 19 June 2020 12:16 > *To:* Anton Louw <Anton.Louw(a)voxtelecom.co.za>; users(a)ovirt.org > *Cc:* Stephen Hutchinson <Stephen.Hutchinson(a)voxtelecom.co.za> > *Subject:* Re: [ovirt-users] KeyCloak Integration > > > > On Fri, 2020-06-19 at 10:03 +0000, Anton Louw wrote: > > > > Hi Artur, > > > > Sure, please see below output: > > > > [root@virt ~]# curl -vvv -H "Accept:application/json" ' > https://virt.example.co.za/ovirt-engine/sso/oauth/token?grant_type=password… > ' > > * About to connect() to virt.example.co.za port 443 (#0) > > * Trying 127.0.0.1... > > * Connected to virt.example.co.za (127.0.0.1) port 443 (#0) > > * Initializing NSS with certpath: sql:/etc/pki/nssdb > > * CAfile: /etc/pki/tls/certs/ca-bundle.crt > > CApath: none > > * SSL connection using TLS_ECDHE_RSA_WITH_AES_256_GCM_SHA384 > > * Server certificate: > > * subject: CN=*.example.co.za,OU=Domain Control Validated > > * start date: Sep 25 07:46:12 2019 GMT > > * expire date: Oct 02 07:39:01 2020 GMT > > * common name: *example.co.za > > * issuer: CN=Starfield Secure Certificate Authority - G2,OU= > http://certs.starfieldtech.com/repository/,O="Starfield Technologies, > Inc.",L=Scottsdale,ST=Arizona,C=US > > > GET > /ovirt-engine/sso/oauth/token?grant_type=password&username=myuser&password=mypass&scope=ovirt-app-api > HTTP/1.1 > > > User-Agent: curl/7.29.0 > > > Host: virt.example.co.za > > > Accept:application/json > > > > > < HTTP/1.1 400 Bad Request > > < Date: Fri, 19 Jun 2020 09:52:11 GMT > > < Server: Apache/2.4.6 (CentOS) OpenSSL/1.0.2k-fips > > < Set-Cookie: locale=en_US; path=/; secure; HttpOnly; Max-Age=2147483647; > Expires=Wed, 07-Jul-2088 13:06:18 GMT > > < X-XSS-PROTECTION: 1; MODE=BLOCK > > < X-CONTENT-TYPE-OPTIONS: NOSNIFF > > < X-FRAME-OPTIONS: SAMEORIGIN > > < Content-Type: application/json > > < Content-Length: 233 > > < Connection: close > > < > > * Closing connection 0 > > {"error_code":"access_denied","error":"Cannot authenticate user Invalid > scopes: ovirt-app-api ovirt-ext=token-info:authz-search > ovirt-ext=token-info:public-authz-search ovirt-ext=token-info:validate > ovirt-ext=token:password-access."} > > > > 1) Test connection using python script (from the blog post ) using sdk. I > suspect it will not work either. > > Testing from Python gives me the same error as well. > > > > 2) I saw some errors in the log on revoking token. Please go to keycloak > admin panel, and under users kill all its active sessions. Then, please > without logging in to engine admin UI, use that curl to obtain token. > > Tested this again, but still getting the below: > > {"error_code":"access_denied","error":"Cannot authenticate user Invalid > scopes: ovirt-app-api ovirt-ext=token-info:authz-search > ovirt-ext=token-info:public-authz-search ovirt-ext=token-info:validate > ovirt-ext=token:password-access."} > > > > Thanks for these test ... unfortunately nothing helped > > > > > > 3) Does it work without OVN integration enabled? > > Can you explain a bit more? How can I disable OVN integration to test this? > > > > I had in mind reverting OVN vs Keycloak integration done according to > "Configuring OVN" chapter in > https://blogs.ovirt.org/2019/01/federate-ovirt-engine-authentication-to-ope… > > Unless, of course, you skipped it. > > > > Most likely you found a bug. Have you ever been able to obtain token for > api access with keycloak integration (even with you previous environments)? > > I am now trying to understand what happened and how to reproduce it before > submitting the bug into http://bugzilla.redhat.com > > > > > > *Anton Louw* > > *Cloud Engineer: Storage and Virtualization* at *Vox* > ------------------------------ > > *T:* 087 805 0000 | *D:* 087 805 1572 > *M:* N/A > *E:* anton.louw(a)voxtelecom.co.za > *A:* Rutherford Estate, 1 Scott Street, Waverley, Johannesburg > www.vox.co.za > > > > [image: F] <https://www.facebook.com/voxtelecomZA> > > > > [image: T] <https://www.twitter.com/voxtelecom> > > > > [image: I] <https://www.instagram.com/voxtelecomza> > > > > [image: L] <https://www.linkedin.com/company/voxtelecom> > > > > [image: Y] <https://www.youtube.com/user/VoxTelecom> > > > > > > > > Thanks > > > > > > *Anton Louw* > > *Cloud Engineer: Storage and Virtualization* at *Vox* > ------------------------------ > > *T:* 087 805 0000 | *D:* 087 805 1572 > *M:* N/A > *E:* anton.louw(a)voxtelecom.co.za > *A:* Rutherford Estate, 1 Scott Street, Waverley, Johannesburg > www.vox.co.za > > > > [image: F] <https://www.facebook.com/voxtelecomZA> > > > > [image: T] <https://www.twitter.com/voxtelecom> > > > > [image: I] <https://www.instagram.com/voxtelecomza> > > > > [image: L] <https://www.linkedin.com/company/voxtelecom> > > > > [image: Y] <https://www.youtube.com/user/VoxTelecom> > > > > > > *From:* Artur Socha <asocha(a)redhat.com> > *Sent:* 19 June 2020 11:40 > *To:* Anton Louw <Anton.Louw(a)voxtelecom.co.za>; users(a)ovirt.org > *Cc:* Stephen Hutchinson <Stephen.Hutchinson(a)voxtelecom.co.za> > *Subject:* Re: [ovirt-users] KeyCloak Integration > > > > On Fri, 2020-06-19 at 08:34 +0000, Anton Louw wrote: > > > > Hi Artur, > > > > Thank you for the quick response. > > > > I have actually tried creating another user, but I still get the same > error. I have attached the output of curl -vvv as well as the logs the > engine and keycloak logs. > > > > This `curl -vvv ...` is actually is incorrect because it is missing -H > before 'Accept' header. However, previous attempts that led to this error > seemed to be fine. Could you just re-send output of the correct curl? > > > > There are few things we can test to try to narrow down the root cause: > > > > 1) Test connection using python script (from the blog post ) using sdk. I > suspect it will not work either. > > > > 2) I saw some errors in the log on revoking token. Please go to keycloak > admin panel, and under users kill all its active sessions. Then, please > without logging in to engine admin UI, use that curl to obtain token. > > > > 3) Does it work without OVN integration enabled? > > > > Artur > > > > > > > > Thank you > > > > > > *Anton Louw* > > *Cloud Engineer: Storage and Virtualization* at *Vox* > ------------------------------ > > *T:* 087 805 0000 | *D:* 087 805 1572 > *M:* N/A > *E:* anton.louw(a)voxtelecom.co.za > *A:* Rutherford Estate, 1 Scott Street, Waverley, Johannesburg > www.vox.co.za > > > > [image: F] <https://www.facebook.com/voxtelecomZA> > > > > [image: T] <https://www.twitter.com/voxtelecom> > > > > [image: I] <https://www.instagram.com/voxtelecomza> > > > > [image: L] <https://www.linkedin.com/company/voxtelecom> > > > > [image: Y] <https://www.youtube.com/user/VoxTelecom> > > > > > > *From:* Artur Socha <asocha(a)redhat.com> > *Sent:* 19 June 2020 10:23 > *To:* Anton Louw <Anton.Louw(a)voxtelecom.co.za>; users(a)ovirt.org > *Subject:* Re: [ovirt-users] KeyCloak Integration > > > > O > > n Fri, 2020-06-19 at 07:35 +0000, Anton Louw via Users wrote: > > > > Hi Everybody, > > > > Hi Anton, > > > > So I have implemented KeyCloak into our oVirt environment, which works, up > until a point. So WebUI access works, but when calling the API, using: > > *curl -k -H "Accept: application/json" '* > *https://virt.example.co.za/ovirt-engine/sso/oauth/token?grant_type=password&username=admin@openidchttp&password=mypass&scope=ovirt-app-api* > <https://virt.example.co.za/ovirt-engine/sso/oauth/token?grant_type=password…> > *'* > > > > I get the below error: > > > > *{"error_description":"Cannot authenticate user Invalid scopes: > ovirt-app-api ovirt-ext=revoke:revoke-all ovirt-ext=token-info:authz-search > ovirt-ext=token-info:public-authz-search ovirt-ext=token-info:validate > ovirt-ext=token:password-access.","error":"access_denied"}* > > > > If my configs are removed, and I use “admin@internal” for my username, > then it works. > > > > I followed the below article step by step, and I double checked that all > the scopes are added into KeyCloak (ovirt-app-api and ovirt-app-admin) > > > > > https://blogs.ovirt.org/2019/01/federate-ovirt-engine-authentication-to-ope… > > > > Anybody have any ideas? > > > > It is my blind shot but could create & check another user? > > > > One more thing to check please use curl -vvv to check if there are any > redirects along the way. > > I will check keycloak settings on my setup - perhaps there is something > non-obvious that could have been missed. > > > > Any chance to get a bit more logs from engine.log and even from keycloak? > Perhaps there is something there that could help. > > > > Artur > > > > > > Thank you > > > > *Anton Louw* > > *Cloud Engineer: Storage and Virtualization* at *Vox* > ------------------------------ > > *T:* 087 805 0000 | *D:* 087 805 1572 > *M:* N/A > *E:* anton.louw(a)voxtelecom.co.za > *A:* Rutherford Estate, 1 Scott Street, Waverley, Johannesburg > www.vox.co.za > > > > [image: F] <https://www.facebook.com/voxtelecomZA> > > > > [image: T] <https://www.twitter.com/voxtelecom> > > > > [image: I] <https://www.instagram.com/voxtelecomza> > > > > [image: L] <https://www.linkedin.com/company/voxtelecom> > > > > [image: Y] <https://www.youtube.com/user/VoxTelecom> > > > > > > [image: #VoxBrand] > <https://www.vox.co.za/fibre/fibre-to-the-home/?prod=HOME> > > > *Disclaimer* > > The contents of this email are confidential to the sender and the intended > recipient. Unless the contents are clearly and entirely of a personal > nature, they are subject to copyright in favour of the holding company of > the Vox group of companies. Any recipient who receives this email in error > should immediately report the error to the sender and permanently delete > this email from all storage devices. > > This email has been scanned for viruses and malware, and may have been > automatically archived by *Mimecast Ltd*, an innovator in Software as a > Service (SaaS) for business. Providing a *safer* and *more useful* place > for your human generated data. Specializing in; Security, archiving and > compliance. To find out more Click Here > <https://www.voxtelecom.co.za/security/mimecast/?prod=Enterprise>. > > > > _______________________________________________ > > Users mailing list -- > > * <users(a)ovirt.org>* > > *users(a)ovirt.org <users(a)ovirt.org>* > > > > > > > > > > > > > > > > To unsubscribe send an email to > > * <users-leave(a)ovirt.org>* > > *users-leave(a)ovirt.org <users-leave(a)ovirt.org>* > > > > > > > > > > > > > > > > Privacy Statement: > > * <https://www.ovirt.org/privacy-policy.html>* > > *https://www.ovirt.org/privacy-policy.html > <https://www.ovirt.org/privacy-policy.html>* > > > > > > > > > > > > > > > > oVirt Code of Conduct: > > * <https://www.ovirt.org/community/about/community-guidelines/>* > > *https://www.ovirt.org/community/about/community-guidelines/ > <https://www.ovirt.org/community/about/community-guidelines/>* > > > > > > > > > > > > > > > > List Archives: > > * > <https://lists.ovirt.org/archives/list/users@ovirt.org/message/CC54IPZLYJYE2…>* > > *https://lists.ovirt.org/archives/list/users@ovirt.org/message/CC54IPZLYJYE2B3NP4LT4TN4CJX4C7BU/ > <https://lists.ovirt.org/archives/list/users@ovirt.org/message/CC54IPZLYJYE2…>* > > > > > > > > > > > > > > > > > > > > > > > > > > > -- Artur Socha Senior Software Engineer, RHV Red Hat

1 0

Storage Domain won't activate
by Gillingham, Eric J (US 393D) 05 Sep '20

05 Sep '20

I recently removed a host from my cluster to upgrade it to 4.4, after I removed the host from the datacenter VMs started to pause on the second system they all migrated to. Investigating via the engine showed the storage domain was showing as "unknown", when I try to activate it via the engine it cycles to locked then to unknown again. /var/log/sanlock.log contains a repeating: add_lockspace e1270474-108c-4cae-83d6-51698cffebbf:1:/dev/e1270474-108c-4cae-83d6-51698cffebbf/ids:0 conflicts with name of list1 s1 e1270474-108c-4cae-83d6-51698cffebbf:3:/dev/e1270474-108c-4cae-83d6-51698cffebbf/ids:0 vdsm.log contains these (maybe related) snippets: --- 2020-09-03 20:19:53,483+0000 INFO (jsonrpc/6) [vdsm.api] FINISH getAllTasksStatuses error=Secured object is not in safe state from=::ffff:137.79.52.43,36326, flow_id=18031a91, task_id=8e92f059-743a-48c8-aa9d-e7c4c836337b (api:52) 2020-09-03 20:19:53,483+0000 ERROR (jsonrpc/6) [storage.TaskManager.Task] (Task='8e92f059-743a-48c8-aa9d-e7c4c836337b') Unexpected error (task:875) Traceback (most recent call last): File "/usr/lib/python2.7/site-packages/vdsm/storage/task.py", line 882, in _run return fn(*args, **kargs) File "<string>", line 2, in getAllTasksStatuses File "/usr/lib/python2.7/site-packages/vdsm/common/api.py", line 50, in method ret = func(*args, **kwargs) File "/usr/lib/python2.7/site-packages/vdsm/storage/hsm.py", line 2201, in getAllTasksStatuses allTasksStatus = self._pool.getAllTasksStatuses() File "/usr/lib/python2.7/site-packages/vdsm/storage/securable.py", line 77, in wrapper raise SecureError("Secured object is not in safe state") SecureError: Secured object is not in safe state 2020-09-03 20:19:53,483+0000 INFO (jsonrpc/6) [storage.TaskManager.Task] (Task='8e92f059-743a-48c8-aa9d-e7c4c836337b') aborting: Task is aborted: u'Secured object is not in safe state' - code 100 (task:1181) 2020-09-03 20:19:53,483+0000 ERROR (jsonrpc/6) [storage.Dispatcher] FINISH getAllTasksStatuses error=Secured object is not in safe state (dispatcher:87) Traceback (most recent call last): File "/usr/lib/python2.7/site-packages/vdsm/storage/dispatcher.py", line 74, in wrapper result = ctask.prepare(func, *args, **kwargs) File "/usr/lib/python2.7/site-packages/vdsm/storage/task.py", line 108, in wrapper return m(self, *a, **kw) File "/usr/lib/python2.7/site-packages/vdsm/storage/task.py", line 1189, in prepare raise self.error SecureError: Secured object is not in safe state --- 2020-09-03 20:44:23,252+0000 INFO (tasks/2) [storage.ThreadPool.WorkerThread] START task 76415a77-9d29-4b72-ade1-53207cfc503b (cmd=<bound method Task.commit of <vdsm.storage.task.Task instance at 0x7fe99c27dea8>>, args=None) (thre adPool:208) 2020-09-03 20:44:23,266+0000 INFO (tasks/2) [storage.SANLock] Acquiring host id for domain e1270474-108c-4cae-83d6-51698cffebbf (id=1, wait=True) (clusterlock:313) 2020-09-03 20:44:23,267+0000 ERROR (tasks/2) [storage.TaskManager.Task] (Task='76415a77-9d29-4b72-ade1-53207cfc503b') Unexpected error (task:875) Traceback (most recent call last): File "/usr/lib/python2.7/site-packages/vdsm/storage/task.py", line 882, in _run return fn(*args, **kargs) File "/usr/lib/python2.7/site-packages/vdsm/storage/task.py", line 336, in run return self.cmd(*self.argslist, **self.argsdict) File "/usr/lib/python2.7/site-packages/vdsm/storage/sp.py", line 317, in startSpm self.masterDomain.acquireHostId(self.id) File "/usr/lib/python2.7/site-packages/vdsm/storage/sd.py", line 957, in acquireHostId self._manifest.acquireHostId(hostId, wait) File "/usr/lib/python2.7/site-packages/vdsm/storage/sd.py", line 501, in acquireHostId self._domainLock.acquireHostId(hostId, wait) File "/usr/lib/python2.7/site-packages/vdsm/storage/clusterlock.py", line 344, in acquireHostId raise se.AcquireHostIdFailure(self._sdUUID, e) AcquireHostIdFailure: Cannot acquire host id: ('e1270474-108c-4cae-83d6-51698cffebbf', SanlockException(22, 'Sanlock lockspace add failure', 'Invalid argument')) --- Another symptom is in the hosts view of the engine SPM bounces between "Normal" and "Contending". When it's Normal if I select Management -> Select as SPM I get "Error while executing action: Cannot force select SPM. Unknown Data Center status." I've tried rebooting the one remaining host in the cluster no to avail, hosted-engine --reinitialize-lockspace also seems to not solve the issue. I'm kind of stumped as to what else to try, would appreciate any guidance on how to resolve this. Thank You

5 9

Failed to connect to server (code: 1006) connecting to second host noVNC console
by James Loker-Steele 04 Sep '20

04 Sep '20

Hi All, Using OLVM and having an issue connecting to a noVNC console on a secondary host. Im getting the Failed to connect. We are using the standalone host as the deployment of the self hosted kept failing, something about the machine not being able to run virtualised machines even though it can. Added a second kvm host to the management engine. Can create new machines, disks etc. Just when trying to connect to the console using noVNC. Any help would be appreciated.

1 1

Upgrade 4.3 to 4.4 node to manager communication error
by Pierre pit 03 Sep '20

03 Sep '20

I have a communication problem between all the nodes and the manager following the upgrade from 4.3 to 4.4. I followed the procedure of update 4.3 to 4.4 everything worked correctly, according to the import export scripts as well as the installation setup on the new manager in 4.4, all is ok. Only after connection to the manager, all the nodes are in a down state, there is no more communication between the manager newly installed in 4.4 and the nodes still in production in 4.3. In the manager I have this message for all the nodes: ` VDSM virtdell8 command Get Host Capabilities failed: PKIX path validation failed: java.security.cert.CertPathValidatorException: Algorithm constraints check failed on signature algorithm: SHA256withRSA` And on the nodes: ` 2020-09-01 17:38:13,083+0200 ERROR (Reactor thread) [ProtocolDetector.SSLHandshakeDispatcher] ssl handshake: SSLError, address: ::ffff:XXX.XXX.XXX.XXX (sslutils:264) vdsm[4400]: ERROR ssl handshake: SSLError, address: ::ffff:XXX.XXX.XXX.XXX` After a search on the forums I found a similar error on version 4.2 only the solution of comment `ssl_excludes` in the `/etc/vdsm/vdsm.conf` file but does not apply to my problem. I unfortunately had to backtrack because it was no longer possible to control ovirt and use the manager for our production. the new machine with the manager in 4.4 is offline while a solution is found Do you know where should I look in order to solve this problem? thank you in advance Pierre

4 3

Re: [ovirt 4.2]The snapshot is deleted after restoring the snapshot
by Strahil Nikolov 03 Sep '20

03 Sep '20

If you have snapshots like A -> B -> C and you restore A , it is normal to loose B & C. After all when you restore A , B & C never happened. Otherwise , ovirt should clone the snapshots in separate images and that is not the requested, right ? Best Regards, Strahil Nikolov В четвъртък, 3 септември 2020 г., 10:03:30 Гринуич+3, liuweijie(a)sunyainfo.com <liuweijie(a)sunyainfo.com> написа: Dear All: I used ovirt api (POST /ovirt-engine/api/vms/{vm:id}/snapshots) to create several snapshots. When I call the snapshot recovery api (POST /ovirt-engine/api/vms/{vm:id }/snapshots/{snapshot:id}/restore) to restoe the first snapshot, I found that the next few snapshots were deleted, and it was normal when I restored the last snapshot. Why does the snapshot with the earlier recovery time delete the subsequent snapshot? The vm's os is centos. Hope get help from you soon, Thank you. Yours sincerely， Evelyn _______________________________________________ Users mailing list -- users(a)ovirt.org To unsubscribe send an email to users-leave(a)ovirt.org Privacy Statement: https://www.ovirt.org/privacy-policy.html oVirt Code of Conduct: https://www.ovirt.org/community/about/community-guidelines/ List Archives: https://lists.ovirt.org/archives/list/users@ovirt.org/message/LJL6NUYBPGJAF…

1 0

Hosted engine + OVN
by Sverker Abrahamsson 03 Sep '20

03 Sep '20

Was it ever solved to install hosted engine with ovn? I tried a few years ago, got it almost to work but then gave up.

2 1

How to Backup a VM
by Stefan Wolf 03 Sep '20

03 Sep '20

Hello to all I try to backup a normal VM. But It seems that I don't really understand the concept. At first I found the possibility to backup with the api https://www.ovirt.org/documentation/administration_guide/#Setting_a_storage…. Create a snapshot of the VM, finding the ID of the snapshot and the configuration of the VM makes sense to me. But at this point, I would download the config an the snapshot and put it to my backup storage. And not create a new VM attach the disk and run a backup with backup programm. And for restoring do the sam way backwards. If i look at other project, there seems do be a way to download the snapshot and configfile, or am I wrong? Maybe someone can explain to me why I should use additional software to install in an additional machine. Or even better someone can explain to me how I don't have to use additional backup software. And to the same topic backup. There is in the documentation the possibility to set up a backup storage It is nearly the same, create a snapshot, or clone the machine and export it to backup storage > Export the new virtual machine to a backup domain. See Exporting a Virtual Machine to a Data Domain in the Virtual Machine Management Guide. Sadly there is just writen what to do, not how, the link points to 404 page. maybe someone can explain to me how to use backup storage thank you very much shb

5 15

[ovirt 4.2]The snapshot is deleted after restoring the snapshot
by liuweijie＠sunyainfo.com 03 Sep '20

03 Sep '20

Dear All: I used ovirt api (POST /ovirt-engine/api/vms/{vm:id}/snapshots) to create several snapshots. When I call the snapshot recovery api (POST /ovirt-engine/api/vms/{vm:id }/snapshots/{snapshot:id}/restore) to restoe the first snapshot, I found that the next few snapshots were deleted, and it was normal when I restored the last snapshot. Why does the snapshot with the earlier recovery time delete the subsequent snapshot? The vm's os is centos. Hope get help from you soon, Thank you. Yours sincerely， Evelyn

1 0

Ovirt Host Crashed
by Darin Schmidt 02 Sep '20

02 Sep '20

I am running this as an all in one system for a test bed at home. The system crashed which lead me to have to reinstall the OS (CentOS 8) and I imported the data stores but I cannot find any way to import the VM's that were in the DATA store. I havent had a chance to backup/export the VMs. I havent been able to find anything in the documentation on how to import these VMs. Any suggestions or links to what Im looking for? I had to create a new DATA store as the option for importing a local data store wasnt an option, I assume its because the host was down. THen I imported the old data store.

2 1

Mellanox OFED with oVirt
by Vinícius Ferrão 02 Sep '20

02 Sep '20

Hello, Anyone had success using Mellanox OFED with oVirt? Already learned some things: 1. I can’t use oVirt Node. 2. Mellanox OFED cannot be installed with mlnx-ofed-all since it breaks dnf. We need to rely on the upstream RDMA implementation. 3. The way to go is running: dnf install mlnx-ofed-dpdk-upstream-libs But after the installation I ended up with broken dnf: [root@c4140 ~]# dnf update Updating Subscription Management repositories. Last metadata expiration check: 0:03:54 ago on Tue 01 Sep 2020 11:52:41 PM -03. Error: Problem: both package mlnx-ofed-all-user-only-5.1-0.6.6.0.rhel8.2.noarch and mlnx-ofed-all-5.1-0.6.6.0.rhel8.2.noarch obsolete glusterfs-rdma - cannot install the best update candidate for package glusterfs-rdma-6.0-37.el8.x86_64 - package ovirt-host-4.4.1-4.el8ev.x86_64 requires glusterfs-rdma, but none of the providers can be installed - package mlnx-ofed-all-5.1-0.6.6.0.rhel8.2.noarch obsoletes glusterfs-rdma provided by glusterfs-rdma-6.0-37.el8.x86_64 - package glusterfs-rdma-3.12.2-40.2.el8.x86_64 requires glusterfs(x86-64) = 3.12.2-40.2.el8, but none of the providers can be installed - package glusterfs-rdma-6.0-15.el8.x86_64 requires glusterfs(x86-64) = 6.0-15.el8, but none of the providers can be installed - package glusterfs-rdma-6.0-20.el8.x86_64 requires glusterfs(x86-64) = 6.0-20.el8, but none of the providers can be installed - cannot install both glusterfs-3.12.2-40.2.el8.x86_64 and glusterfs-6.0-37.el8.x86_64 - cannot install both glusterfs-6.0-15.el8.x86_64 and glusterfs-6.0-37.el8.x86_64 - cannot install both glusterfs-6.0-20.el8.x86_64 and glusterfs-6.0-37.el8.x86_64 - cannot install the best update candidate for package ovirt-host-4.4.1-4.el8ev.x86_64 - cannot install the best update candidate for package glusterfs-6.0-37.el8.x86_64 (try to add '--allowerasing' to command line to replace conflicting packages or '--skip-broken' to skip uninstallable packages or '--nobest' to use not only best candidate packages) That are the packages installed: [root@c4140 ~]# rpm -qa *mlnx* mlnx-dpdk-19.11.0-1.51066.x86_64 mlnx-ofa_kernel-devel-5.1-OFED.5.1.0.6.6.1.rhel8u2.x86_64 mlnx-ethtool-5.4-1.51066.x86_64 mlnx-dpdk-devel-19.11.0-1.51066.x86_64 mlnx-ofa_kernel-5.1-OFED.5.1.0.6.6.1.rhel8u2.x86_64 mlnx-dpdk-doc-19.11.0-1.51066.noarch mlnx-dpdk-tools-19.11.0-1.51066.x86_64 mlnx-ofed-dpdk-upstream-libs-5.1-0.6.6.0.rhel8.2.noarch kmod-mlnx-ofa_kernel-5.1-OFED.5.1.0.6.6.1.rhel8u2.x86_64 mlnx-iproute2-5.6.0-1.51066.x86_64 And finally this is the repo that I’m using: [root@c4140 ~]# cat /etc/yum.repos.d/mellanox_mlnx_ofed.repo # # Mellanox Technologies Ltd. public repository configuration file. # For more information, refer to http://linux.mellanox.com # [mlnx_ofed_latest_base] name=Mellanox Technologies rhel8.2-$basearch mlnx_ofed latest baseurl=http://linux.mellanox.com/public/repo/mlnx_ofed/latest/rhel8.2/$bas… enabled=1 gpgkey=http://www.mellanox.com/downloads/ofed/RPM-GPG-KEY-Mellanox gpgcheck=1 So anyone had success with this?

3 2

How can Gluster be a HCI default, when it's hardly ever working?
by thomas＠hoberg.net 02 Sep '20

02 Sep '20

I've just tried to verify what you said here. As a base line I started with the 1nHCI Gluster setup. From four VMs, two legacy, two Q35 on the single node Gluster, one survived the import, one failed silently with an empty disk, two failed somewhere in the middle of qemu-img trying to write the image to the Gluster storage. For each of those two, this always happened at the same block number, a unique one per machine, not in random places, as if qemu-img reading and writing the very same image could not agree. That's two types of error and a 75% failure rate I created another domain, basically using an NFS automount export from one of the HCI nodes (a 4.3 node serving as 4.4 storage) and imported the very same VMs (source all 4.3) transported via a re-attached export domain to 4.4. Three of the for imports worked fine, no error with qemu-img writing to NFS. All VMs had full disk images and launched, which verified that there is nothing wrong with the exports at least. But there was still one, that failed with the same qemu-img error. I then tried to move the disks from NFS to Gluster, which internally is also done via qemu-img, and I had those fail every time. Gluster or HCI seems a bit of a Russian roulette for migrations, and I am wondering how much it is better for normal operations. I'm still going to try moving via a backup domain (on NFS) and moving between that and Gluster, to see if it makes any difference. I really haven't done a lot of stress testing yet with oVirt, but this experience doesn't build confidence.

2 2

HP blade G7, with be2net.
by Remulo 01 Sep '20

01 Sep '20

Hello I have some blades with 10GE interface that have / need the Emulex be2net driver, however it is no longer available on Redhat8 / Centos8. Is there any way to install ovirt to work on these machines? Here are some links to problems with Redhat. Emulex NIC using be2net driver https://access.redhat.com/solutions/1229853 https://access.redhat.com/solutions/514353 I did a lot of research and couldn't get a functional driver for ovirt. Thank you. -- Atenciosamente, Rêmulo Ferreira.

3 2

Problem installing Windows VM on 4.4.1
by fgarat＠gmail.com 01 Sep '20

01 Sep '20

Hi, I'm having problem after I upgraded to 4.4.1 with Windows machines. The installation sees no disk. Even IDE disk doesn't get detected and installation won't move forward no matter what driver i use for the disk. Any one else having this issue?. Regards, Facundo

4 3

How can you avoid breaking 4.3.11 legacy VMs imported in 4.4.1 during a migration?
by thomas＠hoberg.net 01 Sep '20

01 Sep '20

Testing the 4.3 to 4.4 migration... what I describe here is facts is mostly observations and conjecture, could be wrong, just makes writing easier... While 4.3 seems to maintain a default emulated machine type (pc-i440fx-rhel7.6.0 by default), it doesn't actually allow setting it in the cluster settings: Could be built-in, could be inherited from the default template... Most of my VMs were created with the default on 4.3. oVirt 4.4 presets that to pc-q35-rhel8.1.0 and that has implications: 1. Any VM imported from an export on a 4.3 farm, will get upgraded to Q35, which unfortunately breaks things, e.g. network adapters getting renamed as the first issue I stumbled on some Debian machines 2. If you try to compensate by lowering the cluster default from Q35 to pc-i44fx the hosted-engine will fail, because it was either built or came as Q35 and can no longer find critical devices: It evidently doesn't take/use the VM configuration data it had at the last shutdown, but seems to re-generate it according to some obscure logic, which fails here. I've tried creating a bit of backward compatibility by creating another template based on pc-i440fx, but at the time of the import, I cannot switch the template. If I try to downgrade the cluster, the hosted-engine will fail to start and I can't change the template of the hosted-engine to something Q35. Currently this leaves me in a position where I can't separate the move of VMs from 4.3 to 4.4 and the upgrade of the virtual hardware, which is a different mess for every OS in the mix of VMs. Recommendations, tips anyone? P.S. A hypervisor reconstructing the virtual hardware from anywhere but storage at every launch, is difficult to trust IMHO.

3 6

Upgrade Doc typo.
by carl langlois 01 Sep '20

01 Sep '20

Hi, Not sure if this is relevant but in the ovirt doc in the 4.2 to 4.3 upgrade the repo rpm is specified with 4,4 rpm . Thanks Carl [image: image.png]

2 2

Re: Promiscuous Mode
by thomas＠hoberg.net 31 Aug '20

31 Aug '20

Not sure it will actually do that, but if you create a new network profile, you can select a 'port mirroring' option: It is my understanding that is what you need. You may also want to deselect the filtering rules in the same place.

1 0

can't mount an export domain with 4K block size (VDO) as PosixFS (xfs)
by thomas＠hoberg.net 31 Aug '20

31 Aug '20

After this (https://devconfcz2020a.sched.com/event/YOtG/ovirt-4k-teaching-an-old-dog-ne…) I sure do not expect this (log below): Actually I am trying to evaluate just how portable oVirt storage is and this case I had prepared a USB3 HDD with VDO, which I could literally move between farms to transport VMs. Logical disks are typically large for simplicity within the VMs, QCOW2 and VDO assumed to compensate for this 'lack of planning' while the allocated storage easily fits the HDD. Once I got beyond the initial issues, I came across this somewhat unexpected issue: VDO storage uses 4k blocks all around, but evidently when you mount an export domain (or I guess any domain) as POSIX, 512byte blocks are assumed somewhere and 4k blocks rejected. I'd say that is a bug in VDSM, right? Or is there anything in the mount options to fix this? 020-08-31 18:44:40,424+0200 INFO (periodic/0) [vdsm.api] START repoStats(domains=()) from=internal, task_id=7a293dec-85b3-4b82-92b7-4e7d03b40343 (api:48) 2020-08-31 18:44:40,425+0200 INFO (periodic/0) [vdsm.api] FINISH repoStats return={'9992dc21-edf2-4951-9020-7c78f1220e02': {'code': 0, 'lastCheck': '1.6', 'delay': '0.00283651', 'valid': True, 'version': 5, 'acquired': True, 'actual': True}, '25d95783-44df-4fda-b642-55fe09162149': {'code': 0, 'lastCheck': '2.2', 'delay': '0.00256944', 'valid': True, 'version': 5, 'acquired': True, 'actual': True}, '148d9e9e-d7b8-4220-9ee8-057da96e608c': {'code': 0, 'lastCheck': '2.2', 'delay': '0.00050217', 'valid': True, 'version': 5, 'acquired': True, 'actual': True}} from=internal, task_id=7a293dec-85b3-4b82-92b7-4e7d03b40343 (api:54) 2020-08-31 18:44:41,121+0200 INFO (jsonrpc/7) [jsonrpc.JsonRpcServer] RPC call Host.ping2 succeeded in 0.00 seconds (__init__:312) 2020-08-31 18:44:41,412+0200 INFO (jsonrpc/0) [jsonrpc.JsonRpcServer] RPC call Host.ping2 succeeded in 0.00 seconds (__init__:312) 2020-08-31 18:44:41,414+0200 INFO (jsonrpc/2) [vdsm.api] START repoStats(domains=['9992dc21-edf2-4951-9020-7c78f1220e02']) from=::1,44782, task_id=657c820e-e2eb-4b1d-b2fb-1772b3af6f32 (api:48) 2020-08-31 18:44:41,414+0200 INFO (jsonrpc/2) [vdsm.api] FINISH repoStats return={'9992dc21-edf2-4951-9020-7c78f1220e02': {'code': 0, 'lastCheck': '2.6', 'delay': '0.00283651', 'valid': True, 'version': 5, 'acquired': True, 'actual': True}} from=::1,44782, task_id=657c820e-e2eb-4b1d-b2fb-1772b3af6f32 (api:54) 2020-08-31 18:44:41,414+0200 INFO (jsonrpc/2) [jsonrpc.JsonRpcServer] RPC call Host.getStorageRepoStats succeeded in 0.00 seconds (__init__:312) 2020-08-31 18:44:41,965+0200 INFO (jsonrpc/3) [IOProcessClient] (Global) Starting client (__init__:308) 2020-08-31 18:44:41,984+0200 INFO (ioprocess/87332) [IOProcess] (Global) Starting ioprocess (__init__:434) 2020-08-31 18:44:42,006+0200 INFO (jsonrpc/3) [storage.StorageDomainCache] Removing domain fe9fb0db-2743-457a-80f0-9a4edc509e9d from storage domain cache (sdc:211) 2020-08-31 18:44:42,006+0200 INFO (jsonrpc/3) [storage.StorageDomainCache] Invalidating storage domain cache (sdc:74) 2020-08-31 18:44:42,006+0200 INFO (jsonrpc/3) [vdsm.api] FINISH connectStorageServer return={'statuslist': [{'id': '00000000-0000-0000-0000-000000000000', 'status': 0}]} from=::ffff:192.168.0.87,40378, flow_id=05fe72ef-c8ac-4e03-8453-5171e5fc5f8b, task_id=b92d4144-a9db-4423-ba6b-35934d4f9200 (api:54) 2020-08-31 18:44:42,008+0200 INFO (jsonrpc/3) [jsonrpc.JsonRpcServer] RPC call StoragePool.connectStorageServer succeeded in 3.11 seconds (__init__:312) 2020-08-31 18:44:42,063+0200 INFO (jsonrpc/5) [vdsm.api] START getStorageDomainsList(spUUID='00000000-0000-0000-0000-000000000000', domainClass=3, storageType='', remotePath='/dev/mapper/vdo1', options=None) from=::ffff:192.168.0.87,40378, flow_id=eff548f9-b663-45dd-b8a5-5854f9b5dde8, task_id=98ca66a5-edee-40d4-9253-6a46409241cc (api:48) 2020-08-31 18:44:42,063+0200 INFO (jsonrpc/5) [storage.StorageDomainCache] Refreshing storage domain cache (resize=True) (sdc:80) 2020-08-31 18:44:42,063+0200 INFO (jsonrpc/5) [storage.ISCSI] Scanning iSCSI devices (iscsi:442) 2020-08-31 18:44:42,101+0200 INFO (jsonrpc/1) [jsonrpc.JsonRpcServer] RPC call Host.ping2 succeeded in 0.00 seconds (__init__:312) 2020-08-31 18:44:42,128+0200 INFO (jsonrpc/5) [storage.ISCSI] Scanning iSCSI devices: 0.06 seconds (utils:390) 2020-08-31 18:44:42,129+0200 INFO (jsonrpc/5) [storage.HBA] Scanning FC devices (hba:60) 2020-08-31 18:44:42,182+0200 INFO (jsonrpc/6) [jsonrpc.JsonRpcServer] RPC call Host.ping2 succeeded in 0.00 seconds (__init__:312) 2020-08-31 18:44:42,184+0200 INFO (jsonrpc/4) [vdsm.api] START repoStats(domains=['9992dc21-edf2-4951-9020-7c78f1220e02']) from=::1,44782, task_id=7b3f1eb8-f7e3-4467-93a1-27ef9721c90c (api:48) 2020-08-31 18:44:42,184+0200 INFO (jsonrpc/4) [vdsm.api] FINISH repoStats return={'9992dc21-edf2-4951-9020-7c78f1220e02': {'code': 0, 'lastCheck': '3.4', 'delay': '0.00283651', 'valid': True, 'version': 5, 'acquired': True, 'actual': True}} from=::1,44782, task_id=7b3f1eb8-f7e3-4467-93a1-27ef9721c90c (api:54) 2020-08-31 18:44:42,184+0200 INFO (jsonrpc/4) [jsonrpc.JsonRpcServer] RPC call Host.getStorageRepoStats succeeded in 0.00 seconds (__init__:312) 2020-08-31 18:44:42,277+0200 INFO (jsonrpc/5) [storage.HBA] Scanning FC devices: 0.15 seconds (utils:390) 2020-08-31 18:44:42,309+0200 INFO (jsonrpc/5) [storage.Multipath] Resizing multipath devices (multipath:104) 2020-08-31 18:44:42,312+0200 INFO (jsonrpc/5) [storage.Multipath] Resizing multipath devices: 0.00 seconds (utils:390) 2020-08-31 18:44:42,312+0200 INFO (jsonrpc/5) [storage.StorageDomainCache] Refreshing storage domain cache: 0.25 seconds (utils:390) 2020-08-31 18:44:42,418+0200 INFO (tmap-569/0) [IOProcessClient] (_dev_mapper_vdo1) Starting client (__init__:308) 2020-08-31 18:44:42,445+0200 INFO (tmap-569/1) [IOProcessClient] (glusterSD/nucvirt.mtk.hoberg.net:_data) Starting client (__init__:308) 2020-08-31 18:44:42,449+0200 INFO (ioprocess/87496) [IOProcess] (_dev_mapper_vdo1) Starting ioprocess (__init__:434) 2020-08-31 18:44:42,457+0200 INFO (tmap-569/2) [IOProcessClient] (glusterSD/nucvirt.mtk.hoberg.net:_vmstore) Starting client (__init__:308) 2020-08-31 18:44:42,465+0200 INFO (ioprocess/87508) [IOProcess] (glusterSD/nucvirt.mtk.hoberg.net:_data) Starting ioprocess (__init__:434) 2020-08-31 18:44:42,478+0200 INFO (tmap-569/3) [IOProcessClient] (glusterSD/nucvirt.mtk.hoberg.net:_engine) Starting client (__init__:308) 2020-08-31 18:44:42,486+0200 INFO (ioprocess/87516) [IOProcess] (glusterSD/nucvirt.mtk.hoberg.net:_vmstore) Starting ioprocess (__init__:434) 2020-08-31 18:44:42,514+0200 INFO (ioprocess/87533) [IOProcess] (glusterSD/nucvirt.mtk.hoberg.net:_engine) Starting ioprocess (__init__:434) 2020-08-31 18:44:42,522+0200 INFO (jsonrpc/5) [IOProcessClient] (fe9fb0db-2743-457a-80f0-9a4edc509e9d) Starting client (__init__:308) 2020-08-31 18:44:42,553+0200 INFO (ioprocess/87552) [IOProcess] (fe9fb0db-2743-457a-80f0-9a4edc509e9d) Starting ioprocess (__init__:434) 2020-08-31 18:44:42,582+0200 ERROR (jsonrpc/5) [storage.HSM] Unexpected error (hsm:2843) Traceback (most recent call last): File "/usr/lib/python3.6/site-packages/vdsm/storage/hsm.py", line 2829, in getStorageDomainsList dom = sdCache.produce(sdUUID=sdUUID) File "/usr/lib/python3.6/site-packages/vdsm/storage/sdc.py", line 115, in produce domain.getRealDomain() File "/usr/lib/python3.6/site-packages/vdsm/storage/sdc.py", line 51, in getRealDomain return self._cache._realProduce(self._sdUUID) File "/usr/lib/python3.6/site-packages/vdsm/storage/sdc.py", line 139, in _realProduce domain = self._findDomain(sdUUID) File "/usr/lib/python3.6/site-packages/vdsm/storage/sdc.py", line 156, in _findDomain return findMethod(sdUUID) File "/usr/lib/python3.6/site-packages/vdsm/storage/nfsSD.py", line 146, in findDomain return NfsStorageDomain(NfsStorageDomain.findDomainPath(sdUUID)) File "/usr/lib/python3.6/site-packages/vdsm/storage/fileSD.py", line 390, in __init__ manifest.block_size, storage_block_size) File "/usr/lib/python3.6/site-packages/vdsm/storage/sd.py", line 939, in _validate_storage_block_size raise se.StorageDomainBlockSizeMismatch(block_size, storage_block_size) vdsm.storage.exception.StorageDomainBlockSizeMismatch: Block size does not match storage block size: 'block_size=512, storage_block_size=4096' 2020-08-31 18:44:42,582+0200 INFO (jsonrpc/5) [vdsm.api] FINISH getStorageDomainsList return={'domlist': []} from=::ffff:192.168.0.87,40378, flow_id=eff548f9-b663-45dd-b8a5-5854f9b5dde8, task_id=98ca66a5-edee-40d4-9253-6a46409241cc (api:54) 2020-08-31 18:44:42,582+0200 INFO (jsonrpc/5) [jsonrpc.JsonRpcServer] RPC call Host.getStorageDomains succeeded in 0.52 seconds (__init__:312) 2020-08-31 18:44:42,801+0200 INFO (jsonrpc/7) [vdsm.api] START disconnectStorageServer(domType=6, spUUID='00000000-0000-0000-0000-000000000000', conList=[{'password': '********', 'vfs_type': 'xfs', 'port': '', 'mnt_options': 'rw,relatime,seclabel,attr2,inode64,noquota,_netdev,x-systemd.device-timeout=0', 'iqn': '', 'connection': '/dev/mapper/vdo1', 'ipv6_enabled': 'false', 'id': 'b4a73d6a-344c-446b-a33d-7b665e78578d', 'user': '', 'tpgt': '1'}], options=None) from=::ffff:192.168.0.87,40378, flow_id=6678917a-b29c-416c-aa2d-5abc1d5351a3, task_id=3d0cf36e-427d-4000-8857-a8a2af641b10 (api:48) 2020-08-31 18:44:42,801+0200 INFO (jsonrpc/7) [storage.Mount] unmounting /rhev/data-center/mnt/_dev_mapper_vdo1 (mount:215) 2020-08-31 18:44:42,806+0200 INFO (jsonrpc/0) [vdsm.api] START disconnectStorageServer(domType=6, spUUID='00000000-0000-0000-0000-000000000000', conList=[{'password': '********', 'vfs_type': 'xfs', 'port': '', 'mnt_options': 'rw,relatime,seclabel,attr2,inode64,noquota,_netdev,x-systemd.device-timeout=0', 'iqn': '', 'connection': '/dev/mapper/vdo1', 'ipv6_enabled': 'false', 'id': 'b4a73d6a-344c-446b-a33d-7b665e78578d', 'user': '', 'tpgt': '1'}], options=None) from=::ffff:192.168.0.87,40378, flow_id=bb449405-5b78-4b65-a352-e2e94a269df9, task_id=09783e0a-933c-448d-b0ca-1b5d351eb042 (api:48) 2020-08-31 18:44:42,806+0200 INFO (jsonrpc/0) [storage.Mount] unmounting /rhev/data-center/mnt/_dev_mapper_vdo1 (mount:215) 2020-08-31 18:44:42,821+0200 ERROR (jsonrpc/0) [storage.HSM] Could not disconnect from storageServer (hsm:2528) Traceback (most recent call last): File "/usr/lib/python3.6/site-packages/vdsm/storage/hsm.py", line 2524, in disconnectStorageServer conObj.disconnect() File "/usr/lib/python3.6/site-packages/vdsm/storage/storageServer.py", line 201, in disconnect self._mount.umount(True, True) File "/usr/lib/python3.6/site-packages/vdsm/storage/mount.py", line 217, in umount umount(self.fs_file, force=force, lazy=lazy, freeloop=freeloop) File "/usr/lib/python3.6/site-packages/vdsm/common/supervdsm.py", line 56, in __call__ return callMethod() File "/usr/lib/python3.6/site-packages/vdsm/common/supervdsm.py", line 54, in <lambda> **kwargs) File "<string>", line 2, in umount File "/usr/lib64/python3.6/multiprocessing/managers.py", line 772, in _callmethod raise convert_to_error(kind, result) vdsm.storage.mount.MountError: Command ['/usr/bin/umount', '-f', '-l', '/rhev/data-center/mnt/_dev_mapper_vdo1'] failed with rc=32 out=b'' err=b'umount: /rhev/data-center/mnt/_dev_mapper_vdo1: not mounted.\n' 2020-08-31 18:44:42,821+0200 INFO (jsonrpc/0) [storage.StorageDomainCache] Refreshing storage domain cache (resize=False) (sdc:80)

2 2

Promiscuous Mode
by Anton Louw 31 Aug '20

31 Aug '20

Hi All, I would like to deploy a VM that will monitor a specific network. I know in VMware you can set the port group to allow Promiscuous Mode. Is there something similar I can do in oVirt? Thanks Anton Louw Cloud Engineer: Storage and Virtualization ______________________________________ D: 087 805 1572 | M: N/A A: Rutherford Estate, 1 Scott Street, Waverley, Johannesburg anton.louw(a)voxtelecom.co.za www.vox.co.za

1 0

Error exporting into ova
by Gianluca Cecchi 31 Aug '20

31 Aug '20

Hello, I'm playing with export_vm_as_ova.py downloaded from the examples github: https://github.com/oVirt/ovirt-engine-sdk/blob/master/sdk/examples/export_v… My environment is oVirt 4.3.3.7 with iSCSI storage domain. It fails leaving an ova.tmp file In webadmin gui: Starting to export Vm enginecopy1 as a Virtual Appliance 7/19/1911:55:12 AM VDSM ov301 command TeardownImageVDS failed: Cannot deactivate Logical Volume: ('General Storage Exception: ("5 [] [\' Logical volume fa33df49-b09d-4f86-9719-ede649542c21/0420ef47-0ad0-4cf9-babd-d89383f7536b in use.\']\\nfa33df49-b09d-4f86-9719-ede649542c21/[\'a7480dc5-b5ca-4cb3-986d-77bc12165be4\', \'0420ef47-0ad0-4cf9-babd-d89383f7536b\']",)',) 7/19/1912:25:36 PM Failed to export Vm enginecopy1 as a Virtual Appliance to path /save_ova/base/dump/myvm2.ova on Host ov301 7/19/1912:25:37 PM During export I have this qemu-img process creating the disk over the loop device: root 30878 30871 0 11:55 pts/2 00:00:00 su -p -c qemu-img convert -T none -O qcow2 '/rhev/data-center/mnt/blockSD/fa33df49-b09d-4f86-9719-ede649542c21/images/59a4a324-4c99-4ff5-abb1-e9bbac83292a/0420ef47-0ad0-4cf9-babd-d89383f7536b' '/dev/loop1' vdsm vdsm 30882 30878 10 11:55 ? 00:00:00 qemu-img convert -T none -O qcow2 /rhev/data-center/mnt/blockSD/fa33df49-b09d-4f86-9719-ede649542c21/images/59a4a324-4c99-4ff5-abb1-e9bbac83292a/0420ef47-0ad0-4cf9-babd-d89383f7536b /dev/loop1 The ova.tmp file is getting filled while command runs eg: [root@ov301 ]# du -sh /save_ova/base/dump/myvm2.ova.tmp 416M /save_ova/base/dump/myvm2.ova.tmp [root@ov301 sysctl.d]# [root@ov301 sysctl.d]# du -sh /save_ova/base/dump/myvm2.ova.tmp 911M /save_ova/base/dump/myvm2.ova.tmp [root@ov301 ]# and the final generated / not completed file is in this state: [root@ov301 ]# qemu-img info /save_ova/base/dump/myvm2.ova.tmp image: /save_ova/base/dump/myvm2.ova.tmp file format: raw virtual size: 30G (32217446400 bytes) disk size: 30G [root@ov301 sysctl.d]# But I notice that the timestamp of the file is about 67 minutes after start of job and well after the notice of its failure.... [root@ov301 sysctl.d]# ll /save_ova/base/dump/ total 30963632 -rw-------. 1 root root 32217446400 Jul 19 13:02 myvm2.ova.tmp [root@ov301 sysctl.d]# [root@ov301 sysctl.d]# du -sh /save_ova/base/dump/myvm2.ova.tmp 30G /save_ova/base/dump/myvm2.ova.tmp [root@ov301 sysctl.d]# In engine.log the first error I see is 30 minutes after start 2019-07-19 12:25:31,563+02 ERROR [org.ovirt.engine.core.common.utils.ansible.AnsibleExecutor] (EE-ManagedThreadFactory-engineScheduled-Thread-64) [2001ddf4] Ansible playbook execution failed: Timeout occurred while executing Ansible playbook. 2019-07-19 12:25:31,563+02 INFO [org.ovirt.engine.core.common.utils.ansible.AnsibleExecutor] (EE-ManagedThreadFactory-engineScheduled-Thread-64) [2001ddf4] Ansible playbook command has exited with value: 1 2019-07-19 12:25:31,564+02 ERROR [org.ovirt.engine.core.bll.CreateOvaCommand] (EE-ManagedThreadFactory-engineScheduled-Thread-64) [2001ddf4] Failed to create OVA. Please check logs for more details: /var/log/ovirt-engine/ova/ovirt-export-ova-ansible-20190719115531-ov301-2001ddf4.log 2019-07-19 12:25:31,565+02 INFO [org.ovirt.engine.core.vdsbroker.vdsbroker.TeardownImageVDSCommand] (EE-ManagedThreadFactory-engineScheduled-Thread-64) [2001ddf4] START, TeardownImageVDSCommand(HostName = ov301, ImageActionsVDSCommandParameters:{hostId='8ef1ce6f-4e38-486c-b3a4-58235f1f1d06'}), log id: 3d2246f7 2019-07-19 12:25:36,569+02 ERROR [org.ovirt.engine.core.dal.dbbroker.auditloghandling.AuditLogDirector] (EE-ManagedThreadFactory-engineScheduled-Thread-64) [2001ddf4] EVENT_ID: VDS_BROKER_COMMAND_FAILURE(10,802), VDSM ov301 command TeardownImageVDS failed: Cannot deactivate Logical Volume: ('General Storage Exception: ("5 [] [\' Logical volume fa33df49-b09d-4f86-9719-ede649542c21/0420ef47-0ad0-4cf9-babd-d89383f7536b in use.\']\\nfa33df49-b09d-4f86-9719-ede649542c21/[\'a7480dc5-b5ca-4cb3-986d-77bc12165be4\', \'0420ef47-0ad0-4cf9-babd-d89383f7536b\']",)',) In ansible playbook suggested log file I don't see anything useful. It ends with timestamps when the script has been launched. Last lines are: 2019-07-19 11:55:33,877 p=5699 u=ovirt | TASK [ovirt-ova-export-pre-pack : Retrieving the temporary path for the OVA file] *** 2019-07-19 11:55:34,198 p=5699 u=ovirt | changed: [ov301] => { "changed": true, "dest": "/save_ova/base/dump/myvm2.ova.tmp", "gid": 0, "group": "root", "mode": "0600", "owner": "root", "secontext": "system_u:object_r:nfs_t:s0", "size": 32217446912, "state": "file", "uid": 0 } 2019-07-19 11:55:34,204 p=5699 u=ovirt | TASK [ovirt-ova-pack : Run packing script] ************************************* It seems 30 minutes... for timeout? About what, ansible job? Or possibly implicit user session created when running the python script? The snapshot has been correctly deleted (as I see also in engine.log), I don't see it in webadmin gui. Any known problem? Just for test I executed again at 14:24 and I see same Ansible error at 14:54 The snapshot gets deleted, while the qemu-img command still continues.... [root@ov301 sysctl.d]# ps -ef | grep qemu-img root 13504 13501 0 14:24 pts/1 00:00:00 su -p -c qemu-img convert -T none -O qcow2 '/rhev/data-center/mnt/blockSD/fa33df49-b09d-4f86-9719-ede649542c21/images/59a4a324-4c99-4ff5-abb1-e9bbac83292a/0420ef47-0ad0-4cf9-babd-d89383f7536b' '/dev/loop0' vdsm vdsm 13505 13504 3 14:24 ? 00:01:26 qemu-img convert -T none -O qcow2 /rhev/data-center/mnt/blockSD/fa33df49-b09d-4f86-9719-ede649542c21/images/59a4a324-4c99-4ff5-abb1-e9bbac83292a/0420ef47-0ad0-4cf9-babd-d89383f7536b /dev/loop0 root 17587 24530 0 15:05 pts/0 00:00:00 grep --color=auto qemu-img [root@ov301 sysctl.d]# [root@ov301 sysctl.d]# du -sh /save_ova/base/dump/myvm2.ova.tmp 24G /save_ova/base/dump/myvm2.ova.tmp [root@ov301 sysctl.d]# ll /save_ova/base/dump/myvm2.ova.tmp -rw-------. 1 root root 32217446400 Jul 19 15:14 /save_ova/base/dump/myvm2.ova.tmp [root@ov301 sysctl.d]# and then continues until image copy completes, but at this time the job has already aborted and so the completion of the ova composition doesn't go ahead... and I remain with the ova.tmp file... How to extend timeout? Thanks in advance, Gianluca

4 11

Hosted Engine stuck in Firmware
by Vinícius Ferrão 30 Aug '20

30 Aug '20

Hello, I’ve an strange issue with oVirt 4.4.1 The hosted engine is stuck in the UEFI firmware trying to “never” boot. I think this happened when I changed the default VM mode for the cluster inside the datacenter. There’s a way to fix this without redeploying the engine?

3 3

deprecating export domain?
by Charles Kozler 30 Aug '20

30 Aug '20

Hello, I recently read on this list from a redhat member that export domain is either being deprecated or looking at being deprecated To that end, can you share details? Can you share any notes/postings/bz's that document this? I would imagine something like this would be discussed in larger audience This seems like a somewhat significant change to make and I am curious where this is scheduled? Currently, a lot of my backups rely explicitly on an export domain for online snapshots, so I'd like to plan accordingly Thanks!

11 21

shutdown virtual machine did not display in the list when using virsh list --all and virsh list --state-shutoff
by Kyre 30 Aug '20

30 Aug '20

Hello, I am using ovirt 4.3.8. I want to use the virsh command to boot the shutdown virtual machine, but when using virsh list --all and virsh list --state-shutoff, the shutdown virtual machine did not display in the list. What is the cause of this, and how can I display the shutdown virtual machine. virsh # list --all Please enter your authentication name: root Please enter your password:  Id    Name                           State ----------------------------------------------------  4     win10E2                  running virsh # list --state-shutoff  Id    Name                           State ----------------------------------------------------

4 3

Ovirt Node 4.4.2 Engine Deployment - Notification Settings
by David White 30 Aug '20

30 Aug '20

I finally got oVirt node installed with gluster on a single node. So that's great progress! Once that step was complete... I noticed that the Engine Deployment wizard asks for SMTP settings for where to send notifications. I was kind of surprised that it doesn't allow one to enter any credentials. It looks like this requires an unauthenticated local relay. I don't like that. :) See attached screenshot. Has there been any talk about adding this into the wizard deployment? Sent with ProtonMail Secure Email.

2 1

i/o wait and slow system
by info＠dsdm.ch 29 Aug '20

29 Aug '20

Hello, I'm running a home setup with 3 nodes and 2 SATA SSDs per node. As storage I'm running glusterfs and 40GBit/s links. Software Version:4.3.9.4-1.el7 I've a lot of I/O Wait on the nodes (20%) and on the VMs (50%). gluster volume top vmstore write-perf bs 2014 count 1024 | grep Through Throughput 635.54 MBps time 0.0032 secs Throughput 614.89 MBps time 0.0034 secs Throughput 622.31 MBps time 0.0033 secs Throughput 643.07 MBps time 0.0032 secs Throughput 621.75 MBps time 0.0033 secs Throughput 609.26 MBps time 0.0034 secs gluster volume top vmstore read-perf bs 2014 count 1024 | grep Through Throughput 1274.62 MBps time 0.0016 secs Throughput 1320.32 MBps time 0.0016 secs Throughput 1203.93 MBps time 0.0017 secs Throughput 1293.81 MBps time 0.0016 secs Throughput 1213.14 MBps time 0.0017 secs Throughput 1193.48 MBps time 0.0017 secs Volume Name: vmstore Type: Distributed-Replicate Volume ID: 195e2a05-9667-4b8b-b0b7-82294631de50 Status: Started Snapshot Count: 0 Number of Bricks: 2 x 3 = 6 Transport-type: tcp Bricks: Brick1: 10.9.9.101:/gluster_bricks/vmstore/vmstore Brick2: 10.9.9.102:/gluster_bricks/vmstore/vmstore Brick3: 10.9.9.103:/gluster_bricks/vmstore/vmstore Brick4: 10.9.9.101:/gluster_bricks/S4CYNF0M219849L/S4CYNF0M219849L Brick5: 10.9.9.102:/gluster_bricks/S4CYNF0M219836L/S4CYNF0M219836L Brick6: 10.9.9.103:/gluster_bricks/S4CYNF0M219801Y/S4CYNF0M219801Y Options Reconfigured: performance.client-io-threads: on nfs.disable: on transport.address-family: inet performance.strict-o-direct: on performance.quick-read: off performance.read-ahead: off performance.io-cache: off performance.low-prio-threads: 32 network.remote-dio: off cluster.eager-lock: enable cluster.quorum-type: auto cluster.server-quorum-type: server cluster.data-self-heal-algorithm: full cluster.locking-scheme: granular cluster.shd-max-threads: 8 cluster.shd-wait-qlength: 10000 features.shard: on user.cifs: off cluster.choose-local: off client.event-threads: 4 server.event-threads: 4 network.ping-timeout: 30 storage.owner-uid: 36 storage.owner-gid: 36 cluster.granular-entry-heal: enable Please help me to analyse the root cause. Many thanks Metz

4 14

Missing model_FLAGS on specific host
by Vinícius Ferrão 28 Aug '20

28 Aug '20

Hi, I’ve an strange issue in one of my hosts, it’s missing a lot of CPU flags that oVirt seems to require: [root@c4140 ~]# vdsm-client Host getCapabilities | egrep "cpuFlags|cpuModel" "cpuFlags": "ssse3,mca,ept,pdpe1gb,vmx,clwb,smep,msr,acpi,pge,sse4_2,nopl,cqm_mbm_total,cx16,avx512vl,aperfmperf,xsaves,3dnowprefetch,nonstop_tsc,cmov,mce,intel_pt,avx512f,fpu,pku,tsc,sdbg,erms,pse36,md_clear,apic,sse,pcid,clflushopt,xtopology,pts,monitor,vpid,cpuid,hle,mba,ss,cqm,avx2,ibpb,xgetbv1,flush_l1d,mmx,epb,pti,fxsr,dca,nx,syscall,stibp,mtrr,cx8,sse2,avx,sep,intel_ppin,lm,tm,bts,adx,bmi1,smx,popcnt,pclmulqdq,lahf_lm,mpx,rdseed,cqm_llc,avx512cd,cdp_l3,f16c,invpcid,fsgsbase,cpuid_fault,tm2,smap,dts,pse,xsave,sse4_1,constant_tsc,pat,tsc_deadline_timer,vnmi,avx512dq,dtes64,xsaveopt,ida,pdcm,tpr_shadow,pln,de,x2apic,avx512bw,pae,rdrand,clflush,rdtscp,art,cqm_mbm_local,pebs,ssbd,movbe,pbe,tsc_adjust,vme,ht,est,bmi2,cat_l3,dtherm,ospke,rdt_a,aes,ibrs,rep_good,fma,xtpr,ds_cpl,abm,xsavec,invpcid_single,flexpriority,cqm_occup_llc,pni,rtm,arat,arch_perfmon", "cpuModel": "Intel(R) Xeon(R) Gold 6130 CPU @ 2.10GHz", In a properly working host it have flags like those ones: model_Westmere-IBRS,model_kvm32,model_core2duo,model_Opteron_G1,model_Broadwell,model_qemu64,model_Broadwell-noTSX,model_Nehalem-IBRS,model_Haswell-IBRS,model_pentium2,model_Broadwell-IBRS,model_Haswell-noTSX,model_Haswell,model_Haswell-noTSX-IBRS,model_Conroe,model_pentium,model_n270,model_Nehalem,model_IvyBridge-IBRS,model_kvm64,model_SandyBridge,model_pentium3,model_Broadwell-noTSX-IBRS,model_qemu32,model_486,model_IvyBridge,model_SandyBridge-IBRS,model_Westmere,model_Penryn,model_Opteron_G2,model_coreduo", But in this machine it’s totally missing. I know this model_ flags are an oVirt thing, since they aren’t default on the CPUs. The host machine is a Dell C4140 compute node, the firmware is fully updated, so I’ve done the basic to figure out what’s happening. Thanks,

1 1

ovirt 4.4 self-hosted deployment questions
by Michael Thomas 28 Aug '20

28 Aug '20

I have not been able to find answers to a couple of questions in the self-hosted engine documentation[1]. * When installing a new Enterprise Linux host for ovirt, what are the network requirements? Specifically, am I supposed to set up the ovirtmgmt bridge myself on new hosts, or am I supposed to let that be handled by the engine when I add the new host to the engine? * In the 'New Host' dialog on the engine management page, does the Hostname/IP that I enter have to be the host's name on the ovirtmgmt LAN? If so, then it seems to me that I need to configure the ovirtmgmt bridge myself on new hosts. * Does the engine need to be able to route outside of the cluster (eg to the WAN), or is it allowed to restrict the engine's routing to the local cluster? * In the 'New Host' dialog on the engine management page, what is the meaning of 'Choose hosted engine deployment action'? From the way it is phrased, it sounds like this will create a second engine in my cluster, which doesn't make sense. Or does this mean that the new host will be able to run the Engine VM in a HA manner? In my current test deployment I have 3 subnets in my cluster. Network WAN is the WAN. Network CLUSTER is for communication between cluster compute nodes, storage servers, and management servers. Network OVIRT is for ovirt management and VM migration between hosts. My first self-hosted engine host is connected to networks CLUSTER and OVIRT. The engine VM is only connected to network OVIRT through a bridge on the host, but has a gateway that lets it route traffic to network CLUSTER (but not network WAN). Is this an appropriate network setup for ovirt, or should there be no distinction between the CLUSTER and OVIRT networks? --Mike [1]https://www.ovirt.org/documentation/installing_ovirt_as_a_self-hosted_eng…

2 1

Gluster Volume Type Distributed
by Dominique Deschênes 28 Aug '20

28 Aug '20

Hi Everyone, I would like to use Distrbuted Volume type but the volume type is Gray out. I can only use the replicate type. It's normal ? 3 ovirt Servers 4.4.1-2020080418 Can I configure a replicate volume for the engine domain and Distributed for the data domain? Thank you Dominique Deschênes Ingénieur chargé de projets, Responsable TI 816, boulevard Guimond, Longueuil J4G 1T5 450 670-8383 x105 450 670-2259

4 3

Failed to connect to the hypervisor
by Eugène Ngontang 28 Aug '20

28 Aug '20

Hi, I'm facing a strange issue with ovirt host. I can't list nodes status running virsh, vdsdm seems bad. I can't even delete the hosted engine, nor ping it, but I can still access my infra vm inside it[image: 😉] My outputs : > [root@moe ~]# /usr/sbin/ovirt-hosted-engine-cleanup > This will de-configure the host to run ovirt-hosted-engine-setup from > scratch. > Caution, this operation should be used with care. > Are you sure you want to proceed? [y/n] > y > -=== Destroy hosted-engine VM ===- > You must run deploy first > setlocale: No such file or directory > error: failed to connect to the hypervisor > error: Failed to connect socket to '/var/run/libvirt/libvirt-sock': No > such file or directory > -=== Stop HA services ===- > -=== Shutdown sanlock ===- > shutdown force 1 wait 0 > shutdown done -111 > -=== Disconnecting the hosted-engine storage domain ===- > You must run deploy first > -=== De-configure VDSM networks ===- > ovirtmgmt > ovirtmgmt > A previously configured management bridge has been found on the system, > this will try to de-configure it. Under certain circumstances you can loose > network connection. > Caution, this operation should be used with care. > Are you sure you want to proceed? [y/n] > y > -=== Stop other services ===- > -=== De-configure external daemons ===- > -=== Removing configuration files ===- > ? /etc/init/libvirtd.conf already missing > ? /etc/libvirt/nwfilter/vdsm-no-mac-spoofing.xml already missing > ? /etc/ovirt-hosted-engine/answers.conf already missing > ? /etc/ovirt-hosted-engine/hosted-engine.conf already missing > ? /etc/vdsm/vdsm.conf already missing > ? /etc/pki/vdsm/*/*.pem already missing > ? /etc/pki/CA/cacert.pem already missing > ? /etc/pki/libvirt/*.pem already missing > ? /etc/pki/libvirt/private/*.pem already missing > ? /etc/pki/ovirt-vmconsole/*.pem already missing > ? /var/cache/libvirt/* already missing > ? /var/run/ovirt-hosted-engine-ha/* already missing > [root@moe ~]# ping 192.168.33.143 > PING 192.168.33.143 (192.168.33.143) 56(84) bytes of data. > 64 bytes from 192.168.33.143: icmp_seq=1 ttl=64 time=0.132 ms > 64 bytes from 192.168.33.143: icmp_seq=2 ttl=64 time=0.082 ms > 64 bytes from 192.168.33.143: icmp_seq=3 ttl=64 time=0.072 ms > 64 bytes from 192.168.33.143: icmp_seq=4 ttl=64 time=0.083 ms > 64 bytes from 192.168.33.143: icmp_seq=5 ttl=64 time=0.075 ms > 64 bytes from 192.168.33.143: icmp_seq=6 ttl=64 time=0.082 ms > ^C > --- 192.168.33.143 ping statistics --- > 6 packets transmitted, 6 received, 0% packet loss, time 4999ms > rtt min/avg/max/mdev = 0.072/0.087/0.132/0.022 ms > [root@moe ~]# nodectl check > Status: WARN > Bootloader ... OK > Layer boot entries ... OK > Valid boot entries ... OK > Mount points ... OK > Separate /var ... OK > Discard is used ... OK > Basic storage ... OK > Initialized VG ... OK > Initialized Thin Pool ... OK > Initialized LVs ... OK > Thin storage ... OK > Checking available space in thinpool ... OK > Checking thinpool auto-extend ... OK > vdsmd ... BAD > [root@moe ~]# virsh -r list > setlocale: No such file or directory > error: failed to connect to the hypervisor > error: Failed to connect socket to '/var/run/libvirt/libvirt-sock-ro': No > such file or directory > [root@moe ~]# Have you guys ever faced this issue? How can I access VM inside my virtual data center while the VM Manager is broken? Any solution to work around this please? Regards, Eugène NG -- LesCDN <http://lescdn.com> engontang(a)lescdn.com ------------------------------------------------------------ *Aux hommes il faut un chef, et au* * chef il faut des hommes!L'habit ne fait pas le moine, mais lorsqu'on te voit on te juge!*

1 0

Compatibilty issue
by carl langlois 28 Aug '20

28 Aug '20

Hi, I have upgraded ovirt to 4.3 and now I have some vm that I can start because of the cluster compatibility version. Those VM where in save state when I updated the cluster compatibility version. Now they are in a down state but if i try to run it I get this message. Cannot run VM. The Custom Compatibility Version of VM vmfpgatest (4.2) is not supported in Data Center compatibility version 4.3. any suggestions? Regards Carl

1 0

TroubleshootingoVirt Node deploy FQDN not reachable
by David White 27 Aug '20

27 Aug '20

In a recent thread, Roberto mentioned seeing the error message "FQDN Not Reachable" when trying to deploy oVirt Node 4.4.1, but was able to get past that error when using ovirt-node-ng-installer-4.4.2-2020080612.el8.iso. I experienced the same problems on oVirt Node install 4.4.1, so I tried the latest release of 4.4.2. When that failed, I went back and installed from the exact same image as Roberto said worked on the 4.4.2 branch: ovirt-node-ng-installer-4.4.2-2020080612.el8.iso Unfortunately, that's still not working for me - so that tells me I'm probably doing something wrong. Given the following facts: [root@dev1-centos ~]# hostname dev1-centos.office.barredowlweb.com [root@dev1-centos ~]# host dev1-centos.office.barredowlweb.com dev1-centos.office.barredowlweb.com has address 192.168.2.96 I am trying to install oVirt using the Hyperconverged Gluster Wizard for a Single Node. In the "Host1" box, I enter the full hostname: dev1-centos.office.barredowlweb.com And that's when I get the FQDN error message. Am I missing something here? Sent with ProtonMail Secure Email.

4 5

Is the Hosted Engine setup finished + Can't connect vdsm storage
by Bielej SRE 27 Aug '20

27 Aug '20

Hi, We have 6 node oVirt setup with Compatibility version 4.3 running on CentOS 7.6.1810 Recently We have found out some very interesting log entries on a few of our nodes. /var/log/ovirt-hosted-engine-ha/broker.log: MainThread::INFO::2020-08-27 12:51:50,279::storage_backends::345::ovirt_hosted_engine_ha.lib.storage_backends::(connect) Connecting the storage MainThread::INFO::2020-08-27 12:51:50,280::storage_server::349::ovirt_hosted_engine_ha.lib.storage_server.StorageServer::(connect_storage_server) Connecting storage server MainThread::warning::2020-08-27 12:51:50,284::storage_broker::97::ovirt_hosted_engine_ha.broker.storage_broker.StorageBroker::(__init__) Can't connect vdsm storage: 'NoneType' object has no attribute 'startswith' /var/log/messages Aug 27 12:52:11 ildeco25 vdsm[6909]: ERROR failed to retrieve Hosted Engine HA score '[Errno 2] No such file or directory'Is the Hosted Engine setup finished? We have inherited that setup from the previous team but I'm fairly certain the the Hosted Engine setup is finished. Also, this line is not present on all hosts. Should We worry about ovirt-ha-broker service not running on some hosts? host24 Active: inactive (dead) host25 Active: failed (Result: start-limit) since Thu 2020-08-27 12:56:13 CEST; 2s ago host26 Active: active (running) since Sun 2020-08-23 22:55:01 CEST; 3 days ago host27 Active: inactive (dead) host28 Active: failed (Result: start-limit) since Thu 2020-08-27 12:56:18 CEST; 4s ago host29 Active: inactive (dead) Currently the hosted engine VM is running on host26. I'd be very grateful for any help. Kind regards, Michal Bielejewski

2 1

Impossible to install Windows VMs on 4.4.1
by fgarat＠gmail.com 27 Aug '20

27 Aug '20

Hi, I'm having problem after I upgraded to 4.4.1 with Windows machines. The installation sees no disk. Even IDE disk doesn't get detected and installation won't move forward no matter what driver i use for the disk. Any one else having this issue?. Regards, Facundo

3 4

Host has no default route
by Wesley Stewart 27 Aug '20

27 Aug '20

I am trying to add a host to a single host cluster. Everything has gone pretty well so far, except when the host gets to the end of the installation process, I see an exclamation mark indicating "Host has no default route". The installation "finishes" and the host goes into a non-responsive state. I am not really sure where to check. I get a: host *name* installation has failed. Network error during communication with host. Running 4.3.6 Thanks!

2 1

[ANN] oVirt 4.4.2 Fifth Release Candidate is now available for testing
by Lev Veyde 27 Aug '20

27 Aug '20

oVirt 4.4.2 Fifth Release Candidate is now available for testing The oVirt Project is pleased to announce the availability of oVirt 4.4.2 Fifth Release Candidate for testing, as of August 27th, 2020. This update is the second in a series of stabilization updates to the 4.4 series. Important notes before you try it Please note this is a pre-release build. The oVirt Project makes no guarantees as to its suitability or usefulness. This pre-release must not be used in production. Installation instructions For installation instructions and additional information please refer to: https://ovirt.org/documentation/ This release is available now on x86_64 architecture for: * Red Hat Enterprise Linux 8.2 or newer * CentOS Linux (or similar) 8.2 or newer This release supports Hypervisor Hosts on x86_64 and ppc64le architectures for: * Red Hat Enterprise Linux 8.2 or newer * CentOS Linux (or similar) 8.2 or newer * oVirt Node 4.4 based on CentOS Linux 8.2 (available for x86_64 only) See the release notes [1] for installation instructions and a list of new features and bugs fixed. Notes: - oVirt Appliance is already available for CentOS Linux 8 - oVirt Node NG is already available for CentOS Linux 8 Additional Resources: * Read more about the oVirt 4.4.2 release highlights: http://www.ovirt.org/release/4.4.2/ * Get more oVirt project updates on Twitter: https://twitter.com/ovirt * Check out the latest project news on the oVirt blog: http://www.ovirt.org/blog/ [1] http://www.ovirt.org/release/4.4.2/ [2] http://resources.ovirt.org/pub/ovirt-4.4-pre/iso/ -- Lev Veyde Senior Software Engineer, RHCE | RHCVA | MCITP Red Hat Israel <https://www.redhat.com> lev(a)redhat.com | lveyde(a)redhat.com <https://red.ht/sig> TRIED. TESTED. TRUSTED. <https://redhat.com/trusted>

3 3

LDAP/AD issue
by kim.kargaard＠noroff.no 26 Aug '20

26 Aug '20

Hi all, We have had our ovirt instance connected to our internal AD for users to log into the VM portal for the last year, linked to studentdomene.noroff.no. This has been working without any problems. We had it set up and the DNS server had a forward record to the DC's. All good. Then, of course, the institution decided to introduce student emails and they decided to add the domain stud.noroff.no for student emails and made this the primary domain in the AD. The problem is that when this is changed, students can no longer log into the engine. I have of course changed the ldap settings and added a forward record on the DNS to the new domain. However, it seems that the domain is studentdomene.noroff.no, but with an added UPN suffix with stud.noroff.no When students try to log in, with the config changes, they get this error in the browser: server_error: An error occurred while attempting to query DNS in order to retrieve SRV records with name '_ldap._tcp.stud.noroff.no': NameNotFoundException(DNS name not found [response code 3]), ldapSDKVersion=4.0.7, revision=b28fb50058dfe2864171df2448ad2ad2b4c2ad58 Any ideas on how to solve this issue? My config looks like this: sudo cat /etc/ovirt-engine/aaa/Students.properties [sudo] password for noroffadmin: include = <ad.properties> vars.domain = studentdomene.noroff.no vars.user = CN=ovirt auth,CN=Users,DC=stud,DC=noroff,DC=no vars.password = PASSWORD pool.default.auth.simple.bindDN = ${global:vars.user} pool.default.auth.simple.password = ${global:vars.password} pool.default.serverset.type = srvrecord pool.default.serverset.srvrecord.domain = ${global:vars.domain} my forward on the DNS server looks like this: sudo cat /etc/named/named.conf.local [sudo] password for noroffadmin: zone "platform.noroff.no"{ type master; file "/etc/named/zones/db.platform.noroff.no"; # zone file path }; zone "stud.noroff.no" { type forward; forward only; forwarders { 172.24.111.20; 172.27.111.20; 172.21.111.20; 172.16.111.20; }; }; zone "studentdomene.noroff.no" { type forward; forward only; forwarders { 172.24.111.20; 172.27.111.20; 172.21.111.20; 172.16.111.20; }; }; zone "122.16.172.in-addr.arpa" { type master; file "/etc/named/zones/db.122.16.172"; # 172.16.122.0/24 subnet }; Any pointers would be greatly appreciated :) Kim

2 1

oVirt SHE smart-automated backup
by Vrgotic, Marko 26 Aug '20

26 Aug '20

Dear oVirt, For a while now I have been running backup of oVirt engine via ansible. And that works just fine. However, its still recommended, in SHE HA environment, especially for Restore purpose to have one of HA nodes in Maintenance mode, clean from Virtual load. Has anyone so far been able to automate oVirt engine backup in following or similar way: 1. Register on which HA node is SHE atm 2. Pick the other HA node to go into maintenance mode – in order to have one of HA nodes without virtual load 3. Verify it’s in maintenance mode 4. Execute engine-backup 5. Remove the maintenance mode of the node from step 2 Even if you are half way through, I would be happy to get involved and contribute. Kindly awaiting your reply. ----- kind regards/met vriendelijke groeten Marko Vrgotic Sr. System Engineer @ System Administration ActiveVideo e: m.vrgotic(a)activevideo.com<mailto:m.vrgotic@activevideo.com> w: www.activevideo.com<http://www.activevideo.com> ActiveVideo Networks BV. Mediacentrum 3745 Joop van den Endeplein 1.1217 WJ Hilversum, The Netherlands. The information contained in this message may be legally privileged and confidential. It is intended to be read only by the individual or entity to whom it is addressed or by their designee. If the reader of this message is not the intended recipient, you are on notice that any distribution of this message, in any form, is strictly prohibited. If you have received this message in error, please immediately notify the sender and/or ActiveVideo Networks, LLC by telephone at +1 408.931.9200 and delete or destroy any copy of this message.

2 4

oVirt HCI 4.4.1 DR cannot import target volume after disaster
by wodel youchi 25 Aug '20

25 Aug '20

Hi, I have installed two HCI platforms. Then I set up the disaster-recovery configuration On source platform I did : # gluster volume set all cluster.enable-shared-storage enable # gluster system:: execute gsec_create # gluster volume geo-replication *source* gnodesen2-01.example.com::dest create push-pem # gluster volume geo-replication *source* gnodesen2-01.example.com::dest config use_meta_volume true On target platform I did : # gluster volume set *dest* features.shard enable I then realized these two tests : *1 - Sync data using the schedule_georep.py* a - Use the : "python3 /usr/share/glusterfs/scripts/schedule_georep.py source gnodesen2-01.example.com dest " command to sync data between the volumes b - After the sync is complete, execute "gluster volume dest set features.read-only off" c - Execute *import* (not create) *dest* volume on the target platform manually through the Admin-UI, Result : The domain could not be added and I get this error on vdsm log : " 2020-08-24 10:18:05,644+0100 ERROR (jsonrpc/5) [storage.HSM] Unexpected error (hsm:2843) Traceback (most recent call last): File "/usr/lib/python3.6/site-packages/vdsm/storage/sd.py", line 456, in getVersion version = self.getMetaParam(DMDK_VERSION) File "/usr/lib/python3.6/site-packages/vdsm/storage/sd.py", line 452, in getMetaParam return self._metadata[key] File "/usr/lib/python3.6/site-packages/vdsm/storage/persistent.py", line 114, in __getitem__ return dec(self._dict[key]) File "/usr/lib/python3.6/site-packages/vdsm/storage/persistent.py", line 225, in __getitem__ return self._metadata[key] KeyError: 'VERSION' During handling of the above exception, another exception occurred: Traceback (most recent call last): File "/usr/lib/python3.6/site-packages/vdsm/storage/hsm.py", line 2829, in getStorageDomainsList dom = sdCache.produce(sdUUID=sdUUID) File "/usr/lib/python3.6/site-packages/vdsm/storage/sdc.py", line 115, in produce domain.getRealDomain() File "/usr/lib/python3.6/site-packages/vdsm/storage/sdc.py", line 51, in getRealDomain return self._cache._realProduce(self._sdUUID) File "/usr/lib/python3.6/site-packages/vdsm/storage/sdc.py", line 139, in _realProduce domain = self._findDomain(sdUUID) File "/usr/lib/python3.6/site-packages/vdsm/storage/sdc.py", line 156, in _findDomain return findMethod(sdUUID) File "/usr/lib/python3.6/site-packages/vdsm/storage/glusterSD.py", line 62, in findDomain return GlusterStorageDomain(GlusterStorageDomain.findDomainPath(sdUUID)) File "/usr/lib/python3.6/site-packages/vdsm/storage/fileSD.py", line 385, in __init__ manifest = self.manifestClass(domainPath) File "/usr/lib/python3.6/site-packages/vdsm/storage/fileSD.py", line 162, in __init__ sd.StorageDomainManifest.__init__(self, sdUUID, domaindir, metadata) File "/usr/lib/python3.6/site-packages/vdsm/storage/sd.py", line 352, in __init__ version = self.getVersion() * File "/usr/lib/python3.6/site-packages/vdsm/storage/sd.py", line 458, in getVersion raise se.MetaDataKeyNotFoundError("key={}".format(DMDK_VERSION))vdsm.storage.exception.MetaDataKeyNotFoundError: Meta Data key not found error: ('key=VERSION',)* 2020-08-24 10:18:05,644+0100 INFO (jsonrpc/5) [vdsm.api] FINISH getStorageDomainsList return={'domlist': []} from=::ffff:10.80.101.27,48282, flow_id=5209d576-ee09-4de3-bc46-0c186c6ad52a, task_id=bc93aaae-5449-4d0d-9884-f47ea542a963 (api:54) 2020-08-24 10:18:05,645+0100 INFO (jsonrpc/5) [jsonrpc.JsonRpcServer] RPC call Host.getStorageDomains succeeded in 3.01 seconds (__init__:312) 2020-08-24 10:18:05,754+0100 INFO (jsonrpc/4) [vdsm.api] START disconnectStorageServer(domType=7, spUUID='00000000-0000-0000-0000-000000000000', conList=[{'password': '********', 'vfs_type': 'glusterfs', 'port': '', 'mnt_options': 'backup-volfile-servers=10.90.101.23:10.90.101.25', 'iqn': '', 'connection': '10.90.101.21:/dest', 'ipv6_enabled': 'false', 'id': 'cc24df41-19b0-41ed-ae85-e091e40cc612', 'user': '', 'tpgt': '1'}], options=None) from=::ffff:10.80.101.27,48282, flow_id=5fcaf7d7-d3b6-4430-b5f9-28e6bd95656a, task_id=73654a43-c86b-4250-95d7-3e3399e5bc72 (api:48) 2020-08-24 10:18:05,754+0100 INFO (jsonrpc/4) [storage.Mount] unmounting /rhev/data-center/mnt/glusterSD/10.90.101.21:_dest (mount:215) " *Note :* I had to modify the line 105 from /usr/share/glusterfs/scripts/schedule_georep.py to be able to use it with python3 : - ey = "_".join([func.func_name] + list(args)) + key = "_".join([func.__name__] + list(args)) *2 - Sync data using the adminUI :* a - Create a schedule on the adminUI and wait until the sync is complete. b - Switch off read-only on target volume : gluster volume set dest features.read-only off (on the target volume) c - Execute ansible failover scripts : ansible-playbook dr-rhv-failover.yml --tags="fail_over" command (on the ansible machine) Result : 020-08-24 15:51:58,874+0100 INFO (jsonrpc/0) [IOProcessClient] (3a5d378b-8043-4072-9c58-e838343d6028) Starting client (init:308) 2020-08-24 15:51:58,886+0100 INFO (ioprocess/140212) [IOProcess] (3a5d378b-8043-4072-9c58-e838343d6028) Starting ioprocess (init:434) 2020-08-24 15:51:58,888+0100 INFO (jsonrpc/0)** [vdsm.api] FINISH createStorageDomain error=Storage domain is not empty - requires cleaning: ('gnodesen2-01.example.com:/dest',) from=::ffff:10.80.101.27,45118, flow_id=765c5009, task_id=ed3a34f9-5e81-40f4-a592-78e585c1f937 (api:52) 2020-08-24 15:51:58,888+0100 ERROR (jsonrpc/0) [storage.TaskManager.Task] (Task='ed3a34f9-5e81-40f4-a592-78e585c1f937') Unexpected error (task:880) Traceback (most recent call last): File "/usr/lib/python3.6/site-packages/vdsm/storage/task.py", line 887, in _run return fn(args, *kargs) File "<decorator-gen-121>", line 2, in createStorageDomain File "/usr/lib/python3.6/site-packages/vdsm/common/api.py", line 50, in method ret = func(args, *kwargs) File "/usr/lib/python3.6/site-packages/vdsm/storage/hsm.py", line 2644, in createStorageDomain max_hosts=max_hosts) File "/usr/lib/python3.6/site-packages/vdsm/storage/nfsSD.py", line 95, in create version) **File "/usr/lib/python3.6/site-packages/vdsm/storage/nfsSD.py", line 60, in _preCreateValidation *raise se.StorageDomainNotEmpty(typeSpecificArg)vdsm.storage.exception.StorageDomainNotEmpty: Storage domain is not empty - requires cleaning: ('gnodesen2-01.example.com:/dest',)2020-08-24 15:51:58,888+0100 INFO (jsonrpc/0) [storage.TaskManager.Task] (Task='ed3a34f9-5e81-40f4-a592-78e585c1f937') aborting: Task is aborted: "value=Storage domain is not empty - requires cleaning: ('gnodesen2-01.example.com:/dest',) abortedcode=361" (task:1190)* 2020-08-24 15:51:58,888+0100 ERROR (jsonrpc/0) [storage.Dispatcher] FINISH createStorageDomain error=Storage domain is not empty - requires cleaning: ('gnodesen2-01.example.com:/dest',) (dispatcher:83) 2020-08-24 15:51:58,889+0100 INFO (jsonrpc/0) [jsonrpc.JsonRpcServer] RPC* call StorageDomain.create failed *(error 361) in 0.20 seconds (init:312) 2020-08-24 15:51:58,934+0100 INFO (jsonrpc/2) [vdsm.api] START disconnectStorageServer(domType=7, spUUID='00000000-0000-0000-0000-000000000000', conList=[{'password': '****', 'vfs_type': 'glusterfs', 'port': '', 'iqn': '', 'connection': 'gnodesen2-01.example.com:/dest', 'ipv6_enabled': 'false', 'id': 'd104177f-f391-45bc-a8a7-905da90d4e7d', 'user': '', 'tpgt': '1'}], options=None) from=::ffff:10.80.101.27,45118, flow_id=f8c31837-fd32-48f2-bed9-6138ea4c6bb0, task_id=badf6391-06cd-4e11-ade5-95dd10b86af5 (api:48) 2020-08-24 15:51:58,934+0100 INFO (jsonrpc/2) [storage.Mount] unmounting /rhev/data-center/mnt/glusterSD/gnodesen2-01.example.com:_dest (mount:215) Any ideas? Regards.

1 0