January 2019 - Users - oVirt List Archives

Problems with procedure in blog post...
by Jason P. Thomas 13 Feb '19

13 Feb '19

rgolan asked me to post this here for him. I'm trying the procedure from this blog post: https://ovirt.org/blog/2019/01/ovirt-openshift-part-1.html Ultimate goal is to install ovirt metrics store. I've tried vars.yaml with and without the ovirt metrics stuff commented out. The errors seem to revolve around downloading the centos qcow2c image. With image_path: "{{ lookup('env', 'HOME') }}/Downloads/{{ template_name }}.qcow2" defined, I get the following error: fatal: [localhost]: FAILED! => {"msg": "The task includes an option with an undefined variable. The error was: 'dict object' has no attribute 'isdir'\n\nThe error appears to have been in '/usr/share/ansible/roles/ovirt.image-template/tasks/qcow2_image.yml': line 12, column 3, but may\nbe elsewhere in the file depending on the exact syntax problem.\n\nThe offending line appears to be:\n\n\n- name: Download the qcow image\n ^ here\n"} If I comment out the above image_path line and uncomment the one at the bottom of vars.yaml that specifies /var/tmp, I get this error: fatal: [localhost]: FAILED! => {"changed": false, "cmd": "/usr/bin/file /var/tmp/CentOS-7-x86_64-GenericCloud.qcow2c", "msg": "[Errno 2] No such file or directory", "rc": 2} --Jason

3 4

Issues adding iscsi storage domain
by Leo David 12 Feb '19

12 Feb '19

Hello everyone, Trying to setup an iscsi target as a storage domain, and it seems not to be possible. Discovered the hosts, the targets are displayed. Selected one target, clicked the "Login" arrow, spinner runs a bit , and the arrow gets grayed out. But no LUNs are displayed, to select from. From this step, I can't go further, if I hit "OK" nothing happends. Just as a thought....ssh into the ovirt node used as initiator, and lsblk command shows the block device as present. So i have to cancel the "New Domain" windows without being able to add the domain, but that iscsi block device still remains present on the hosts. Using 4.2.8 Any thoughts ? Thank you very much ! Leo -- Best regards, Leo David

2 5

Open_vSwitch no key error after upgrading to 4.2.8
by Jayme 08 Feb '19

08 Feb '19

I upgraded oVirt to 4.2.8 and now I am spammed with the following message in all host syslog. How can I stop/fix this error? ovs-vsctl: ovs|00001|db_ctl_base|ERR|no key "odl_os_hostconfig_hostid" in Open_vSwitch record "." column external_ids

4 4

need network design advice for iSCSI
by John Florian 08 Feb '19

08 Feb '19

I have an existing 4.2 setup with 2 hosts, both with a quad-gbit NIC and a QNAP TS-569 Pro NAS with twin gbit NIC and five 7k2 drives. At present, the I have 5 VLANs, each with their own subnet as: 1. my "main" net (VLAN 1, 172.16.7.0/24) 2. ovirtmgmt (VLAN 100, 192.168.100.0/24) 3. four storage nets (VLANs 101-104, 192.168.101.0/24 - 192.168.104.0/24) On the NAS, I enslaved both NICs into a 802.3ad LAG and then bound an IP address for each of the four storage nets giving me: * bond0.101@bond0: 192.168.101.101 * bond0.102@bond0: 192.168.102.102 * bond0.103@bond0: 192.168.103.103 * bond0.104@bond0: 192.168.104.104 The hosts are similar, but with all four NICs enslaved into a 802.3ad LAG: Host 1: * bond0.101@bond0: 192.168.101.203 * bond0.102@bond0: 192.168.102.203 * bond0.103@bond0: 192.168.103.203 * bond0.104@bond0: 192.168.104.203 Host 2: * bond0.101@bond0: 192.168.101.204 * bond0.102@bond0: 192.168.102.204 * bond0.103@bond0: 192.168.103.204 * bond0.104@bond0: 192.168.104.204 I believe my performance could be better though. While running bonnie++ on a VM, the NAS reports top disk throughput around 70MB/s and the network (both NICs) topping out around 90MB/s. I suspect I'm being hurt by the load balancing across the NICs. I've played with various load balancing options for the LAGs (src-dst-ip and src-dst-mac) but with little difference in effect. Watching the resource monitor on the NAS, I can see that one NIC almost exclusive does transmits while the other is almost exclusively receives. Here's the bonnie report (my apologies to those reading plain-text here): Bonnie++ Benchmark results *Version 1.97* *Sequential Output* *Sequential Input* *Random Seeks* *Sequential Create* *Random Create* Size Per Char Block Rewrite Per Char Block Num Files Create Read Delete Create Read Delete K/sec % CPU K/sec % CPU K/sec % CPU K/sec % CPU K/sec % CPU /sec % CPU /sec % CPU /sec % CPU /sec % CPU /sec % CPU /sec % CPU /sec % CPU unamed 4G 267 97 75284 21 22775 8 718 97 43559 7 189.5 8 16 6789 60 +++++ +++ 24948 75 14792 86 +++++ +++ 18163 51 Latency 69048us 754ms 898ms 61246us 311ms 1126ms Latency 33937us 1132us 1299us 528us 22us 458us I keep seeing MPIO mentioned for iSCSI deployments and now I'm trying to get my head around how to best set that up or to even know if it would be helpful. I only have one switch (a Catalyst 3750g) in this small setup so fault tolerance at that level isn't a goal. So... what would the recommendation be? I've never done MPIO before but know where it's at in the web UI at least. -- John Florian

5 9

sun.security.validator
by suporte＠logicworks.pt 06 Feb '19

06 Feb '19

Hi, I'm running Version 4.2.3.8-1.el7, and after reboot the engine machine no longer could login into administration portal with this error: sun.security.validator.ValidatorException: PKIX path validation faile java.security.cert.CertPathValidatorException: validity check failed I'm using a self signed cert. Any idea? Thanks -- Jose Ferradeira http://www.logicworks.pt

5 5

Ovirt cluster unstable; gluster to blame (again)
by Jim Kusznir 04 Feb '19

04 Feb '19

hi all: Once again my production ovirt cluster is collapsing in on itself. My servers are intermittently unavailable or degrading, customers are noticing and calling in. This seems to be yet another gluster failure that I haven't been able to pin down. I posted about this a while ago, but didn't get anywhere (no replies that I found). The problem started out as a glusterfsd process consuming large amounts of ram (up to the point where ram and swap were exhausted and the kernel OOM killer killed off the glusterfsd process). For reasons not clear to me at this time, that resulted in any VMs running on that host and that gluster volume to be paused with I/O error (the glusterfs process is usually unharmed; why it didn't continue I/O with other servers is confusing to me). I have 3 servers and a total of 4 gluster volumes (engine, iso, data, and data-hdd). The first 3 are replica 2+arb; the 4th (data-hdd) is replica 3. The first 3 are backed by an LVM partition (some thin provisioned) on an SSD; the 4th is on a seagate hybrid disk (hdd + some internal flash for acceleration). data-hdd is the only thing on the disk. Servers are Dell R610 with the PERC/6i raid card, with the disks individually passed through to the OS (no raid enabled). The above RAM usage issue came from the data-hdd volume. Yesterday, I cought one of the glusterfsd high ram usage before the OOM-Killer had to run. I was able to migrate the VMs off the machine and for good measure, reboot the entire machine (after taking this opportunity to run the software updates that ovirt said were pending). Upon booting back up, the necessary volume healing began. However, this time, the healing caused all three servers to go to very, very high load averages (I saw just under 200 on one server; typically they've been 40-70) with top reporting IO Wait at 7-20%. Network for this volume is a dedicated gig network. According to bwm-ng, initially the network bandwidth would hit 50MB/s (yes, bytes), but tailed off to mostly in the kB/s for a while. All machines' load averages were still 40+ and gluster volume heal data-hdd info reported 5 items needing healing. Server's were intermittently experiencing IO issues, even on the 3 gluster volumes that appeared largely unaffected. Even the OS activities on the hosts itself (logging in, running commands) would often be very delayed. The ovirt engine was seemingly randomly throwing engine down / engine up / engine failed notifications. Responsiveness on ANY VM was horrific most of the time, with random VMs being inaccessible. I let the gluster heal run overnight. By morning, there were still 5 items needing healing, all three servers were still experiencing high load, and servers were still largely unstable. I've noticed that all of my ovirt outages (and I've had a lot, way more than is acceptable for a production cluster) have come from gluster. I still have 3 VMs who's hard disk images have become corrupted by my last gluster crash that I haven't had time to repair / rebuild yet (I believe this crash was caused by the OOM issue previously mentioned, but I didn't know it at the time). Is gluster really ready for production yet? It seems so unstable to me.... I'm looking at replacing gluster with a dedicated NFS server likely FreeNAS. Any suggestions? What is the "right" way to do production storage on this (3 node cluster)? Can I get this gluster volume stable enough to get my VMs to run reliably again until I can deploy another storage solution? --Jim

11 23

ETL service aggregation to hourly tables has encountered an error. Please consult the service log for more details.
by melnyksergii＠gmail.com 04 Feb '19

04 Feb '19

Dears, I have an a some error in Ovirt 4.2.7 In dash I see: ETL service aggregation to hourly tables has encountered an error. Please consult the service log for more details. In log ovirt engine server: 2019-01-14 15:59:59|rwL6AB|euUXph|wfcjQ7|OVIRT_ENGINE_DWH|HourlyTimeKeepingJob|Default|5|tWarn|tWarn_1|2019-01-14 15:59:59| ETL service aggregation to hourly tables has encountered an error. lastHourAgg value =Mon Jan 14 14:00:00 EET 2019 and runTime = Mon Jan 14 15:59:59 EET 2019 .Please consult the service log for more details.|42 In some sources people said the problem is in PostgreSQL DB, but I don't understand how can I fix this problem? Thanks

4 9

oVirt can't communicate with vSphere
by Benjamin Selinger 03 Feb '19

03 Feb '19

Trying to get my ovirt cluster connected to my vSphere cluster to import my guests. When trying to connect, I get: "VDSM ovirt1 command GetVmsNamesFromExternalProviderVDS failed: internal error: curl_easy_perform() returned an error: Couldn't connect to server (7) : Failed connect to 10.0.0.55:443; Connection timed out" ovirt1 node (192.168.1.195) is at location A while vSphere (10.0.0.55) is at location B. I added a static route on ovirt1 for 10.0.0.0/8 via 192.168.1.13, which has a VPN connection back to 10.0.0.0/8. ICMP from ovirt1 gets through just fine, but any other traffic never leaves ovirt1 (ie: 443 traffic never arrives at 1.13). I'm assuming there's some firewall rule somewhere, blocking anything other than outbound ICMP, but I have been unable to find it. Any suggestions?

2 3

Mounting ISO in subfolder
by Giulio Casella 01 Feb '19

01 Feb '19

Hi, I noticed that if I create a directory in root of ISO domain and put an image in it, you can see it in admin portal image list (listed as "foo/bar.iso"), but mounting that iso in a VM ("change CD") fails "Error while executing action Change CD: Drive image file could not be found" I can easily reproduce this behaviour in ovirt 4.2.8 and rhv 4.2.7. Is it a bug? Cheers, gc

5 9

Storage Domain Ovirt 4.3 RC EPYC
by Darin Schmidt 01 Feb '19

01 Feb '19

Uncaught exception occurred. Please try reloading the page. Details: (TypeError) : Cannot read property 'Vg' of null Please have your administrator check the UI logs Im trying to setup Ovirt 4.3 RC because its the only one that supports AMD EPYC. The issue Im having is that I creates a raid 5 array of 5 120GB SSDs using mdadm and mounted it at /VM. I put the Host into maintenance, select Configure Local Storage, and it fails telling me: New Local Storage Domain: Storage format is unsupported I formatted the array to ext4. This is a self hosted system. Anyone have any ideas? [root@ovirt VM]# uname -r 3.10.0-957.1.3.el7.x86_64 [root@ovirt VM]# cat /etc/centos-release CentOS Linux release 7.6.1810 (Core) [root@ovirt VM]# rpm -qa|grep ovirt ovirt-engine-backend-4.3.0.2-1.el7.noarch ovirt-imageio-common-1.4.6-1.el7.x86_64 ovirt-engine-setup-plugin-websocket-proxy-4.3.0.4-1.el7.noarch ovirt-ansible-vm-infra-1.1.12-1.el7.noarch ovirt-ansible-infra-1.1.11-1.el7.noarch ovirt-engine-setup-plugin-ovirt-engine-common-4.3.0.4-1.el7.noarch libgovirt-0.3.4-1.el7.x86_64 ovirt-hosted-engine-setup-2.3.3-1.el7.noarch ovirt-iso-uploader-4.3.0-1.el7.noarch ovirt-engine-4.3.0.2-1.el7.noarch python2-ovirt-host-deploy-1.8.0-1.el7.noarch ovirt-ansible-repositories-1.1.4-1.el7.noarch python-ovirt-engine-sdk4-4.3.0-2.el7.x86_64 ovirt-engine-webadmin-portal-4.3.0.2-1.el7.noarch ovirt-engine-vmconsole-proxy-helper-4.3.0.4-1.el7.noarch ovirt-vmconsole-proxy-1.0.6-2.el7.noarch ovirt-ansible-image-template-1.1.9-1.el7.noarch ovirt-ansible-shutdown-env-1.0.3-1.el7.noarch ovirt-engine-dwh-4.3.0-1.el7.noarch ovirt-ansible-roles-1.1.6-1.el7.noarch ovirt-host-deploy-common-1.8.0-1.el7.noarch ovirt-engine-api-explorer-0.0.3-1.el7.noarch ovirt-engine-setup-base-4.3.0.4-1.el7.noarch ovirt-engine-extensions-api-impl-4.3.0.4-1.el7.noarch cockpit-ovirt-dashboard-0.12.1-1.el7.noarch ovirt-engine-tools-backup-4.3.0.2-1.el7.noarch ovirt-engine-restapi-4.3.0.2-1.el7.noarch ovirt-ansible-hosted-engine-setup-1.0.8-1.el7.noarch ovirt-engine-websocket-proxy-4.3.0.4-1.el7.noarch ovirt-hosted-engine-ha-2.3.1-1.el7.noarch ovirt-ansible-cluster-upgrade-1.1.11-1.el7.noarch ovirt-cockpit-sso-0.1.1-1.el7.noarch ovirt-engine-ui-extensions-1.0.2-1.el7.noarch ovirt-engine-setup-4.3.0.4-1.el7.noarch ovirt-imageio-daemon-1.4.6-1.el7.noarch ovirt-host-4.3.0-2.el7.x86_64 ovirt-vmconsole-1.0.6-2.el7.noarch ovirt-imageio-proxy-1.4.6-1.el7.noarch ovirt-engine-wildfly-overlay-14.0.1-3.el7.noarch ovirt-web-ui-1.5.0-1.el7.noarch ovirt-provider-ovn-1.2.19-1.el7.noarch ovirt-engine-dbscripts-4.3.0.2-1.el7.noarch ovirt-ansible-engine-setup-1.1.7-1.el7.noarch ovirt-engine-setup-plugin-ovirt-engine-4.3.0.4-1.el7.noarch ovirt-provider-ovn-driver-1.2.19-1.el7.noarch ovirt-engine-extension-aaa-jdbc-1.1.8-1.el7.noarch python2-ovirt-setup-lib-1.2.0-1.el7.noarch ovirt-host-deploy-java-1.8.0-1.el7.noarch ovirt-engine-tools-4.3.0.2-1.el7.noarch cockpit-machines-ovirt-176-4.el7.centos.noarch ovirt-engine-metrics-1.2.0.3-1.el7.noarch ovirt-imageio-proxy-setup-1.4.6-1.el7.noarch ovirt-vmconsole-host-1.0.6-2.el7.noarch ovirt-ansible-manageiq-1.1.13-1.el7.noarch ovirt-engine-wildfly-14.0.1-3.el7.x86_64 ovirt-engine-setup-plugin-vmconsole-proxy-helper-4.3.0.4-1.el7.noarch ovirt-host-dependencies-4.3.0-2.el7.x86_64 ovirt-ansible-disaster-recovery-1.1.4-1.el7.noarch ovirt-engine-dwh-setup-4.3.0-1.el7.noarch ovirt-ansible-v2v-conversion-host-1.9.1-1.el7.noarch python2-ovirt-engine-lib-4.3.0.4-1.el7.noarch ovirt-release43-pre-4.3.0-0.7.rc4.el7.noarch

3 5