- Users - oVirt List Archives

Network Performance Issues
by Bryan Sockel 08 Feb '19

08 Feb '19

Hi, I currently have a 4 node ovirt cluster running. Each node is configured with an active passive network setup, with each link being 10 GB. After looking over my performance metrics collected via observium. I am noticing the network traffic rarely exceeds 100 MB. I am noticing this across all four of my servers and my 2 storage arrays that are also connected via the same 10 GB links. What is the best way to trouble shoot this problem? Thanks

2 1

need network design advice for iSCSI
by John Florian 08 Feb '19

08 Feb '19

I have an existing 4.2 setup with 2 hosts, both with a quad-gbit NIC and a QNAP TS-569 Pro NAS with twin gbit NIC and five 7k2 drives. At present, the I have 5 VLANs, each with their own subnet as: 1. my "main" net (VLAN 1, 172.16.7.0/24) 2. ovirtmgmt (VLAN 100, 192.168.100.0/24) 3. four storage nets (VLANs 101-104, 192.168.101.0/24 - 192.168.104.0/24) On the NAS, I enslaved both NICs into a 802.3ad LAG and then bound an IP address for each of the four storage nets giving me: * bond0.101@bond0: 192.168.101.101 * bond0.102@bond0: 192.168.102.102 * bond0.103@bond0: 192.168.103.103 * bond0.104@bond0: 192.168.104.104 The hosts are similar, but with all four NICs enslaved into a 802.3ad LAG: Host 1: * bond0.101@bond0: 192.168.101.203 * bond0.102@bond0: 192.168.102.203 * bond0.103@bond0: 192.168.103.203 * bond0.104@bond0: 192.168.104.203 Host 2: * bond0.101@bond0: 192.168.101.204 * bond0.102@bond0: 192.168.102.204 * bond0.103@bond0: 192.168.103.204 * bond0.104@bond0: 192.168.104.204 I believe my performance could be better though. While running bonnie++ on a VM, the NAS reports top disk throughput around 70MB/s and the network (both NICs) topping out around 90MB/s. I suspect I'm being hurt by the load balancing across the NICs. I've played with various load balancing options for the LAGs (src-dst-ip and src-dst-mac) but with little difference in effect. Watching the resource monitor on the NAS, I can see that one NIC almost exclusive does transmits while the other is almost exclusively receives. Here's the bonnie report (my apologies to those reading plain-text here): Bonnie++ Benchmark results *Version 1.97* *Sequential Output* *Sequential Input* *Random Seeks* *Sequential Create* *Random Create* Size Per Char Block Rewrite Per Char Block Num Files Create Read Delete Create Read Delete K/sec % CPU K/sec % CPU K/sec % CPU K/sec % CPU K/sec % CPU /sec % CPU /sec % CPU /sec % CPU /sec % CPU /sec % CPU /sec % CPU /sec % CPU unamed 4G 267 97 75284 21 22775 8 718 97 43559 7 189.5 8 16 6789 60 +++++ +++ 24948 75 14792 86 +++++ +++ 18163 51 Latency 69048us 754ms 898ms 61246us 311ms 1126ms Latency 33937us 1132us 1299us 528us 22us 458us I keep seeing MPIO mentioned for iSCSI deployments and now I'm trying to get my head around how to best set that up or to even know if it would be helpful. I only have one switch (a Catalyst 3750g) in this small setup so fault tolerance at that level isn't a goal. So... what would the recommendation be? I've never done MPIO before but know where it's at in the web UI at least. -- John Florian

5 9

ovirt-node-4.3, deployment fails when moving hosted engine vm to gluster storage.
by feral 07 Feb '19

07 Feb '19

I have no idea what's wrong at this point. Very vanilla install of 3 nodes. Run the Hyperconverged wizard, completes fine. Run the engine deployment, takes hours, eventually fails with : [ INFO ] TASK [oVirt.hosted-engine-setup : Check engine VM health] [ ERROR ] fatal: [localhost]: FAILED! => {"attempts": 120, "changed": true, "cmd": ["hosted-engine", "--vm-status", "--json"], "delta": "0:00:00.340985", "end": "2019-02-06 11:44:48.836431", "rc": 0, "start": "2019-02-06 11:44:48.495446", "stderr": "", "stderr_lines": [], "stdout": "{\"1\": {\"conf_on_shared_storage\": true, \"live-data\": true, \"extra\": \"metadata_parse_version=1\\nmetadata_feature_version=1\\ntimestamp=12994 (Wed Feb 6 11:44:44 2019)\\nhost-id=1\\nscore=3400\\nvm_conf_refresh_time=12995 (Wed Feb 6 11:44:44 2019)\\nconf_on_shared_storage=True\\nmaintenance=False\\nstate=EngineStop\\nstopped=False\\n\", \"hostname\": \"ovirt-431.localdomain\", \"host-id\": 1, \"engine-status\": {\"reason\": \"failed liveliness check\", \"health\": \"bad\", \"vm\": \"up\", \"detail\": \"Up\"}, \"score\": 3400, \"stopped\": false, \"maintenance\": false, \"crc32\": \"5474927a\", \"local_conf_timestamp\": 12995, \"host-ts\": 12994}, \"global_maintenance\": false}", "stdout_lines": ["{\"1\": {\"conf_on_shared_storage\": true, \"live-data\": true, \"extra\": \"metadata_parse_version=1\\nmetadata_feature_version=1\\ntimestamp=12994 (Wed Feb 6 11:44:44 2019)\\nhost-id=1\\nscore=3400\\nvm_conf_refresh_time=12995 (Wed Feb 6 11:44:44 2019)\\nconf_on_shared_storage=True\\nmaintenance=False\\nstate=EngineStop\\nstopped=False\\n\", \"hostname\": \"ovirt-431.localdomain\", \"host-id\": 1, \"engine-status\": {\"reason\": \"failed liveliness check\", \"health\": \"bad\", \"vm\": \"up\", \"detail\": \"Up\"}, \"score\": 3400, \"stopped\": false, \"maintenance\": false, \"crc32\": \"5474927a\", \"local_conf_timestamp\": 12995, \"host-ts\": 12994}, \"global_maintenance\": false}"]} [ INFO ] TASK [oVirt.hosted-engine-setup : Check VM status at virt level] [ INFO ] changed: [localhost] [ INFO ] TASK [oVirt.hosted-engine-setup : debug] [ INFO ] ok: [localhost] [ INFO ] TASK [oVirt.hosted-engine-setup : Fail if engine VM is not running] [ INFO ] skipping: [localhost] [ INFO ] TASK [oVirt.hosted-engine-setup : Get target engine VM IP address] [ INFO ] changed: [localhost] [ INFO ] TASK [oVirt.hosted-engine-setup : Get VDSM's target engine VM stats] [ INFO ] changed: [localhost] [ INFO ] TASK [oVirt.hosted-engine-setup : Convert stats to JSON format] [ INFO ] ok: [localhost] [ INFO ] TASK [oVirt.hosted-engine-setup : Get target engine VM IP address from VDSM stats] [ INFO ] ok: [localhost] [ INFO ] TASK [oVirt.hosted-engine-setup : debug] [ INFO ] ok: [localhost] [ INFO ] TASK [oVirt.hosted-engine-setup : Fail if Engine IP is different from engine's he_fqdn resolved IP] [ INFO ] skipping: [localhost] [ INFO ] TASK [oVirt.hosted-engine-setup : Fail is for any other reason the engine didn't started] [ ERROR ] fatal: [localhost]: FAILED! => {"changed": false, "msg": "The engine failed to start inside the engine VM; please check engine.log."} --------------------------------------------------- I can't check the engine.log as I can't connect to the VM once this failure occurs. I can ssh in prior to the VM being moved to gluster storage, but as soon as it starts doing so, the VM never comes back online. -- _____ Fact: 1. Ninjas are mammals. 2. Ninjas fight ALL the time. 3. The purpose of the ninja is to flip out and kill people.

2 6

oVirt users using Gluster: feedback needed
by Sandro Bonazzola 07 Feb '19

07 Feb '19

Hi, Gluster community opened a survey for the whole month of February 2019: https://www.gluster.org/gluster-community-survey-february-2019/ If you're using Gluster with oVirt please be sure to give gluster community feedback helping improve for next versions. Thanks, -- SANDRO BONAZZOLA MANAGER, SOFTWARE ENGINEERING, EMEA R&D RHV Red Hat EMEA <https://www.redhat.com/> sbonazzo(a)redhat.com <https://red.ht/sig>

1 0

ovn bad gateway after update from 4.2.7 to 4.2.8
by Gianluca Cecchi 07 Feb '19

07 Feb '19

Hello, at this moment (about two days ago) I have updated only engine (external, not self hosted) from 4.2.7.5 to 4.2.8.2 As soon as I'm starting for the first time a VM with an ovn based nic I get what below in ovirt-provider-ovn.log In admin gui, if I try for example to start via "run once" I get: " Error while executing action Run VM once: Failed to communicate with the external provider, see log for additional details. " Any clue? Thanks, Gianluca 2019-01-29 17:23:20,554 root Starting server 2019-01-29 17:23:20,554 root Version: 1.2.18-1 2019-01-29 17:23:20,555 root Build date: 20190114151850 2019-01-29 17:23:20,555 root Githash: dae4c1d 2019-01-29 18:04:15,575 root Starting server 2019-01-29 18:04:15,576 root Version: 1.2.18-1 2019-01-29 18:04:15,576 root Build date: 20190114151850 2019-01-29 18:04:15,576 root Githash: dae4c1d 2019-02-01 14:26:58,316 root From: ::ffff:127.0.0.1:49582 Request: GET /v2.0/ports 2019-02-01 14:26:58,317 root HTTPSConnectionPool(host='engine-host', port=443): Max retries exceeded with url: /ovirt-engine/sso/oauth/token-info (Caused by NewConnectionError('<urllib3.connection.VerifiedHTTPSConnection object at 0x7fe806166b90>: Failed to establish a new connection: [Errno -2] Name or service not known',)) Traceback (most recent call last): File "/usr/share/ovirt-provider-ovn/handlers/base_handler.py", line 134, in _handle_request method, path_parts, content File "/usr/share/ovirt-provider-ovn/handlers/selecting_handler.py", line 175, in handle_request return self.call_response_handler(handler, content, parameters) File "/usr/share/ovirt-provider-ovn/handlers/neutron.py", line 33, in call_response_handler TOKEN_HTTP_HEADER_FIELD_NAME, '')): File "/usr/share/ovirt-provider-ovn/auth/plugin_facade.py", line 31, in validate_token return auth.core.plugin.validate_token(token) File "/usr/share/ovirt-provider-ovn/auth/plugins/ovirt/authorization_by_username.py", line 36, in validate_token return self._is_user_name(token, _admin_user_name()) File "/usr/share/ovirt-provider-ovn/auth/plugins/ovirt/authorization_by_username.py", line 47, in _is_user_name timeout=AuthorizationByUserName._timeout()) File "/usr/share/ovirt-provider-ovn/auth/plugins/ovirt/sso.py", line 131, in get_token_info timeout=timeout File "/usr/share/ovirt-provider-ovn/auth/plugins/ovirt/sso.py", line 54, in wrapper response = func(*args, **kwargs) File "/usr/share/ovirt-provider-ovn/auth/plugins/ovirt/sso.py", line 47, in wrapper raise BadGateway(e) BadGateway: HTTPSConnectionPool(host='engine-host', port=443): Max retries exceeded with url: /ovirt-engine/sso/oauth/token-info (Caused by NewConnectionError('<urllib3.connection.VerifiedHTTPSConnection object at 0x7fe806166b90>: Failed to establish a new connection: [Errno -2] Name or service not known',)) 2019-02-01 14:27:26,968 root From: ::ffff:127.0.0.1:49590 Request: GET /v2.0/ports 2019-02-01 14:27:26,969 root HTTPSConnectionPool(host='engine-host', port=443): Max retries exceeded with url: /ovirt-engine/sso/oauth/token-info (Caused by NewConnectionError('<urllib3.connection.VerifiedHTTPSConnection object at 0x7fe80618df50>: Failed to establish a new connection: [Errno -2] Name or service not known',)) Traceback (most recent call last): File "/usr/share/ovirt-provider-ovn/handlers/base_handler.py", line 134, in _handle_request method, path_parts, content File "/usr/share/ovirt-provider-ovn/handlers/selecting_handler.py", line 175, in handle_request return self.call_response_handler(handler, content, parameters) File "/usr/share/ovirt-provider-ovn/handlers/neutron.py", line 33, in call_response_handler TOKEN_HTTP_HEADER_FIELD_NAME, '')): File "/usr/share/ovirt-provider-ovn/auth/plugin_facade.py", line 31, in validate_token return auth.core.plugin.validate_token(token) File "/usr/share/ovirt-provider-ovn/auth/plugins/ovirt/authorization_by_username.py", line 36, in validate_token return self._is_user_name(token, _admin_user_name()) File "/usr/share/ovirt-provider-ovn/auth/plugins/ovirt/authorization_by_username.py", line 47, in _is_user_name timeout=AuthorizationByUserName._timeout()) File "/usr/share/ovirt-provider-ovn/auth/plugins/ovirt/sso.py", line 131, in get_token_info timeout=timeout File "/usr/share/ovirt-provider-ovn/auth/plugins/ovirt/sso.py", line 54, in wrapper response = func(*args, **kwargs) File "/usr/share/ovirt-provider-ovn/auth/plugins/ovirt/sso.py", line 47, in wrapper raise BadGateway(e) BadGateway: HTTPSConnectionPool(host='engine-host', port=443): Max retries exceeded with url: /ovirt-engine/sso/oauth/token-info (Caused by NewConnectionError('<urllib3.connection.VerifiedHTTPSConnection object at 0x7fe80618df50>: Failed to establish a new connection: [Errno -2] Name or service not known',)) 2019-02-01 14:29:17,412 root From: ::ffff:127.0.0.1:49616 Request: GET /v2.0/ports 2019-02-01 14:29:17,412 root HTTPSConnectionPool(host='engine-host', port=443): Max retries exceeded with url: /ovirt-engine/sso/oauth/token-info (Caused by NewConnectionError('<urllib3.connection.VerifiedHTTPSConnection object at 0x7fe80618de50>: Failed to establish a new connection: [Errno -2] Name or service not known',)) Traceback (most recent call last): File "/usr/share/ovirt-provider-ovn/handlers/base_handler.py", line 134, in _handle_request method, path_parts, content File "/usr/share/ovirt-provider-ovn/handlers/selecting_handler.py", line 175, in handle_request return self.call_response_handler(handler, content, parameters) File "/usr/share/ovirt-provider-ovn/handlers/neutron.py", line 33, in call_response_handler TOKEN_HTTP_HEADER_FIELD_NAME, '')): File "/usr/share/ovirt-provider-ovn/auth/plugin_facade.py", line 31, in validate_token return auth.core.plugin.validate_token(token) File "/usr/share/ovirt-provider-ovn/auth/plugins/ovirt/authorization_by_username.py", line 36, in validate_token return self._is_user_name(token, _admin_user_name()) File "/usr/share/ovirt-provider-ovn/auth/plugins/ovirt/authorization_by_username.py", line 47, in _is_user_name timeout=AuthorizationByUserName._timeout()) File "/usr/share/ovirt-provider-ovn/auth/plugins/ovirt/sso.py", line 131, in get_token_info timeout=timeout File "/usr/share/ovirt-provider-ovn/auth/plugins/ovirt/sso.py", line 54, in wrapper response = func(*args, **kwargs) File "/usr/share/ovirt-provider-ovn/auth/plugins/ovirt/sso.py", line 47, in wrapper raise BadGateway(e) BadGateway: HTTPSConnectionPool(host='engine-host', port=443): Max retries exceeded with url: /ovirt-engine/sso/oauth/token-info (Caused by NewConnectionError('<urllib3.connection.VerifiedHTTPSConnection object at 0x7fe80618de50>: Failed to establish a new connection: [Errno -2] Name or service not known',))

4 8

vm autostart and priorities
by Hetz Ben Hamo 07 Feb '19

07 Feb '19

Hi, I'm migrating my VM's to oVirt from ESXI and I've only got one left on the ESXi side - a VM which provides me DNS, mail and DHCP services. Without this VM's, I can't access the internet. The only issue that prevents me from migrating this VM to oVirt and "kill" the ESXi machine, is that I need to start this VM's upon oVirt node power up (auto start), so if the power goes down and comes back, I'll have my infrastructure up and running automatically within minutes. Is there a way to auto start specific VM's or better - start VM's based on specific orders upon boot of the nodes? I think it's a very important feature. Thanks

3 2

Re: ovirt upgrade 4.3 - dashboard is dispalyed 2x in top left corner menu
by Greg Sheremeta 07 Feb '19

07 Feb '19

On Thu, Feb 7, 2019 at 7:39 AM Martin Humaj <mhumaj(a)gmail.com> wrote: > Hi Greg, > > Thanks a lot for quick reply. > > yum update ovirt-engine-ui-extensions, fixed the issue > > The duplicate is gone. > Good to hear, thanks. For posterity, can you tell me if you did the update today or a couple days ago? And, are you using hosted engine? I want to make sure it's not still broken for everyone else :) > > martin > > On Thu, Feb 7, 2019 at 1:32 PM Greg Sheremeta <gshereme(a)redhat.com> wrote: > >> It should be fixed now. When did you update? >> >> Are you using hosted engine? >> >> On Thu, Feb 7, 2019, 6:36 AM <mhumaj(a)gmail.com> wrote: >> >>> After the upgrade to ovirt 4.3, I can see two entry for dashboard in top >>> left corner menu. Is there any way how to get rid of the duplicity? >>> >>> thanks >>> _______________________________________________ >>> Users mailing list -- users(a)ovirt.org >>> To unsubscribe send an email to users-leave(a)ovirt.org >>> Privacy Statement: https://www.ovirt.org/site/privacy-policy/ >>> oVirt Code of Conduct: >>> https://www.ovirt.org/community/about/community-guidelines/ >>> List Archives: >>> https://lists.ovirt.org/archives/list/users@ovirt.org/message/GN4ER4YD6KTQL… >>> >> -- GREG SHEREMETA SENIOR SOFTWARE ENGINEER - TEAM LEAD - RHV UX Red Hat NA <https://www.redhat.com/> gshereme(a)redhat.com IRC: gshereme <https://red.ht/sig>

2 2

qxl issue on OVirt 4.3.0
by Robert Crawford 07 Feb '19

07 Feb '19

On 4.3.0 with the 1809 CentOS image from the ISO Domain the VM Guest will fail to boot and freeze on initialized qxl 0.1.0

2 5

ovirt upgrade 4.3 - dashboard is dispalyed 2x in top left corner menu
by mhumaj＠gmail.com 07 Feb '19

07 Feb '19

After the upgrade to ovirt 4.3, I can see two entry for dashboard in top left corner menu. Is there any way how to get rid of the duplicity? thanks

2 1

Cluster upgrade
by Misak Khachatryan 07 Feb '19

07 Feb '19

Hi, I've successfully upgraded to 4.3, but when I'm trying to upgrade Cluster version I'm getting this: "Error while executing action: Cannot change Cluster Compatibility Version to higher version when there are active Hosts with lower version. -Please move Host virt2 with lower version to maintenance first." Any clues? Best regards, Misak Khachatryan

2 4