[ovirt-users] Re: External networks issue after upgrading to 4.3.6

unfortunately I get the following error when running the playbook. TASK [ovirt-provider-ovn-driver : Configure OVN for oVirt] **************************************************************************************************************************************************** fatal: [threatrealm3.sec.ouc.ac.cy]: FAILED! => { "changed": true, "cmd": [ "vdsm-tool", "ovn-config", "192.168.1.21", "ovirt-provider-ovn" ], "delta": "0:00:02.993452", "end": "2019-10-09 14:09:57.555302", "rc": 1, "start": "2019-10-09 14:09:54.561850" } STDERR: Traceback (most recent call last): File "/usr/bin/vdsm-tool", line 220, in main return tool_command[cmd]["command"](*args) File "/usr/lib/python2.7/site-packages/vdsm/tool/ovn_config.py", line 64, in ovn_config ip_address = get_ip_addr(get_network(network_caps(), net_name)) File "/usr/lib/python2.7/site-packages/vdsm/tool/ovn_config.py", line 118, in get_network raise NetworkNotFoundError(net_name) NetworkNotFoundError: ovirt-provider-ovn MSG: non-zero return code fatal: [threatrealm2.sec.ouc.ac.cy]: FAILED! => { "changed": true, "cmd": [ "vdsm-tool", "ovn-config", "192.168.1.21", "ovirt-provider-ovn" ], "delta": "0:00:03.033376", "end": "2019-10-09 14:09:57.650353", "rc": 1, "start": "2019-10-09 14:09:54.616977" } STDERR: Traceback (most recent call last): File "/usr/bin/vdsm-tool", line 220, in main return tool_command[cmd]["command"](*args) File "/usr/lib/python2.7/site-packages/vdsm/tool/ovn_config.py", line 64, in ovn_config ip_address = get_ip_addr(get_network(network_caps(), net_name)) File "/usr/lib/python2.7/site-packages/vdsm/tool/ovn_config.py", line 118, in get_network raise NetworkNotFoundError(net_name) NetworkNotFoundError: ovirt-provider-ovn MSG: non-zero return code PLAY RECAP **************************************************************************************************************************************************************************************************** localhost : ok=1 changed=0 unreachable=0 failed=0 skipped=2 rescued=0 ignored=0 threatrealm2.sec.ouc.ac.cy : ok=5 changed=0 unreachable=0 failed=1 skipped=0 rescued=0 ignored=0 threatrealm3.sec.ouc.ac.cy : ok=5 changed=0 unreachable=0 failed=1 skipped=0 rescued=0 ignored=0 On Wed, Oct 9, 2019 at 11:36 AM Miguel Duarte de Mora Barroso < mdbarroso(a)redhat.com&gt; wrote: > On Wed, Oct 9, 2019 at 10:13 AM ada per <adaper3(a)gmail.com&gt; wrote: > > > > Can you please share the /var/log/openvswitch/ovn-controller.log from > the host? > > If you configure a subnet on the network, gets the VM an IP address? > > > > > > the /var/log/openvswitch/ovn-controller.log its > actually empty. So Iam Attaching yesterdays log (it wasn’t connecting but > after removing and reinstalling the hosts it connects) > > > > Even with specifying a subnet still no IPs. > > > > > > btw the openvswitch is active > > > > openvswitch.service - Open vSwitch > > > > Loaded: loaded > (/usr/lib/systemd/system/openvswitch.service; enabled; vendor preset: > disabled) > > > > Active: active (exited) since Tue 2019-10-08 16:02:12 > EEST; 18h ago > > > > > > > > You upgraded from which version ? > > > > Upgraded from 4 .3.4 to 4.3.6.7-1.el7 it currently runs > openvswitch 2.10.1 > > I find it strange that you weren't prompted to update ovs to > openvswitch-2.11.0-4.el7 , which is the currently tagged version for > ovirt-4.3 and master. > > If you do yum update openvswitch, doesn't it spot the update possibility ? > > > > > > > >> I have another environment running on a previous version of ovirt > and it works perfectly there. I think its a bug. > > > > Which version of ovirt is being run on that environment ? What's the > > openvswitch / openvswitch-ovn-central version running on that env ? > > > > Ovirt Version 4.3.4.3-1.el7 > > > > > > > > Openvswitch 2.10.1 ( so both environments run the same > version of openvswitch) > > > > > > > > > > > > Thinking a couple steps ahead, knowing the list of registered chassis > > could be useful. Please also tell us the output of the command (run it > > on your engine node): > > $ ovn-sbctl list chassis > > > > > > > > # ovn-sbctl list chassis > > > > net_mlx5: cannot load glue library: libibverbs.so.1: cannot open shared > object file: No such file or directory > > > > net_mlx5: cannot initialize PMD due to missing run-time dependency on > rdma-core libraries (libibverbs, libmlx5) > > > > PMD: net_mlx4: cannot load glue library: libibverbs.so.1: cannot open > shared obj ect file: No such file or directory > > > > PMD: net_mlx4: cannot initialize PMD due to missing run-time dependency > on rdma-core libraries (libibverbs, libmlx4) > > Sorry for the ugly logged errors; they're just noise. > > Despite that, the absence of registered chassis is quite surely why > your environment is misbehaving - it looks like the ovirt update > messed up the previous config. > > Could you follow the instructions at [0] to re-configure OVN ? After > the playbook finishes, please re-check your chassis table (ovn-sbctl > list chassis). You should find exactly one entry per ovirt-node. > > [0] - > https://www.ovirt.org/documentation/admin-guide/chap-External_Providers.h... > > > > > > > > > > > > > > > > > On Tue, Oct 8, 2019 at 6:10 PM Miguel Duarte de Mora Barroso < > mdbarroso(a)redhat.com&gt; wrote: > >> > >> On Tue, Oct 8, 2019 at 5:02 PM Miguel Duarte de Mora Barroso > >> <mdbarroso(a)redhat.com&gt; wrote: > >> > > >> > On Tue, Oct 8, 2019 at 4:53 PM Dominik Holler <dholler(a)redhat.com&gt; > wrote: > >> > > > >> > > > >> > > > >> > > On Tue, Oct 8, 2019 at 4:25 PM ada per <adaper3(a)gmail.com&gt; wrote: > >> > >> > >> > >> After upgrading to the latest stable version the external > networks lost all functionality. Under providers-->ovn-network-provider the > test runs successfully. > >> > > >> > You upgraded from which version ? > >> > > >> > >> > >> > >> But when im creating an external provider network, attaching it > to a router as a LAN setting up dhcp lease it is not reachable from other > VMs in the same network. Hosts and hosted engine can't seem to ping it > either. > >> > >> I tried disabling firewalls on both hosted engines, VMs and host > and still nothing. > >> > >> When configuring Logical networks or VLANs they work perfectly > tje one problem is the external networks. > >> > >> > >> > >> I have another environment running on a previous version of ovirt > and it works perfectly there. I think its a bug. > >> > > >> > Which version of ovirt is being run on that environment ? What's the > >> > openvswitch / openvswitch-ovn-central version running on that env ? > >> > >> Thinking a couple steps ahead, knowing the list of registered chassis > >> could be useful. Please also tell us the output of the command (run it > >> on your engine node): > >> $ ovn-sbctl list chassis > >> > >> > > >> > >> > >> > > > >> > > Can you please share the /var/log/openvswitch/ovn-controller.log > from the host? > >> > > If you configure a subnet on the network, gets the VM an IP > address? > >> > > > >> > > > >> > >> > >> > >> > >> > >> Thanks for your help >