External networks issue after upgrading to 4.3.6
After upgrading to the latest stable version the external networks lost all functionality. Under providers-->ovn-network-provider the test runs successfully. But when im creating an external provider network, attaching it to a router as a LAN setting up dhcp lease it is not reachable from other VMs in the same network. Hosts and hosted engine can't seem to ping it either. I tried disabling firewalls on both hosted engines, VMs and host and still nothing. When configuring Logical networks or VLANs they work perfectly tje one problem is the external networks. I have another environment running on a previous version of ovirt and it works perfectly there. I think its a bug. Thanks for your help
On Tue, Oct 8, 2019 at 4:25 PM ada per <adaper3@gmail.com> wrote:
After upgrading to the latest stable version the external networks lost all functionality. Under providers-->ovn-network-provider the test runs successfully.
But when im creating an external provider network, attaching it to a router as a LAN setting up dhcp lease it is not reachable from other VMs in the same network. Hosts and hosted engine can't seem to ping it either. I tried disabling firewalls on both hosted engines, VMs and host and still nothing. When configuring Logical networks or VLANs they work perfectly tje one problem is the external networks.
I have another environment running on a previous version of ovirt and it works perfectly there. I think its a bug.
Can you please share the /var/log/openvswitch/ovn-controller.log from the host? If you configure a subnet on the network, gets the VM an IP address?
Thanks for your help
On Tue, Oct 8, 2019 at 4:53 PM Dominik Holler <dholler@redhat.com> wrote:
On Tue, Oct 8, 2019 at 4:25 PM ada per <adaper3@gmail.com> wrote:
After upgrading to the latest stable version the external networks lost all functionality. Under providers-->ovn-network-provider the test runs successfully.
You upgraded from which version ?
But when im creating an external provider network, attaching it to a router as a LAN setting up dhcp lease it is not reachable from other VMs in the same network. Hosts and hosted engine can't seem to ping it either. I tried disabling firewalls on both hosted engines, VMs and host and still nothing. When configuring Logical networks or VLANs they work perfectly tje one problem is the external networks.
I have another environment running on a previous version of ovirt and it works perfectly there. I think its a bug.
Which version of ovirt is being run on that environment ? What's the openvswitch / openvswitch-ovn-central version running on that env ?
Can you please share the /var/log/openvswitch/ovn-controller.log from the host? If you configure a subnet on the network, gets the VM an IP address?
Thanks for your help
On Tue, Oct 8, 2019 at 5:02 PM Miguel Duarte de Mora Barroso <mdbarroso@redhat.com> wrote:
On Tue, Oct 8, 2019 at 4:53 PM Dominik Holler <dholler@redhat.com> wrote:
On Tue, Oct 8, 2019 at 4:25 PM ada per <adaper3@gmail.com> wrote:
After upgrading to the latest stable version the external networks lost all functionality. Under providers-->ovn-network-provider the test runs successfully.
You upgraded from which version ?
But when im creating an external provider network, attaching it to a router as a LAN setting up dhcp lease it is not reachable from other VMs in the same network. Hosts and hosted engine can't seem to ping it either. I tried disabling firewalls on both hosted engines, VMs and host and still nothing. When configuring Logical networks or VLANs they work perfectly tje one problem is the external networks.
I have another environment running on a previous version of ovirt and it works perfectly there. I think its a bug.
Which version of ovirt is being run on that environment ? What's the openvswitch / openvswitch-ovn-central version running on that env ?
Thinking a couple steps ahead, knowing the list of registered chassis could be useful. Please also tell us the output of the command (run it on your engine node): $ ovn-sbctl list chassis
Can you please share the /var/log/openvswitch/ovn-controller.log from the host? If you configure a subnet on the network, gets the VM an IP address?
Thanks for your help
Can you please share the /var/log/openvswitch/ovn-controller.log from the host? If you configure a subnet on the network, gets the VM an IP address? the /var/log/openvswitch/ovn-controller.log its actually empty. So Iam Attaching yesterdays log (it wasn’t connecting but after removing and reinstalling the hosts it connects) Even with specifying a subnet still no IPs. btw the openvswitch is active openvswitch.service - Open vSwitch Loaded: loaded (/usr/lib/systemd/system/openvswitch.service; enabled; vendor preset: disabled) Active: active (exited) since Tue 2019-10-08 16:02:12 EEST; 18h ago You upgraded from which version ? Upgraded from 4 .3.4 to 4.3.6.7-1.el7 it currently runs openvswitch 2.10.1
I have another environment running on a previous version of ovirt and it works perfectly there. I think its a bug.
Which version of ovirt is being run on that environment ? What's the openvswitch / openvswitch-ovn-central version running on that env ? Ovirt Version 4.3.4.3-1.el7 Openvswitch 2.10.1 ( so both environments run the same version of openvswitch) Thinking a couple steps ahead, knowing the list of registered chassis could be useful. Please also tell us the output of the command (run it on your engine node): $ ovn-sbctl list chassis # ovn-sbctl list chassis net_mlx5: cannot load glue library: libibverbs.so.1: cannot open shared object file: No such file or directory net_mlx5: cannot initialize PMD due to missing run-time dependency on rdma-core libraries (libibverbs, libmlx5) PMD: net_mlx4: cannot load glue library: libibverbs.so.1: cannot open shared obj ect file: No such file or directory PMD: net_mlx4: cannot initialize PMD due to missing run-time dependency on rdma-core libraries (libibverbs, libmlx4) On Tue, Oct 8, 2019 at 6:10 PM Miguel Duarte de Mora Barroso < mdbarroso@redhat.com> wrote:
On Tue, Oct 8, 2019 at 5:02 PM Miguel Duarte de Mora Barroso <mdbarroso@redhat.com> wrote:
On Tue, Oct 8, 2019 at 4:53 PM Dominik Holler <dholler@redhat.com>
On Tue, Oct 8, 2019 at 4:25 PM ada per <adaper3@gmail.com> wrote:
After upgrading to the latest stable version the external networks
lost all functionality. Under providers-->ovn-network-provider the test runs successfully.
You upgraded from which version ?
But when im creating an external provider network, attaching it to a
router as a LAN setting up dhcp lease it is not reachable from other VMs in
wrote: the same network. Hosts and hosted engine can't seem to ping it either.
I tried disabling firewalls on both hosted engines, VMs and host and still nothing. When configuring Logical networks or VLANs they work perfectly tje one problem is the external networks.
I have another environment running on a previous version of ovirt and it works perfectly there. I think its a bug.
Which version of ovirt is being run on that environment ? What's the openvswitch / openvswitch-ovn-central version running on that env ?
Thinking a couple steps ahead, knowing the list of registered chassis could be useful. Please also tell us the output of the command (run it on your engine node): $ ovn-sbctl list chassis
Can you please share the /var/log/openvswitch/ovn-controller.log from
the host?
If you configure a subnet on the network, gets the VM an IP address?
Thanks for your help
On Wed, Oct 9, 2019 at 11:13 AM ada per <adaper3@gmail.com> wrote:
Can you please share the /var/log/openvswitch/ovn-controller.log from the host? If you configure a subnet on the network, gets the VM an IP address?
the /var/log/openvswitch/ovn-controller.log its actually empty. So Iam Attaching yesterdays log (it wasn’t connecting but after removing and reinstalling the hosts it connects)
Even with specifying a subnet still no IPs.
btw the openvswitch is active
openvswitch.service - Open vSwitch
Loaded: loaded (/usr/lib/systemd/system/openvswitch.service; enabled; vendor preset: disabled)
Active: active (exited) since Tue 2019-10-08 16:02:12 EEST; 18h ago
You upgraded from which version ?
Upgraded from 4 .3.4 to 4.3.6.7-1.el7 it currently runs openvswitch 2.10.1
I have another environment running on a previous version of ovirt and it works perfectly there. I think its a bug.
Which version of ovirt is being run on that environment ? What's the openvswitch / openvswitch-ovn-central version running on that env ?
Ovirt Version 4.3.4.3-1.el7
Openvswitch 2.10.1 ( so both environments run the same version of openvswitch)
Thinking a couple steps ahead, knowing the list of registered chassis could be useful. Please also tell us the output of the command (run it on your engine node): $ ovn-sbctl list chassis
# ovn-sbctl list chassis
net_mlx5: cannot load glue library: libibverbs.so.1: cannot open shared object file: No such file or directory
net_mlx5: cannot initialize PMD due to missing run-time dependency on rdma-core libraries (libibverbs, libmlx5)
PMD: net_mlx4: cannot load glue library: libibverbs.so.1: cannot open shared obj ect file: No such file or directory
PMD: net_mlx4: cannot initialize PMD due to missing run-time dependency on rdma-core libraries (libibverbs, libmlx4)
On Tue, Oct 8, 2019 at 6:10 PM Miguel Duarte de Mora Barroso < mdbarroso@redhat.com> wrote:
On Tue, Oct 8, 2019 at 5:02 PM Miguel Duarte de Mora Barroso <mdbarroso@redhat.com> wrote:
On Tue, Oct 8, 2019 at 4:53 PM Dominik Holler <dholler@redhat.com>
On Tue, Oct 8, 2019 at 4:25 PM ada per <adaper3@gmail.com> wrote:
After upgrading to the latest stable version the external networks
lost all functionality. Under providers-->ovn-network-provider the test runs successfully.
You upgraded from which version ?
But when im creating an external provider network, attaching it to a
router as a LAN setting up dhcp lease it is not reachable from other VMs in
wrote: the same network. Hosts and hosted engine can't seem to ping it either.
I tried disabling firewalls on both hosted engines, VMs and host and still nothing. When configuring Logical networks or VLANs they work perfectly tje one problem is the external networks.
I have another environment running on a previous version of ovirt and it works perfectly there. I think its a bug.
Which version of ovirt is being run on that environment ? What's the openvswitch / openvswitch-ovn-central version running on that env ?
Thinking a couple steps ahead, knowing the list of registered chassis could be useful. Please also tell us the output of the command (run it on your engine node): $ ovn-sbctl list chassis
Can you please share the /var/log/openvswitch/ovn-controller.log from
the host?
If you configure a subnet on the network, gets the VM an IP address?
Thanks for your help
On Wed, Oct 9, 2019 at 10:13 AM ada per <adaper3@gmail.com> wrote:
Can you please share the /var/log/openvswitch/ovn-controller.log from the host? If you configure a subnet on the network, gets the VM an IP address?
the /var/log/openvswitch/ovn-controller.log its actually empty. So Iam Attaching yesterdays log (it wasn’t connecting but after removing and reinstalling the hosts it connects)
Even with specifying a subnet still no IPs.
btw the openvswitch is active
openvswitch.service - Open vSwitch
Loaded: loaded (/usr/lib/systemd/system/openvswitch.service; enabled; vendor preset: disabled)
Active: active (exited) since Tue 2019-10-08 16:02:12 EEST; 18h ago
You upgraded from which version ?
Upgraded from 4 .3.4 to 4.3.6.7-1.el7 it currently runs openvswitch 2.10.1
I find it strange that you weren't prompted to update ovs to openvswitch-2.11.0-4.el7 , which is the currently tagged version for ovirt-4.3 and master. If you do yum update openvswitch, doesn't it spot the update possibility ?
I have another environment running on a previous version of ovirt and it works perfectly there. I think its a bug.
Which version of ovirt is being run on that environment ? What's the openvswitch / openvswitch-ovn-central version running on that env ?
Ovirt Version 4.3.4.3-1.el7
Openvswitch 2.10.1 ( so both environments run the same version of openvswitch)
Thinking a couple steps ahead, knowing the list of registered chassis could be useful. Please also tell us the output of the command (run it on your engine node): $ ovn-sbctl list chassis
# ovn-sbctl list chassis
net_mlx5: cannot load glue library: libibverbs.so.1: cannot open shared object file: No such file or directory
net_mlx5: cannot initialize PMD due to missing run-time dependency on rdma-core libraries (libibverbs, libmlx5)
PMD: net_mlx4: cannot load glue library: libibverbs.so.1: cannot open shared obj ect file: No such file or directory
PMD: net_mlx4: cannot initialize PMD due to missing run-time dependency on rdma-core libraries (libibverbs, libmlx4)
Sorry for the ugly logged errors; they're just noise. Despite that, the absence of registered chassis is quite surely why your environment is misbehaving - it looks like the ovirt update messed up the previous config. Could you follow the instructions at [0] to re-configure OVN ? After the playbook finishes, please re-check your chassis table (ovn-sbctl list chassis). You should find exactly one entry per ovirt-node. [0] - https://www.ovirt.org/documentation/admin-guide/chap-External_Providers.html...
On Tue, Oct 8, 2019 at 6:10 PM Miguel Duarte de Mora Barroso <mdbarroso@redhat.com> wrote:
On Tue, Oct 8, 2019 at 5:02 PM Miguel Duarte de Mora Barroso <mdbarroso@redhat.com> wrote:
On Tue, Oct 8, 2019 at 4:53 PM Dominik Holler <dholler@redhat.com> wrote:
On Tue, Oct 8, 2019 at 4:25 PM ada per <adaper3@gmail.com> wrote:
After upgrading to the latest stable version the external networks lost all functionality. Under providers-->ovn-network-provider the test runs successfully.
You upgraded from which version ?
But when im creating an external provider network, attaching it to a router as a LAN setting up dhcp lease it is not reachable from other VMs in the same network. Hosts and hosted engine can't seem to ping it either. I tried disabling firewalls on both hosted engines, VMs and host and still nothing. When configuring Logical networks or VLANs they work perfectly tje one problem is the external networks.
I have another environment running on a previous version of ovirt and it works perfectly there. I think its a bug.
Which version of ovirt is being run on that environment ? What's the openvswitch / openvswitch-ovn-central version running on that env ?
Thinking a couple steps ahead, knowing the list of registered chassis could be useful. Please also tell us the output of the command (run it on your engine node): $ ovn-sbctl list chassis
Can you please share the /var/log/openvswitch/ovn-controller.log from the host? If you configure a subnet on the network, gets the VM an IP address?
Thanks for your help
unfortunately I get the following error when running the playbook. TASK [ovirt-provider-ovn-driver : Configure OVN for oVirt] **************************************************************************************************************************************************** fatal: [threatrealm3.sec.ouc.ac.cy]: FAILED! => { "changed": true, "cmd": [ "vdsm-tool", "ovn-config", "192.168.1.21", "ovirt-provider-ovn" ], "delta": "0:00:02.993452", "end": "2019-10-09 14:09:57.555302", "rc": 1, "start": "2019-10-09 14:09:54.561850" } STDERR: Traceback (most recent call last): File "/usr/bin/vdsm-tool", line 220, in main return tool_command[cmd]["command"](*args) File "/usr/lib/python2.7/site-packages/vdsm/tool/ovn_config.py", line 64, in ovn_config ip_address = get_ip_addr(get_network(network_caps(), net_name)) File "/usr/lib/python2.7/site-packages/vdsm/tool/ovn_config.py", line 118, in get_network raise NetworkNotFoundError(net_name) NetworkNotFoundError: ovirt-provider-ovn MSG: non-zero return code fatal: [threatrealm2.sec.ouc.ac.cy]: FAILED! => { "changed": true, "cmd": [ "vdsm-tool", "ovn-config", "192.168.1.21", "ovirt-provider-ovn" ], "delta": "0:00:03.033376", "end": "2019-10-09 14:09:57.650353", "rc": 1, "start": "2019-10-09 14:09:54.616977" } STDERR: Traceback (most recent call last): File "/usr/bin/vdsm-tool", line 220, in main return tool_command[cmd]["command"](*args) File "/usr/lib/python2.7/site-packages/vdsm/tool/ovn_config.py", line 64, in ovn_config ip_address = get_ip_addr(get_network(network_caps(), net_name)) File "/usr/lib/python2.7/site-packages/vdsm/tool/ovn_config.py", line 118, in get_network raise NetworkNotFoundError(net_name) NetworkNotFoundError: ovirt-provider-ovn MSG: non-zero return code PLAY RECAP **************************************************************************************************************************************************************************************************** localhost : ok=1 changed=0 unreachable=0 failed=0 skipped=2 rescued=0 ignored=0 threatrealm2.sec.ouc.ac.cy : ok=5 changed=0 unreachable=0 failed=1 skipped=0 rescued=0 ignored=0 threatrealm3.sec.ouc.ac.cy : ok=5 changed=0 unreachable=0 failed=1 skipped=0 rescued=0 ignored=0 On Wed, Oct 9, 2019 at 11:36 AM Miguel Duarte de Mora Barroso < mdbarroso@redhat.com> wrote:
On Wed, Oct 9, 2019 at 10:13 AM ada per <adaper3@gmail.com> wrote:
Can you please share the /var/log/openvswitch/ovn-controller.log from
the host?
If you configure a subnet on the network, gets the VM an IP address?
the /var/log/openvswitch/ovn-controller.log its actually empty. So Iam Attaching yesterdays log (it wasn’t connecting but after removing and reinstalling the hosts it connects)
Even with specifying a subnet still no IPs.
btw the openvswitch is active
openvswitch.service - Open vSwitch
Loaded: loaded (/usr/lib/systemd/system/openvswitch.service; enabled; vendor preset: disabled)
Active: active (exited) since Tue 2019-10-08 16:02:12 EEST; 18h ago
You upgraded from which version ?
Upgraded from 4 .3.4 to 4.3.6.7-1.el7 it currently runs openvswitch 2.10.1
I find it strange that you weren't prompted to update ovs to openvswitch-2.11.0-4.el7 , which is the currently tagged version for ovirt-4.3 and master.
If you do yum update openvswitch, doesn't it spot the update possibility ?
I have another environment running on a previous version of ovirt and
it works perfectly there. I think its a bug.
Which version of ovirt is being run on that environment ? What's the openvswitch / openvswitch-ovn-central version running on that env ?
Ovirt Version 4.3.4.3-1.el7
Openvswitch 2.10.1 ( so both environments run the same
version of openvswitch)
Thinking a couple steps ahead, knowing the list of registered chassis could be useful. Please also tell us the output of the command (run it on your engine node): $ ovn-sbctl list chassis
# ovn-sbctl list chassis
net_mlx5: cannot load glue library: libibverbs.so.1: cannot open shared
object file: No such file or directory
net_mlx5: cannot initialize PMD due to missing run-time dependency on
rdma-core libraries (libibverbs, libmlx5)
PMD: net_mlx4: cannot load glue library: libibverbs.so.1: cannot open
shared obj ect file: No such file or directory
PMD: net_mlx4: cannot initialize PMD due to missing run-time dependency
on rdma-core libraries (libibverbs, libmlx4)
Sorry for the ugly logged errors; they're just noise.
Despite that, the absence of registered chassis is quite surely why your environment is misbehaving - it looks like the ovirt update messed up the previous config.
Could you follow the instructions at [0] to re-configure OVN ? After the playbook finishes, please re-check your chassis table (ovn-sbctl list chassis). You should find exactly one entry per ovirt-node.
[0] - https://www.ovirt.org/documentation/admin-guide/chap-External_Providers.html...
On Tue, Oct 8, 2019 at 6:10 PM Miguel Duarte de Mora Barroso <
mdbarroso@redhat.com> wrote:
On Tue, Oct 8, 2019 at 5:02 PM Miguel Duarte de Mora Barroso <mdbarroso@redhat.com> wrote:
On Tue, Oct 8, 2019 at 4:53 PM Dominik Holler <dholler@redhat.com>
wrote:
On Tue, Oct 8, 2019 at 4:25 PM ada per <adaper3@gmail.com> wrote:
After upgrading to the latest stable version the external networks
lost all functionality. Under providers-->ovn-network-provider the test runs successfully.
You upgraded from which version ?
But when im creating an external provider network, attaching it to
a router as a LAN setting up dhcp lease it is not reachable from other VMs in the same network. Hosts and hosted engine can't seem to ping it either.
I tried disabling firewalls on both hosted engines, VMs and host and still nothing. When configuring Logical networks or VLANs they work perfectly tje one problem is the external networks.
I have another environment running on a previous version of ovirt and it works perfectly there. I think its a bug.
Which version of ovirt is being run on that environment ? What's the openvswitch / openvswitch-ovn-central version running on that env ?
Thinking a couple steps ahead, knowing the list of registered chassis could be useful. Please also tell us the output of the command (run it on your engine node): $ ovn-sbctl list chassis
Can you please share the /var/log/openvswitch/ovn-controller.log
from the host?
If you configure a subnet on the network, gets the VM an IP address?
Thanks for your help
I am also attaching a screenshot of the host network setup. Apparently the hosts can see the external networks and they get synchronized properly. Is there a chance that the issue might be from open-stack security? i disabled the port sec though On Wed, Oct 9, 2019 at 2:12 PM ada per <adaper3@gmail.com> wrote:
unfortunately I get the following error when running the playbook.
TASK [ovirt-provider-ovn-driver : Configure OVN for oVirt] **************************************************************************************************************************************************** fatal: [threatrealm3.sec.ouc.ac.cy]: FAILED! => { "changed": true, "cmd": [ "vdsm-tool", "ovn-config", "192.168.1.21", "ovirt-provider-ovn" ], "delta": "0:00:02.993452", "end": "2019-10-09 14:09:57.555302", "rc": 1, "start": "2019-10-09 14:09:54.561850" }
STDERR:
Traceback (most recent call last): File "/usr/bin/vdsm-tool", line 220, in main return tool_command[cmd]["command"](*args) File "/usr/lib/python2.7/site-packages/vdsm/tool/ovn_config.py", line 64, in ovn_config ip_address = get_ip_addr(get_network(network_caps(), net_name)) File "/usr/lib/python2.7/site-packages/vdsm/tool/ovn_config.py", line 118, in get_network raise NetworkNotFoundError(net_name) NetworkNotFoundError: ovirt-provider-ovn
MSG:
non-zero return code
fatal: [threatrealm2.sec.ouc.ac.cy]: FAILED! => { "changed": true, "cmd": [ "vdsm-tool", "ovn-config", "192.168.1.21", "ovirt-provider-ovn" ], "delta": "0:00:03.033376", "end": "2019-10-09 14:09:57.650353", "rc": 1, "start": "2019-10-09 14:09:54.616977" }
STDERR:
Traceback (most recent call last): File "/usr/bin/vdsm-tool", line 220, in main return tool_command[cmd]["command"](*args) File "/usr/lib/python2.7/site-packages/vdsm/tool/ovn_config.py", line 64, in ovn_config ip_address = get_ip_addr(get_network(network_caps(), net_name)) File "/usr/lib/python2.7/site-packages/vdsm/tool/ovn_config.py", line 118, in get_network raise NetworkNotFoundError(net_name) NetworkNotFoundError: ovirt-provider-ovn
MSG:
non-zero return code
PLAY RECAP **************************************************************************************************************************************************************************************************** localhost : ok=1 changed=0 unreachable=0 failed=0 skipped=2 rescued=0 ignored=0 threatrealm2.sec.ouc.ac.cy : ok=5 changed=0 unreachable=0 failed=1 skipped=0 rescued=0 ignored=0 threatrealm3.sec.ouc.ac.cy : ok=5 changed=0 unreachable=0 failed=1 skipped=0 rescued=0 ignored=0
On Wed, Oct 9, 2019 at 11:36 AM Miguel Duarte de Mora Barroso < mdbarroso@redhat.com> wrote:
On Wed, Oct 9, 2019 at 10:13 AM ada per <adaper3@gmail.com> wrote:
Can you please share the /var/log/openvswitch/ovn-controller.log from
the host?
If you configure a subnet on the network, gets the VM an IP address?
the /var/log/openvswitch/ovn-controller.log its actually empty. So Iam Attaching yesterdays log (it wasn’t connecting but after removing and reinstalling the hosts it connects)
Even with specifying a subnet still no IPs.
btw the openvswitch is active
openvswitch.service - Open vSwitch
Loaded: loaded (/usr/lib/systemd/system/openvswitch.service; enabled; vendor preset: disabled)
Active: active (exited) since Tue 2019-10-08 16:02:12 EEST; 18h ago
You upgraded from which version ?
Upgraded from 4 .3.4 to 4.3.6.7-1.el7 it currently runs openvswitch 2.10.1
I find it strange that you weren't prompted to update ovs to openvswitch-2.11.0-4.el7 , which is the currently tagged version for ovirt-4.3 and master.
If you do yum update openvswitch, doesn't it spot the update possibility ?
I have another environment running on a previous version of ovirt
and it works perfectly there. I think its a bug.
Which version of ovirt is being run on that environment ? What's the openvswitch / openvswitch-ovn-central version running on that env ?
Ovirt Version 4.3.4.3-1.el7
Openvswitch 2.10.1 ( so both environments run the same
version of openvswitch)
Thinking a couple steps ahead, knowing the list of registered chassis could be useful. Please also tell us the output of the command (run it on your engine node): $ ovn-sbctl list chassis
# ovn-sbctl list chassis
net_mlx5: cannot load glue library: libibverbs.so.1: cannot open shared
object file: No such file or directory
net_mlx5: cannot initialize PMD due to missing run-time dependency on
rdma-core libraries (libibverbs, libmlx5)
PMD: net_mlx4: cannot load glue library: libibverbs.so.1: cannot open
shared obj ect file: No such file or directory
PMD: net_mlx4: cannot initialize PMD due to missing run-time dependency
on rdma-core libraries (libibverbs, libmlx4)
Sorry for the ugly logged errors; they're just noise.
Despite that, the absence of registered chassis is quite surely why your environment is misbehaving - it looks like the ovirt update messed up the previous config.
Could you follow the instructions at [0] to re-configure OVN ? After the playbook finishes, please re-check your chassis table (ovn-sbctl list chassis). You should find exactly one entry per ovirt-node.
[0] - https://www.ovirt.org/documentation/admin-guide/chap-External_Providers.html...
On Tue, Oct 8, 2019 at 6:10 PM Miguel Duarte de Mora Barroso <
mdbarroso@redhat.com> wrote:
On Tue, Oct 8, 2019 at 5:02 PM Miguel Duarte de Mora Barroso <mdbarroso@redhat.com> wrote:
On Tue, Oct 8, 2019 at 4:53 PM Dominik Holler <dholler@redhat.com>
wrote:
On Tue, Oct 8, 2019 at 4:25 PM ada per <adaper3@gmail.com> wrote: > > After upgrading to the latest stable version the external
networks lost all functionality. Under providers-->ovn-network-provider the test runs successfully.
You upgraded from which version ?
> > But when im creating an external provider network, attaching it to a router as a LAN setting up dhcp lease it is not reachable from other VMs in the same network. Hosts and hosted engine can't seem to ping it either. > I tried disabling firewalls on both hosted engines, VMs and host and still nothing. > When configuring Logical networks or VLANs they work perfectly tje one problem is the external networks. > > I have another environment running on a previous version of ovirt and it works perfectly there. I think its a bug.
Which version of ovirt is being run on that environment ? What's the openvswitch / openvswitch-ovn-central version running on that env ?
Thinking a couple steps ahead, knowing the list of registered chassis could be useful. Please also tell us the output of the command (run it on your engine node): $ ovn-sbctl list chassis
>
Can you please share the /var/log/openvswitch/ovn-controller.log
from the host?
If you configure a subnet on the network, gets the VM an IP address?
> > > Thanks for your help
On Wed, Oct 9, 2019 at 1:28 PM ada per <adaper3@gmail.com> wrote:
I am also attaching a screenshot of the host network setup. Apparently the hosts can see the external networks and they get synchronized properly. Is there a chance that the issue might be from open-stack security? i disabled the port sec though
On Wed, Oct 9, 2019 at 2:12 PM ada per <adaper3@gmail.com> wrote:
unfortunately I get the following error when running the playbook.
TASK [ovirt-provider-ovn-driver : Configure OVN for oVirt] **************************************************************************************************************************************************** fatal: [threatrealm3.sec.ouc.ac.cy]: FAILED! => { "changed": true, "cmd": [ "vdsm-tool", "ovn-config", "192.168.1.21", "ovirt-provider-ovn" ], "delta": "0:00:02.993452", "end": "2019-10-09 14:09:57.555302", "rc": 1, "start": "2019-10-09 14:09:54.561850" }
Here you should run with ovirtmgmt as network name - from your network's screenshot, I think that's the correct thing to do. This basically specifies on which network will OVN set up its overlay. Also, it's important to make sure that the IP you're using as "ovn_central" - 192.168.1.21 - is the ovirt-engine IP address on the network you're passing as argument - ovirtmgmt.
STDERR:
Traceback (most recent call last): File "/usr/bin/vdsm-tool", line 220, in main return tool_command[cmd]["command"](*args) File "/usr/lib/python2.7/site-packages/vdsm/tool/ovn_config.py", line 64, in ovn_config ip_address = get_ip_addr(get_network(network_caps(), net_name)) File "/usr/lib/python2.7/site-packages/vdsm/tool/ovn_config.py", line 118, in get_network raise NetworkNotFoundError(net_name) NetworkNotFoundError: ovirt-provider-ovn
MSG:
non-zero return code
fatal: [threatrealm2.sec.ouc.ac.cy]: FAILED! => { "changed": true, "cmd": [ "vdsm-tool", "ovn-config", "192.168.1.21", "ovirt-provider-ovn" ], "delta": "0:00:03.033376", "end": "2019-10-09 14:09:57.650353", "rc": 1, "start": "2019-10-09 14:09:54.616977" }
STDERR:
Traceback (most recent call last): File "/usr/bin/vdsm-tool", line 220, in main return tool_command[cmd]["command"](*args) File "/usr/lib/python2.7/site-packages/vdsm/tool/ovn_config.py", line 64, in ovn_config ip_address = get_ip_addr(get_network(network_caps(), net_name)) File "/usr/lib/python2.7/site-packages/vdsm/tool/ovn_config.py", line 118, in get_network raise NetworkNotFoundError(net_name) NetworkNotFoundError: ovirt-provider-ovn
MSG:
non-zero return code
PLAY RECAP **************************************************************************************************************************************************************************************************** localhost : ok=1 changed=0 unreachable=0 failed=0 skipped=2 rescued=0 ignored=0 threatrealm2.sec.ouc.ac.cy : ok=5 changed=0 unreachable=0 failed=1 skipped=0 rescued=0 ignored=0 threatrealm3.sec.ouc.ac.cy : ok=5 changed=0 unreachable=0 failed=1 skipped=0 rescued=0 ignored=0
On Wed, Oct 9, 2019 at 11:36 AM Miguel Duarte de Mora Barroso <mdbarroso@redhat.com> wrote:
On Wed, Oct 9, 2019 at 10:13 AM ada per <adaper3@gmail.com> wrote:
Can you please share the /var/log/openvswitch/ovn-controller.log from the host? If you configure a subnet on the network, gets the VM an IP address?
the /var/log/openvswitch/ovn-controller.log its actually empty. So Iam Attaching yesterdays log (it wasn’t connecting but after removing and reinstalling the hosts it connects)
Even with specifying a subnet still no IPs.
btw the openvswitch is active
openvswitch.service - Open vSwitch
Loaded: loaded (/usr/lib/systemd/system/openvswitch.service; enabled; vendor preset: disabled)
Active: active (exited) since Tue 2019-10-08 16:02:12 EEST; 18h ago
You upgraded from which version ?
Upgraded from 4 .3.4 to 4.3.6.7-1.el7 it currently runs openvswitch 2.10.1
I find it strange that you weren't prompted to update ovs to openvswitch-2.11.0-4.el7 , which is the currently tagged version for ovirt-4.3 and master.
If you do yum update openvswitch, doesn't it spot the update possibility ?
I have another environment running on a previous version of ovirt and it works perfectly there. I think its a bug.
Which version of ovirt is being run on that environment ? What's the openvswitch / openvswitch-ovn-central version running on that env ?
Ovirt Version 4.3.4.3-1.el7
Openvswitch 2.10.1 ( so both environments run the same version of openvswitch)
Thinking a couple steps ahead, knowing the list of registered chassis could be useful. Please also tell us the output of the command (run it on your engine node): $ ovn-sbctl list chassis
# ovn-sbctl list chassis
net_mlx5: cannot load glue library: libibverbs.so.1: cannot open shared object file: No such file or directory
net_mlx5: cannot initialize PMD due to missing run-time dependency on rdma-core libraries (libibverbs, libmlx5)
PMD: net_mlx4: cannot load glue library: libibverbs.so.1: cannot open shared obj ect file: No such file or directory
PMD: net_mlx4: cannot initialize PMD due to missing run-time dependency on rdma-core libraries (libibverbs, libmlx4)
Sorry for the ugly logged errors; they're just noise.
Despite that, the absence of registered chassis is quite surely why your environment is misbehaving - it looks like the ovirt update messed up the previous config.
Could you follow the instructions at [0] to re-configure OVN ? After the playbook finishes, please re-check your chassis table (ovn-sbctl list chassis). You should find exactly one entry per ovirt-node.
[0] - https://www.ovirt.org/documentation/admin-guide/chap-External_Providers.html...
On Tue, Oct 8, 2019 at 6:10 PM Miguel Duarte de Mora Barroso <mdbarroso@redhat.com> wrote:
On Tue, Oct 8, 2019 at 5:02 PM Miguel Duarte de Mora Barroso <mdbarroso@redhat.com> wrote:
On Tue, Oct 8, 2019 at 4:53 PM Dominik Holler <dholler@redhat.com> wrote: > > > > On Tue, Oct 8, 2019 at 4:25 PM ada per <adaper3@gmail.com> wrote: >> >> After upgrading to the latest stable version the external networks lost all functionality. Under providers-->ovn-network-provider the test runs successfully.
You upgraded from which version ?
>> >> But when im creating an external provider network, attaching it to a router as a LAN setting up dhcp lease it is not reachable from other VMs in the same network. Hosts and hosted engine can't seem to ping it either. >> I tried disabling firewalls on both hosted engines, VMs and host and still nothing. >> When configuring Logical networks or VLANs they work perfectly tje one problem is the external networks. >> >> I have another environment running on a previous version of ovirt and it works perfectly there. I think its a bug.
Which version of ovirt is being run on that environment ? What's the openvswitch / openvswitch-ovn-central version running on that env ?
Thinking a couple steps ahead, knowing the list of registered chassis could be useful. Please also tell us the output of the command (run it on your engine node): $ ovn-sbctl list chassis
>> > > Can you please share the /var/log/openvswitch/ovn-controller.log from the host? > If you configure a subnet on the network, gets the VM an IP address? > > >> >> >> Thanks for your help
Done!! thank you. it worked now but the ovn-sbctl list chassis finds nothing results fom playbook PLAY RECAP **************************************************************************************************************************************************************************************************** localhost : ok=1 changed=0 unreachable=0 failed=0 skipped=2 rescued=0 ignored=0 threatrealm2.sec.ouc.ac.cy : ok=6 changed=1 unreachable=0 failed=0 skipped=3 rescued=0 ignored=0 threatrealm3.sec.ouc.ac.cy : ok=6 changed=1 unreachable=0 failed=0 skipped=3 rescued=0 ignored=0 results form ovn-sbctl list chassis: net_mlx5: cannot load glue library: libibverbs.so.1: cannot open shared object file: No such file or directory net_mlx5: cannot initialize PMD due to missing run-time dependency on rdma-core libraries (libibverbs, libmlx5) PMD: net_mlx4: cannot load glue library: libibverbs.so.1: cannot open shared object file: No such file or directory PMD: net_mlx4: cannot initialize PMD due to missing run-time dependency on rdma-core libraries (libibverbs, libmlx4)
On Wed, Oct 9, 2019 at 1:50 PM ada per <adaper3@gmail.com> wrote:
Done!! thank you. it worked now but the ovn-sbctl list chassis finds nothing
Good. Some other things: - what's the output of 'ip a' on ovirt-engine ? - ssh into one of the hosts in that cluster and do 'ovs-vsctl list Open' With this we can check that the OVN configuration is correct.
results fom playbook
PLAY RECAP **************************************************************************************************************************************************************************************************** localhost : ok=1 changed=0 unreachable=0 failed=0 skipped=2 rescued=0 ignored=0 threatrealm2.sec.ouc.ac.cy : ok=6 changed=1 unreachable=0 failed=0 skipped=3 rescued=0 ignored=0 threatrealm3.sec.ouc.ac.cy : ok=6 changed=1 unreachable=0 failed=0 skipped=3 rescued=0 ignored=0
results form ovn-sbctl list chassis:
net_mlx5: cannot load glue library: libibverbs.so.1: cannot open shared object file: No such file or directory net_mlx5: cannot initialize PMD due to missing run-time dependency on rdma-core libraries (libibverbs, libmlx5) PMD: net_mlx4: cannot load glue library: libibverbs.so.1: cannot open shared object file: No such file or directory PMD: net_mlx4: cannot initialize PMD due to missing run-time dependency on rdma-core libraries (libibverbs, libmlx4)
ip a 1: lo: <LOOPBACK,UP,LOWER_UP> mtu 65536 qdisc noqueue state UNKNOWN group default qlen 1000 link/loopback 00:00:00:00:00:00 brd 00:00:00:00:00:00 inet 127.0.0.1/8 scope host lo valid_lft forever preferred_lft forever inet6 ::1/128 scope host valid_lft forever preferred_lft forever 2: eth0: <BROADCAST,MULTICAST,UP,LOWER_UP> mtu 1500 qdisc mq state UP group default qlen 1000 link/ether 00:16:3e:32:c8:ff brd ff:ff:ff:ff:ff:ff inet 192.168.1.21/24 brd 192.168.1.255 scope global eth0 valid_lft forever preferred_lft forever inet6 fe80::216:3eff:fe32:c8ff/64 scope link valid_lft forever preferred_lft forever 3: eth1: <BROADCAST,MULTICAST,UP,LOWER_UP> mtu 1500 qdisc mq state UP group default qlen 1000 link/ether 56:6f:74:a4:00:0d brd ff:ff:ff:ff:ff:ff inet6 fe80::abc9:3c51:1a61:ab7a/64 scope link noprefixroute valid_lft forever preferred_lft forever and [root@threatrealm3 ~]# ovs-vsctl list Open _uuid : 637b29cb-3874-4356-9d16-f4e0e4ddf212 bridges : [f805bd1c-3d65-4f46-a1f4-2a337bfbfe60] cur_cfg : 538 datapath_types : [netdev, system] db_version : "7.16.1" dpdk_initialized : false dpdk_version : "DPDK 18.11.0" external_ids : {hostname="threatrealm3.sec.ouc.ac.cy", ovn-bridge-mapping s="", ovn-encap-ip="192.168.1.26", ovn-encap-type=geneve, ovn-remote="ssl:192.16 8.1.21:6642", rundir="/var/run/openvswitch", system-id="d3240f0e-50ca-43e9-b7de- c1a36587f609"} iface_types : [erspan, geneve, gre, internal, "ip6erspan", "ip6gre", lis p, patch, stt, system, tap, vxlan] manager_options : [] next_cfg : 538 other_config : {} ovs_version : "2.11.0" ssl : [] statistics : {} system_type : centos system_version : "7" [root@threatrealm2 ~]# ovs-vsctl list Open _uuid : 79d5588a-c68d-4615-8ce7-7be785fe8bfd bridges : [e52f952a-e7cb-4ead-83c7-18b0c5b0d2e9] cur_cfg : 997 datapath_types : [netdev, system] db_version : "7.16.1" dpdk_initialized : false dpdk_version : "DPDK 18.11.0" external_ids : {hostname="threatrealm2.sec.ouc.ac.cy", ovn-bridge-mappings="", ovn-encap-ip="192.168.1.22", ovn-encap-type=geneve, ovn-remote="ssl:192.168.1.21:6642", rundir="/var/run/openvswitch", system-id="96e6f622-2834-4e91-ab31-0df3f21788f2"} iface_types : [erspan, geneve, gre, internal, "ip6erspan", "ip6gre", lisp, patch, stt, system, tap, vxlan] manager_options : [] next_cfg : 997 other_config : {} ovs_version : "2.11.0" ssl : [] statistics : {} system_type : centos system_version : "7" On Wed, Oct 9, 2019 at 4:16 PM Miguel Duarte de Mora Barroso < mdbarroso@redhat.com> wrote:
On Wed, Oct 9, 2019 at 1:50 PM ada per <adaper3@gmail.com> wrote:
Done!! thank you. it worked now but the ovn-sbctl list chassis finds
nothing
Good.
Some other things: - what's the output of 'ip a' on ovirt-engine ? - ssh into one of the hosts in that cluster and do 'ovs-vsctl list Open'
With this we can check that the OVN configuration is correct.
results fom playbook
PLAY RECAP
****************************************************************************************************************************************************************************************************
localhost : ok=1 changed=0 unreachable=0 failed=0 skipped=2 rescued=0 ignored=0 threatrealm2.sec.ouc.ac.cy : ok=6 changed=1 unreachable=0 failed=0 skipped=3 rescued=0 ignored=0 threatrealm3.sec.ouc.ac.cy : ok=6 changed=1 unreachable=0 failed=0 skipped=3 rescued=0 ignored=0
results form ovn-sbctl list chassis:
net_mlx5: cannot load glue library: libibverbs.so.1: cannot open shared object file: No such file or directory net_mlx5: cannot initialize PMD due to missing run-time dependency on rdma-core libraries (libibverbs, libmlx5) PMD: net_mlx4: cannot load glue library: libibverbs.so.1: cannot open shared object file: No such file or directory PMD: net_mlx4: cannot initialize PMD due to missing run-time dependency on rdma-core libraries (libibverbs, libmlx4)
any input??? On Wed, 9 Oct 2019, 16:50 ada per, <adaper3@gmail.com> wrote:
ip a 1: lo: <LOOPBACK,UP,LOWER_UP> mtu 65536 qdisc noqueue state UNKNOWN group default qlen 1000 link/loopback 00:00:00:00:00:00 brd 00:00:00:00:00:00 inet 127.0.0.1/8 scope host lo valid_lft forever preferred_lft forever inet6 ::1/128 scope host valid_lft forever preferred_lft forever 2: eth0: <BROADCAST,MULTICAST,UP,LOWER_UP> mtu 1500 qdisc mq state UP group default qlen 1000 link/ether 00:16:3e:32:c8:ff brd ff:ff:ff:ff:ff:ff inet 192.168.1.21/24 brd 192.168.1.255 scope global eth0 valid_lft forever preferred_lft forever inet6 fe80::216:3eff:fe32:c8ff/64 scope link valid_lft forever preferred_lft forever 3: eth1: <BROADCAST,MULTICAST,UP,LOWER_UP> mtu 1500 qdisc mq state UP group default qlen 1000 link/ether 56:6f:74:a4:00:0d brd ff:ff:ff:ff:ff:ff inet6 fe80::abc9:3c51:1a61:ab7a/64 scope link noprefixroute valid_lft forever preferred_lft forever
and
[root@threatrealm3 ~]# ovs-vsctl list Open _uuid : 637b29cb-3874-4356-9d16-f4e0e4ddf212 bridges : [f805bd1c-3d65-4f46-a1f4-2a337bfbfe60] cur_cfg : 538 datapath_types : [netdev, system] db_version : "7.16.1" dpdk_initialized : false dpdk_version : "DPDK 18.11.0" external_ids : {hostname="threatrealm3.sec.ouc.ac.cy", ovn-bridge-mapping s="", ovn-encap-ip="192.168.1.26", ovn-encap-type=geneve, ovn-remote="ssl:192.16
8.1.21:6642", rundir="/var/run/openvswitch", system-id="d3240f0e-50ca-43e9-b7de-
c1a36587f609"} iface_types : [erspan, geneve, gre, internal, "ip6erspan", "ip6gre", lis p, patch, stt, system, tap, vxlan] manager_options : [] next_cfg : 538 other_config : {} ovs_version : "2.11.0" ssl : [] statistics : {} system_type : centos system_version : "7"
[root@threatrealm2 ~]# ovs-vsctl list Open _uuid : 79d5588a-c68d-4615-8ce7-7be785fe8bfd bridges : [e52f952a-e7cb-4ead-83c7-18b0c5b0d2e9] cur_cfg : 997 datapath_types : [netdev, system] db_version : "7.16.1" dpdk_initialized : false dpdk_version : "DPDK 18.11.0" external_ids : {hostname="threatrealm2.sec.ouc.ac.cy", ovn-bridge-mappings="", ovn-encap-ip="192.168.1.22", ovn-encap-type=geneve, ovn-remote="ssl:192.168.1.21:6642", rundir="/var/run/openvswitch", system-id="96e6f622-2834-4e91-ab31-0df3f21788f2"} iface_types : [erspan, geneve, gre, internal, "ip6erspan", "ip6gre", lisp, patch, stt, system, tap, vxlan] manager_options : [] next_cfg : 997 other_config : {} ovs_version : "2.11.0" ssl : [] statistics : {} system_type : centos system_version : "7"
On Wed, Oct 9, 2019 at 4:16 PM Miguel Duarte de Mora Barroso < mdbarroso@redhat.com> wrote:
On Wed, Oct 9, 2019 at 1:50 PM ada per <adaper3@gmail.com> wrote:
Done!! thank you. it worked now but the ovn-sbctl list chassis finds
nothing
Good.
Some other things: - what's the output of 'ip a' on ovirt-engine ? - ssh into one of the hosts in that cluster and do 'ovs-vsctl list Open'
With this we can check that the OVN configuration is correct.
results fom playbook
PLAY RECAP
****************************************************************************************************************************************************************************************************
localhost : ok=1 changed=0 unreachable=0 failed=0 skipped=2 rescued=0 ignored=0 threatrealm2.sec.ouc.ac.cy : ok=6 changed=1 unreachable=0 failed=0 skipped=3 rescued=0 ignored=0 threatrealm3.sec.ouc.ac.cy : ok=6 changed=1 unreachable=0 failed=0 skipped=3 rescued=0 ignored=0
results form ovn-sbctl list chassis:
net_mlx5: cannot load glue library: libibverbs.so.1: cannot open shared object file: No such file or directory net_mlx5: cannot initialize PMD due to missing run-time dependency on rdma-core libraries (libibverbs, libmlx5) PMD: net_mlx4: cannot load glue library: libibverbs.so.1: cannot open shared object file: No such file or directory PMD: net_mlx4: cannot initialize PMD due to missing run-time dependency on rdma-core libraries (libibverbs, libmlx4)
On Fri, Oct 11, 2019 at 8:24 AM ada per <adaper3@gmail.com> wrote:
any input???
What is written to the ovn-controller.log now? I expect that there is an helpful error explaning the problem while connecting to 192.161.21:6642.
On Wed, 9 Oct 2019, 16:50 ada per, <adaper3@gmail.com> wrote:
ip a 1: lo: <LOOPBACK,UP,LOWER_UP> mtu 65536 qdisc noqueue state UNKNOWN group default qlen 1000 link/loopback 00:00:00:00:00:00 brd 00:00:00:00:00:00 inet 127.0.0.1/8 scope host lo valid_lft forever preferred_lft forever inet6 ::1/128 scope host valid_lft forever preferred_lft forever 2: eth0: <BROADCAST,MULTICAST,UP,LOWER_UP> mtu 1500 qdisc mq state UP group default qlen 1000 link/ether 00:16:3e:32:c8:ff brd ff:ff:ff:ff:ff:ff inet 192.168.1.21/24 brd 192.168.1.255 scope global eth0 valid_lft forever preferred_lft forever inet6 fe80::216:3eff:fe32:c8ff/64 scope link valid_lft forever preferred_lft forever 3: eth1: <BROADCAST,MULTICAST,UP,LOWER_UP> mtu 1500 qdisc mq state UP group default qlen 1000 link/ether 56:6f:74:a4:00:0d brd ff:ff:ff:ff:ff:ff inet6 fe80::abc9:3c51:1a61:ab7a/64 scope link noprefixroute valid_lft forever preferred_lft forever
and
[root@threatrealm3 ~]# ovs-vsctl list Open _uuid : 637b29cb-3874-4356-9d16-f4e0e4ddf212 bridges : [f805bd1c-3d65-4f46-a1f4-2a337bfbfe60] cur_cfg : 538 datapath_types : [netdev, system] db_version : "7.16.1" dpdk_initialized : false dpdk_version : "DPDK 18.11.0" external_ids : {hostname="threatrealm3.sec.ouc.ac.cy", ovn-bridge-mapping s="", ovn-encap-ip="192.168.1.26", ovn-encap-type=geneve, ovn-remote="ssl:192.16
8.1.21:6642", rundir="/var/run/openvswitch", system-id="d3240f0e-50ca-43e9-b7de-
c1a36587f609"} iface_types : [erspan, geneve, gre, internal, "ip6erspan", "ip6gre", lis p, patch, stt, system, tap, vxlan] manager_options : [] next_cfg : 538 other_config : {} ovs_version : "2.11.0" ssl : [] statistics : {} system_type : centos system_version : "7"
[root@threatrealm2 ~]# ovs-vsctl list Open _uuid : 79d5588a-c68d-4615-8ce7-7be785fe8bfd bridges : [e52f952a-e7cb-4ead-83c7-18b0c5b0d2e9] cur_cfg : 997 datapath_types : [netdev, system] db_version : "7.16.1" dpdk_initialized : false dpdk_version : "DPDK 18.11.0" external_ids : {hostname="threatrealm2.sec.ouc.ac.cy", ovn-bridge-mappings="", ovn-encap-ip="192.168.1.22", ovn-encap-type=geneve, ovn-remote="ssl:192.168.1.21:6642", rundir="/var/run/openvswitch", system-id="96e6f622-2834-4e91-ab31-0df3f21788f2"} iface_types : [erspan, geneve, gre, internal, "ip6erspan", "ip6gre", lisp, patch, stt, system, tap, vxlan] manager_options : [] next_cfg : 997 other_config : {} ovs_version : "2.11.0" ssl : [] statistics : {} system_type : centos system_version : "7"
On Wed, Oct 9, 2019 at 4:16 PM Miguel Duarte de Mora Barroso < mdbarroso@redhat.com> wrote:
On Wed, Oct 9, 2019 at 1:50 PM ada per <adaper3@gmail.com> wrote:
Done!! thank you. it worked now but the ovn-sbctl list chassis finds
nothing
Good.
Some other things: - what's the output of 'ip a' on ovirt-engine ? - ssh into one of the hosts in that cluster and do 'ovs-vsctl list Open'
With this we can check that the OVN configuration is correct.
results fom playbook
PLAY RECAP
****************************************************************************************************************************************************************************************************
localhost : ok=1 changed=0 unreachable=0 failed=0 skipped=2 rescued=0 ignored=0 threatrealm2.sec.ouc.ac.cy : ok=6 changed=1 unreachable=0 failed=0 skipped=3 rescued=0 ignored=0 threatrealm3.sec.ouc.ac.cy : ok=6 changed=1 unreachable=0 failed=0 skipped=3 rescued=0 ignored=0
results form ovn-sbctl list chassis:
net_mlx5: cannot load glue library: libibverbs.so.1: cannot open shared object file: No such file or directory net_mlx5: cannot initialize PMD due to missing run-time dependency on rdma-core libraries (libibverbs, libmlx5) PMD: net_mlx4: cannot load glue library: libibverbs.so.1: cannot open shared object file: No such file or directory PMD: net_mlx4: cannot initialize PMD due to missing run-time dependency on rdma-core libraries (libibverbs, libmlx4)
Thank you for all your help! Unfortunately the log is empty! Im now reinstalling the environment from scratch On Fri, Oct 11, 2019 at 9:32 AM Dominik Holler <dholler@redhat.com> wrote:
On Fri, Oct 11, 2019 at 8:24 AM ada per <adaper3@gmail.com> wrote:
any input???
What is written to the ovn-controller.log now? I expect that there is an helpful error explaning the problem while connecting to 192.161.21:6642.
On Wed, 9 Oct 2019, 16:50 ada per, <adaper3@gmail.com> wrote:
ip a 1: lo: <LOOPBACK,UP,LOWER_UP> mtu 65536 qdisc noqueue state UNKNOWN group default qlen 1000 link/loopback 00:00:00:00:00:00 brd 00:00:00:00:00:00 inet 127.0.0.1/8 scope host lo valid_lft forever preferred_lft forever inet6 ::1/128 scope host valid_lft forever preferred_lft forever 2: eth0: <BROADCAST,MULTICAST,UP,LOWER_UP> mtu 1500 qdisc mq state UP group default qlen 1000 link/ether 00:16:3e:32:c8:ff brd ff:ff:ff:ff:ff:ff inet 192.168.1.21/24 brd 192.168.1.255 scope global eth0 valid_lft forever preferred_lft forever inet6 fe80::216:3eff:fe32:c8ff/64 scope link valid_lft forever preferred_lft forever 3: eth1: <BROADCAST,MULTICAST,UP,LOWER_UP> mtu 1500 qdisc mq state UP group default qlen 1000 link/ether 56:6f:74:a4:00:0d brd ff:ff:ff:ff:ff:ff inet6 fe80::abc9:3c51:1a61:ab7a/64 scope link noprefixroute valid_lft forever preferred_lft forever
and
[root@threatrealm3 ~]# ovs-vsctl list Open _uuid : 637b29cb-3874-4356-9d16-f4e0e4ddf212 bridges : [f805bd1c-3d65-4f46-a1f4-2a337bfbfe60] cur_cfg : 538 datapath_types : [netdev, system] db_version : "7.16.1" dpdk_initialized : false dpdk_version : "DPDK 18.11.0" external_ids : {hostname="threatrealm3.sec.ouc.ac.cy", ovn-bridge-mapping s="", ovn-encap-ip="192.168.1.26", ovn-encap-type=geneve, ovn-remote="ssl:192.16
8.1.21:6642", rundir="/var/run/openvswitch", system-id="d3240f0e-50ca-43e9-b7de-
c1a36587f609"} iface_types : [erspan, geneve, gre, internal, "ip6erspan", "ip6gre", lis p, patch, stt, system, tap, vxlan] manager_options : [] next_cfg : 538 other_config : {} ovs_version : "2.11.0" ssl : [] statistics : {} system_type : centos system_version : "7"
[root@threatrealm2 ~]# ovs-vsctl list Open _uuid : 79d5588a-c68d-4615-8ce7-7be785fe8bfd bridges : [e52f952a-e7cb-4ead-83c7-18b0c5b0d2e9] cur_cfg : 997 datapath_types : [netdev, system] db_version : "7.16.1" dpdk_initialized : false dpdk_version : "DPDK 18.11.0" external_ids : {hostname="threatrealm2.sec.ouc.ac.cy", ovn-bridge-mappings="", ovn-encap-ip="192.168.1.22", ovn-encap-type=geneve, ovn-remote="ssl:192.168.1.21:6642", rundir="/var/run/openvswitch", system-id="96e6f622-2834-4e91-ab31-0df3f21788f2"} iface_types : [erspan, geneve, gre, internal, "ip6erspan", "ip6gre", lisp, patch, stt, system, tap, vxlan] manager_options : [] next_cfg : 997 other_config : {} ovs_version : "2.11.0" ssl : [] statistics : {} system_type : centos system_version : "7"
On Wed, Oct 9, 2019 at 4:16 PM Miguel Duarte de Mora Barroso < mdbarroso@redhat.com> wrote:
On Wed, Oct 9, 2019 at 1:50 PM ada per <adaper3@gmail.com> wrote:
Done!! thank you. it worked now but the ovn-sbctl list chassis finds
nothing
Good.
Some other things: - what's the output of 'ip a' on ovirt-engine ? - ssh into one of the hosts in that cluster and do 'ovs-vsctl list Open'
With this we can check that the OVN configuration is correct.
results fom playbook
PLAY RECAP
****************************************************************************************************************************************************************************************************
localhost : ok=1 changed=0 unreachable=0 failed=0 skipped=2 rescued=0 ignored=0 threatrealm2.sec.ouc.ac.cy : ok=6 changed=1 unreachable=0 failed=0 skipped=3 rescued=0 ignored=0 threatrealm3.sec.ouc.ac.cy : ok=6 changed=1 unreachable=0 failed=0 skipped=3 rescued=0 ignored=0
results form ovn-sbctl list chassis:
net_mlx5: cannot load glue library: libibverbs.so.1: cannot open shared object file: No such file or directory net_mlx5: cannot initialize PMD due to missing run-time dependency on rdma-core libraries (libibverbs, libmlx5) PMD: net_mlx4: cannot load glue library: libibverbs.so.1: cannot open shared object file: No such file or directory PMD: net_mlx4: cannot initialize PMD due to missing run-time dependency on rdma-core libraries (libibverbs, libmlx4)
Hi everyone, Im having the same issue after upgrading from 4.3.4 to 4.3.6 . tried to do what you mention with no luck. here are the outputs: from one host: 2019-11-13T00:49:18.233Z|00003|reconnect|INFO|unix:/var/run/openvswitch/db.sock: connected 2019-11-13T00:49:18.235Z|00004|reconnect|INFO|ssl:192.168.57.239:6642: connecting... 2019-11-13T00:49:18.248Z|00005|reconnect|INFO|ssl:192.168.57.239:6642: connected 2019-11-13T00:49:18.253Z|00006|ofctrl|INFO|unix:/var/run/openvswitch/br-int.mgmt: connecting to switch 2019-11-13T00:49:18.253Z|00007|rconn|INFO|unix:/var/run/openvswitch/br-int.mgmt: connecting... 2019-11-13T00:49:18.253Z|00008|rconn|INFO|unix:/var/run/openvswitch/br-int.mgmt: connected 2019-11-13T00:49:18.253Z|00009|pinctrl|INFO|unix:/var/run/openvswitch/br-int.mgmt: connecting to switch 2019-11-13T00:49:18.253Z|00010|rconn|INFO|unix:/var/run/openvswitch/br-int.mgmt: connecting... 2019-11-13T00:49:18.255Z|00011|rconn|INFO|unix:/var/run/openvswitch/br-int.mgmt: connected 2019-11-13T00:49:18.264Z|00012|dpif_netlink|INFO|The kernel module does not support meters. after running the playbook with the params: localhost : ok=1 changed=0 unreachable=0 failed=0 skipped=2 rescued=0 ignored=0 multivac00.phy.nyx : ok=6 changed=1 unreachable=0 failed=0 skipped=3 rescued=0 ignored=0 multivac01.phy.nyx : ok=6 changed=1 unreachable=0 failed=0 skipped=3 rescued=0 ignored=0 from one host: _uuid : ddb5474b-f5be-4f06-ac8c-812b061821ae bridges : [3d5ee016-a77c-44d6-a559-82515170be4c] cur_cfg : 1 datapath_types : [netdev, system] db_version : "7.16.1" dpdk_initialized : false dpdk_version : "DPDK 18.11.0" external_ids : {hostname="multivac01.phy.nyx", ovn-bridge-mappings="", ovn-encap-ip="192.168.57.241", ovn-encap-type=geneve, ovn-remote="ssl:192.168.57.239:6642", rundir="/var/run/openvswitch", system-id="302680af-2677-41ef-9eaf-62eb75c75748"} iface_types : [erspan, geneve, gre, internal, "ip6erspan", "ip6gre", lisp, patch, stt, system, tap, vxlan] manager_options : [] next_cfg : 1 other_config : {} ovs_version : "2.11.0" ssl : [] statistics : {} system_type : centos system_version : "7" ovn-sbctl list chassis keeps there for pretty long time with no output. thanks for the help in advance =) JP El vie., 11 oct. 2019 a las 5:17, ada per (<adaper3@gmail.com>) escribió:
Thank you for all your help! Unfortunately the log is empty! Im now reinstalling the environment from scratch
On Fri, Oct 11, 2019 at 9:32 AM Dominik Holler <dholler@redhat.com> wrote:
On Fri, Oct 11, 2019 at 8:24 AM ada per <adaper3@gmail.com> wrote:
any input???
What is written to the ovn-controller.log now? I expect that there is an helpful error explaning the problem while connecting to 192.161.21:6642.
On Wed, 9 Oct 2019, 16:50 ada per, <adaper3@gmail.com> wrote:
ip a 1: lo: <LOOPBACK,UP,LOWER_UP> mtu 65536 qdisc noqueue state UNKNOWN group default qlen 1000 link/loopback 00:00:00:00:00:00 brd 00:00:00:00:00:00 inet 127.0.0.1/8 scope host lo valid_lft forever preferred_lft forever inet6 ::1/128 scope host valid_lft forever preferred_lft forever 2: eth0: <BROADCAST,MULTICAST,UP,LOWER_UP> mtu 1500 qdisc mq state UP group default qlen 1000 link/ether 00:16:3e:32:c8:ff brd ff:ff:ff:ff:ff:ff inet 192.168.1.21/24 brd 192.168.1.255 scope global eth0 valid_lft forever preferred_lft forever inet6 fe80::216:3eff:fe32:c8ff/64 scope link valid_lft forever preferred_lft forever 3: eth1: <BROADCAST,MULTICAST,UP,LOWER_UP> mtu 1500 qdisc mq state UP group default qlen 1000 link/ether 56:6f:74:a4:00:0d brd ff:ff:ff:ff:ff:ff inet6 fe80::abc9:3c51:1a61:ab7a/64 scope link noprefixroute valid_lft forever preferred_lft forever
and
[root@threatrealm3 ~]# ovs-vsctl list Open _uuid : 637b29cb-3874-4356-9d16-f4e0e4ddf212 bridges : [f805bd1c-3d65-4f46-a1f4-2a337bfbfe60] cur_cfg : 538 datapath_types : [netdev, system] db_version : "7.16.1" dpdk_initialized : false dpdk_version : "DPDK 18.11.0" external_ids : {hostname="threatrealm3.sec.ouc.ac.cy", ovn-bridge-mapping s="", ovn-encap-ip="192.168.1.26", ovn-encap-type=geneve, ovn-remote="ssl:192.16
8.1.21:6642", rundir="/var/run/openvswitch", system-id="d3240f0e-50ca-43e9-b7de-
c1a36587f609"} iface_types : [erspan, geneve, gre, internal, "ip6erspan", "ip6gre", lis p, patch, stt, system, tap, vxlan] manager_options : [] next_cfg : 538 other_config : {} ovs_version : "2.11.0" ssl : [] statistics : {} system_type : centos system_version : "7"
[root@threatrealm2 ~]# ovs-vsctl list Open _uuid : 79d5588a-c68d-4615-8ce7-7be785fe8bfd bridges : [e52f952a-e7cb-4ead-83c7-18b0c5b0d2e9] cur_cfg : 997 datapath_types : [netdev, system] db_version : "7.16.1" dpdk_initialized : false dpdk_version : "DPDK 18.11.0" external_ids : {hostname="threatrealm2.sec.ouc.ac.cy", ovn-bridge-mappings="", ovn-encap-ip="192.168.1.22", ovn-encap-type=geneve, ovn-remote="ssl:192.168.1.21:6642", rundir="/var/run/openvswitch", system-id="96e6f622-2834-4e91-ab31-0df3f21788f2"} iface_types : [erspan, geneve, gre, internal, "ip6erspan", "ip6gre", lisp, patch, stt, system, tap, vxlan] manager_options : [] next_cfg : 997 other_config : {} ovs_version : "2.11.0" ssl : [] statistics : {} system_type : centos system_version : "7"
On Wed, Oct 9, 2019 at 4:16 PM Miguel Duarte de Mora Barroso < mdbarroso@redhat.com> wrote:
On Wed, Oct 9, 2019 at 1:50 PM ada per <adaper3@gmail.com> wrote:
Done!! thank you. it worked now but the ovn-sbctl list chassis
finds nothing
Good.
Some other things: - what's the output of 'ip a' on ovirt-engine ? - ssh into one of the hosts in that cluster and do 'ovs-vsctl list Open'
With this we can check that the OVN configuration is correct.
results fom playbook
PLAY RECAP
****************************************************************************************************************************************************************************************************
localhost : ok=1 changed=0 unreachable=0 failed=0 skipped=2 rescued=0 ignored=0 threatrealm2.sec.ouc.ac.cy : ok=6 changed=1 unreachable=0 failed=0 skipped=3 rescued=0 ignored=0 threatrealm3.sec.ouc.ac.cy : ok=6 changed=1 unreachable=0 failed=0 skipped=3 rescued=0 ignored=0
results form ovn-sbctl list chassis:
net_mlx5: cannot load glue library: libibverbs.so.1: cannot open shared object file: No such file or directory net_mlx5: cannot initialize PMD due to missing run-time dependency on rdma-core libraries (libibverbs, libmlx5) PMD: net_mlx4: cannot load glue library: libibverbs.so.1: cannot open shared object file: No such file or directory PMD: net_mlx4: cannot initialize PMD due to missing run-time dependency on rdma-core libraries (libibverbs, libmlx4)
_______________________________________________
Users mailing list -- users@ovirt.org To unsubscribe send an email to users-leave@ovirt.org Privacy Statement: https://www.ovirt.org/site/privacy-policy/ oVirt Code of Conduct: https://www.ovirt.org/community/about/community-guidelines/ List Archives: https://lists.ovirt.org/archives/list/users@ovirt.org/message/YMXTTJUPCOVUVW...
participants (4)
-
ada per -
Dominik Holler -
Juan Pablo -
Miguel Duarte de Mora Barroso