[ovirt-users] multiple ip routing table issue
Edward Clay
edward.clay at uk2group.com
Tue Nov 21 23:26:04 UTC 2017
On Tue, 2017-11-21 at 16:01 -0700, Edward Clay wrote:
> On Wed, 2017-11-22 at 00:17 +0200, Edward Haas wrote:
> > On Tue, Nov 21, 2017 at 6:16 PM, Edward Clay <edward.clay at uk2group.
> > com> wrote:
> > > On Tue, 2017-11-21 at 09:00 +0200, Edward Haas wrote:
> > > > On Tue, Nov 21, 2017 at 1:24 AM, Edward Clay <edward.clay at uk2gr
> > > > oup.com> wrote:
> > > > > Hello,
> > > > >
> > > > > We have an issue where hosts are configured with the public
> > > > > facing nework interface as the ovirtmgmt network and it's
> > > > > default route is added to a ovirt created table but not to
> > > > > the main routing table. From my searching I've found this
> > > > > snippet from https://www.ovirt.org/develop/release-management
> > > > > /features/network/multiple-gateways/ which seems to explain
> > > > > why I can't ping anything or communicate with any other
> > > > > system needing a default route.
> > > >
> > > > By default, the default route is set on the ovirtmgmt network
> > > > (the default one, defined on the interface/ip which you added
> > > > the host to Engine).
> > > > Do you have a different network set up which you will like to
> > > > set the default route on?
> > > >
> > > >
> > > > > "And finally, here's the host's main routing table. Any
> > > > > traffic coming in to the host will use the ip rules and an
> > > > > interface's routing table. The main routing table is only
> > > > > used for traffic originating from the host."
> > > > >
> > > > > I'm seeing the following main and custom ovirt created
> > > > > tables.
> > > > >
> > > > > main:
> > > > > # ip route show table main
> > > > > 10.0.0.0/8 via 10.4.16.1 dev enp3s0.106
> > > > > 10.4.16.0/24 dev enp3s0.106 proto kernel scope link src
> > > > > 10.4.16.15
> > > > > 1.1.1.0/24 dev PUBLICB proto kernel scope link src
> > > > > 1.1.1.1 169.254.0.0/16 dev enp6s0 scope link metric 1002
> > > > > 169.254.0.0/16 dev enp3s0 scope link metric 1003
> > > > > 169.254.0.0/16 dev enp7s0 scope link metric 1004
> > > > > 169.254.0.0/16 dev enp3s0.106 scope link metric 1020
> > > > > 169.254.0.0/16 dev PRIVATE scope link metric 1022
> > > > > 169.254.0.0/16 dev PUBLIC scope link metric 1024
> > > > >
> > > > > table 1138027711
> > > > > # ip route show table 1138027711
> > > > > default via 1.1.1.1 dev PUBLIC
> > > > > 1.1.1.0/24 via 1.1.1.1 dev PUBLIC
> > > > >
> > > > > If I manually execute the following command to add the
> > > > > default route as well to the main table I can ping ouside of
> > > > > the local network.
> > > > >
> > > > > ip route add 0.0.0.0/0 via 1.1.1.1 dev PUBLIC
> > > > >
> > > > > If I attempt to modify the /etc/sysconfig/network-
> > > > > scripts/route-PUBLIC ad reboot the server ad one would think
> > > > > this file is recreated by vdsm on boot.
> > > > >
> > > > > What I'm looking for is the correct way to setup a default
> > > > > gateway for the main routing table so the hosts can get OS
> > > > > updates and communicate with the outside world.
> > > >
> > > > Providing the output from "ip addr" may help clear up some
> > > > things.
> > > > It looks like you have on the host the default route set as
> > > > 10.4.16.1 (on enp3s0.106), could you elaborate what this
> > > > interface is?
> > >
> > > We have setup vlan taging to utilize the 2 internetal network
> > > interfaces (originally enp6s0 and enp7s0) to be configured with
> > > mulitiple networks each. We eventually added 10Gb nics to all
> > > servers to improve san glusterfs performance which is enp3s0
> > > which replaced enp6s0 in our setup.
> > >
> > > enp3s0.106 = ovirtmgmt network access to private internal
> > > networks only
> > > enp3s0.206 = private network bridge PRIVATE used for private
> > > internal network access for VMs
> > > enp7s0.606 = is used for public access for both VMs (bridge) and
> > > each host/cp/san in our ovirt setup named PUBLIC
> > >
> > > # ip addr show
> > > 1: lo: <LOOPBACK,UP,LOWER_UP> mtu 65536 qdisc noqueue state
> > > UNKNOWN qlen 1
> > > link/loopback 00:00:00:00:00:00 brd 00:00:00:00:00:00
> > > inet 127.0.0.1/8 scope host lo
> > > valid_lft forever preferred_lft forever
> > > inet6 ::1/128 scope host
> > > valid_lft forever preferred_lft forever
> > > 2: enp6s0: <BROADCAST,MULTICAST,UP,LOWER_UP> mtu 1500 qdisc
> > > pfifo_fast state UP qlen 1000
> > > link/ether 00:25:90:38:d6:2c brd ff:ff:ff:ff:ff:ff
> > > inet6 fe80::225:90ff:fe38:d62c/64 scope link
> > > valid_lft forever preferred_lft forever
> > > 3: enp3s0: <BROADCAST,MULTICAST,UP,LOWER_UP> mtu 1500 qdisc mq
> > > state UP qlen 1000
> > > link/ether 90:e2:ba:1d:a4:00 brd ff:ff:ff:ff:ff:ff
> > > inet6 fe80::92e2:baff:fe1d:a400/64 scope link
> > > valid_lft forever preferred_lft forever
> > > 4: enp7s0: <BROADCAST,MULTICAST,UP,LOWER_UP> mtu 1500 qdisc
> > > pfifo_fast state UP qlen 1000
> > > link/ether 00:25:90:38:d6:2d brd ff:ff:ff:ff:ff:ff
> > > 20: enp3s0.106 at enp3s0: <BROADCAST,MULTICAST,UP,LOWER_UP> mtu 1500
> > > qdisc noqueue state UP qlen 1000
> > > link/ether 90:e2:ba:1d:a4:00 brd ff:ff:ff:ff:ff:ff
> > > inet 10.4.16.15/24 brd 10.4.16.255 scope global enp3s0.106
> > > valid_lft forever preferred_lft forever
> > > 21: enp3s0.206 at enp3s0: <BROADCAST,MULTICAST,UP,LOWER_UP> mtu 1500
> > > qdisc noqueue master PRIVATEB state UP qlen 1000
> > > link/ether 90:e2:ba:1d:a4:00 brd ff:ff:ff:ff:ff:ff
> > > 22: PRIVATE: <BROADCAST,MULTICAST,UP,LOWER_UP> mtu 1500 qdisc
> > > noqueue state UP qlen 1000
> > > link/ether 90:e2:ba:1d:a4:00 brd ff:ff:ff:ff:ff:ff
> > > 23: enp7s0.606 at enp7s0: <BROADCAST,MULTICAST,UP,LOWER_UP> mtu 1500
> > > qdisc noqueue master PUBLICB state UP qlen 1000
> > > link/ether 00:25:90:38:d6:2d brd ff:ff:ff:ff:ff:ff
> > > 24: PUBLIC: <BROADCAST,MULTICAST,UP,LOWER_UP> mtu 1500 qdisc
> > > noqueue state UP qlen 1000
> > > link/ether 00:25:90:38:d6:2d brd ff:ff:ff:ff:ff:ff
> > > inet 1.1.1.10/24 brd 1.1.1.255 scope global PUBLICB
> > > valid_lft forever preferred_lft forever
> > > 25: ;vdsmdummy;: <BROADCAST,MULTICAST> mtu 1500 qdisc noop state
> > > DOWN qlen 1000
> > > link/ether 0e:32:93:dd:a4:55 brd ff:ff:ff:ff:ff:ff
> > >
> > >
> > > So all this being said I just need to reconfigure things in a way
> > > that the PUBLIC interface has a default route in the main routing
> > > table. Otherwise all ovirt host are unable to communicate with
> > > the outside world until I manually add a default route to 1.1.1.1
> > > via the PUBLIC interface. Is that possible.
> >
> > It is available in oVirt 4.2 as a network cluster role.
> > The option to assign a default route role to a network: https://www
> > .ovirt.org/documentation/admin-guide/chap-
> > Logical_Networks/#designate-a-specific-traffic-type-for-a-logical-
> > network-with-the-manage-networks-window
> >
> > On 4.1, it is available as a network custom property and its
> > support is limited: See https://bugzilla.redhat.com/show_bug.cgi?id
> > =1200963#c43 and https://gerrit.ovirt.org/#/c/66127
> > Make sure you do not define two networks with the flag on.
> >
> >
> Thanks for the prompt reply. I've taken a look at the link you
> provided for 4.1 and I'm not sure how or where I'm supposed to set
> thid custom property for the interface. Is the patch mentioned in
> the link you provide already included in 4.1 or do I need to do
> something additional to make this work.
>
> I've attempted to edit an existing host network by clicking "setup
> host network" but the page times out with a "page unresponsive" I
> can either exit or wait. Waiting doesn't seem to produce good
> results. Is this where I would adde/edit this custom property?
Looks like I should of read a bit harder before replying. I found the
following two commands on the ovirt change 66127 page.
https://gerrit.ovirt.org/#/c/66127/Note that prior to using a custom
property, one has to define it on
Engine by:
sudo engine-config -g CustomDeviceProperties
sudo engine-config -s
CustomDeviceProperties='{type=interface;prop={default_reoute=^(true|fal
se)$}}' --cver=4.0
and restart of ovirt-engine.
I've done the above and I can edit the vNIC profile on all networks
that have them. The ovirtmgmt network does not have a vNIC profile to
edit. Is this theh correct location to make this change?
How do I make if false for the ovirtmgmt network. The option to add a
new vNIC profile to it isn't available.
Also once this value is set what do I do next?
> >
> > > > Thanks,
> > > > Edy.
> > > >
> > > > > _______________________________________________
> > > > >
> > > > > Users mailing list
> > > > >
> > > > > Users at ovirt.org
> > > > >
> > > > > http://lists.ovirt.org/mailman/listinfo/users
> > > > >
> > > > >
> > > --
> > >
-------------- next part --------------
An HTML attachment was scrubbed...
URL: <http://lists.ovirt.org/pipermail/users/attachments/20171121/3db2fc76/attachment.html>
More information about the Users
mailing list