[ovirt-users] multiple ip routing table issue
Edward Clay
edward.clay at uk2group.com
Wed Nov 22 17:43:48 UTC 2017
On Wed, 2017-11-22 at 10:46 +0200, Edward Haas wrote:
> On Wed, Nov 22, 2017 at 1:26 AM, Edward Clay <edward.clay at uk2group.co
> m> wrote:
> > On Tue, 2017-11-21 at 16:01 -0700, Edward Clay wrote:
> > > On Wed, 2017-11-22 at 00:17 +0200, Edward Haas wrote:
> > > > On Tue, Nov 21, 2017 at 6:16 PM, Edward Clay <edward.clay at uk2gr
> > > > oup.com> wrote:
> > > > > On Tue, 2017-11-21 at 09:00 +0200, Edward Haas wrote:
> > > > > > On Tue, Nov 21, 2017 at 1:24 AM, Edward Clay <edward.clay at u
> > > > > > k2group.com> wrote:
> > > > > > > Hello,
> > > > > > >
> > > > > > > We have an issue where hosts are configured with the
> > > > > > > public facing nework interface as the ovirtmgmt network
> > > > > > > and it's default route is added to a ovirt created table
> > > > > > > but not to the main routing table. From my searching
> > > > > > > I've found this snippet from https://www.ovirt.org/develo
> > > > > > > p/release-management/features/network/multiple-gateways/
> > > > > > > which seems to explain why I can't ping anything or communicate with any other system needing a default route.
> > > > > >
> > > > > > By default, the default route is set on the ovirtmgmt
> > > > > > network (the default one, defined on the interface/ip which
> > > > > > you added the host to Engine).
> > > > > > Do you have a different network set up which you will like
> > > > > > to set the default route on?
> > > > > >
> > > > > >
> > > > > > > "And finally, here's the host's main routing table. Any
> > > > > > > traffic coming in to the host will use the ip rules and
> > > > > > > an interface's routing table. The main routing table is
> > > > > > > only used for traffic originating from the host."
> > > > > > >
> > > > > > > I'm seeing the following main and custom ovirt created
> > > > > > > tables.
> > > > > > >
> > > > > > > main:
> > > > > > > # ip route show table main
> > > > > > > 10.0.0.0/8 via 10.4.16.1 dev enp3s0.106
> > > > > > > 10.4.16.0/24 dev enp3s0.106 proto kernel scope link src
> > > > > > > 10.4.16.15
> > > > > > > 1.1.1.0/24 dev PUBLICB proto kernel scope link src
> > > > > > > 1.1.1.1 169.254.0.0/16 dev enp6s0 scope link metric 1002
> > > > > > > 169.254.0.0/16 dev enp3s0 scope link metric 1003
> > > > > > > 169.254.0.0/16 dev enp7s0 scope link metric 1004
> > > > > > > 169.254.0.0/16 dev enp3s0.106 scope link metric 1020
> > > > > > > 169.254.0.0/16 dev PRIVATE scope link metric 1022
> > > > > > > 169.254.0.0/16 dev PUBLIC scope link metric 1024
> > > > > > >
> > > > > > > table 1138027711
> > > > > > > # ip route show table 1138027711
> > > > > > > default via 1.1.1.1 dev PUBLIC
> > > > > > > 1.1.1.0/24 via 1.1.1.1 dev PUBLIC
> > > > > > >
> > > > > > > If I manually execute the following command to add the
> > > > > > > default route as well to the main table I can ping ouside
> > > > > > > of the local network.
> > > > > > >
> > > > > > > ip route add 0.0.0.0/0 via 1.1.1.1 dev PUBLIC
> > > > > > >
> > > > > > > If I attempt to modify the /etc/sysconfig/network-
> > > > > > > scripts/route-PUBLIC ad reboot the server ad one would
> > > > > > > think this file is recreated by vdsm on boot.
> > > > > > >
> > > > > > > What I'm looking for is the correct way to setup a
> > > > > > > default gateway for the main routing table so the hosts
> > > > > > > can get OS updates and communicate with the outside
> > > > > > > world.
> > > > > >
> > > > > > Providing the output from "ip addr" may help clear up some
> > > > > > things.
> > > > > > It looks like you have on the host the default route set as
> > > > > > 10.4.16.1 (on enp3s0.106), could you elaborate what this
> > > > > > interface is?
> > > > >
> > > > > We have setup vlan taging to utilize the 2 internetal network
> > > > > interfaces (originally enp6s0 and enp7s0) to be configured
> > > > > with mulitiple networks each. We eventually added 10Gb nics
> > > > > to all servers to improve san glusterfs performance which is
> > > > > enp3s0 which replaced enp6s0 in our setup.
> > > > >
> > > > > enp3s0.106 = ovirtmgmt network access to private internal
> > > > > networks only
> > > > > enp3s0.206 = private network bridge PRIVATE used for private
> > > > > internal network access for VMs
> > > > > enp7s0.606 = is used for public access for both VMs (bridge)
> > > > > and each host/cp/san in our ovirt setup named PUBLIC
> > > > >
> > > > > # ip addr show
> > > > > 1: lo: <LOOPBACK,UP,LOWER_UP> mtu 65536 qdisc noqueue state
> > > > > UNKNOWN qlen 1
> > > > > link/loopback 00:00:00:00:00:00 brd 00:00:00:00:00:00
> > > > > inet 127.0.0.1/8 scope host lo
> > > > > valid_lft forever preferred_lft forever
> > > > > inet6 ::1/128 scope host
> > > > > valid_lft forever preferred_lft forever
> > > > > 2: enp6s0: <BROADCAST,MULTICAST,UP,LOWER_UP> mtu 1500 qdisc
> > > > > pfifo_fast state UP qlen 1000
> > > > > link/ether 00:25:90:38:d6:2c brd ff:ff:ff:ff:ff:ff
> > > > > inet6 fe80::225:90ff:fe38:d62c/64 scope link
> > > > > valid_lft forever preferred_lft forever
> > > > > 3: enp3s0: <BROADCAST,MULTICAST,UP,LOWER_UP> mtu 1500 qdisc
> > > > > mq state UP qlen 1000
> > > > > link/ether 90:e2:ba:1d:a4:00 brd ff:ff:ff:ff:ff:ff
> > > > > inet6 fe80::92e2:baff:fe1d:a400/64 scope link
> > > > > valid_lft forever preferred_lft forever
> > > > > 4: enp7s0: <BROADCAST,MULTICAST,UP,LOWER_UP> mtu 1500 qdisc
> > > > > pfifo_fast state UP qlen 1000
> > > > > link/ether 00:25:90:38:d6:2d brd ff:ff:ff:ff:ff:ff
> > > > > 20: enp3s0.106 at enp3s0: <BROADCAST,MULTICAST,UP,LOWER_UP> mtu
> > > > > 1500 qdisc noqueue state UP qlen 1000
> > > > > link/ether 90:e2:ba:1d:a4:00 brd ff:ff:ff:ff:ff:ff
> > > > > inet 10.4.16.15/24 brd 10.4.16.255 scope global
> > > > > enp3s0.106
> > > > > valid_lft forever preferred_lft forever
> > > > > 21: enp3s0.206 at enp3s0: <BROADCAST,MULTICAST,UP,LOWER_UP> mtu
> > > > > 1500 qdisc noqueue master PRIVATEB state UP qlen 1000
> > > > > link/ether 90:e2:ba:1d:a4:00 brd ff:ff:ff:ff:ff:ff
> > > > > 22: PRIVATE: <BROADCAST,MULTICAST,UP,LOWER_UP> mtu 1500 qdisc
> > > > > noqueue state UP qlen 1000
> > > > > link/ether 90:e2:ba:1d:a4:00 brd ff:ff:ff:ff:ff:ff
> > > > > 23: enp7s0.606 at enp7s0: <BROADCAST,MULTICAST,UP,LOWER_UP> mtu
> > > > > 1500 qdisc noqueue master PUBLICB state UP qlen 1000
> > > > > link/ether 00:25:90:38:d6:2d brd ff:ff:ff:ff:ff:ff
> > > > > 24: PUBLIC: <BROADCAST,MULTICAST,UP,LOWER_UP> mtu 1500 qdisc
> > > > > noqueue state UP qlen 1000
> > > > > link/ether 00:25:90:38:d6:2d brd ff:ff:ff:ff:ff:ff
> > > > > inet 1.1.1.10/24 brd 1.1.1.255 scope global PUBLICB
> > > > > valid_lft forever preferred_lft forever
> > > > > 25: ;vdsmdummy;: <BROADCAST,MULTICAST> mtu 1500 qdisc noop
> > > > > state DOWN qlen 1000
> > > > > link/ether 0e:32:93:dd:a4:55 brd ff:ff:ff:ff:ff:ff
> > > > >
> > > > >
> > > > > So all this being said I just need to reconfigure things in a
> > > > > way that the PUBLIC interface has a default route in the main
> > > > > routing table. Otherwise all ovirt host are unable to
> > > > > communicate with the outside world until I manually add a
> > > > > default route to 1.1.1.1 via the PUBLIC interface. Is that
> > > > > possible.
> > > >
> > > > It is available in oVirt 4.2 as a network cluster role.
> > > > The option to assign a default route role to a network: https:/
> > > > /www.ovirt.org/documentation/admin-guide/chap-
> > > > Logical_Networks/#designate-a-specific-traffic-type-for-a-
> > > > logical-network-with-the-manage-networks-window
> > > >
> > > > On 4.1, it is available as a network custom property and its
> > > > support is limited: See https://bugzilla.redhat.com/show_bug.cg
> > > > i?id=1200963#c43 and https://gerrit.ovirt.org/#/c/66127
> > > > Make sure you do not define two networks with the flag on.
> > > >
> > > >
> > > Thanks for the prompt reply. I've taken a look at the link you
> > > provided for 4.1 and I'm not sure how or where I'm supposed to
> > > set thid custom property for the interface. Is the patch
> > > mentioned in the link you provide already included in 4.1 or do I
> > > need to do something additional to make this work.
> > >
> > > I've attempted to edit an existing host network by clicking
> > > "setup host network" but the page times out with a "page
> > > unresponsive" I can either exit or wait. Waiting doesn't seem
> > > to produce good results. Is this where I would adde/edit this
> > > custom property?
> >
> > Looks like I should of read a bit harder before replying. I found
> > the following two commands on the ovirt change 66127 page.
> >
> > https://gerrit.ovirt.org/#/c/66127/
> > Note that prior to using a custom property, one has to define it on
> > Engine by:
> > sudo engine-config -g CustomDeviceProperties
> > sudo engine-config -s
> > CustomDeviceProperties='{type=interface;prop={default_reoute=^(true
> > |false)$}}' --cver=4.0
> > and restart of ovirt-engine.
>
> Note the misspell: It should be "default_route"
>
> > I've done the above and I can edit the vNIC profile on all networks
> > that have them. The ovirtmgmt network does not have a vNIC profile
> > to edit. Is this theh correct location to make this change?
>
> In the commit message, it mentions "management network attachement"
> and not the vNic profile.
> It should appear at the same place you set the IP address for the
> network on the host.
Looks like my issues with trying to change the hosts network config
timing out was due to chromium browser. I was able to make the needed
changes using firefox. I've been able to make the needed changes to
one hosts and see that it is good.
thanks for the help.
>
> > How do I make if false for the ovirtmgmt network. The option to
> > add a new vNIC profile to it isn't available.
> >
> >
> > Also once this value is set what do I do next?
> >
> > > >
> > > > > > Thanks,
> > > > > > Edy.
> > > > > >
> > > > > > > _______________________________________________
> > > > > > >
> > > > > > > Users mailing list
> > > > > > >
> > > > > > > Users at ovirt.org
> > > > > > >
> > > > > > > http://lists.ovirt.org/mailman/listinfo/users
> > > > > > >
> > > > > > >
> > > > > --
> > > > >
--
Edward Clay
Systems Adminstrator
UK2 Group -- US Operations
Phone: 1-800-222-2165
E-Mail: edward.clay at uk2group.com
-------------- next part --------------
An HTML attachment was scrubbed...
URL: <http://lists.ovirt.org/pipermail/users/attachments/20171122/19a2a5bb/attachment.html>
More information about the Users
mailing list