[ovirt-users] oVIRT 4 / OVN / Communication issues of instances between nodes.
Devin Acosta
devin at pabstatencio.com
Mon Dec 5 21:17:35 UTC 2016
Lance,
I found some interesting logs, we have (3) oVIRT nodes.
We are running:
CentOS Linux release 7.2.1511 (Core)
Linux hostname 3.10.0-327.36.3.el7.x86_64 #1 SMP Mon Oct 24 16:09:20 UTC
2016 x86_64 x86_64 x86_64 GNU/Linux
What is interesting now is my 2nd node now won't hand out DHCP any more,
however it does have a strange "tunnel" error message on node2 also, see
below. What is bizarre node1 nor node3 show that bizarre tunnel message?
[ovirt-node1]
[root at ovirt-node1 ~]# lsmod | grep vport
vport_geneve 12815 1
geneve 13381 1 vport_geneve
openvswitch 84535 1 vport_geneve
[ovirt-node1 / ovn-controller.log]
2016-12-05T20:47:56.761Z|00001|vlog|INFO|opened log file
/var/log/openvswitch/ovn-controller.log
2016-12-05T20:47:56.762Z|00002|reconnect|INFO|unix:/var/run/openvswitch/db.sock:
connecting...
2016-12-05T20:47:56.762Z|00003|reconnect|INFO|unix:/var/run/openvswitch/db.sock:
connected
2016-12-05T20:47:56.764Z|00004|reconnect|INFO|tcp:172.20.192.77:6642:
connecting...
2016-12-05T20:47:56.764Z|00005|reconnect|INFO|tcp:172.20.192.77:6642:
connected
2016-12-05T20:47:56.768Z|00006|binding|INFO|Claiming lport
91d4f4f5-4b9f-42c0-aa2c-8a101474bb84 for this chassis.
2016-12-05T20:47:56.768Z|00007|binding|INFO|Claiming 00:1a:4a:16:01:5e
dynamic
2016-12-05T20:47:56.768Z|00008|binding|INFO|Claiming lport
71ef81f1-7c20-4c68-b536-d274703f7541 for this chassis.
2016-12-05T20:47:56.768Z|00009|binding|INFO|Claiming 00:1a:4a:16:01:61
dynamic
2016-12-05T20:47:56.768Z|00010|ofctrl|INFO|unix:/var/run/openvswitch/br-int.mgmt:
connecting to switch
2016-12-05T20:47:56.768Z|00011|rconn|INFO|unix:/var/run/openvswitch/br-int.mgmt:
connecting...
2016-12-05T20:47:56.768Z|00012|pinctrl|INFO|unix:/var/run/openvswitch/br-int.mgmt:
connecting to switch
2016-12-05T20:47:56.768Z|00013|rconn|INFO|unix:/var/run/openvswitch/br-int.mgmt:
connecting...
2016-12-05T20:47:56.768Z|00014|ofctrl|INFO|dropping duplicate flow:
table_id=29, priority=50, metadata=0x2,dl_dst=00:1a:4a:16:01:62,
actions=set_field:0x5->reg15,resubmit(,32)
2016-12-05T20:47:56.770Z|00015|rconn|INFO|unix:/var/run/openvswitch/br-int.mgmt:
connected
2016-12-05T20:47:56.770Z|00016|rconn|INFO|unix:/var/run/openvswitch/br-int.mgmt:
connected
2016-12-05T20:47:56.770Z|00017|ofctrl|INFO|dropping duplicate flow:
table_id=29, priority=50, metadata=0x2,dl_dst=00:1a:4a:16:01:62,
actions=set_field:0x5->reg15,resubmit(,32)
2016-12-05T20:47:56.771Z|00018|ofctrl|INFO|dropping duplicate flow:
table_id=29, priority=50, metadata=0x2,dl_dst=00:1a:4a:16:01:62,
actions=set_field:0x5->reg15,resubmit(,32)
2016-12-05T20:47:56.772Z|00019|ofctrl|INFO|dropping duplicate flow:
table_id=29, priority=50, metadata=0x2,dl_dst=00:1a:4a:16:01:62,
actions=set_field:0x5->reg15,resubmit(,32)
2016-12-05T20:47:56.773Z|00020|ofctrl|INFO|dropping duplicate flow:
table_id=32, priority=150, reg10=0x2/0x2, actions=resubmit(,33)
2016-12-05T20:47:56.774Z|00021|ofctrl|INFO|OpenFlow error: OFPT_ERROR
(OF1.3) (xid=0x17): OFPBAC_BAD_TYPE
OFPT_FLOW_MOD (OF1.3) (xid=0x17):
(***truncated to 64 bytes from 120***)
00000000 04 0e 00 78 00 00 00 17-00 00 00 00 00 00 00 00 |...x............|
00000010 00 00 00 00 00 00 00 00-32 00 00 00 00 00 00 64 |........2......d|
00000020 ff ff ff ff ff ff ff ff-ff ff ff ff 00 00 00 00 |................|
00000030 00 01 00 22 80 00 0a 02-86 dd 00 01 01 08 00 00 |..."............|
2016-12-05T20:47:56.774Z|00022|ofctrl|INFO|OpenFlow error: OFPT_ERROR
(OF1.3) (xid=0x1f): OFPBAC_BAD_TYPE
OFPT_FLOW_MOD (OF1.3) (xid=0x1f):
(***truncated to 64 bytes from 160***)
00000000 04 0e 00 a0 00 00 00 1f-00 00 00 00 00 00 00 00 |................|
00000010 00 00 00 00 00 00 00 00-36 00 00 00 00 00 00 64 |........6......d|
00000020 ff ff ff ff ff ff ff ff-ff ff ff ff 00 00 00 00 |................|
00000030 00 01 00 22 80 00 0a 02-08 00 00 01 01 08 00 00 |..."............|
2016-12-05T20:47:56.774Z|00023|ofctrl|INFO|OpenFlow error: OFPT_ERROR
(OF1.3) (xid=0x21): OFPBAC_BAD_TYPE
OFPT_FLOW_MOD (OF1.3) (xid=0x21):
(***truncated to 64 bytes from 136***)
00000000 04 0e 00 88 00 00 00 21-00 00 00 00 00 00 00 00 |.......!........|
00000010 00 00 00 00 00 00 00 00-19 00 00 00 00 00 00 64 |...............d|
00000020 ff ff ff ff ff ff ff ff-ff ff ff ff 00 00 00 00 |................|
00000030 00 01 00 22 80 00 0a 02-86 dd 00 01 01 08 00 00 |..."............|
2016-12-05T20:47:56.774Z|00024|ofctrl|INFO|OpenFlow error: OFPT_ERROR
(OF1.3) (xid=0x2c): OFPBAC_BAD_TYPE
OFPT_FLOW_MOD (OF1.3) (xid=0x2c):
(***truncated to 64 bytes from 160***)
00000000 04 0e 00 a0 00 00 00 2c-00 00 00 00 00 00 00 00 |.......,........|
00000010 00 00 00 00 00 00 00 00-19 00 00 00 00 00 00 64 |...............d|
00000020 ff ff ff ff ff ff ff ff-ff ff ff ff 00 00 00 00 |................|
00000030 00 01 00 22 80 00 0a 02-86 dd 00 01 01 08 00 00 |..."............|
2016-12-05T20:47:56.774Z|00025|ofctrl|INFO|OpenFlow error: OFPT_ERROR
(OF1.3) (xid=0x2d): OFPBAC_BAD_TYPE
OFPT_FLOW_MOD (OF1.3) (xid=0x2d):
(***truncated to 64 bytes from 136***)
00000000 04 0e 00 88 00 00 00 2d-00 00 00 00 00 00 00 00 |.......-........|
00000010 00 00 00 00 00 00 00 00-36 00 00 00 00 00 00 64 |........6......d|
00000020 ff ff ff ff ff ff ff ff-ff ff ff ff 00 00 00 00 |................|
00000030 00 01 00 22 80 00 0a 02-08 00 00 01 01 08 00 00 |..."............|
2016-12-05T20:47:56.774Z|00026|ofctrl|INFO|OpenFlow error: OFPT_ERROR
(OF1.3) (xid=0x31): OFPBAC_BAD_TYPE
OFPT_FLOW_MOD (OF1.3) (xid=0x31):
(***truncated to 64 bytes from 120***)
00000000 04 0e 00 78 00 00 00 31-00 00 00 00 00 00 00 00 |...x...1........|
00000010 00 00 00 00 00 00 00 00-32 00 00 00 00 00 00 64 |........2......d|
00000020 ff ff ff ff ff ff ff ff-ff ff ff ff 00 00 00 00 |................|
00000030 00 01 00 22 80 00 0a 02-08 00 00 01 01 08 00 00 |..."............|
2016-12-05T20:47:56.774Z|00027|ofctrl|INFO|OpenFlow error: OFPT_ERROR
(OF1.3) (xid=0x39): OFPBAC_BAD_TYPE
OFPT_FLOW_MOD (OF1.3) (xid=0x39):
(***truncated to 64 bytes from 136***)
00000000 04 0e 00 88 00 00 00 39-00 00 00 00 00 00 00 00 |.......9........|
00000010 00 00 00 00 00 00 00 00-36 00 00 00 00 00 00 64 |........6......d|
00000020 ff ff ff ff ff ff ff ff-ff ff ff ff 00 00 00 00 |................|
00000030 00 01 00 22 80 00 0a 02-86 dd 00 01 01 08 00 00 |..."............|
2016-12-05T20:47:56.774Z|00028|ofctrl|INFO|OpenFlow error: OFPT_ERROR
(OF1.3) (xid=0x47): OFPBAC_BAD_TYPE
OFPT_FLOW_MOD (OF1.3) (xid=0x47):
(***truncated to 64 bytes from 160***)
00000000 04 0e 00 a0 00 00 00 47-00 00 00 00 00 00 00 00 |.......G........|
00000010 00 00 00 00 00 00 00 00-19 00 00 00 00 00 00 64 |...............d|
00000020 ff ff ff ff ff ff ff ff-ff ff ff ff 00 00 00 00 |................|
00000030 00 01 00 22 80 00 0a 02-08 00 00 01 01 08 00 00 |..."............|
2016-12-05T20:47:56.774Z|00029|ofctrl|INFO|OpenFlow error: OFPT_ERROR
(OF1.3) (xid=0x49): OFPBAC_BAD_TYPE
OFPT_FLOW_MOD (OF1.3) (xid=0x49):
(***truncated to 64 bytes from 120***)
00000000 04 0e 00 78 00 00 00 49-00 00 00 00 00 00 00 00 |...x...I........|
00000010 00 00 00 00 00 00 00 00-15 00 00 00 00 00 00 64 |...............d|
00000020 ff ff ff ff ff ff ff ff-ff ff ff ff 00 00 00 00 |................|
00000030 00 01 00 22 80 00 0a 02-08 00 00 01 01 08 00 00 |..."............|
2016-12-05T20:47:56.774Z|00030|ofctrl|INFO|OpenFlow error: OFPT_ERROR
(OF1.3) (xid=0x4a): OFPBAC_BAD_TYPE
OFPT_FLOW_MOD (OF1.3) (xid=0x4a):
(***truncated to 64 bytes from 160***)
00000000 04 0e 00 a0 00 00 00 4a-00 00 00 00 00 00 00 00 |.......J........|
00000010 00 00 00 00 00 00 00 00-36 00 00 00 00 00 00 64 |........6......d|
00000020 ff ff ff ff ff ff ff ff-ff ff ff ff 00 00 00 00 |................|
00000030 00 01 00 22 80 00 0a 02-86 dd 00 01 01 08 00 00 |..."............|
2016-12-05T20:47:56.774Z|00031|ofctrl|INFO|OpenFlow error: OFPT_ERROR
(OF1.3) (xid=0x53): OFPBAC_BAD_TYPE
OFPT_FLOW_MOD (OF1.3) (xid=0x53):
(***truncated to 64 bytes from 136***)
00000000 04 0e 00 88 00 00 00 53-00 00 00 00 00 00 00 00 |.......S........|
00000010 00 00 00 00 00 00 00 00-19 00 00 00 00 00 00 64 |...............d|
00000020 ff ff ff ff ff ff ff ff-ff ff ff ff 00 00 00 00 |................|
00000030 00 01 00 22 80 00 0a 02-08 00 00 01 01 08 00 00 |..."............|
2016-12-05T20:47:56.774Z|00032|ofctrl|INFO|OpenFlow error: OFPT_ERROR
(OF1.3) (xid=0x61): OFPBAC_BAD_TYPE
OFPT_FLOW_MOD (OF1.3) (xid=0x61):
(***truncated to 64 bytes from 120***)
00000000 04 0e 00 78 00 00 00 61-00 00 00 00 00 00 00 00 |...x...a........|
00000010 00 00 00 00 00 00 00 00-15 00 00 00 00 00 00 64 |...............d|
00000020 ff ff ff ff ff ff ff ff-ff ff ff ff 00 00 00 00 |................|
00000030 00 01 00 22 80 00 0a 02-86 dd 00 01 01 08 00 00 |..."............|
[ovirt-node2]
# lsmod | grep vport
vport_geneve 12815 1
geneve 13381 1 vport_geneve
openvswitch 84535 1 vport_geneve
[ovirt-node2 in the ovs-vswitchd.log]
What is of interest, is the "tunnel" error line, not sure where it's
getting IP 50.48.48.48, not even an IP we use on this network, nor do i
know where it's getting IP 47.51.48.55 ?
2016-12-05T20:35:04.499Z|00001|tunnel(revalidator25)|WARN|receive tunnel
port not found
(tun_id=0x3230203838373438,tun_src=47.51.48.55,tun_dst=50.48.48.48,tun_ipv6_src=::,tun_ipv6_dst=::,tun_gbp_id=0,tun_gbp_flags=0,tun_tos=20,tun_ttl=114,tun_flags=csum|key,vlan_tci=0x0000,dl_src=00:00:00:00:00:00,dl_dst=00:00:00:00:00:00,dl_type=0x1234)
2016-12-05T20:35:04.345Z|00001|vlog|INFO|opened log file
/var/log/openvswitch/ovs-vswitchd.log
2016-12-05T20:35:04.347Z|00002|ovs_numa|INFO|Discovered 16 CPU cores on
NUMA node 0
2016-12-05T20:35:04.347Z|00003|ovs_numa|INFO|Discovered 16 CPU cores on
NUMA node 1
2016-12-05T20:35:04.347Z|00004|ovs_numa|INFO|Discovered 2 NUMA nodes and 32
CPU cores
2016-12-05T20:35:04.348Z|00005|reconnect|INFO|unix:/var/run/openvswitch/db.sock:
connecting...
2016-12-05T20:35:04.348Z|00006|reconnect|INFO|unix:/var/run/openvswitch/db.sock:
connected
2016-12-05T20:35:04.350Z|00007|ofproto_dpif|INFO|system at ovs-system:
Datapath supports recirculation
2016-12-05T20:35:04.350Z|00008|ofproto_dpif|INFO|system at ovs-system: MPLS
label stack length probed as 1
2016-12-05T20:35:04.350Z|00009|ofproto_dpif|INFO|system at ovs-system:
Datapath does not support truncate action
2016-12-05T20:35:04.350Z|00010|ofproto_dpif|INFO|system at ovs-system:
Datapath supports unique flow ids
2016-12-05T20:35:04.350Z|00011|ofproto_dpif|INFO|system at ovs-system:
Datapath does not support ct_state
2016-12-05T20:35:04.350Z|00012|ofproto_dpif|INFO|system at ovs-system:
Datapath does not support ct_zone
2016-12-05T20:35:04.350Z|00013|ofproto_dpif|INFO|system at ovs-system:
Datapath does not support ct_mark
2016-12-05T20:35:04.350Z|00014|ofproto_dpif|INFO|system at ovs-system:
Datapath does not support ct_label
2016-12-05T20:35:04.350Z|00015|ofproto_dpif|INFO|system at ovs-system:
Datapath does not support ct_state_nat
2016-12-05T20:35:04.493Z|00016|bridge|INFO|bridge br-int: added interface
vnet0 on port 5
2016-12-05T20:35:04.493Z|00001|ofproto_dpif_upcall(handler1)|INFO|received
packet on unassociated datapath port 0
2016-12-05T20:35:04.493Z|00017|bridge|INFO|bridge br-int: added interface
br-int on port 65534
2016-12-05T20:35:04.493Z|00018|bridge|INFO|bridge br-int: using datapath ID
000016d6e0b66442
2016-12-05T20:35:04.493Z|00019|connmgr|INFO|br-int: added service
controller "punix:/var/run/openvswitch/br-int.mgmt"
2016-12-05T20:35:04.494Z|00020|bridge|INFO|ovs-vswitchd (Open vSwitch)
2.6.90
2016-12-05T20:35:04.499Z|00001|tunnel(revalidator25)|WARN|receive tunnel
port not found
(tun_id=0x3230203838373438,tun_src=47.51.48.55,tun_dst=50.48.48.48,tun_ipv6_src=::,tun_ipv6_dst=::,tun_gbp_id=0,tun_gbp_flags=0,tun_tos=20,tun_ttl=114,tun_flags=csum|key,vlan_tci=0x0000,dl_src=00:00:00:00:00:00,dl_dst=00:00:00:00:00:00,dl_type=0x1234)
2016-12-05T20:35:09.657Z|00021|bridge|INFO|bridge br-int: added interface
ovn-c0dc09-0 on port 6
2016-12-05T20:35:09.657Z|00022|bridge|INFO|bridge br-int: added interface
ovn-252778-0 on port 7
2016-12-05T20:35:09.660Z|00023|ofproto_dpif|WARN|Rejecting ct action
because datapath does not support ct action (your kernel module may be out
of date)
2016-12-05T20:35:09.660Z|00024|connmgr|INFO|br-int<->unix: sending
OFPBAC_BAD_TYPE error reply to OFPT_FLOW_MOD message
2016-12-05T20:35:09.660Z|00025|ofproto_dpif|WARN|Rejecting ct action
because datapath does not support ct action (your kernel module may be out
of date)
2016-12-05T20:35:09.660Z|00026|connmgr|INFO|br-int<->unix: sending
OFPBAC_BAD_TYPE error reply to OFPT_FLOW_MOD message
2016-12-05T20:35:09.660Z|00027|ofproto_dpif|WARN|Rejecting ct action
because datapath does not support ct action (your kernel module may be out
of date)
2016-12-05T20:35:09.660Z|00028|connmgr|INFO|br-int<->unix: sending
OFPBAC_BAD_TYPE error reply to OFPT_FLOW_MOD message
2016-12-05T20:35:09.660Z|00029|ofproto_dpif|WARN|Rejecting ct action
because datapath does not support ct action (your kernel module may be out
of date)
2016-12-05T20:35:09.660Z|00030|connmgr|INFO|br-int<->unix: sending
OFPBAC_BAD_TYPE error reply to OFPT_FLOW_MOD message
2016-12-05T20:35:09.660Z|00031|ofproto_dpif|WARN|Rejecting ct action
because datapath does not support ct action (your kernel module may be out
of date)
2016-12-05T20:35:09.660Z|00032|connmgr|INFO|br-int<->unix: sending
OFPBAC_BAD_TYPE error reply to OFPT_FLOW_MOD message
2016-12-05T20:35:09.660Z|00033|connmgr|INFO|br-int<->unix: sending
OFPBAC_BAD_TYPE error reply to OFPT_FLOW_MOD message
2016-12-05T20:35:09.660Z|00034|connmgr|INFO|br-int<->unix: sending
OFPBAC_BAD_TYPE error reply to OFPT_FLOW_MOD message
2016-12-05T20:35:09.660Z|00035|connmgr|INFO|br-int<->unix: sending
OFPBAC_BAD_TYPE error reply to OFPT_FLOW_MOD message
2016-12-05T20:35:09.660Z|00036|connmgr|INFO|br-int<->unix: sending
OFPBAC_BAD_TYPE error reply to OFPT_FLOW_MOD message
2016-12-05T20:35:09.660Z|00037|connmgr|INFO|br-int<->unix: sending
OFPBAC_BAD_TYPE error reply to OFPT_FLOW_MOD message
2016-12-05T20:35:14.495Z|00038|memory|INFO|281084 kB peak resident set size
after 10.2 seconds
2016-12-05T20:35:14.495Z|00039|memory|INFO|handlers:23 ofconns:2 ports:4
revalidators:9 rules:76 udpif keys:1
2016-12-05T20:35:19.659Z|00040|connmgr|INFO|br-int<->unix: 73 flow_mods 10
s ago (72 adds, 1 deletes)
2016-12-05T20:37:43.716Z|00041|connmgr|INFO|br-int<->unix: 8 flow_mods 10 s
ago (7 deletes, 1 modifications)
2016-12-05T20:39:02.163Z|00042|bridge|INFO|bridge br-int: added interface
ovn-c0dc09-0 on port 8
2016-12-05T20:39:12.165Z|00043|connmgr|INFO|br-int<->unix: 8 flow_mods 10 s
ago (7 adds, 1 modifications)
2016-12-05T20:45:34.604Z|00044|connmgr|INFO|br-int<->unix: 8 flow_mods 10 s
ago (7 deletes, 1 modifications)
2016-12-05T20:47:56.774Z|00045|bridge|INFO|bridge br-int: added interface
ovn-c0dc09-0 on port 9
2016-12-05T20:48:06.776Z|00046|connmgr|INFO|br-int<->unix: 8 flow_mods 10 s
ago (7 adds, 1 modifications)
[ovirt-node3 / ovs-vswitchd.log]
2016-12-05T21:15:19.969Z|00001|vlog|INFO|opened log file
/var/log/openvswitch/ovs-vswitchd.log
2016-12-05T21:15:19.973Z|00002|ovs_numa|INFO|Discovered 16 CPU cores on
NUMA node 0
2016-12-05T21:15:19.973Z|00003|ovs_numa|INFO|Discovered 16 CPU cores on
NUMA node 1
2016-12-05T21:15:19.973Z|00004|ovs_numa|INFO|Discovered 2 NUMA nodes and 32
CPU cores
2016-12-05T21:15:19.973Z|00005|reconnect|INFO|unix:/var/run/openvswitch/db.sock:
connecting...
2016-12-05T21:15:19.973Z|00006|reconnect|INFO|unix:/var/run/openvswitch/db.sock:
connected
2016-12-05T21:15:19.978Z|00007|ofproto_dpif|INFO|system at ovs-system:
Datapath supports recirculation
2016-12-05T21:15:19.978Z|00008|ofproto_dpif|INFO|system at ovs-system: MPLS
label stack length probed as 1
2016-12-05T21:15:19.978Z|00009|ofproto_dpif|INFO|system at ovs-system:
Datapath does not support truncate action
2016-12-05T21:15:19.978Z|00010|ofproto_dpif|INFO|system at ovs-system:
Datapath supports unique flow ids
2016-12-05T21:15:19.978Z|00011|ofproto_dpif|INFO|system at ovs-system:
Datapath does not support ct_state
2016-12-05T21:15:19.978Z|00012|ofproto_dpif|INFO|system at ovs-system:
Datapath does not support ct_zone
2016-12-05T21:15:19.978Z|00013|ofproto_dpif|INFO|system at ovs-system:
Datapath does not support ct_mark
2016-12-05T21:15:19.978Z|00014|ofproto_dpif|INFO|system at ovs-system:
Datapath does not support ct_label
2016-12-05T21:15:19.978Z|00015|ofproto_dpif|INFO|system at ovs-system:
Datapath does not support ct_state_nat
2016-12-05T21:15:20.148Z|00001|ofproto_dpif_upcall(handler1)|INFO|received
packet on unassociated datapath port 0
2016-12-05T21:15:20.148Z|00016|bridge|INFO|bridge br-int: added interface
vnet0 on port 7
2016-12-05T21:15:20.148Z|00017|bridge|INFO|bridge br-int: added interface
br-int on port 65534
2016-12-05T21:15:20.148Z|00018|bridge|INFO|bridge br-int: using datapath ID
0000921726222e4b
2016-12-05T21:15:20.148Z|00019|connmgr|INFO|br-int: added service
controller "punix:/var/run/openvswitch/br-int.mgmt"
2016-12-05T21:15:20.150Z|00020|bridge|INFO|ovs-vswitchd (Open vSwitch)
2.6.90
2016-12-05T21:15:30.150Z|00021|memory|INFO|281052 kB peak resident set size
after 10.2 seconds
2016-12-05T21:15:30.150Z|00022|memory|INFO|handlers:23 ports:2
revalidators:9 rules:4
2016-12-05T21:15:47.900Z|00023|bridge|INFO|bridge br-int: added interface
ovn-456949-0 on port 8
2016-12-05T21:15:47.900Z|00024|bridge|INFO|bridge br-int: added interface
ovn-c0dc09-0 on port 9
2016-12-05T21:15:47.901Z|00025|ofproto_dpif|WARN|Rejecting ct action
because datapath does not support ct action (your kernel module may be out
of date)
2016-12-05T21:15:47.901Z|00026|connmgr|INFO|br-int<->unix: sending
OFPBAC_BAD_TYPE error reply to OFPT_FLOW_MOD message
2016-12-05T21:15:47.901Z|00027|ofproto_dpif|WARN|Rejecting ct action
because datapath does not support ct action (your kernel module may be out
of date)
2016-12-05T21:15:47.901Z|00028|connmgr|INFO|br-int<->unix: sending
OFPBAC_BAD_TYPE error reply to OFPT_FLOW_MOD message
2016-12-05T21:15:47.901Z|00029|ofproto_dpif|WARN|Rejecting ct action
because datapath does not support ct action (your kernel module may be out
of date)
2016-12-05T21:15:47.901Z|00030|connmgr|INFO|br-int<->unix: sending
OFPBAC_BAD_TYPE error reply to OFPT_FLOW_MOD message
2016-12-05T21:15:47.901Z|00031|ofproto_dpif|WARN|Rejecting ct action
because datapath does not support ct action (your kernel module may be out
of date)
2016-12-05T21:15:47.901Z|00032|connmgr|INFO|br-int<->unix: sending
OFPBAC_BAD_TYPE error reply to OFPT_FLOW_MOD message
2016-12-05T21:15:47.901Z|00033|ofproto_dpif|WARN|Rejecting ct action
because datapath does not support ct action (your kernel module may be out
of date)
2016-12-05T21:15:47.901Z|00034|connmgr|INFO|br-int<->unix: sending
OFPBAC_BAD_TYPE error reply to OFPT_FLOW_MOD message
2016-12-05T21:15:47.901Z|00035|connmgr|INFO|br-int<->unix: sending
OFPBAC_BAD_TYPE error reply to OFPT_FLOW_MOD message
2016-12-05T21:15:47.901Z|00036|connmgr|INFO|br-int<->unix: sending
OFPBAC_BAD_TYPE error reply to OFPT_FLOW_MOD message
2016-12-05T21:15:47.902Z|00037|connmgr|INFO|br-int<->unix: sending
OFPBAC_BAD_TYPE error reply to OFPT_FLOW_MOD message
2016-12-05T21:15:47.902Z|00038|connmgr|INFO|br-int<->unix: sending
OFPBAC_BAD_TYPE error reply to OFPT_FLOW_MOD message
2016-12-05T21:15:47.902Z|00039|connmgr|INFO|br-int<->unix: sending
OFPBAC_BAD_TYPE error reply to OFPT_FLOW_MOD message
2016-12-05T21:15:57.901Z|00040|connmgr|INFO|br-int<->unix: 73 flow_mods 10
s ago (72 adds, 1 deletes)
[ovirt-node3 ovn-controller.log]
2016-12-05T21:15:47.858Z|00001|vlog|INFO|opened log file
/var/log/openvswitch/ovn-controller.log
2016-12-05T21:15:47.860Z|00002|reconnect|INFO|unix:/var/run/openvswitch/db.sock:
connecting...
2016-12-05T21:15:47.860Z|00003|reconnect|INFO|unix:/var/run/openvswitch/db.sock:
connected
2016-12-05T21:15:47.864Z|00004|reconnect|INFO|tcp:172.20.192.77:6642:
connecting...
2016-12-05T21:15:47.864Z|00005|reconnect|INFO|tcp:172.20.192.77:6642:
connected
2016-12-05T21:15:47.867Z|00006|binding|INFO|Claiming lport
6b289418-8b8e-42b4-8334-c71584afcd3e for this chassis.
2016-12-05T21:15:47.867Z|00007|binding|INFO|Claiming 00:1a:4a:16:01:5c
dynamic
2016-12-05T21:15:47.867Z|00008|ofctrl|INFO|unix:/var/run/openvswitch/br-int.mgmt:
connecting to switch
2016-12-05T21:15:47.867Z|00009|rconn|INFO|unix:/var/run/openvswitch/br-int.mgmt:
connecting...
2016-12-05T21:15:47.868Z|00010|pinctrl|INFO|unix:/var/run/openvswitch/br-int.mgmt:
connecting to switch
2016-12-05T21:15:47.868Z|00011|rconn|INFO|unix:/var/run/openvswitch/br-int.mgmt:
connecting...
2016-12-05T21:15:47.868Z|00012|ofctrl|INFO|dropping duplicate flow:
table_id=29, priority=50, metadata=0x2,dl_dst=00:1a:4a:16:01:62,
actions=set_field:0x5->reg15,resubmit(,32)
2016-12-05T21:15:47.869Z|00013|rconn|INFO|unix:/var/run/openvswitch/br-int.mgmt:
connected
2016-12-05T21:15:47.869Z|00014|rconn|INFO|unix:/var/run/openvswitch/br-int.mgmt:
connected
2016-12-05T21:15:47.870Z|00015|ofctrl|INFO|dropping duplicate flow:
table_id=29, priority=50, metadata=0x2,dl_dst=00:1a:4a:16:01:62,
actions=set_field:0x5->reg15,resubmit(,32)
2016-12-05T21:15:47.871Z|00016|ofctrl|INFO|dropping duplicate flow:
table_id=29, priority=50, metadata=0x2,dl_dst=00:1a:4a:16:01:62,
actions=set_field:0x5->reg15,resubmit(,32)
2016-12-05T21:15:47.872Z|00017|ofctrl|INFO|dropping duplicate flow:
table_id=29, priority=50, metadata=0x2,dl_dst=00:1a:4a:16:01:62,
actions=set_field:0x5->reg15,resubmit(,32)
2016-12-05T21:15:47.901Z|00018|ofctrl|INFO|dropping duplicate flow:
table_id=29, priority=50, metadata=0x2,dl_dst=00:1a:4a:16:01:62,
actions=set_field:0x5->reg15,resubmit(,32)
2016-12-05T21:15:47.901Z|00019|ofctrl|INFO|OpenFlow error: OFPT_ERROR
(OF1.3) (xid=0x16): OFPBAC_BAD_TYPE
OFPT_FLOW_MOD (OF1.3) (xid=0x16):
(***truncated to 64 bytes from 120***)
00000000 04 0e 00 78 00 00 00 16-00 00 00 00 00 00 00 00 |...x............|
00000010 00 00 00 00 00 00 00 00-32 00 00 00 00 00 00 64 |........2......d|
00000020 ff ff ff ff ff ff ff ff-ff ff ff ff 00 00 00 00 |................|
00000030 00 01 00 22 80 00 0a 02-86 dd 00 01 01 08 00 00 |..."............|
2016-12-05T21:15:47.902Z|00020|ofctrl|INFO|OpenFlow error: OFPT_ERROR
(OF1.3) (xid=0x1c): OFPBAC_BAD_TYPE
OFPT_FLOW_MOD (OF1.3) (xid=0x1c):
(***truncated to 64 bytes from 160***)
00000000 04 0e 00 a0 00 00 00 1c-00 00 00 00 00 00 00 00 |................|
00000010 00 00 00 00 00 00 00 00-36 00 00 00 00 00 00 64 |........6......d|
00000020 ff ff ff ff ff ff ff ff-ff ff ff ff 00 00 00 00 |................|
00000030 00 01 00 22 80 00 0a 02-08 00 00 01 01 08 00 00 |..."............|
2016-12-05T21:15:47.902Z|00021|ofctrl|INFO|OpenFlow error: OFPT_ERROR
(OF1.3) (xid=0x1e): OFPBAC_BAD_TYPE
OFPT_FLOW_MOD (OF1.3) (xid=0x1e):
(***truncated to 64 bytes from 136***)
00000000 04 0e 00 88 00 00 00 1e-00 00 00 00 00 00 00 00 |................|
00000010 00 00 00 00 00 00 00 00-19 00 00 00 00 00 00 64 |...............d|
00000020 ff ff ff ff ff ff ff ff-ff ff ff ff 00 00 00 00 |................|
00000030 00 01 00 22 80 00 0a 02-86 dd 00 01 01 08 00 00 |..."............|
2016-12-05T21:15:47.902Z|00022|ofctrl|INFO|OpenFlow error: OFPT_ERROR
(OF1.3) (xid=0x27): OFPBAC_BAD_TYPE
OFPT_FLOW_MOD (OF1.3) (xid=0x27):
(***truncated to 64 bytes from 160***)
00000000 04 0e 00 a0 00 00 00 27-00 00 00 00 00 00 00 00 |.......'........|
00000010 00 00 00 00 00 00 00 00-19 00 00 00 00 00 00 64 |...............d|
00000020 ff ff ff ff ff ff ff ff-ff ff ff ff 00 00 00 00 |................|
00000030 00 01 00 22 80 00 0a 02-86 dd 00 01 01 08 00 00 |..."............|
2016-12-05T21:15:47.902Z|00023|ofctrl|INFO|OpenFlow error: OFPT_ERROR
(OF1.3) (xid=0x28): OFPBAC_BAD_TYPE
OFPT_FLOW_MOD (OF1.3) (xid=0x28):
(***truncated to 64 bytes from 136***)
00000000 04 0e 00 88 00 00 00 28-00 00 00 00 00 00 00 00 |.......(........|
00000010 00 00 00 00 00 00 00 00-36 00 00 00 00 00 00 64 |........6......d|
00000020 ff ff ff ff ff ff ff ff-ff ff ff ff 00 00 00 00 |................|
00000030 00 01 00 22 80 00 0a 02-08 00 00 01 01 08 00 00 |..."............|
2016-12-05T21:15:47.902Z|00024|ofctrl|INFO|OpenFlow error: OFPT_ERROR
(OF1.3) (xid=0x2b): OFPBAC_BAD_TYPE
OFPT_FLOW_MOD (OF1.3) (xid=0x2b):
(***truncated to 64 bytes from 120***)
00000000 04 0e 00 78 00 00 00 2b-00 00 00 00 00 00 00 00 |...x...+........|
00000010 00 00 00 00 00 00 00 00-32 00 00 00 00 00 00 64 |........2......d|
00000020 ff ff ff ff ff ff ff ff-ff ff ff ff 00 00 00 00 |................|
00000030 00 01 00 22 80 00 0a 02-08 00 00 01 01 08 00 00 |..."............|
2016-12-05T21:15:47.902Z|00025|ofctrl|INFO|OpenFlow error: OFPT_ERROR
(OF1.3) (xid=0x31): OFPBAC_BAD_TYPE
OFPT_FLOW_MOD (OF1.3) (xid=0x31):
(***truncated to 64 bytes from 136***)
00000000 04 0e 00 88 00 00 00 31-00 00 00 00 00 00 00 00 |.......1........|
00000010 00 00 00 00 00 00 00 00-36 00 00 00 00 00 00 64 |........6......d|
00000020 ff ff ff ff ff ff ff ff-ff ff ff ff 00 00 00 00 |................|
00000030 00 01 00 22 80 00 0a 02-86 dd 00 01 01 08 00 00 |..."............|
2016-12-05T21:15:47.902Z|00026|ofctrl|INFO|OpenFlow error: OFPT_ERROR
(OF1.3) (xid=0x3d): OFPBAC_BAD_TYPE
OFPT_FLOW_MOD (OF1.3) (xid=0x3d):
(***truncated to 64 bytes from 160***)
00000000 04 0e 00 a0 00 00 00 3d-00 00 00 00 00 00 00 00 |.......=........|
00000010 00 00 00 00 00 00 00 00-19 00 00 00 00 00 00 64 |...............d|
00000020 ff ff ff ff ff ff ff ff-ff ff ff ff 00 00 00 00 |................|
00000030 00 01 00 22 80 00 0a 02-08 00 00 01 01 08 00 00 |..."............|
2016-12-05T21:15:47.902Z|00027|ofctrl|INFO|OpenFlow error: OFPT_ERROR
(OF1.3) (xid=0x3f): OFPBAC_BAD_TYPE
OFPT_FLOW_MOD (OF1.3) (xid=0x3f):
(***truncated to 64 bytes from 120***)
00000000 04 0e 00 78 00 00 00 3f-00 00 00 00 00 00 00 00 |...x...?........|
00000010 00 00 00 00 00 00 00 00-15 00 00 00 00 00 00 64 |...............d|
00000020 ff ff ff ff ff ff ff ff-ff ff ff ff 00 00 00 00 |................|
00000030 00 01 00 22 80 00 0a 02-08 00 00 01 01 08 00 00 |..."............|
2016-12-05T21:15:47.902Z|00028|ofctrl|INFO|OpenFlow error: OFPT_ERROR
(OF1.3) (xid=0x40): OFPBAC_BAD_TYPE
OFPT_FLOW_MOD (OF1.3) (xid=0x40):
(***truncated to 64 bytes from 160***)
00000000 04 0e 00 a0 00 00 00 40-00 00 00 00 00 00 00 00 |....... at ........|
00000010 00 00 00 00 00 00 00 00-36 00 00 00 00 00 00 64 |........6......d|
00000020 ff ff ff ff ff ff ff ff-ff ff ff ff 00 00 00 00 |................|
00000030 00 01 00 22 80 00 0a 02-86 dd 00 01 01 08 00 00 |..."............|
2016-12-05T21:15:47.902Z|00029|ofctrl|INFO|OpenFlow error: OFPT_ERROR
(OF1.3) (xid=0x46): OFPBAC_BAD_TYPE
OFPT_FLOW_MOD (OF1.3) (xid=0x46):
(***truncated to 64 bytes from 136***)
00000000 04 0e 00 88 00 00 00 46-00 00 00 00 00 00 00 00 |.......F........|
00000010 00 00 00 00 00 00 00 00-19 00 00 00 00 00 00 64 |...............d|
00000020 ff ff ff ff ff ff ff ff-ff ff ff ff 00 00 00 00 |................|
00000030 00 01 00 22 80 00 0a 02-08 00 00 01 01 08 00 00 |..."............|
2016-12-05T21:15:47.902Z|00030|ofctrl|INFO|OpenFlow error: OFPT_ERROR
(OF1.3) (xid=0x54): OFPBAC_BAD_TYPE
OFPT_FLOW_MOD (OF1.3) (xid=0x54):
(***truncated to 64 bytes from 120***)
00000000 04 0e 00 78 00 00 00 54-00 00 00 00 00 00 00 00 |...x...T........|
00000010 00 00 00 00 00 00 00 00-15 00 00 00 00 00 00 64 |...............d|
00000020 ff ff ff ff ff ff ff ff-ff ff ff ff 00 00 00 00 |................|
00000030 00 01 00 22 80 00 0a 02-86 dd 00 01 01 08 00 00 |..."............|
Your help is greatly appreciated!
Devin
On Mon, Dec 5, 2016 at 12:31 PM, Lance Richardson <lrichard at redhat.com>
wrote:
> > From: "Devin Acosta" <devin at pabstatencio.com>
> > To: "Marcin Mirecki" <mmirecki at redhat.com>
> > Cc: "users" <Users at ovirt.org>
> > Sent: Monday, December 5, 2016 12:11:46 PM
> > Subject: Re: [ovirt-users] oVIRT 4 / OVN / Communication issues of
> instances between nodes.
> >
> > Marcin,
> >
> > Also I noticed in your original post it mentions:
> >
> > ip link - the result should include a link called genev_sys_ ...
> >
> > I noticed that on my hosts I don't see any links with name: genev_sys_ ??
> > Could this be a problem?
> >
> > lo:
> > enp4s0f0:
> > enp4s0f1:
> > enp7s0f0:
> > enp7s0f1:
> > bond0:
> > DEV-NOC:
> > ovirtmgmt:
> > bond0.700 at bond0:
> > DEV-VM-NET:
> > bond0.705 at bond0:
> > ;vdsmdummy;:
> > vnet0:
> > vnet1:
> > vnet2:
> > vnet3:
> > vnet4:
> > ovs-system:
> > br-int:
> > vnet5:
> > vnet6:
> >
>
> Hi Devin,
>
> What distribution and kernel version are you using?
>
> One thing you could check is whether the vport_geneve kernel module
> is being loaded, e.g. you should see something like:
>
> $ lsmod | grep vport
> vport_geneve 12560 1
> openvswitch 246755 5 vport_geneve
>
> If vport_geneve is not loaded, you could "sudo modprobe vport_geneve"
> to make sure it's available and can be loaded.
>
> The first 100 lines or so of ovs-vswitchd.log might have some useful
> information about where things are going wrong.
>
> It does sound as though there is some issue with geneve tunnels,
> which would certainly explain issues with inter-node traffic.
>
> Regards,
>
> Lance
>
--
Devin Acosta
Red Hat Certified Architect, LinuxStack
602-354-1220 || devin at linuxguru.co
-------------- next part --------------
An HTML attachment was scrubbed...
URL: <http://lists.ovirt.org/pipermail/users/attachments/20161205/79537d5d/attachment-0001.html>
More information about the Users
mailing list