On Tue, Sep 13, 2016 at 10:41 PM, Luca 'remix_tj' Lorenzetto < lorenzetto.luca@gmail.com> wrote:
On Tue, Sep 13, 2016 at 9:20 PM, Edward Haas <ehaas@redhat.com> wrote:
On Mon, Sep 12, 2016 at 3:14 PM, Luca 'remix_tj' Lorenzetto <lorenzetto.luca@gmail.com> wrote:
On Mon, Sep 12, 2016 at 2:11 PM, Colin Coe <colin.coe@gmail.com> wrote:
So is the problem getting traffic between two different VLANs working? If so, have you double checked your routing?
No, the issue is that VM cannot still ping the gateway of it's own network.
Resuming:
VM can ping the host interface assigned to the logical network (eno5) VM cannot ping gateway
Host can ping gateway and VM
External machine (e.g. gateway) cannot ping VM External machine can ping the host interface assigned to the logical network (eno5)
I may be missing something here, and a diagram would have helped. Assuming this is your setup, see how vlan tagging matters and translates
to
the network:
[pc]---<vlan 828, access>[switch]<trunk, tag 828>----<trunk>[eno5]-[eno5.828]<tag stripped>[bridge]---<vtap>-[vNIC - regular iface, no vlan]
In this setup, eno5 and eno5.828 should not have any IP defined, that is not legal. You need the address on the bridge. And you are not suppose to add it manually, but set it through Engine.
Yes, i reported a wrong information. Sorry.
I added the ip address starting via the engine to the bridge Development.
Based on your description, setting an address on eno5, means that your gw/pc is not residing on a VLAN, or you have no trunk between eno to it. [cut] and fix the data flow diagram if it is not correct.
The data flow now is the following, after disabling VLAN settings on engine and changing the port as native interface.
[pc]---<vlan 828, access>[switch]<vlan828, access>----[eno5]---[bridge]---<vtap>-[vNIC - regular iface, no vlan]
Before was as you reported. VLAN usage has been disable to check if something was wrong on my setup on the host.
As you can see on my previous emails, seems that the issue is about the linux bridge, which is not forwarding ARP packets from the external network (traffic incoming to eno5) to the vtap (vnet0, which reports mac address different from the one seen inside the VM)
Perhaps it will be better to just add here the output of from the host: ip addr
1: lo: <LOOPBACK,UP,LOWER_UP> mtu 65536 qdisc noqueue state UNKNOWN link/loopback 00:00:00:00:00:00 brd 00:00:00:00:00:00 inet 127.0.0.1/8 scope host lo valid_lft forever preferred_lft forever 2: eno1: <BROADCAST,MULTICAST,SLAVE,UP,LOWER_UP> mtu 1500 qdisc mq master bond0 state UP qlen 1000 link/ether 00:21:5a:9b:ba:8d brd ff:ff:ff:ff:ff:ff 3: eno2: <BROADCAST,MULTICAST,SLAVE,UP,LOWER_UP> mtu 1500 qdisc mq master bond0 state UP qlen 1000 link/ether 00:21:5a:9b:ba:8d brd ff:ff:ff:ff:ff:ff 4: eno3: <BROADCAST,MULTICAST,SLAVE,UP,LOWER_UP> mtu 1500 qdisc mq master bond1 state UP qlen 1000 link/ether 00:21:5a:9b:ba:91 brd ff:ff:ff:ff:ff:ff 5: eno4: <BROADCAST,MULTICAST,SLAVE,UP,LOWER_UP> mtu 1500 qdisc mq master bond1 state UP qlen 1000 link/ether 00:21:5a:9b:ba:91 brd ff:ff:ff:ff:ff:ff 6: eno5: <BROADCAST,MULTICAST,UP,LOWER_UP> mtu 1500 qdisc mq master Development state UP qlen 1000 link/ether 00:21:5a:9b:ba:95 brd ff:ff:ff:ff:ff:ff 7: eno6: <BROADCAST,MULTICAST,UP,LOWER_UP> mtu 1500 qdisc mq state UP qlen 1000 link/ether 00:21:5a:9b:ba:97 brd ff:ff:ff:ff:ff:ff 8: bond0: <BROADCAST,MULTICAST,MASTER,UP,LOWER_UP> mtu 1500 qdisc noqueue master ovirtmgmt state UP link/ether 00:21:5a:9b:ba:8d brd ff:ff:ff:ff:ff:ff 9: bond1: <BROADCAST,MULTICAST,MASTER,UP,LOWER_UP> mtu 1500 qdisc noqueue state UP link/ether 00:21:5a:9b:ba:91 brd ff:ff:ff:ff:ff:ff inet 172.25.44.57/22 brd 172.25.47.255 scope global bond1 valid_lft forever preferred_lft forever 10: ;vdsmdummy;: <BROADCAST,MULTICAST> mtu 1500 qdisc noop state DOWN link/ether 00:00:00:00:00:00 brd ff:ff:ff:ff:ff:ff 12: ovirtmgmt: <BROADCAST,MULTICAST,UP,LOWER_UP> mtu 1500 qdisc noqueue state UP link/ether 00:21:5a:9b:ba:8d brd ff:ff:ff:ff:ff:ff inet 10.5.12.48/22 brd 10.5.15.255 scope global ovirtmgmt valid_lft forever preferred_lft forever 31: Development: <BROADCAST,MULTICAST,UP,LOWER_UP> mtu 1500 qdisc noqueue state UP link/ether 00:21:5a:9b:ba:95 brd ff:ff:ff:ff:ff:ff inet 10.5.30.12/22 brd 10.5.31.255 scope global Development valid_lft forever preferred_lft forever 32: vnet0: <BROADCAST,MULTICAST,UP,LOWER_UP> mtu 1500 qdisc pfifo_fast master Development state UNKNOWN qlen 500 link/ether fe:1a:4a:16:01:51 brd ff:ff:ff:ff:ff:ff
brctl show
bridge name bridge id STP enabled interfaces ;vdsmdummy; 8000.000000000000 no Development 8000.00215a9bba95 no eno5 vnet0 ovirtmgmt 8000.00215a9bba8d no bond0
vdsClient -s 0 getVdsCaps
HBAInventory = {'FC': [{'model': '554FLB - HP FlexFabric 10Gb 2-port 554FLB Adapter', 'wwnn': '5001438002a3b2f7', 'wwpn': '5001438002a3b2f6'}, {'model': '554FLB - HP FlexFabric 10Gb 2-port 554FLB Adapter', 'wwnn': '5001438002a3b2f9', 'wwpn': '5001438002a3b2f8'}], 'iSCSI': [{'InitiatorName': 'iqn.1994-05.com.redhat:33f12e4cd757'}]} ISCSIInitiatorName = 'iqn.1994-05.com.redhat:33f12e4cd757' additionalFeatures = [] autoNumaBalancing = 0 bondings = {'bond0': {'active_slave': 'eno1', 'addr': '', 'cfg': {'BONDING_OPTS': '', 'BOOTPROTO': 'none', 'BRIDGE': 'ovirtmgmt', 'DEVICE': 'bond0', 'HOTPLUG': 'no', 'IPV6INIT': 'no', 'MTU': '1500', 'NM_CONTROLLED': 'no', 'ONBOOT': 'yes'}, 'dhcpv4': False, 'dhcpv6': False, 'gateway': '', 'hwaddr': '00:21:5a:9b:ba:8d', 'ipv4addrs': [], 'ipv6addrs': [], 'ipv6gateway': '::', 'mtu': '1500', 'netmask': '', 'opts': {'miimon': '100', 'mode': '1'}, 'slaves': ['eno1', 'eno2']}, 'bond1': {'active_slave': 'eno3', 'addr': '172.25.44.57', 'cfg': {'BONDING_MASTER': 'yes', 'BONDING_OPTS': '', 'BOOTPROTO': 'none', 'DEFROUTE': 'yes', 'DEVICE': 'bond1', 'IPADDR': '172.25.44.57', 'IPV4_FAILURE_FATAL': 'no', 'IPV6INIT': 'no', 'NAME': 'bond1', 'PREFIX': '22', 'TYPE': 'Bond'}, 'dhcpv4': False, 'dhcpv6': False, 'gateway': '', 'hwaddr': '00:21:5a:9b:ba:91', 'ipv4addrs': ['172.25.44.57/22'], 'ipv6addrs': [], 'ipv6gateway': '::', 'mtu': '1500', 'netmask': '255.255.252.0', 'opts': {'miimon': '100', 'mode': '1'}, 'slaves': ['eno3', 'eno4']}} bridges = {'Development': {'addr': '10.5.30.12', 'cfg': {'BOOTPROTO': 'none', 'DEFROUTE': 'no', 'DELAY': '0', 'DEVICE': 'Development', 'GATEWAY': '10.5.31.254', 'HOTPLUG': 'no', 'IPADDR': '10.5.30.12', 'IPV6INIT': 'no', 'MTU': '1500', 'NETMASK': '255.255.252.0', 'NM_CONTROLLED': 'no', 'ONBOOT': 'yes', 'STP': 'off', 'TYPE': 'Bridge'}, 'dhcpv4': False, 'dhcpv6': False, 'gateway': '10.5.31.254', 'ipv4addrs': ['10.5.30.12/22'], 'ipv6addrs': [], 'ipv6gateway': '::', 'mtu': '1500', 'netmask': '255.255.252.0', 'opts': {'ageing_time': '30000', 'bridge_id': '8000.00215a9bba95', 'default_pvid': '1', 'forward_delay': '0', 'gc_timer': '840', 'group_addr': '1:80:c2:0:0:0', 'group_fwd_mask': '0x0', 'hash_elasticity': '4', 'hash_max': '512', 'hello_time': '200', 'hello_timer': '37', 'max_age': '2000', 'multicast_last_member_count': '2',
'multicast_last_member_interval': '100',
'multicast_membership_interval': '26000', 'multicast_querier': '0', 'multicast_querier_interval': '25500', 'multicast_query_interval': '12500',
'multicast_query_response_interval': '1000', 'multicast_router': '1', 'multicast_snooping': '1', 'multicast_startup_query_count': '2',
'multicast_startup_query_interval': '3125', 'nf_call_arptables': '0', 'nf_call_ip6tables': '0', 'nf_call_iptables': '0', 'priority': '32768', 'root_id': '8000.00215a9bba95', 'root_path_cost': '0', 'root_port': '0', 'stp_state': '0', 'tcn_timer': '0', 'topology_change': '0', 'topology_change_detected': '0', 'topology_change_timer': '0', 'vlan_filtering': '0'}, 'ports': ['eno5', 'vnet0'], 'stp': 'off'}, 'ovirtmgmt': {'addr': '10.5.12.48', 'cfg': {'BOOTPROTO': 'none', 'DEFROUTE': 'yes', 'DELAY': '0', 'DEVICE': 'ovirtmgmt', 'GATEWAY': '10.5.15.254', 'HOTPLUG': 'no', 'IPADDR': '10.5.12.48', 'IPV6INIT': 'no', 'MTU': '1500', 'NETMASK': '255.255.252.0', 'NM_CONTROLLED': 'no', 'ONBOOT': 'yes', 'STP': 'off', 'TYPE': 'Bridge'}, 'dhcpv4': False, 'dhcpv6': False, 'gateway': '10.5.15.254', 'ipv4addrs': ['10.5.12.48/22'], 'ipv6addrs': [], 'ipv6gateway': '::', 'mtu': '1500', 'netmask': '255.255.252.0', 'opts': {'ageing_time': '30000', 'bridge_id': '8000.00215a9bba8d', 'default_pvid': '1', 'forward_delay': '0', 'gc_timer': '1141', 'group_addr': '1:80:c2:0:0:0', 'group_fwd_mask': '0x0', 'hash_elasticity': '4', 'hash_max': '512', 'hello_time': '200', 'hello_timer': '36', 'max_age': '2000', 'multicast_last_member_count': '2',
'multicast_last_member_interval': '100',
'multicast_membership_interval': '26000', 'multicast_querier': '0', 'multicast_querier_interval': '25500', 'multicast_query_interval': '12500',
'multicast_query_response_interval': '1000', 'multicast_router': '1', 'multicast_snooping': '1', 'multicast_startup_query_count': '2',
'multicast_startup_query_interval': '3125', 'nf_call_arptables': '0', 'nf_call_ip6tables': '0', 'nf_call_iptables': '0', 'priority': '32768', 'root_id': '8000.00215a9bba8d', 'root_path_cost': '0', 'root_port': '0', 'stp_state': '0', 'tcn_timer': '0', 'topology_change': '0', 'topology_change_detected': '0', 'topology_change_timer': '0', 'vlan_filtering': '0'}, 'ports': ['bond0'], 'stp': 'off'}} clusterLevels = ['3.4', '3.5', '3.6'] cpuCores = '6' cpuFlags = 'fpu,vme,de,pse,tsc,msr,pae,mce,cx8,apic,sep,mtrr,pge,mca, cmov,pat,pse36,clflush,dts,acpi,mmx,fxsr,sse,sse2,ss,ht, tm,pbe,syscall,nx,pdpe1gb,rdtscp,lm,constant_tsc,arch_ perfmon,pebs,bts,rep_good,nopl,xtopology,nonstop_tsc, aperfmperf,eagerfpu,pni,pclmulqdq,dtes64,monitor,ds_ cpl,vmx,smx,est,tm2,ssse3,cx16,xtpr,pdcm,pcid,dca,sse4_ 1,sse4_2,x2apic,popcnt,tsc_deadline_timer,aes,xsave,avx, lahf_lm,ida,arat,epb,pln,pts,dtherm,tpr_shadow,vnmi,flexpriority,ept,vpid, xsaveopt,model_Nehalem,model_Conroe,model_coreduo,model_ core2duo,model_Penryn,model_Westmere,model_n270,model_SandyBridge' cpuModel = 'Intel(R) Xeon(R) CPU E5-2640 0 @ 2.50GHz' cpuSockets = '1' cpuSpeed = '2500.000' cpuThreads = '12' emulatedMachines = ['pc-i440fx-rhel7.1.0', 'rhel6.3.0', 'pc-q35-rhel7.2.0', 'pc-i440fx-rhel7.0.0', 'rhel6.1.0', 'rhel6.6.0', 'rhel6.2.0', 'pc', 'pc-q35-rhel7.0.0', 'pc-q35-rhel7.1.0', 'q35', 'pc-i440fx-rhel7.2.0', 'rhel6.4.0', 'rhel6.0.0', 'rhel6.5.0'] guestOverhead = '65' hooks = {'before_device_create': {'50_vmfex': {'md5': 'e05994261acaea7dcf4b88ea0e81f1f5'}}, 'before_device_migrate_destination': {'50_vmfex': {'md5': 'e05994261acaea7dcf4b88ea0e81f1f5'}}, 'before_nic_hotplug': {'50_vmfex': {'md5': 'e05994261acaea7dcf4b88ea0e81f1f5'}}} hostdevPassthrough = 'false' kdumpStatus = 1 kvmEnabled = 'true' lastClient = '127.0.0.1' lastClientIface = 'lo' liveMerge = 'true' liveSnapshot = 'true' memSize = '23908' netConfigDirty = 'False' networks = {'Development': {'addr': '10.5.30.12', 'bridged': True, 'cfg': {'BOOTPROTO': 'none', 'DEFROUTE': 'no', 'DELAY': '0', 'DEVICE': 'Development', 'GATEWAY': '10.5.31.254', 'HOTPLUG': 'no', 'IPADDR': '10.5.30.12', 'IPV6INIT': 'no', 'MTU': '1500', 'NETMASK': '255.255.252.0', 'NM_CONTROLLED': 'no', 'ONBOOT': 'yes', 'STP': 'off', 'TYPE': 'Bridge'}, 'dhcpv4': False, 'dhcpv6': False, 'gateway': '10.5.31.254', 'iface': 'Development', 'ipv4addrs': ['10.5.30.12/22'], 'ipv6addrs': [], 'ipv6gateway': '::', 'mtu': '1500', 'netmask': '255.255.252.0', 'ports': ['eno5', 'vnet0'], 'stp': 'off'}, 'ovirtmgmt': {'addr': '10.5.12.48', 'bridged': True, 'cfg': {'BOOTPROTO': 'none', 'DEFROUTE': 'yes', 'DELAY': '0', 'DEVICE': 'ovirtmgmt', 'GATEWAY': '10.5.15.254', 'HOTPLUG': 'no', 'IPADDR': '10.5.12.48', 'IPV6INIT': 'no', 'MTU': '1500', 'NETMASK': '255.255.252.0', 'NM_CONTROLLED': 'no', 'ONBOOT': 'yes', 'STP': 'off', 'TYPE': 'Bridge'}, 'dhcpv4': False, 'dhcpv6': False, 'gateway': '10.5.15.254', 'iface': 'ovirtmgmt', 'ipv4addrs': ['10.5.12.48/22'], 'ipv6addrs': [], 'ipv6gateway': '::', 'mtu': '1500', 'netmask': '255.255.252.0', 'ports': ['bond0'], 'stp': 'off'}} nics = {'eno1': {'addr': '', 'cfg': {'BOOTPROTO': 'none', 'DEVICE': 'eno1', 'HWADDR': '00:21:5a:9b:ba:8d', 'IPV6INIT': 'no', 'MASTER': 'bond0', 'MTU': '1500', 'NM_CONTROLLED': 'no', 'ONBOOT': 'yes', 'SLAVE': 'yes'}, 'dhcpv4': False, 'dhcpv6': False, 'gateway': '', 'hwaddr': '00:21:5a:9b:ba:8d', 'ipv4addrs': [], 'ipv6addrs': [], 'ipv6gateway': '::', 'mtu': '1500', 'netmask': '', 'permhwaddr': '00:21:5a:9b:ba:8d', 'speed': 10000}, 'eno2': {'addr': '', 'cfg': {'BOOTPROTO': 'none', 'DEVICE': 'eno2', 'HWADDR': '00:21:5a:9b:ba:8f', 'IPV6INIT': 'no', 'MASTER': 'bond0', 'MTU': '1500', 'NM_CONTROLLED': 'no', 'ONBOOT': 'yes', 'SLAVE': 'yes'}, 'dhcpv4': False, 'dhcpv6': False, 'gateway': '', 'hwaddr': '00:21:5a:9b:ba:8d', 'ipv4addrs': [], 'ipv6addrs': [], 'ipv6gateway': '::', 'mtu': '1500', 'netmask': '', 'permhwaddr': '00:21:5a:9b:ba:8f', 'speed': 10000}, 'eno3': {'addr': '', 'cfg': {'BOOTPROTO': 'none', 'DEFROUTE': 'yes', 'DEVICE': 'eno3', 'IPV4_FAILURE_FATAL': 'no', 'IPV6INIT': 'no', 'MASTER': 'bond1', 'NAME': 'bond1-slave', 'ONBOOT': 'yes', 'PEERDNS': 'yes', 'PEERROUTES': 'yes', 'SLAVE': 'yes', 'TYPE': 'Ethernet'}, 'dhcpv4': False, 'dhcpv6': False, 'gateway': '', 'hwaddr': '00:21:5a:9b:ba:91', 'ipv4addrs': [], 'ipv6addrs': [], 'ipv6gateway': '::', 'mtu': '1500', 'netmask': '', 'permhwaddr': '00:21:5a:9b:ba:91', 'speed': 10000}, 'eno4': {'addr': '', 'cfg': {'BOOTPROTO': 'none', 'DEFROUTE': 'yes', 'DEVICE': 'eno4', 'IPV4_FAILURE_FATAL': 'no', 'IPV6INIT': 'no', 'MASTER': 'bond1', 'NAME': 'bond1-slave2', 'ONBOOT': 'yes', 'PEERDNS': 'yes', 'PEERROUTES': 'yes', 'SLAVE': 'yes', 'TYPE': 'Ethernet'}, 'dhcpv4': False, 'dhcpv6': False, 'gateway': '', 'hwaddr': '00:21:5a:9b:ba:91', 'ipv4addrs': [], 'ipv6addrs': [], 'ipv6gateway': '::', 'mtu': '1500', 'netmask': '', 'permhwaddr': '00:21:5a:9b:ba:93', 'speed': 10000}, 'eno5': {'addr': '', 'cfg': {'BOOTPROTO': 'none', 'BRIDGE': 'Development', 'DEVICE': 'eno5', 'HWADDR': '00:21:5a:9b:ba:95', 'IPV6INIT': 'no', 'MTU': '1500', 'NM_CONTROLLED': 'no', 'ONBOOT': 'yes'}, 'dhcpv4': False, 'dhcpv6': False, 'gateway': '', 'hwaddr': '00:21:5a:9b:ba:95', 'ipv4addrs': [], 'ipv6addrs': [], 'ipv6gateway': '::', 'mtu': '1500', 'netmask': '', 'speed': 10000}, 'eno6': {'addr': '', 'cfg': {'BOOTPROTO': 'none', 'DEVICE': 'eno6', 'HWADDR': '00:21:5a:9b:ba:97', 'MTU': '1500', 'NM_CONTROLLED': 'no', 'ONBOOT': 'yes'}, 'dhcpv4': False, 'dhcpv6': False, 'gateway': '', 'hwaddr': '00:21:5a:9b:ba:97', 'ipv4addrs': [], 'ipv6addrs': [], 'ipv6gateway': '::', 'mtu': '1500', 'netmask': '', 'speed': 10000}} numaNodeDistance = {'0': [10]} numaNodes = {'0': {'cpus': [0, 1, 2, 3, 4, 5, 6, 7, 8, 9, 10, 11], 'totalMemory': '23908'}} onlineCpus = '0,1,2,3,4,5,6,7,8,9,10,11' operatingSystem = {'name': 'RHEL', 'release': '9.el7', 'version': '7.2'} packages2 = {'glusterfs-cli': {'buildtime': 1447142472, 'release': '1.el7', 'version': '3.7.6'}, 'kernel': {'buildtime': 1453345829.0, 'release': '327.4.5.el7.x86_64', 'version': '3.10.0'}, 'librbd1': {'buildtime': 1442598293, 'release': '17.el7cp', 'version': '0.80.8'}, 'libvirt': {'buildtime': 1456935391, 'release': '13.el7_2.4', 'version': '1.2.17'}, 'mom': {'buildtime': 1461328785, 'release': '1.1.el7', 'version': '0.5.3'}, 'qemu-img': {'buildtime': 1460115917, 'release': '31.el7_2.10.1', 'version': '2.3.0'}, 'qemu-kvm': {'buildtime': 1460115917, 'release': '31.el7_2.10.1', 'version': '2.3.0'}, 'spice-server': {'buildtime': 1461678243, 'release': '15.el7_2.1', 'version': '0.12.4'}, 'vdsm': {'buildtime': 1464098638, 'release': '1.el7', 'version': '4.17.28'}} reservedMem = '321' rngSources = ['random'] selinux = {'mode': '-1'} software_revision = '1' software_version = '4.17' supportedENGINEs = ['3.4', '3.5', '3.6'] uuid = 'CB758278-ED02-4BF4-A2F2-FD1EFCC73D3A' version_name = 'Snow Man' vlans = {} vmTypes = ['kvm']
I hope in some help. The behavior is very strange and i've no idea on how to move.
If I understood corectly, the VM cannot ping the gw/pc even without vlans in place. Do you have mutiple macs on the VM itself? (like a bridge or some sub interfaces) The only thing that comes to my mind is the mac-spoofing that is enabled by default, it restricts the VM from using more than one mac address to communicate through a vnic. Except that, I do not have a good idea on what may be wrong. Your best friend is tcpdump, make sure you do not see any vlan/tagged packets on the bridge. (if a tagged packet enters from eno5, it will pass to the bridge as is, even passed to the vtap as is... the VM will drop it as it does not expect a tag there)
Luca
-- "E' assurdo impiegare gli uomini di intelligenza eccellente per fare calcoli che potrebbero essere affidati a chiunque se si usassero delle macchine" Gottfried Wilhelm von Leibnitz, Filosofo e Matematico (1646-1716)
"Internet è la più grande biblioteca del mondo. Ma il problema è che i libri sono tutti sparsi sul pavimento" John Allen Paulos, Matematico (1945-vivente)
Luca 'remix_tj' Lorenzetto, http://www.remixtj.net , < lorenzetto.luca@gmail.com>