[ovirt-users] Windows Networking Issues -

Tue Dec 1 09:31:11 EST 2015

I wanted to follow up on this after I found my resolution.

I started to see kernel errors when I migrated all but my windows host off
a hypervisor and generated traffic.  I then took those errors and started
looking back at all of the hypervisors only to find this error was on each
of them; also it actively reporting on systems with Windows VMs.  Tracing
back the logs lead me to the bug reports below where I learned that this
issue had been re-introduced to the 2.6.32-X kernel.

ABRT info -
Nov 30 19:28:42 server2.example.com kernel: WARNING: at net/core/dev.c:1915
skb_warn_bad_offload+0x99/0xb0() (Tainted: G        W  -- ------------   )
Nov 30 19:28:42 server2.example.com kernel: Hardware name: PowerEdge M620
Nov 30 19:28:42 server2.example.com kernel: : caps=(0x40c9, 0x0) len=1514
data_len=1460 ip_summed=1
Nov 30 19:28:42 server2.example.com kernel: Modules linked in: sch_prio
act_mirred cls_u32 sch_ingress ebt_arp xt_physdev ipt_REJECT
nf_conntrack_ipv4 nf_defrag_ipv4 xt_multiport iptable_filter ip_tables fuse
nfs lockd fscache auth_rpcgss nfs_acl sunrpc vfat fat bonding ebtable_nat
ebtables be2iscsi iscsi_boot_sysfs bnx2i cnic uio cxgb4i iw_cxgb4 cxgb4
cxgb3i libcxgbi iw_cxgb3 cxgb3 ib_iser rdma_cm ib_cm iw_cm ib_sa ib_mad
ib_core ib_addr iscsi_tcp libiscsi_tcp libiscsi scsi_transport_iscsi
mpt3sas mpt2sas scsi_transport_sas raid_class mptctl mptbase dell_rbu
autofs4 bridge 8021q garp stp llc ip6t_REJECT nf_conntrack_ipv6
nf_defrag_ipv6 xt_state nf_conntrack ip6table_filter ip6_tables ipv6
dm_round_robin dm_multipath vhost_net macvtap macvlan tun kvm_intel kvm sg
ipmi_devintf sr_mod cdrom joydev power_meter acpi_ipmi ipmi_si
ipmi_msghandler iTCO_wdt iTCO_vendor_support ixgbe dca ptp pps_core mdio
dcdbas sb_edac edac_core lpc_ich mfd_core shpchp usb_storage ext4 jbd2
mbcache sd_mod crc_t10dif megaraid_sas wmi ahci dm_mirror dm_region_hash
dm_log dm_mod [last unloaded: ip_tables]
Nov 30 19:28:42 server2.example.com kernel: Pid: 0, comm: swapper Tainted:
G        W  -- ------------    2.6.32-573.7.1.el6.x86_64 #1
Nov 30 19:28:42 server2.example.com kernel: Call Trace:
Nov 30 19:28:42 server2.example.com kernel: <IRQ>  [<ffffffff81077461>] ?
warn_slowpath_common+0x91/0xe0
Nov 30 19:28:42 server2.example.com kernel: [<ffffffff81077566>] ?
warn_slowpath_fmt+0x46/0x60
Nov 30 19:28:42 server2.example.com kernel: [<ffffffff81296a55>] ?
__ratelimit+0xd5/0x120
Nov 30 19:28:42 server2.example.com kernel: [<ffffffff8146b2e9>] ?
skb_warn_bad_offload+0x99/0xb0
Nov 30 19:28:42 server2.example.com kernel: [<ffffffff8146f9b1>] ?
__skb_gso_segment+0x71/0xc0
Nov 30 19:28:42 server2.example.com kernel: [<ffffffff8146fa13>] ?
skb_gso_segment+0x13/0x20
Nov 30 19:28:42 server2.example.com kernel: [<ffffffff8146fabb>] ?
dev_hard_start_xmit+0x9b/0x490
Nov 30 19:28:42 server2.example.com kernel: [<ffffffff8148d17a>] ?
sch_direct_xmit+0x15a/0x1c0
Nov 30 19:28:42 server2.example.com kernel: [<ffffffff81470158>] ?
dev_queue_xmit+0x228/0x320
Nov 30 19:28:42 server2.example.com kernel: [<ffffffffa034d8b0>] ?
__br_forward+0x0/0xd0 [bridge]
Nov 30 19:28:42 server2.example.com kernel: [<ffffffffa034d818>] ?
br_dev_queue_push_xmit+0x88/0xc0 [bridge]
Nov 30 19:28:42 server2.example.com kernel: [<ffffffffa034d8a8>] ?
br_forward_finish+0x58/0x60 [bridge]
Nov 30 19:28:42 server2.example.com kernel: [<ffffffffa034d95a>] ?
__br_forward+0xaa/0xd0 [bridge]
Nov 30 19:28:42 server2.example.com kernel: [<ffffffff8145f788>] ?
skb_clone+0x58/0xb0
Nov 30 19:28:42 server2.example.com kernel: [<ffffffffa034d4de>] ?
deliver_clone+0x3e/0x60 [bridge]
Nov 30 19:28:42 server2.example.com kernel: [<ffffffffa034d9c1>] ?
br_forward+0x41/0x70 [bridge]
Nov 30 19:28:42 server2.example.com kernel: [<ffffffffa034e87e>] ?
br_handle_frame_finish+0x17e/0x330 [bridge]
Nov 30 19:28:42 server2.example.com kernel: [<ffffffffa034ebf0>] ?
br_handle_frame+0x1c0/0x270 [bridge]
Nov 30 19:28:42 server2.example.com kernel: [<ffffffffa034ea30>] ?
br_handle_frame+0x0/0x270 [bridge]
Nov 30 19:28:42 server2.example.com kernel: [<ffffffff8146aaf7>] ?
__netif_receive_skb+0x1c7/0x570
Nov 30 19:28:42 server2.example.com kernel: [<ffffffff8146e3d8>] ?
netif_receive_skb+0x58/0x60
Nov 30 19:28:42 server2.example.com kernel: [<ffffffff8146e4e0>] ?
napi_skb_finish+0x50/0x70
Nov 30 19:28:42 server2.example.com kernel: [<ffffffff8146eaf9>] ?
napi_gro_receive_gr+0x39/0x50
Nov 30 19:28:42 server2.example.com kernel: [<ffffffff81512aeb>] ?
vlan_gro_receive+0x1b/0x30
Nov 30 19:28:42 server2.example.com kernel: [<ffffffffa0185e25>] ?
ixgbe_clean_rx_irq+0x995/0xc70 [ixgbe]
Nov 30 19:28:42 server2.example.com kernel: [<ffffffffa018663a>] ?
ixgbe_poll+0x40a/0x760 [ixgbe]
Nov 30 19:28:42 server2.example.com kernel: [<ffffffff81470443>] ?
net_rx_action+0x103/0x2f0
Nov 30 19:28:42 server2.example.com kernel: [<ffffffff810ad4bd>] ?
ktime_get+0x6d/0x100
Nov 30 19:28:42 server2.example.com kernel: [<ffffffff8107ffa1>] ?
__do_softirq+0xc1/0x1e0
Nov 30 19:28:42 server2.example.com kernel: [<ffffffff810ed920>] ?
handle_IRQ_event+0x60/0x170
Nov 30 19:28:42 server2.example.com kernel: [<ffffffff8100c38c>] ?
call_softirq+0x1c/0x30
Nov 30 19:28:42 server2.example.com kernel: [<ffffffff8100fbd5>] ?
do_softirq+0x65/0xa0
Nov 30 19:28:42 server2.example.com kernel: [<ffffffff8107fe55>] ?
irq_exit+0x85/0x90
Nov 30 19:28:42 server2.example.com kernel: [<ffffffff815426d5>] ?
do_IRQ+0x75/0xf0
Nov 30 19:28:42 server2.example.com kernel: [<ffffffff8100ba53>] ?
ret_from_intr+0x0/0x11
Nov 30 19:28:42 server2.example.com kernel: <EOI>  [<ffffffff812f109e>] ?
intel_idle+0xfe/0x1b0
Nov 30 19:28:42 server2.example.com kernel: [<ffffffff812f1081>] ?
intel_idle+0xe1/0x1b0
Nov 30 19:28:42 server2.example.com kernel: [<ffffffff8143376a>] ?
cpuidle_idle_call+0x7a/0xe0
Nov 30 19:28:42 server2.example.com kernel: [<ffffffff81009fe6>] ?
cpu_idle+0xb6/0x110
Nov 30 19:28:42 server2.example.com kernel: [<ffffffff8151f21a>] ?
rest_init+0x7a/0x80
Nov 30 19:28:42 server2.example.com kernel: [<ffffffff81c38122>] ?
start_kernel+0x424/0x431
Nov 30 19:28:42 server2.example.com kernel: [<ffffffff81c3733a>] ?
x86_64_start_reservations+0x125/0x129
Nov 30 19:28:42 server2.example.com kernel: [<ffffffff81c37453>] ?
x86_64_start_kernel+0x115/0x124
Nov 30 19:28:42 server2.example.com kernel: ---[ end trace 38b6983a05ecf4b4
]---

I was able to resolve this by re-configuring the network cards em1/em2 with
lro = off.
ethtool to disable lro -
ethtool -K em1 lro off; ethtool -K em2 lro off

Bug reports -
1270892 - https://bugzilla.redhat.com/show_bug.cgi?id=1270892
772317 - https://bugzilla.redhat.com/show_bug.cgi?id=772317
-------------- next part --------------
An HTML attachment was scrubbed...
URL: <http://lists.ovirt.org/pipermail/users/attachments/20151201/c453a68c/attachment-0001.html>