From jon at rosslug.org.uk Sat Jan 2 06:05:40 2016 Content-Type: multipart/mixed; boundary="===============6817363720204117397==" MIME-Version: 1.0 From: Jon Archer To: users at ovirt.org Subject: Re: [ovirt-users] R: Re: Network instability after upgrade 3.6.0 -> 3.6.1 Date: Sat, 02 Jan 2016 11:05:35 +0000 Message-ID: <9F04B853-E046-43FA-93FF-EC6901393A40@rosslug.org.uk> In-Reply-To: 20151230094402.GA29152@redhat.com --===============6817363720204117397== Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: quoted-printable ------6F542WX18VLPXEJTSAO8OF9DUPZW5N Content-Transfer-Encoding: 8bit Content-Type: text/plain; charset=3DUTF-8 Hi, I'm not near the server for a while but network is set up 2x broadcom nic with whatever driver works out of the box. Both set as slaves to bond0 which is in mode=3D4 with no explicit options. Switch is a fairly basic tp-link but works almost identically to a Cisco. H= as the 2 ports set up in a portchannel. Short and long, this config works in earlier kernel, but not with kernel sh= ipped in 7.2 Release notes for RH7.2 suggest some work on bonding has been done, wonder = if default options (LACP speed?) have changed? Jon On 30 December 2015 09:44:02 GMT+00:00, Dan Kenigsberg wrote: >On Tue, Dec 29, 2015 at 09:57:07PM +0000, Jon Archer wrote: >> Hi Stefano, >> = >> It's definitely not the switch, it seems to be the latest kernel >package >> (kernel-3.10.0-327.3.1.el7.x86_64) which stops bonding working >correctly, >> reverting back to the previous kernel brings the network up in >802.3ad mode >> (4). >> = >> I know, from reading the release notes of 7.2, that there were some >changes >> to the bonding bits in the kernel so i'm guessing maybe some defaults >have >> changed. >> = >> I'll keep digging and post back as soon as i have something. >> = >> Jon >> = >> On 29/12/15 19:55, Stefano Danzi wrote: >> >Hi! I didn't solve yet. I'm still using mode 2 on bond interface. >What's >> >your switch model and firmware version? > >Hi Jon and Stefano, > >We've been testing bond mode 4 with (an earlier) >kernel-3.10.0-327.el7.x86_64 and experienced no such behaviour. > >However, to better identify the suspected kernel bug, could you provide >more information regarding your network connectivity? > >What is the make of your NICs? Which driver do you use? > >Do you set special ethtool opts (LRO with bridge was broken in 7.2.0 >kernel if I am not mistaken)? > >You have the ovirtmgmt bridge on top of your bond, right? > >Can you share your ifcfg*? -- = Sent from my Android device with K-9 Mail. Please excuse my brevity. ------6F542WX18VLPXEJTSAO8OF9DUPZW5N Content-Type: text/html; charset=3Dutf-8 Content-Transfer-Encoding: 8bit Hi,

I'm not near the server for a while but network is set up

2x broadcom nic with whatever driver works out of the box.

Both set as slaves to bond0 which is in mode=3D4 with no explicit options.<= br>
Switch is a fairly basic tp-link but works almost identically to a Cisco. H= as the 2 ports set up in a portchannel.

Short and long, this config works in earlier kernel, but not with kernel sh= ipped in 7.2

Release notes for RH7.2 suggest some work on bonding has been done, wonder = if default options (LACP speed?) have changed?

Jon

On 30 December 2015 09:44:02 GMT+00:0= 0, Dan Kenigsberg <danken(a)redhat.com> wrote:

On Tue, Dec 29, 2015 at 09:57:07PM +0000, Jon Archer =
wrote:
 Hi Stefano,
 
 It's definitely not the switch, it seems to be the latest kernel =
package
 (kernel-3.10.0-327.3.1.el7.x86_64) which stops bonding workin=
g correctly,
 reverting back to the previous kernel brings the network=
 up in 802.3ad mode
 (4).
 
 I know, from reading the releas=
e notes of 7.2, that there were some changes
 to the bonding bits in t=
he kernel so i'm guessing maybe some defaults have
 changed.
  I'll keep digging and post back as soon as i have something.
 
 Jon
 
 On 29/12/15 19:55, Stefano Danzi wrote:
Hi! I didn't solve yet. I'm still usin=
g mode 2 on bond interfac
 e.
What's
your switch model and firmware version?

Hi Jon and Stefano,

We've been testing bond mode 4=
 with (an earlier)
kernel-3.10.0-327.el7.x86_64 and experienced no suc=
h behaviour.

However, to better identify the suspected kernel bu=
g, could you provide
more information regarding your network connectiv=
ity?

What is the make of your NICs? Which driver do you use?

Do you set special ethtool opts (LRO with bridge was broken in 7.2.=
0
kernel if I am not mistaken)?

You have the ovirtmgmt brid=
ge on top of your bond, right?

Can you share your ifcfg*?
<=
br />

--
Sent from my Android device with K-9 Mail. Please excuse my brevity.= ------6F542WX18VLPXEJTSAO8OF9DUPZW5N-- --===============6817363720204117397== Content-Type: multipart/alternative MIME-Version: 1.0 Content-Transfer-Encoding: base64 Content-Disposition: attachment; filename="attachment.bin" LS0tLS0tNkY1NDJXWDE4VkxQWEVKVFNBTzhPRjlEVVBaVzVOCkNvbnRlbnQtVHJhbnNmZXItRW5j b2Rpbmc6IDhiaXQKQ29udGVudC1UeXBlOiB0ZXh0L3BsYWluOwogY2hhcnNldD1VVEYtOAoKSGks CgpJJ20gbm90IG5lYXIgdGhlIHNlcnZlciBmb3IgYSB3aGlsZSBidXQgbmV0d29yayBpcyBzZXQg dXAKCjJ4IGJyb2FkY29tIG5pYyB3aXRoIHdoYXRldmVyIGRyaXZlciB3b3JrcyBvdXQgb2YgdGhl IGJveC4KCkJvdGggc2V0IGFzIHNsYXZlcyB0byBib25kMCB3aGljaCBpcyBpbiBtb2RlPTQgd2l0 aCBubyBleHBsaWNpdCBvcHRpb25zLgoKU3dpdGNoIGlzIGEgZmFpcmx5IGJhc2ljIHRwLWxpbmsg YnV0IHdvcmtzIGFsbW9zdCBpZGVudGljYWxseSB0byBhIENpc2NvLiBIYXMgdGhlIDIgcG9ydHMg c2V0IHVwIGluIGEgcG9ydGNoYW5uZWwuCgpTaG9ydCBhbmQgbG9uZywgdGhpcyBjb25maWcgd29y a3MgaW4gZWFybGllciBrZXJuZWwsIGJ1dCBub3Qgd2l0aCBrZXJuZWwgc2hpcHBlZCBpbiA3LjIK ClJlbGVhc2Ugbm90ZXMgZm9yIFJINy4yIHN1Z2dlc3Qgc29tZSB3b3JrIG9uIGJvbmRpbmcgaGFz IGJlZW4gZG9uZSwgd29uZGVyIGlmIGRlZmF1bHQgb3B0aW9ucyAoTEFDUCBzcGVlZD8pIGhhdmUg Y2hhbmdlZD8KCkpvbgoKT24gMzAgRGVjZW1iZXIgMjAxNSAwOTo0NDowMiBHTVQrMDA6MDAsIERh biBLZW5pZ3NiZXJnIDxkYW5rZW5AcmVkaGF0LmNvbT4gd3JvdGU6Cj5PbiBUdWUsIERlYyAyOSwg MjAxNSBhdCAwOTo1NzowN1BNICswMDAwLCBKb24gQXJjaGVyIHdyb3RlOgo+PiBIaSBTdGVmYW5v LAo+PiAKPj4gSXQncyBkZWZpbml0ZWx5IG5vdCB0aGUgc3dpdGNoLCBpdCBzZWVtcyB0byBiZSB0 aGUgbGF0ZXN0IGtlcm5lbAo+cGFja2FnZQo+PiAoa2VybmVsLTMuMTAuMC0zMjcuMy4xLmVsNy54 ODZfNjQpIHdoaWNoIHN0b3BzIGJvbmRpbmcgd29ya2luZwo+Y29ycmVjdGx5LAo+PiByZXZlcnRp bmcgYmFjayB0byB0aGUgcHJldmlvdXMga2VybmVsIGJyaW5ncyB0aGUgbmV0d29yayB1cCBpbgo+ ODAyLjNhZCBtb2RlCj4+ICg0KS4KPj4gCj4+IEkga25vdywgZnJvbSByZWFkaW5nIHRoZSByZWxl YXNlIG5vdGVzIG9mIDcuMiwgdGhhdCB0aGVyZSB3ZXJlIHNvbWUKPmNoYW5nZXMKPj4gdG8gdGhl IGJvbmRpbmcgYml0cyBpbiB0aGUga2VybmVsIHNvIGknbSBndWVzc2luZyBtYXliZSBzb21lIGRl ZmF1bHRzCj5oYXZlCj4+IGNoYW5nZWQuCj4+IAo+PiBJJ2xsIGtlZXAgZGlnZ2luZyBhbmQgcG9z dCBiYWNrIGFzIHNvb24gYXMgaSBoYXZlIHNvbWV0aGluZy4KPj4gCj4+IEpvbgo+PiAKPj4gT24g MjkvMTIvMTUgMTk6NTUsIFN0ZWZhbm8gRGFuemkgd3JvdGU6Cj4+ID5IaSEgSSBkaWRuJ3Qgc29s dmUgeWV0LiBJJ20gc3RpbGwgdXNpbmcgbW9kZSAyIG9uIGJvbmQgaW50ZXJmYWNlLgo+V2hhdCdz Cj4+ID55b3VyIHN3aXRjaCBtb2RlbCBhbmQgZmlybXdhcmUgdmVyc2lvbj8KPgo+SGkgSm9uIGFu ZCBTdGVmYW5vLAo+Cj5XZSd2ZSBiZWVuIHRlc3RpbmcgYm9uZCBtb2RlIDQgd2l0aCAoYW4gZWFy bGllcikKPmtlcm5lbC0zLjEwLjAtMzI3LmVsNy54ODZfNjQgYW5kIGV4cGVyaWVuY2VkIG5vIHN1 Y2ggYmVoYXZpb3VyLgo+Cj5Ib3dldmVyLCB0byBiZXR0ZXIgaWRlbnRpZnkgdGhlIHN1c3BlY3Rl ZCBrZXJuZWwgYnVnLCBjb3VsZCB5b3UgcHJvdmlkZQo+bW9yZSBpbmZvcm1hdGlvbiByZWdhcmRp bmcgeW91ciBuZXR3b3JrIGNvbm5lY3Rpdml0eT8KPgo+V2hhdCBpcyB0aGUgbWFrZSBvZiB5b3Vy IE5JQ3M/IFdoaWNoIGRyaXZlciBkbyB5b3UgdXNlPwo+Cj5EbyB5b3Ugc2V0IHNwZWNpYWwgZXRo dG9vbCBvcHRzIChMUk8gd2l0aCBicmlkZ2Ugd2FzIGJyb2tlbiBpbiA3LjIuMAo+a2VybmVsIGlm IEkgYW0gbm90IG1pc3Rha2VuKT8KPgo+WW91IGhhdmUgdGhlIG92aXJ0bWdtdCBicmlkZ2Ugb24g dG9wIG9mIHlvdXIgYm9uZCwgcmlnaHQ/Cj4KPkNhbiB5b3Ugc2hhcmUgeW91ciBpZmNmZyo/Cgot LSAKU2VudCBmcm9tIG15IEFuZHJvaWQgZGV2aWNlIHdpdGggSy05IE1haWwuIFBsZWFzZSBleGN1 c2UgbXkgYnJldml0eS4KLS0tLS0tNkY1NDJXWDE4VkxQWEVKVFNBTzhPRjlEVVBaVzVOCkNvbnRl bnQtVHlwZTogdGV4dC9odG1sOwogY2hhcnNldD11dGYtOApDb250ZW50LVRyYW5zZmVyLUVuY29k aW5nOiA4Yml0Cgo8aHRtbD48aGVhZD48L2hlYWQ+PGJvZHk+SGksPGJyPgo8YnI+CkkmIzM5O20g bm90IG5lYXIgdGhlIHNlcnZlciBmb3IgYSB3aGlsZSBidXQgbmV0d29yayBpcyBzZXQgdXA8YnI+ Cjxicj4KMnggYnJvYWRjb20gbmljIHdpdGggd2hhdGV2ZXIgZHJpdmVyIHdvcmtzIG91dCBvZiB0 aGUgYm94Ljxicj4KPGJyPgpCb3RoIHNldCBhcyBzbGF2ZXMgdG8gYm9uZDAgd2hpY2ggaXMgaW4g bW9kZT00IHdpdGggbm8gZXhwbGljaXQgb3B0aW9ucy48YnI+Cjxicj4KU3dpdGNoIGlzIGEgZmFp cmx5IGJhc2ljIHRwLWxpbmsgYnV0IHdvcmtzIGFsbW9zdCBpZGVudGljYWxseSB0byBhIENpc2Nv LiBIYXMgdGhlIDIgcG9ydHMgc2V0IHVwIGluIGEgcG9ydGNoYW5uZWwuPGJyPgo8YnI+ClNob3J0 IGFuZCBsb25nLCB0aGlzIGNvbmZpZyB3b3JrcyBpbiBlYXJsaWVyIGtlcm5lbCwgYnV0IG5vdCB3 aXRoIGtlcm5lbCBzaGlwcGVkIGluIDcuMjxicj4KPGJyPgpSZWxlYXNlIG5vdGVzIGZvciBSSDcu MiBzdWdnZXN0IHNvbWUgd29yayBvbiBib25kaW5nIGhhcyBiZWVuIGRvbmUsIHdvbmRlciBpZiBk ZWZhdWx0IG9wdGlvbnMgKExBQ1Agc3BlZWQ/KSBoYXZlIGNoYW5nZWQ/PGJyPgo8YnI+Ckpvbjxi cj48YnI+PGRpdiBjbGFzcz0iZ21haWxfcXVvdGUiPk9uIDMwIERlY2VtYmVyIDIwMTUgMDk6NDQ6 MDIgR01UKzAwOjAwLCBEYW4gS2VuaWdzYmVyZyAmbHQ7ZGFua2VuQHJlZGhhdC5jb20mZ3Q7IHdy b3RlOjxibG9ja3F1b3RlIGNsYXNzPSJnbWFpbF9xdW90ZSIgc3R5bGU9Im1hcmdpbjogMHB0IDBw dCAwcHQgMC44ZXg7IGJvcmRlci1sZWZ0OiAxcHggc29saWQgcmdiKDIwNCwgMjA0LCAyMDQpOyBw YWRkaW5nLWxlZnQ6IDFleDsiPgo8cHJlIGNsYXNzPSJrOW1haWwiPk9uIFR1ZSwgRGVjIDI5LCAy MDE1IGF0IDA5OjU3OjA3UE0gKzAwMDAsIEpvbiBBcmNoZXIgd3JvdGU6PGJyIC8+PGJsb2NrcXVv dGUgY2xhc3M9ImdtYWlsX3F1b3RlIiBzdHlsZT0ibWFyZ2luOiAwcHQgMHB0IDFleCAwLjhleDsg Ym9yZGVyLWxlZnQ6IDFweCBzb2xpZCAjNzI5ZmNmOyBwYWRkaW5nLWxlZnQ6IDFleDsiPiBIaSBT dGVmYW5vLDxiciAvPiA8YnIgLz4gSXQncyBkZWZpbml0ZWx5IG5vdCB0aGUgc3dpdGNoLCBpdCBz ZWVtcyB0byBiZSB0aGUgbGF0ZXN0IGtlcm5lbCBwYWNrYWdlPGJyIC8+IChrZXJuZWwtMy4xMC4w LTMyNy4zLjEuZWw3Lng4Nl82NCkgd2hpY2ggc3RvcHMgYm9uZGluZyB3b3JraW5nIGNvcnJlY3Rs eSw8YnIgLz4gcmV2ZXJ0aW5nIGJhY2sgdG8gdGhlIHByZXZpb3VzIGtlcm5lbCBicmluZ3MgdGhl IG5ldHdvcmsgdXAgaW4gODAyLjNhZCBtb2RlPGJyIC8+ICg0KS48YnIgLz4gPGJyIC8+IEkga25v dywgZnJvbSByZWFkaW5nIHRoZSByZWxlYXNlIG5vdGVzIG9mIDcuMiwgdGhhdCB0aGVyZSB3ZXJl IHNvbWUgY2hhbmdlczxiciAvPiB0byB0aGUgYm9uZGluZyBiaXRzIGluIHRoZSBrZXJuZWwgc28g aSdtIGd1ZXNzaW5nIG1heWJlIHNvbWUgZGVmYXVsdHMgaGF2ZTxiciAvPiBjaGFuZ2VkLjxiciAv PiA8YnIgLz4gSSdsbCBrZWVwIGRpZ2dpbmcgYW5kIHBvc3QgYmFjayBhcyBzb29uIGFzIGkgaGF2 ZSBzb21ldGhpbmcuPGJyIC8+IDxiciAvPiBKb248YnIgLz4gPGJyIC8+IE9uIDI5LzEyLzE1IDE5 OjU1LCBTdGVmYW5vIERhbnppIHdyb3RlOjxiciAvPjxibG9ja3F1b3RlIGNsYXNzPSJnbWFpbF9x dW90ZSIgc3R5bGU9Im1hcmdpbjogMHB0IDBwdCAxZXggMC44ZXg7IGJvcmRlci1sZWZ0OiAxcHgg c29saWQgI2FkN2ZhODsgcGFkZGluZy1sZWZ0OiAxZXg7Ij5IaSEgSSBkaWRuJ3Qgc29sdmUgeWV0 LiBJJ20gc3RpbGwgdXNpbmcgbW9kZSAyIG9uIGJvbmQgaW50ZXJmYWMKIGUuCldoYXQnczxiciAv PnlvdXIgc3dpdGNoIG1vZGVsIGFuZCBmaXJtd2FyZSB2ZXJzaW9uPzxiciAvPjwvYmxvY2txdW90 ZT48L2Jsb2NrcXVvdGU+PGJyIC8+SGkgSm9uIGFuZCBTdGVmYW5vLDxiciAvPjxiciAvPldlJ3Zl IGJlZW4gdGVzdGluZyBib25kIG1vZGUgNCB3aXRoIChhbiBlYXJsaWVyKTxiciAvPmtlcm5lbC0z LjEwLjAtMzI3LmVsNy54ODZfNjQgYW5kIGV4cGVyaWVuY2VkIG5vIHN1Y2ggYmVoYXZpb3VyLjxi ciAvPjxiciAvPkhvd2V2ZXIsIHRvIGJldHRlciBpZGVudGlmeSB0aGUgc3VzcGVjdGVkIGtlcm5l bCBidWcsIGNvdWxkIHlvdSBwcm92aWRlPGJyIC8+bW9yZSBpbmZvcm1hdGlvbiByZWdhcmRpbmcg eW91ciBuZXR3b3JrIGNvbm5lY3Rpdml0eT88YnIgLz48YnIgLz5XaGF0IGlzIHRoZSBtYWtlIG9m IHlvdXIgTklDcz8gV2hpY2ggZHJpdmVyIGRvIHlvdSB1c2U/PGJyIC8+PGJyIC8+RG8geW91IHNl dCBzcGVjaWFsIGV0aHRvb2wgb3B0cyAoTFJPIHdpdGggYnJpZGdlIHdhcyBicm9rZW4gaW4gNy4y LjA8YnIgLz5rZXJuZWwgaWYgSSBhbSBub3QgbWlzdGFrZW4pPzxiciAvPjxiciAvPllvdSBoYXZl IHRoZSBvdmlydG1nbXQgYnJpZGdlIG9uIHRvcCBvZiB5b3VyIGJvbmQsIHJpZ2h0PzxiciAvPjxi ciAvPkNhbiB5b3Ugc2hhcmUgeW91ciBpZmNmZyo/PGJyIC8+PGJyIC8+PC9wcmU+PC9ibG9ja3F1 b3RlPjwvZGl2Pjxicj4KLS0gPGJyPgpTZW50IGZyb20gbXkgQW5kcm9pZCBkZXZpY2Ugd2l0aCBL LTkgTWFpbC4gUGxlYXNlIGV4Y3VzZSBteSBicmV2aXR5LjwvYm9keT48L2h0bWw+Ci0tLS0tLTZG NTQyV1gxOFZMUFhFSlRTQU84T0Y5RFVQWlc1Ti0tCgo= --===============6817363720204117397==--