This is a multi-part message in MIME format. ------------MIME-3653757893-462597306-delim Content-Type: text/plain; charset="windows-1252" Content-Transfer-Encoding: quoted-printable On 07/29/2015 12=3A12 PM=2C NUNIN Roberto wrote=3A =3E=3E -----Messaggio originale----- =3E=3E Da=3A Michael S=2E Tsirkin =5Bmailto=3Amst=40redhat=2Ecom=5D =3E=3E Inviato=3A mercoled=EC=20=329 luglio 2015 12=3A03 =3E=3E A=3A NUNIN Roberto =3E=3E Cc=3A Fabian Deutsch=3B users=40ovirt=2Eorg =3E=3E Oggetto=3A Re=3A R=3A =5Bovirt-users=5D R=3A R=3A R=3A R=3A R=3A R= =3A PXE boot of a VM on vdsm don=27t =3E=3E read DHCP offer =3E=3E =3E=3E On Wed=2C Jul 29=2C 2015 at 12=3A00=3A38PM +0200=2C NUNIN Roberto wr= ote=3A =3E=3E=3E=3E -----Messaggio originale----- =3E=3E=3E=3E Da=3A users-bounces=40ovirt=2Eorg =5Bmailto=3Ausers-bounces=40= ovirt=2Eorg=5D Per conto =3E=3E di =3E=3E=3E=3E Michael S=2E Tsirkin =3E=3E=3E=3E Inviato=3A gioved=EC=20=39 luglio 2015 15=3A15 =3E=3E=3E=3E A=3A Fabian Deutsch =3E=3E=3E=3E Cc=3A users=40ovirt=2Eorg =3E=3E=3E=3E Oggetto=3A Re=3A =5Bovirt-users=5D R=3A R=3A R=3A R=3A R=3A R= =3A PXE boot of a VM on vdsm don=27t =3E=3E read =3E=3E=3E=3E DHCP offer =3E=3E=3E=3E =3E=3E=3E=3E On Thu=2C Jul 09=2C 2015 at 08=3A57=3A50AM -0400=2C Fabian Deu= tsch wrote=3A =3E=3E=3E=3E=3E ----- Original Message ----- =3E=3E=3E=3E=3E=3E On Wed=2C Jul 08=2C 2015 at 09=3A11=3A42AM +0300=2C Mich= ael S=2E Tsirkin wrote=3A =3E=3E=3E=3E=3E=3E=3E On Tue=2C Jul 07=2C 2015 at 05=3A13=3A28PM +0100=2C D= an Kenigsberg wrote=3A =3E=3E=3E=3E=3E=3E=3E=3E On Tue=2C Jul 07=2C 2015 at 10=3A14=3A54AM +0200= =2C NUNIN Roberto wrote=3A =3E=3E=3E=3E=3E=3E=3E=3E=3E=3E On Mon=2C Jul 06=2C 2015 at 10=3A33=3A59AM += 0200=2C NUNIN Roberto =3E=3E wrote=3A =3E=3E=3E=3E=3E=3E=3E=3E=3E=3E=3E Hi Dan =3E=3E=3E=3E=3E=3E=3E=3E=3E=3E=3E =3E=3E=3E=3E=3E=3E=3E=3E=3E=3E=3E Sorry for question=3A what do you mean fo= r interface vnetxxxx =3F =3E=3E=3E=3E=3E=3E=3E=3E=3E=3E=3E Currently our path is =3A =3E=3E=3E=3E=3E=3E=3E=3E=3E=3E=3E eno1 - eno2 ---- bond0 ----- bond=2E3500= =28VLAN=29 ------ bridge ----- =3E=3E=3E=3E=3E=3E=3E=3E=3E=3E=3E vm=2E =3E=3E=3E=3E=3E=3E=3E=3E=3E=3E=3E =3E=3E=3E=3E=3E=3E=3E=3E=3E=3E=3E Which one of these =3F =3E=3E=3E=3E=3E=3E=3E=3E=3E=3E=3E Moreover=2C reading Fabian statements abo= ut bonding limits=2C =3E=3E=3E=3E today I =3E=3E=3E=3E=3E=3E=3E=3E=3E=3E=3E can try =3E=3E=3E=3E=3E=3E=3E=3E=3E=3E to switch to a config without bonding=2E =3E=3E=3E=3E=3E=3E=3E=3E=3E=3E =3E=3E=3E=3E=3E=3E=3E=3E=3E=3E =22vm=22 is a complicated term=2E =3E=3E=3E=3E=3E=3E=3E=3E=3E=3E =3E=3E=3E=3E=3E=3E=3E=3E=3E=3E =60brctl show=60 would not show you a =22vm= =22 connected to a bridge=2E =3E=3E=3E=3E When =3E=3E=3E=3E=3E=3E=3E=3E=3E=3E you =3E=3E=3E=3E=3E=3E=3E=3E=3E=3E WOULD see is a vnet888 tap device=2E The=20= =22other side=22 of this =3E=3E device =3E=3E=3E=3E is =3E=3E=3E=3E=3E=3E=3E=3E=3E=3E held by qemu=2C which implement the VM=2E =3E=3E=3E=3E=3E=3E=3E=3E=3E Ok=2C understood and found it=2C vnet2 =3E=3E=3E=3E=3E=3E=3E=3E=3E =3E=3E=3E=3E=3E=3E=3E=3E=3E=3E I=27m asking if the dhcp offer has reached t= hat tap device=2E =3E=3E=3E=3E=3E=3E=3E=3E=3E No=2C the DHCP offer packet do not reach the vn= et2 interface=2C I can =3E=3E see =3E=3E=3E=3E=3E=3E=3E=3E=3E only DHCP DISCOVER=2E =3E=3E=3E=3E=3E=3E=3E=3E Ok=2C so it seems that we have a problem in the ho= st bridging=2E =3E=3E=3E=3E=3E=3E=3E=3E =3E=3E=3E=3E=3E=3E=3E=3E Is it the latest kernel-3=2E10=2E0-229=2E7=2E2=2Ee= l7=2Ex86=5F64 =3F =3E=3E=3E=3E=3E=3E=3E=3E =3E=3E=3E=3E=3E=3E=3E=3E Michael=2C a DHCP DISCOVER is sent out of a just-b= ooted guest=2C and =3E=3E=3E=3E OFFER =3E=3E=3E=3E=3E=3E=3E=3E returns to the bridge=2C but is not propagated to= the tap device=2E =3E=3E=3E=3E=3E=3E=3E=3E Can you suggest how to debug this further=3F =3E=3E=3E=3E=3E=3E=3E Dump packets including the ethernet headers=2E =3E=3E=3E=3E=3E=3E=3E Likely something interfered with them so the eth addr= ess is wrong=2E =3E=3E=3E=3E=3E=3E=3E =3E=3E=3E=3E=3E=3E=3E Since bonding does this sometimes=2C this is the most= likely culprit=2E =3E=3E=3E=3E=3E=3E We=27ve ruled this out already - Roberto reproduces the= issue without a =3E=3E=3E=3E=3E=3E bond=2E =3E=3E=3E=3E=3E To me this looks like either a regression in the host side= bridging=2E But =3E=3E otoh it =3E=3E=3E=3E doesn=27t look =3E=3E=3E=3E=3E like it=27s happening always=2C because otherwise I=27d exp= ect more noise =3E=3E around =3E=3E=3E=3E this issue=2E =3E=3E=3E=3E=3E - fabian =3E=3E=3E=3E Hard to say=2E E=2Eg=2E forwarding delay would do this for a w= hile=2E =3E=3E=3E=3E If eth address of the packets is okay=2C poke at the fbd=2C ma= ybe there=27s =3E=3E=3E=3E something wrong there=2E Maybe stp is detecting a loop - try c= hecking that=2E =3E=3E=3E Someone is checking this =3F =3E=3E=3E In tested config SPT was off=2E =3E=3E Then maybe you have a loop =3A=29 =3E That was already checked=2C the MAC was unique in the VLAN=2E =3E =3E RN =3E Did you also try a reboot of the VM=3F We have the same issue with foreman= and both Libvirt and oVirt=2E On second boot PXE boots properly from DHCP= =2E Haven=27t had the time to investigate yet so we=27re using mostly image based provisioning on oVirt at the moment=2E Met vriendelijke groet=2C With kind regards=2C Jorick Astrego Netbulae Virtualization Experts=20 ---------------- =09Tel=3A 053 20 30 270 =09info=40netbulae=2Eeu =09Staalsteden 4-3A =09KvK= 08198180 =09Fax=3A 053 20 30 271 =09www=2Enetbulae=2Eeu =097547 TA Enschede =09BTW= NL821234584B01 ---------------- ------------MIME-3653757893-462597306-delim Content-Type: text/html; charset="utf-8" Content-Transfer-Encoding: quoted-printable =3Chtml=3E =3Cbody=3E <br> <br> On 07/29/2015 12:12 PM, NUNIN Roberto wrote: <br> <font color=3D"#000000">>> -----Messaggio originale----- </font><= br> <font color=3D"#000000">>> Da: Michael S. Tsirkin [mailto:mst@<a href= =3D"mailto:redhat.com">redhat.com</a>] </font><br> <font color=3D"#000000">>> Inviato: mercoledì 29 luglio 2015 12:= 03 </font><br> <font color=3D"#000000">>> A: NUNIN Roberto </font><br> <font color=3D"#000000">>> Cc: Fabian Deutsch; users@<a href=3D"mailt= o:ovirt.org">ovirt.org</a> </font><br> <font color=3D"#000000">>> Oggetto: Re: R: [ovirt-users] R: R: R: R: = R: R: PXE boot of a VM on vdsm don't </font><br> <font color=3D"#000000">>> read DHCP offer </font><br> <font color=3D"#000000">>> </font><br> <font color=3D"#000000">>> On Wed, Jul 29, 2015 at 12:00:38PM +0200, = NUNIN Roberto wrote: </font><br> <font color=3D"#000000">>>>> -----Messaggio originale----- = </font><br> <font color=3D"#000000">>>>> Da: users-bounces@<a href=3D"mailt= o:ovirt.org">ovirt.org</a> [mailto:users-bounces@<a href=3D"mailto:ovi= rt.org">ovirt.org</a>] Per conto </font><br> <font color=3D"#000000">>> di </font><br> <font color=3D"#000000">>>>> Michael S. Tsirkin </font><br> <font color=3D"#000000">>>>> Inviato: giovedì 9 luglio 201= 5 15:15 </font><br> <font color=3D"#000000">>>>> A: Fabian Deutsch </font><br> <font color=3D"#000000">>>>> Cc: users@<a href=3D"mailto:ovirt.= org">ovirt.org</a> </font><br> <font color=3D"#000000">>>>> Oggetto: Re: [ovirt-users] R: R: R= : R: R: R: PXE boot of a VM on vdsm don't </font><br> <font color=3D"#000000">>> read </font><br> <font color=3D"#000000">>>>> DHCP offer </font><br> <font color=3D"#000000">>>>> </font><br> <font color=3D"#000000">>>>> On Thu, Jul 09, 2015 at 08:57:50AM= -0400, Fabian Deutsch wrote: </font><br> <font color=3D"#000000">>>>>> ----- Original Message -----&#= 13;</font><br> <font color=3D"#000000">>>>>>> On Wed, Jul 08, 2015 at 09= :11:42AM +0300, Michael S. Tsirkin wrote: </font><br> <font color=3D"#000000">>>>>>>> On Tue, Jul 07, 2015 a= t 05:13:28PM +0100, Dan Kenigsberg wrote: </font><br> <font color=3D"#000000">>>>>>>>> On Tue, Jul 07, 20= 15 at 10:14:54AM +0200, NUNIN Roberto wrote: </font><br> <font color=3D"#000000">>>>>>>>>>> On Mon, Ju= l 06, 2015 at 10:33:59AM +0200, NUNIN Roberto </font><br> <font color=3D"#000000">>> wrote: </font><br> <font color=3D"#000000">>>>>>>>>>>> Hi Dan= </font><br> <font color=3D"#000000">>>>>>>>>>>> </= font><br> <font color=3D"#000000">>>>>>>>>>>> Sorry = for question: what do you mean for interface vnetxxxx ? </font><br> <font color=3D"#000000">>>>>>>>>>>> Curren= tly our path is : </font><br> <font color=3D"#000000">>>>>>>>>>>> eno1 -= eno2 ---- bond0 ----- bond.3500 (VLAN) ------ bridge ----- </fon= t><br> <font color=3D"#000000">>>>>>>>>>>> vm.= 3;</font><br> <font color=3D"#000000">>>>>>>>>>>> </= font><br> <font color=3D"#000000">>>>>>>>>>>> Which = one of these ? </font><br> <font color=3D"#000000">>>>>>>>>>>> Moreov= er, reading Fabian statements about bonding limits, </font><br> <font color=3D"#000000">>>>> today I </font><br> <font color=3D"#000000">>>>>>>>>>>> can tr= y </font><br> <font color=3D"#000000">>>>>>>>>>> to switch = to a config without bonding. </font><br> <font color=3D"#000000">>>>>>>>>>> </font=
<br> <font color=3D"#000000">>>>>>>>>>> "vm&q= uot; is a complicated term. </font><br> <font color=3D"#000000">>>>>>>>>>> </font= <br> <font color=3D"#000000">>>>>>>>>>> `brctl sho= w` would not show you a "vm" connected to a bridge. </font><b= r> <font color=3D"#000000">>>>> When </font><br> <font color=3D"#000000">>>>>>>>>>> you </= font><br> <font color=3D"#000000">>>>>>>>>>> WOULD see = is a vnet888 tap device. The "other side" of this </font><br> <font color=3D"#000000">>> device </font><br> <font color=3D"#000000">>>>> is </font><br> <font color=3D"#000000">>>>>>>>>>> held by qe= mu, which implement the VM. </font><br> <font color=3D"#000000">>>>>>>>>> Ok, understood= and found it, vnet2 </font><br> <font color=3D"#000000">>>>>>>>>> </font><br=
<font color=3D"#000000">>>>>>>>>>> I'm asking= if the dhcp offer has reached that tap device. </font><br> <font color=3D"#000000">>>>>>>>>> No, the DHCP o= ffer packet do not reach the vnet2 interface, I can </font><br> <font color=3D"#000000">>> see </font><br> <font color=3D"#000000">>>>>>>>>> only DHCP DISC= OVER. </font><br> <font color=3D"#000000">>>>>>>>> Ok, so it seems th= at we have a problem in the host bridging. </font><br> <font color=3D"#000000">>>>>>>>> </font><br> <font color=3D"#000000">>>>>>>>> Is it the latest k= ernel-3.10.0-229.7.2.el7.x86_64 ? </font><br> <font color=3D"#000000">>>>>>>>> </font><br> <font color=3D"#000000">>>>>>>>> Michael, a DHCP DI= SCOVER is sent out of a just-booted guest, and </font><br> <font color=3D"#000000">>>>> OFFER </font><br> <font color=3D"#000000">>>>>>>>> returns to the bri= dge, but is not propagated to the tap device. </font><br> <font color=3D"#000000">>>>>>>>> Can you suggest ho= w to debug this further? </font><br> <font color=3D"#000000">>>>>>>> Dump packets including= the ethernet headers. </font><br> <font color=3D"#000000">>>>>>>> Likely something inter= fered with them so the eth address is wrong. </font><br> <font color=3D"#000000">>>>>>>> </font><br> <font color=3D"#000000">>>>>>>> Since bonding does thi= s sometimes, this is the most likely culprit. </font><br> <font color=3D"#000000">>>>>>> We've ruled this out alrea= dy - Roberto reproduces the issue without a </font><br> <font color=3D"#000000">>>>>>> bond. </font><br> <font color=3D"#000000">>>>>> To me this looks like either a= regression in the host side bridging. But </font><br> <font color=3D"#000000">>> otoh it </font><br> <font color=3D"#000000">>>>> doesn't look </font><br> <font color=3D"#000000">>>>>> like it's happening always, be= cause otherwise I'd expect more noise </font><br> <font color=3D"#000000">>> around </font><br> <font color=3D"#000000">>>>> this issue. </font><br> <font color=3D"#000000">>>>>> - fabian </font><br> <font color=3D"#000000">>>>> Hard to say. E.g. forwarding delay= would do this for a while. </font><br> <font color=3D"#000000">>>>> If eth address of the packets is o= kay, poke at the fbd, maybe there's </font><br> <font color=3D"#000000">>>>> something wrong there. Maybe stp i= s detecting a loop - try checking that. </font><br> <font color=3D"#000000">>>> Someone is checking this ? </font>= <br> <font color=3D"#000000">>>> In tested config SPT was off. </fo= nt><br> <font color=3D"#000000">>> Then maybe you have a loop :) </font><= br> <font color=3D"#000000">> That was already checked, the MAC was unique i= n the VLAN. </font><br> <font color=3D"#000000">> </font><br> <font color=3D"#000000">> RN </font><br> <font color=3D"#000000">> </font><br> <br> Did you also try a reboot of the VM? We have the same issue with foreman= 3;<br> and both Libvirt and oVirt. On second boot PXE boots properly from DHCP.= 3;<br> <br> Haven't had the time to investigate yet so we're using mostly image <br=
based provisioning on oVirt at the moment. <br> <br> <br> = =3CBR /=3E =3CBR /=3E =3Cb style=3D=22color=3A=23604c78=22=3E=3C/b=3E=3Cbr=3E=3Cspan style=3D=22c= olor=3A=23604c78=3B=22=3E=3Cfont color=3D=22000000=22=3E=3Cspan style=3D=22= mso-fareast-language=3Aen-gb=3B=22 lang=3D=22NL=22=3EMet vriendelijke groet= =2C With kind regards=2C=3Cbr=3E=3Cbr=3E=3C/span=3EJorick Astrego=3C/font= =3E=3C/span=3E=3Cb style=3D=22color=3A=23604c78=22=3E=3Cbr=3E=3Cbr=3ENetbul= ae Virtualization Experts =3C/b=3E=3Cbr=3E=3Chr style=3D=22border=3Anone=3B= border-top=3A1px solid =23ccc=3B=22=3E=3Ctable style=3D=22width=3A 522px=22= =3E=3Ctbody=3E=3Ctr=3E=3Ctd style=3D=22width=3A 130px=3Bfont-size=3A 10px= =22=3ETel=3A 053 20 30 270=3C/td=3E =3Ctd style=3D=22width=3A 130px=3Bf= ont-size=3A 10px=22=3Einfo=40netbulae=2Eeu=3C/td=3E =3Ctd style=3D=22wid= th=3A 130px=3Bfont-size=3A 10px=22=3EStaalsteden 4-3A=3C/td=3E =3Ctd sty= le=3D=22width=3A 130px=3Bfont-size=3A 10px=22=3EKvK 08198180=3C/td=3E=3C/tr= =3E=3Ctr=3E =3Ctd style=3D=22width=3A 130px=3Bfont-size=3A 10px=22=3EFax= =3A 053 20 30 271=3C/td=3E =3Ctd style=3D=22width=3A 130px=3Bfont-size= =3A 10px=22=3Ewww=2Enetbulae=2Eeu=3C/td=3E =3Ctd style=3D=22width=3A 130= px=3Bfont-size=3A 10px=22=3E7547 TA Enschede=3C/td=3E =3Ctd style=3D=22w= idth=3A 130px=3Bfont-size=3A 10px=22=3EBTW NL821234584B01=3C/td=3E=3C/tr=3E= =3C/tbody=3E=3C/table=3E=3Cbr=3E=3Chr style=3D=22border=3Anone=3Bborder-top= =3A1px solid =23ccc=3B=22=3E=3CBR /=3E =3C/body=3E =3C/html=3E ------------MIME-3653757893-462597306-delim--