One interesting thing I noticed today after looking at the logs from the 'ovn-controller' on the oVIRT nodes, is node1 is logging like crazy, the file is over 1.2GB in size already, seems to be looping like crazy? What should i check to see why it is logging and maxing CPU like this? 

2016-12-20T21:16:17.517Z|91605|binding|INFO|Changing chassis for lport d2e19c41-f8ff-49d2-bd8e-01d833e59a59 from 252778b5-bc63-486d-8d82-c3e4350ee650 to c0dc0909-4274-400d-826e-82348fee47e9.
2016-12-20T21:16:17.521Z|91606|poll_loop|INFO|Dropped 5666 log messages in last 6 seconds (most recently, 0 seconds ago) due to excessive rate
2016-12-20T21:16:17.521Z|91607|poll_loop|INFO|wakeup due to [POLLIN] on fd 12 (172.20.192.73:54710<->172.20.192.77:6642) at lib/stream-fd.c:155 (94% CPU usage)
2016-12-20T21:16:17.521Z|91608|binding|INFO|Changing chassis for lport d2e19c41-f8ff-49d2-bd8e-01d833e59a59 from 252778b5-bc63-486d-8d82-c3e4350ee650 to c0dc0909-4274-400d-826e-82348fee47e9.
2016-12-20T21:16:17.524Z|91609|binding|INFO|Changing chassis for lport d2e19c41-f8ff-49d2-bd8e-01d833e59a59 from 252778b5-bc63-486d-8d82-c3e4350ee650 to c0dc0909-4274-400d-826e-82348fee47e9.
2016-12-20T21:16:17.527Z|91610|binding|INFO|Changing chassis for lport d2e19c41-f8ff-49d2-bd8e-01d833e59a59 from 252778b5-bc63-486d-8d82-c3e4350ee650 to c0dc0909-4274-400d-826e-82348fee47e9.
2016-12-20T21:16:17.531Z|91611|binding|INFO|Changing chassis for lport d2e19c41-f8ff-49d2-bd8e-01d833e59a59 from 252778b5-bc63-486d-8d82-c3e4350ee650 to c0dc0909-4274-400d-826e-82348fee47e9.
2016-12-20T21:16:17.534Z|91612|binding|INFO|Changing chassis for lport d2e19c41-f8ff-49d2-bd8e-01d833e59a59 from 252778b5-bc63-486d-8d82-c3e4350ee650 to c0dc0909-4274-400d-826e-82348fee47e9.
2016-12-20T21:16:17.538Z|91613|binding|INFO|Changing chassis for lport d2e19c41-f8ff-49d2-bd8e-01d833e59a59 from 252778b5-bc63-486d-8d82-c3e4350ee650 to c0dc0909-4274-400d-826e-82348fee47e9.
2016-12-20T21:16:17.541Z|91614|binding|INFO|Changing chassis for lport d2e19c41-f8ff-49d2-bd8e-01d833e59a59 from 252778b5-bc63-486d-8d82-c3e4350ee650 to c0dc0909-4274-400d-826e-82348fee47e9.
2016-12-20T21:16:17.545Z|91615|binding|INFO|Changing chassis for lport d2e19c41-f8ff-49d2-bd8e-01d833e59a59 from 252778b5-bc63-486d-8d82-c3e4350ee650 to c0dc0909-4274-400d-826e-82348fee47e9.
2016-12-20T21:16:17.548Z|91616|binding|INFO|Changing chassis for lport d2e19c41-f8ff-49d2-bd8e-01d833e59a59 from 252778b5-bc63-486d-8d82-c3e4350ee650 to c0dc0909-4274-400d-826e-82348fee47e9.
2016-12-20T21:16:17.552Z|91617|binding|INFO|Changing chassis for lport d2e19c41-f8ff-49d2-bd8e-01d833e59a59 from 252778b5-bc63-486d-8d82-c3e4350ee650 to c0dc0909-4274-400d-826e-82348fee47e9.
2016-12-20T21:16:17.555Z|91618|binding|INFO|Changing chassis for lport d2e19c41-f8ff-49d2-bd8e-01d833e59a59 from 252778b5-bc63-486d-8d82-c3e4350ee650 to c0dc0909-4274-400d-826e-82348fee47e9.
2016-12-20T21:16:17.559Z|91619|binding|INFO|Changing chassis for lport d2e19c41-f8ff-49d2-bd8e-01d833e59a59 from 252778b5-bc63-486d-8d82-c3e4350ee650 to c0dc0909-4274-400d-826e-82348fee47e9.
2016-12-20T21:16:17.562Z|91620|binding|INFO|Changing chassis for lport d2e19c41-f8ff-49d2-bd8e-01d833e59a59 from 252778b5-bc63-486d-8d82-c3e4350ee650 to c0dc0909-4274-400d-826e-82348fee47e9.
2016-12-20T21:16:17.566Z|91621|binding|INFO|Changing chassis for lport d2e19c41-f8ff-49d2-bd8e-01d833e59a59 from 252778b5-bc63-486d-8d82-c3e4350ee650 to c0dc0909-4274-400d-826e-82348fee47e9.
2016-12-20T21:16:17.569Z|91622|binding|INFO|Changing chassis for lport d2e19c41-f8ff-49d2-bd8e-01d833e59a59 from 252778b5-bc63-486d-8d82-c3e4350ee650 to c0dc0909-4274-400d-826e-82348fee47e9.
2016-12-20T21:16:17.573Z|91623|binding|INFO|Changing chassis for lport d2e19c41-f8ff-49d2-bd8e-01d833e59a59 from 252778b5-bc63-486d-8d82-c3e4350ee650 to c0dc0909-4274-400d-826e-82348fee47e9.
2016-12-20T21:16:17.576Z|91624|binding|INFO|Changing chassis for lport d2e19c41-f8ff-49d2-bd8e-01d833e59a59 from 252778b5-bc63-486d-8d82-c3e4350ee650 to c0dc0909-4274-400d-826e-82348fee47e9.
2016-12-20T21:16:17.580Z|91625|binding|INFO|Changing chassis for lport d2e19c41-f8ff-49d2-bd8e-01d833e59a59 from 252778b5-bc63-486d-8d82-c3e4350ee650 to c0dc0909-4274-400d-826e-82348fee47e9.
2016-12-20T21:16:17.583Z|91626|binding|INFO|Changing chassis for lport d2e19c41-f8ff-49d2-bd8e-01d833e59a59 from 252778b5-bc63-486d-8d82-c3e4350ee650 to c0dc0909-4274-400d-826e-82348fee47e9.
2016-12-20T21:16:17.587Z|91627|binding|INFO|Changing chassis for lport d2e19c41-f8ff-49d2-bd8e-01d833e59a59 from 252778b5-bc63-486d-8d82-c3e4350ee650 to c0dc0909-4274-400d-826e-82348fee47e9.
2016-12-20T21:16:17.590Z|91628|binding|INFO|Changing chassis for lport d2e19c41-f8ff-49d2-bd8e-01d833e59a59 from 252778b5-bc63-486d-8d82-c3e4350ee650 to c0dc0909-4274-400d-826e-82348fee47e9.
2016-12-20T21:16:17.593Z|91629|binding|INFO|Changing chassis for lport d2e19c41-f8ff-49d2-bd8e-01d833e59a59 from 252778b5-bc63-486d-8d82-c3e4350ee650 to c0dc0909-4274-400d-826e-82348fee47e9.
2016-12-20T21:16:17.597Z|91630|binding|INFO|Changing chassis for lport d2e19c41-f8ff-49d2-bd8e-01d833e59a59 from 252778b5-bc63-486d-8d82-c3e4350ee650 to c0dc0909-4274-400d-826e-82348fee47e9.
2016-12-20T21:16:17.600Z|91631|binding|INFO|Changing chassis for lport d2e19c41-f8ff-49d2-bd8e-01d833e59a59 from 252778b5-bc63-486d-8d82-c3e4350ee650 to c0dc0909-4274-400d-826e-82348fee47e9.
2016-12-20T21:16:17.604Z|91632|binding|INFO|Changing chassis for lport d2e19c41-f8ff-49d2-bd8e-01d833e59a59 from 252778b5-bc63-486d-8d82-c3e4350ee650 to c0dc0909-4274-400d-826e-82348fee47e9.
2016-12-20T21:16:17.607Z|91633|binding|INFO|Changing chassis for lport d2e19c41-f8ff-49d2-bd8e-01d833e59a59 from 252778b5-bc63-486d-8d82-c3e4350ee650 to c0dc0909-4274-400d-826e-82348fee47e9.
2016-12-20T21:16:17.611Z|91634|binding|INFO|Changing chassis for lport d2e19c41-f8ff-49d2-bd8e-01d833e59a59 from 252778b5-bc63-486d-8d82-c3e4350ee650 to c0dc0909-4274-400d-826e-82348fee47e9.

top - 21:17:55 up 27 days,  3:49,  1 user,  load average: 2.33, 2.12, 2.06
Tasks: 576 total,   2 running, 574 sleeping,   0 stopped,   0 zombie
%Cpu(s):  1.4 us,  0.3 sy,  0.0 ni, 98.2 id,  0.0 wa,  0.0 hi,  0.0 si,  0.0 st
KiB Mem : 13189636+total, 72442312 free, 41483504 used, 17970552 buff/cache
KiB Swap:  4194300 total,  4193856 free,      444 used. 89843568 avail Mem

  PID USER      PR  NI    VIRT    RES    SHR S  %CPU %MEM     TIME+ COMMAND
21818 qemu      20   0 33.027g 0.031t  12176 S  94.1 25.5 465:24.64 qemu-kvm
29540 root      10 -10   51096   3844   1176 R  88.2  0.0   6:28.52 ovn-controller
 1571 root      20   0    4376    672    520 S  29.4  0.0   6668:43 rngd
 9282 qemu      20   0 1744736 703420  12108 S   5.9  0.5  35:34.51 qemu-kvm
30690 root      20   0  158228   2564   1496 R   5.9  0.0   0:00.02 top


On Tue, Dec 20, 2016 at 1:47 AM, Numan Siddique <nusiddiq@redhat.com> wrote:
+Russel

On Tue, Dec 20, 2016 at 10:30 AM, Devin Acosta <devin@pabstatencio.com> wrote:

Marcin,Numan,Lance:

I really appreciate all the assistance that you have given me thus far. I wanted to circle back on this topic, even though I sense I know what the answer will be. ;) My Networking team keeps insisting that they want to control DHCP from their side however yet still be able to create virtual Layer 2 networks within oVirt. I understand that it sounds like OVN was never meant for this kind of configuration.

It is not necessary to use native DHCP feature of OVN. You can always disable it. Infact it is disabled, unless dhcp options are added to each logical port. So it's fine to use your own DHCP server. I don't think MTU discovery is supported in OVN.

 
In the setup that we currently have going for the most part Linux boxes are working if we set the MTU to be lower around 1400, however with the Windows boxes we are getting very strange behavior, sometimes we have to set the MTU low to as 1000, but then the next day 1200 works. We aren't fully sure if it's just some strange windows issue with the driver or what. 

One question that was asked of me was if OVS/OVN supports like PMTUD, MTU Protocol discovery? Also I'm suspecting if was to configure oVIRT to use Neutron using OVS I would encounter the same issues I have now with the MTU if they are trying to do DHCP from a virtual appliance on the network? 

Any other comments or suggestions that you can provide on this?

Thanks again.

On Mon, Dec 12, 2016 at 7:14 AM, Numan Siddique <nusiddiq@redhat.com> wrote:
Hi Devin,

Below is one example of creating dhcp options and associating them with logical ports using ovn-nbctl commands. Please see the links shared by Lance for more details.

Lets say you have a network with cidr - 10.0.0.0/24.

One example would be

$ovn-nbctl dhcp-options-create 10.0.0.0/24

Run the command ovn-nbctl dhcp-options-list and store the uuid of it in any variable (DHCP_UUID)
Now create the dhcp options for this DHCP_UUID just created.
There are 4 dhcp options which needs to be defined (they are mandatory)
  - server_id
  - server_mac
     -  router
     - lease_time

since you want to add mtu option as well, you can add the dhcp options as 

$ovn-nbctl dhcp-options-set-options $DHCP_UUID  server_id=10.0.0.1 server_mac=00:00:00:00:00:10 router=10.0.0.1 lease_time=3600 mtu=1400

The above is just an example. You can see the dhcp options set by running
$ovn-nbctl dhcp-options-get-options $DHCP_UUID


​The dhcp options defined here have no value unless you associate these with the logical switch port.

$ovn-nbctl lsp-set-dhcpv4-options $LPORT_NAME $DHCP_UUID​

Suppose if you have 3 logical ports - lp1, lp2 and lp3, you can associate it as
$ovn-nbctl lsp-set-dhcpv4-options lp1 $DHCP_UUID​
$ovn-nbctl lsp-set-dhcpv4-options lp2 $DHCP_UUID​
$ovn-nbctl lsp-set-dhcpv4-options lp3 $DHCP_UUID​


Please let us know if you have any more questions.

You can refer to this blog to get some more inner details of how native DHCP is supported in ovn - https://numansiddiqueblog.wordpress.com/2016/08/09/native-dhcp-support-in-ovn/

Thanks
Numan


On Mon, Dec 12, 2016 at 7:28 PM, Lance Richardson <lrichard@redhat.com> wrote:
Hi Devin,

This blog posting does a good job of explaining how to configure OVN
DHCP support:

    http://blog.spinhirne.com/2016/09/an-introduction-to-ovn-routing.html

The ovn-nb man page lists the DHCP options that can be provided, including
mtu:

    http://openvswitch.org/support/dist-docs/
​​
ovn-
​​
nb.5.html


And the ovn-nbctl man page has details about the command-line interface
for setting DHCP options:

    http://openvswitch.org/support/dist-docs/ovn-nbctl.8.html

I have very little experience using OVN's DHCP support, I've copied Numan
in case I've left anything out.

    Lance
----- Original Message -----
> From: "Marcin Mirecki" <mmirecki@redhat.com>
> To: "Devin Acosta" <devin@pabstatencio.com>
> Cc: "users" <Users@ovirt.org>, "Lance Richardson" <lrichard@redhat.com>
> Sent: Monday, December 12, 2016 4:35:51 AM
> Subject: Re: oVirt / OVN / MTU
>
> Devin,
>
> oVirt does not currently support changing external network mtu from within
> ovirt (it rather relies on the provider handling this internally).
>
> If you are using OVN DHCP (have subnets defined for a network), you can
> modify the OVN DHCP options directly in the OVN database.
> I have never actually tested this myself, but looking at the OVN
> documentation, it should do the job on the ports.
>
> The standard OVN way to do so is to use the "ovn-vsctl set DHCP_Options ..."
> command.
> (Unfortunately as I am trying it now it tells me that modifying DHCP_Options
> is not supported)
> Alternatively, you can use the OVS python API (let me know if you need any
> help on this).
>
> Lance,
> Would changing the dhcp:options:mtu suffice?
> Could you please comment on how to modify the DHCP MTU using the OVN cmd
> line?
>
> Thanks,
> Marcin
>
>
>
> ----- Original Message -----
> > From: "Devin Acosta" <devin@pabstatencio.com>
> > To: "Marcin Mirecki" <mmirecki@redhat.com>, "users" <Users@ovirt.org>,
> > "Lance Richardson" <lrichard@redhat.com>
> > Sent: Monday, December 12, 2016 1:20:59 AM
> > Subject: Fwd: oVirt / OVN / MTU
> >
> > Marcin / Lance,
> >
> > Not sure if the list was working correctly, I couldn't see that my message
> > below made it to the list. If I need to change the MTU settings for OVN /
> > OpenVSwitch to something lower than 1500, what is the best way to do this?
> > We noticed that some instances (ie: Windows 2012R2) are having issues with
> > the default MTU of 1500, I think there is an issue at the upper layers, and
> > we can get it to work if we manually set the MTU on the instance to say
> > 1400. Is there an easy way to do this so that any VM's that come up
> > automatically get MTU of 1400?
> >
> > Devin
> >
> > ---------- Forwarded message ----------
> > From: Devin Acosta <devin@pabstatencio.com>
> > Date: Fri, Dec 9, 2016 at 2:02 PM
> > Subject: oVirt / OVN / MTU
> > To: users <Users@ovirt.org>
> >
> >
> >
> > We are running oVirt 4.0.5 and we have OVN working to provide a Virtual
> > Layer 2 network. We are noticing that because the OVN is using Geneve and
> > between all the firewalls and networks it crosses we are running into an
> > MTU issue. What is the best suggested way to lower say the entire OVN
> > network to say MTU of 1400, and also allow for fragmenting packets?
> >
> >
> > --
> >
> > Devin Acosta
> > Red Hat Certified Architect, LinuxStack
> > 602-354-1220 || devin@linuxguru.co
> >
> >
> >
> > --
> >
> > Devin Acosta
> > Red Hat Certified Architect, LinuxStack
> > 602-354-1220 || devin@linuxguru.co
> >
>




--

Devin Acosta
Red Hat Certified Architect, LinuxStack 




--

Devin Acosta
Red Hat Certified Architect, LinuxStack 
602-354-1220 || devin@linuxguru.co