Chassis "c0dc0909-4274-400d-826e-82348fee47e9"
    hostname: "host1"
    Encap geneve
        ip: "172.20.192.73"
        options: {csum="true"}
    Port_Binding "f409d17f-38bc-4ee9-922b-63dbfc46cd91"
    Port_Binding "29dce6d5-77bd-4a3c-b077-bb002c4347d6"
    Port_Binding "e4f23f60-cc16-4399-a15b-fb05c09c4409"
    Port_Binding "a6d2008f-bb73-437b-bbac-b50487ea5257"
    Port_Binding "ee971db2-cdaf-4fa0-b83c-993371533af9"
    Port_Binding "f5aed0a0-03e0-46ec-92c6-07e5f03e9cc9"
    Port_Binding "4f1cf249-7fb6-4ff2-a7a9-a98bf45af6b8"
    Port_Binding "49725a3e-ef3f-474d-b84a-4975ce55e8fb"
    Port_Binding "bd7c66e9-81d5-4bfe-9331-203908048a6f"
    Port_Binding "301bf657-b6c4-459f-bd88-d21885190651"
    Port_Binding "0745689c-2c70-4b48-bda2-d0c1f8127133"
Chassis "252778b5-bc63-486d-8d82-c3e4350ee650"
    hostname: "host2"
    Encap geneve
        ip: "172.20.192.75"
        options: {csum="true"}
    Port_Binding "d2e19c41-f8ff-49d2-bd8e-01d833e59a59"
Chassis "45694909-7882-4d7a-948a-bdf34e6472cb"
    hostname: "host3"
    Encap geneve
        ip: "172.20.192.74"
        options: {csum="true"}
    Port_Binding "70443ae2-a04b-4d1f-90e4-f12e7e48fe82"
    Port_Binding "fcb6df47-32e6-4f7c-96fd-b598c33bff97"


I see on host #3

2016-12-20T21:35:01.280Z|173706735|binding|INFO|Changing chassis for lport d2e19c41-f8ff-49d2-bd8e-01d833e59a59 from c0dc0909-4274-400d-826e-82348fee47e9 to 252778b5-bc63-486d-8d82-c3e4350ee650.
2016-12-20T21:35:01.283Z|173706736|binding|INFO|Changing chassis for lport d2e19c41-f8ff-49d2-bd8e-01d833e59a59 from c0dc0909-4274-400d-826e-82348fee47e9 to 252778b5-bc63-486d-8d82-c3e4350ee650.
2016-12-20T21:35:01.286Z|173706737|binding|INFO|Changing chassis for lport d2e19c41-f8ff-49d2-bd8e-01d833e59a59 from c0dc0909-4274-400d-826e-82348fee47e9 to 252778b5-bc63-486d-8d82-c3e4350ee650.
2016-12-20T21:35:01.288Z|173706738|binding|INFO|Changing chassis for lport d2e19c41-f8ff-49d2-bd8e-01d833e59a59 from c0dc0909-4274-400d-826e-82348fee47e9 to 252778b5-bc63-486d-8d82-c3e4350ee650.
2016-12-20T21:35:01.291Z|173706739|binding|INFO|Changing chassis for lport d2e19c41-f8ff-49d2-bd8e-01d833e59a59 from c0dc0909-4274-400d-826e-82348fee47e9 to 252778b5-bc63-486d-8d82-c3e4350ee650.
2016-12-20T21:35:01.294Z|173706740|binding|INFO|Changing chassis for lport d2e19c41-f8ff-49d2-bd8e-01d833e59a59 from c0dc0909-4274-400d-826e-82348fee47e9 to 252778b5-bc63-486d-8d82-c3e4350ee650.
2016-12-20T21:35:01.297Z|173706741|binding|INFO|Changing chassis for lport d2e19c41-f8ff-49d2-bd8e-01d833e59a59 from c0dc0909-4274-400d-826e-82348fee47e9 to 252778b5-bc63-486d-8d82-c3e4350ee650.
2016-12-20T21:35:01.299Z|173706742|binding|INFO|Changing chassis for lport d2e19c41-f8ff-49d2-bd8e-01d833e59a59 from c0dc0909-4274-400d-826e-82348fee47e9 to 252778b5-bc63-486d-8d82-c3e4350ee650.
2016-12-20T21:35:01.302Z|173706743|binding|INFO|Changing chassis for lport d2e19c41-f8ff-49d2-bd8e-01d833e59a59 from c0dc0909-4274-400d-826e-82348fee47e9 to 252778b5-bc63-486d-8d82-c3e4350ee650.
2016-12-20T21:35:01.305Z|173706744|binding|INFO|Changing chassis for lport d2e19c41-f8ff-49d2-bd8e-01d833e59a59 from c0dc0909-4274-400d-826e-82348fee47e9 to 252778b5-bc63-486d-8d82-c3e4350ee650.
2016-12-20T21:35:01.308Z|173706745|binding|INFO|Changing chassis for lport d2e19c41-f8ff-49d2-bd8e-01d833e59a59 from c0dc0909-4274-400d-826e-82348fee47e9 to 252778b5-bc63-486d-8d82-c3e4350ee650.
2016-12-20T21:35:01.310Z|173706746|binding|INFO|Changing chassis for lport d2e19c41-f8ff-49d2-bd8e-01d833e59a59 from c0dc0909-4274-400d-826e-82348fee47e9 to 252778b5-bc63-486d-8d82-c3e4350ee650.
2016-12-20T21:35:01.313Z|173706747|binding|INFO|Changing chassis for lport d2e19c41-f8ff-49d2-bd8e-01d833e59a59 from c0dc0909-4274-400d-826e-82348fee47e9 to 252778b5-bc63-486d-8d82-c3e4350ee650.
2016-12-20T21:35:01.316Z|173706748|binding|INFO|Changing chassis for lport d2e19c41-f8ff-49d2-bd8e-01d833e59a59 from c0dc0909-4274-400d-826e-82348fee47e9 to 252778b5-bc63-486d-8d82-c3e4350ee650.
2016-12-20T21:35:01.319Z|173706749|binding|INFO|Changing chassis for lport d2e19c41-f8ff-49d2-bd8e-01d833e59a59 from c0dc0909-4274-400d-826e-82348fee47e9 to 252778b5-bc63-486d-8d82-c3e4350ee650.
2016-12-20T21:35:01.321Z|173706750|binding|INFO|Changing chassis for lport d2e19c41-f8ff-49d2-bd8e-01d833e59a59 from c0dc0909-4274-400d-826e-82348fee47e9 to 252778b5-bc63-486d-8d82-c3e4350ee650.
2016-12-20T21:35:01.324Z|173706751|binding|INFO|Changing chassis for lport d2e19c41-f8ff-49d2-bd8e-01d833e59a59 from c0dc0909-4274-400d-826e-82348fee47e9 to 252778b5-bc63-486d-8d82-c3e4350ee650.
2016-12-20T21:35:01.327Z|173706752|binding|INFO|Changing chassis for lport d2e19c41-f8ff-49d2-bd8e-01d833e59a59 from c0dc0909-4274-400d-826e-82348fee47e9 to 252778b5-bc63-486d-8d82-c3e4350ee650.

on host #, running ovs-vsctl show

61af799c-a621-445e-8183-23dcb38ea3cc
    Bridge br-int
        fail_mode: secure
        Port "vnet0"
            Interface "vnet0"
                error: "could not open network device vnet0 (No such device)"
        Port "ovn-c0dc09-0"
            Interface "ovn-c0dc09-0"
                type: geneve
                options: {csum="true", key=flow, remote_ip="172.20.192.73"}
        Port "ovn-456949-0"
            Interface "ovn-456949-0"
                type: geneve
                options: {csum="true", key=flow, remote_ip="172.20.192.74"}
        Port br-int
            Interface br-int
                type: internal
    ovs_version: "2.6.90"

So some port got stuck on a box? Any idea what would cause this and what i should do to remove it?

On Tue, Dec 20, 2016 at 2:29 PM, Russell Bryant <rbryant@redhat.com> wrote:


On Tue, Dec 20, 2016 at 4:18 PM, Devin Acosta <devin@pabstatencio.com> wrote:

One interesting thing I noticed today after looking at the logs from the 'ovn-controller' on the oVIRT nodes, is node1 is logging like crazy, the file is over 1.2GB in size already, seems to be looping like crazy? What should i check to see why it is logging and maxing CPU like this? 

2016-12-20T21:16:17.517Z|91605|binding|INFO|Changing chassis for lport d2e19c41-f8ff-49d2-bd8e-01d833e59a59 from 252778b5-bc63-486d-8d82-c3e4350ee650 to c0dc0909-4274-400d-826e-82348fee47e9.
2016-12-20T21:16:17.521Z|91606|poll_loop|INFO|Dropped 5666 log messages in last 6 seconds (most recently, 0 seconds ago) due to excessive rate
2016-12-20T21:16:17.521Z|91607|poll_loop|INFO|wakeup due to [POLLIN] on fd 12 (172.20.192.73:54710<->172.20.192.77:6642) at lib/stream-fd.c:155 (94% CPU usage)
2016-12-20T21:16:17.521Z|91608|binding|INFO|Changing chassis for lport d2e19c41-f8ff-49d2-bd8e-01d833e59a59 from 252778b5-bc63-486d-8d82-c3e4350ee650 to c0dc0909-4274-400d-826e-82348fee47e9.

​Check "ovn-sbctl show" to figure out which host is chassis ​252778b5-bc63-486d-8d82-c3e4350ee650.  See if the other host has similar entries in its log.

This could happen if a port is configured on two hosts at the same time.  They will constantly fight over which chassis owns it.

-- 
Russell Bryant



--

Devin Acosta
Red Hat Certified Architect, LinuxStack 
602-354-1220 || devin@linuxguru.co