oVirt 4.3.2 Error: genev_sys_6081 is not present in the system

Hello, I've installed oVirt 4.3.2 and the problem is that it log messages: VDSM ovirt-04 command Get Host Statistics failed: Internal JSON-RPC error: {'reason': '[Errno 19] genev_sys_6081 is not present in the system'} in Open Virtualization Manager. It also keeps on logging in /var/log/messages: Mar 22 11:51:03 ovirt-04 NetworkManager[8725]: <info> [1553244663.9861] device (genev_sys_6081): carrier: link connected Mar 22 11:51:03 ovirt-04 NetworkManager[8725]: <info> [1553244663.9864] manager: (genev_sys_6081): new Generic device (/org/freedesktop/NetworkManager/Devices/34160) Mar 22 11:51:03 ovirt-04 NetworkManager[8725]: <info> [1553244663.9866] device (genev_sys_6081): enslaved to non-master-type device ovs-system; ignoring Mar 22 11:51:03 ovirt-04 NetworkManager[8725]: <info> [1553244663.9906] device (genev_sys_6081): enslaved to non-master-type device ovs-system; ignoring Mar 22 11:51:03 ovirt-04 kernel: device genev_sys_6081 left promiscuous mode Mar 22 11:51:04 ovirt-04 kernel: device genev_sys_6081 entered promiscuous mode Mar 22 11:51:04 ovirt-04 NetworkManager[8725]: <info> [1553244664.0038] device (genev_sys_6081): carrier: link connected Mar 22 11:51:04 ovirt-04 NetworkManager[8725]: <info> [1553244664.0042] manager: (genev_sys_6081): new Generic device (/org/freedesktop/NetworkManager/Devices/34161) Mar 22 11:51:04 ovirt-04 NetworkManager[8725]: <info> [1553244664.0044] device (genev_sys_6081): enslaved to non-master-type device ovs-system; ignoring Mar 22 11:51:04 ovirt-04 NetworkManager[8725]: <info> [1553244664.0082] device (genev_sys_6081): enslaved to non-master-type device ovs-system; ignoring Mar 22 11:51:04 ovirt-04 kernel: device genev_sys_6081 left promiscuous mode Mar 22 11:51:04 ovirt-04 kernel: device genev_sys_6081 entered promiscuous mode Also I can see the following in /var/log/openvswitch: 2019-03-22T08:53:12.413Z|131047|bridge|WARN|could not add network device ovn-034a1c-0 to ofproto (File exists) 2019-03-22T08:53:12.431Z|131048|tunnel|WARN|ovn-a21088-0: attempting to add tunnel port with same config as port 'ovn-0836d7-0' (::->192.168.10.24, key=flow, legacy_l2, dp port=2) 2019-03-22T08:53:12.466Z|131055|connmgr|WARN|accept failed (Too many open files) 2019-03-22T08:53:12.466Z|131056|unixctl|WARN|punix:/var/run/openvswitch/ovs-vswitchd.9103.ctl: accept failed: Too many open files In ovsdb-server.log: 2019-03-22T03:24:12.583Z|05792|jsonrpc|WARN|unix#28684: receive error: Connection reset by peer 2019-03-22T03:24:12.583Z|05793|reconnect|WARN|unix#28684: connection dropped (Connection reset by peer) How to fix this issue with geneve tunnels?

On Fri, 12 Apr 2019 12:31:15 -0000 "Dee Slaw" <v.voroncov@scanru.ru> wrote:
Hello, I've installed oVirt 4.3.2 and the problem is that it log messages:
VDSM ovirt-04 command Get Host Statistics failed: Internal JSON-RPC error: {'reason': '[Errno 19] genev_sys_6081 is not present in the system'} in Open Virtualization Manager.
This might happen if genev_sys_6081 flickers.
It also keeps on logging in /var/log/messages:
Mar 22 11:51:03 ovirt-04 NetworkManager[8725]: <info> [1553244663.9861] device (genev_sys_6081): carrier: link connected Mar 22 11:51:03 ovirt-04 NetworkManager[8725]: <info> [1553244663.9864] manager: (genev_sys_6081): new Generic device (/org/freedesktop/NetworkManager/Devices/34160) Mar 22 11:51:03 ovirt-04 NetworkManager[8725]: <info> [1553244663.9866] device (genev_sys_6081): enslaved to non-master-type device ovs-system; ignoring Mar 22 11:51:03 ovirt-04 NetworkManager[8725]: <info> [1553244663.9906] device (genev_sys_6081): enslaved to non-master-type device ovs-system; ignoring Mar 22 11:51:03 ovirt-04 kernel: device genev_sys_6081 left promiscuous mode Mar 22 11:51:04 ovirt-04 kernel: device genev_sys_6081 entered promiscuous mode Mar 22 11:51:04 ovirt-04 NetworkManager[8725]: <info> [1553244664.0038] device (genev_sys_6081): carrier: link connected Mar 22 11:51:04 ovirt-04 NetworkManager[8725]: <info> [1553244664.0042] manager: (genev_sys_6081): new Generic device (/org/freedesktop/NetworkManager/Devices/34161) Mar 22 11:51:04 ovirt-04 NetworkManager[8725]: <info> [1553244664.0044] device (genev_sys_6081): enslaved to non-master-type device ovs-system; ignoring Mar 22 11:51:04 ovirt-04 NetworkManager[8725]: <info> [1553244664.0082] device (genev_sys_6081): enslaved to non-master-type device ovs-system; ignoring Mar 22 11:51:04 ovirt-04 kernel: device genev_sys_6081 left promiscuous mode Mar 22 11:51:04 ovirt-04 kernel: device genev_sys_6081 entered promiscuous mode
Also I can see the following in /var/log/openvswitch:
2019-03-22T08:53:12.413Z|131047|bridge|WARN|could not add network device ovn-034a1c-0 to ofproto (File exists) 2019-03-22T08:53:12.431Z|131048|tunnel|WARN|ovn-a21088-0: attempting to add tunnel port with same config as port 'ovn-0836d7-0' (::->192.168.10.24, key=flow, legacy_l2, dp port=2)
2019-03-22T08:53:12.466Z|131055|connmgr|WARN|accept failed (Too many open files) 2019-03-22T08:53:12.466Z|131056|unixctl|WARN|punix:/var/run/openvswitch/ovs-vswitchd.9103.ctl: accept failed: Too many open files
Are there too files opened on the host?
In ovsdb-server.log:
2019-03-22T03:24:12.583Z|05792|jsonrpc|WARN|unix#28684: receive error: Connection reset by peer 2019-03-22T03:24:12.583Z|05793|reconnect|WARN|unix#28684: connection dropped (Connection reset by peer)
How to fix this issue with geneve tunnels? _______________________________________________ Users mailing list -- users@ovirt.org To unsubscribe send an email to users-leave@ovirt.org Privacy Statement: https://www.ovirt.org/site/privacy-policy/ oVirt Code of Conduct: https://www.ovirt.org/community/about/community-guidelines/ List Archives: https://lists.ovirt.org/archives/list/users@ovirt.org/message/PVJJPK7MQOFLXJ...

Also get this after install new ovirt node. It stops after about 20 minutes.

Would you please share the last lines of all files in /var/log/openvswitch, the relevant lines of /var/log/message, and the ouput of ss -lap lsof ? Thanks On Sun, 14 Apr 2019 14:17:53 -0000 "Nardus Geldenhuys" <nardusg@gmail.com> wrote:
Also get this after install new ovirt node. It stops after about 20 minutes. _______________________________________________ Users mailing list -- users@ovirt.org To unsubscribe send an email to users-leave@ovirt.org Privacy Statement: https://www.ovirt.org/site/privacy-policy/ oVirt Code of Conduct: https://www.ovirt.org/community/about/community-guidelines/ List Archives: https://lists.ovirt.org/archives/list/users@ovirt.org/message/IGJUPKTPCVJ32N...

Please find attached On Mon, 15 Apr 2019 at 10:10, Dominik Holler <dholler@redhat.com> wrote:
Would you please share the last lines of all files in /var/log/openvswitch, the relevant lines of /var/log/message, and the ouput of ss -lap lsof ? Thanks
On Sun, 14 Apr 2019 14:17:53 -0000 "Nardus Geldenhuys" <nardusg@gmail.com> wrote:
Also get this after install new ovirt node. It stops after about 20 minutes. _______________________________________________ Users mailing list -- users@ovirt.org To unsubscribe send an email to users-leave@ovirt.org Privacy Statement: https://www.ovirt.org/site/privacy-policy/ oVirt Code of Conduct: https://www.ovirt.org/community/about/community-guidelines/ List Archives: https://lists.ovirt.org/archives/list/users@ovirt.org/message/IGJUPKTPCVJ32N...

ss output attached On Mon, 15 Apr 2019 at 10:10, Dominik Holler <dholler@redhat.com> wrote:
Would you please share the last lines of all files in /var/log/openvswitch, the relevant lines of /var/log/message, and the ouput of ss -lap lsof ? Thanks
On Sun, 14 Apr 2019 14:17:53 -0000 "Nardus Geldenhuys" <nardusg@gmail.com> wrote:
Also get this after install new ovirt node. It stops after about 20 minutes. _______________________________________________ Users mailing list -- users@ovirt.org To unsubscribe send an email to users-leave@ovirt.org Privacy Statement: https://www.ovirt.org/site/privacy-policy/ oVirt Code of Conduct: https://www.ovirt.org/community/about/community-guidelines/ List Archives: https://lists.ovirt.org/archives/list/users@ovirt.org/message/IGJUPKTPCVJ32N...

Thanks, looks like message about too many open files is valid. Next question is why the geneve tunnel is failing. Would you share the output of ovs-vsctl list Interface ? Can the affected host can reach the other OVN hosts via their remote_ip on udp port 6081? A check by netcat might give a hint: ovs-vsctl list Interface | grep remote_ip nc -zuv <remote_ip> 6081 On Mon, 15 Apr 2019 12:01:54 +0200 Nardus Geldenhuys <nardusg@gmail.com> wrote:
ss output attached
On Mon, 15 Apr 2019 at 10:10, Dominik Holler <dholler@redhat.com> wrote:
Would you please share the last lines of all files in /var/log/openvswitch, the relevant lines of /var/log/message, and the ouput of ss -lap lsof ? Thanks
On Sun, 14 Apr 2019 14:17:53 -0000 "Nardus Geldenhuys" <nardusg@gmail.com> wrote:
Also get this after install new ovirt node. It stops after about 20 minutes. _______________________________________________ Users mailing list -- users@ovirt.org To unsubscribe send an email to users-leave@ovirt.org Privacy Statement: https://www.ovirt.org/site/privacy-policy/ oVirt Code of Conduct: https://www.ovirt.org/community/about/community-guidelines/ List Archives: https://lists.ovirt.org/archives/list/users@ovirt.org/message/IGJUPKTPCVJ32N...

Hi Please find attached. Yes they can talk to each other on those ports. Ncat: UDP packet sent successfully Regards Nardus On Mon, 15 Apr 2019 at 15:11, Dominik Holler <dholler@redhat.com> wrote:
Thanks, looks like message about too many open files is valid. Next question is why the geneve tunnel is failing. Would you share the output of ovs-vsctl list Interface ?
Can the affected host can reach the other OVN hosts via their remote_ip on udp port 6081? A check by netcat might give a hint: ovs-vsctl list Interface | grep remote_ip nc -zuv <remote_ip> 6081
On Mon, 15 Apr 2019 12:01:54 +0200 Nardus Geldenhuys <nardusg@gmail.com> wrote:
ss output attached
On Mon, 15 Apr 2019 at 10:10, Dominik Holler <dholler@redhat.com> wrote:
Would you please share the last lines of all files in /var/log/openvswitch, the relevant lines of /var/log/message, and the ouput of ss -lap lsof ? Thanks
On Sun, 14 Apr 2019 14:17:53 -0000 "Nardus Geldenhuys" <nardusg@gmail.com> wrote:
Also get this after install new ovirt node. It stops after about 20 minutes. _______________________________________________ Users mailing list -- users@ovirt.org To unsubscribe send an email to users-leave@ovirt.org Privacy Statement: https://www.ovirt.org/site/privacy-policy/ oVirt Code of Conduct: https://www.ovirt.org/community/about/community-guidelines/ List Archives:
https://lists.ovirt.org/archives/list/users@ovirt.org/message/IGJUPKTPCVJ32N...

On Mon, 15 Apr 2019 15:23:19 +0200 Nardus Geldenhuys <nardusg@gmail.com> wrote:
Hi
Please find attached. Yes they can talk to each other on those ports.
Ncat: UDP packet sent successfully
Unfortunately I do not have an idea why your setup does not work. Can you please try to vdsm-tool ovn-config <engine_ip_on_net> <ovirt_name_of_the_172.18.206.x_network> and see if this fixes the issue? Else I just can ask you to open a bug on oVirt or re-try on ovs-discuss mailing list.
Regards
Nardus
On Mon, 15 Apr 2019 at 15:11, Dominik Holler <dholler@redhat.com> wrote:
Thanks, looks like message about too many open files is valid. Next question is why the geneve tunnel is failing. Would you share the output of ovs-vsctl list Interface ?
Can the affected host can reach the other OVN hosts via their remote_ip on udp port 6081? A check by netcat might give a hint: ovs-vsctl list Interface | grep remote_ip nc -zuv <remote_ip> 6081
On Mon, 15 Apr 2019 12:01:54 +0200 Nardus Geldenhuys <nardusg@gmail.com> wrote:
ss output attached
On Mon, 15 Apr 2019 at 10:10, Dominik Holler <dholler@redhat.com> wrote:
Would you please share the last lines of all files in /var/log/openvswitch, the relevant lines of /var/log/message, and the ouput of ss -lap lsof ? Thanks
On Sun, 14 Apr 2019 14:17:53 -0000 "Nardus Geldenhuys" <nardusg@gmail.com> wrote:
Also get this after install new ovirt node. It stops after about 20 minutes. _______________________________________________ Users mailing list -- users@ovirt.org To unsubscribe send an email to users-leave@ovirt.org Privacy Statement: https://www.ovirt.org/site/privacy-policy/ oVirt Code of Conduct: https://www.ovirt.org/community/about/community-guidelines/ List Archives:
https://lists.ovirt.org/archives/list/users@ovirt.org/message/IGJUPKTPCVJ32N...

Same error here - any news on this?

Hey Do use openvswitch? We don't and I did notice that the messages disappear after a while. Regards Nardus

On Mon, 29 Apr 2019 08:58:29 -0000 "Markus Schaufler" <markus.schaufler@digit-all.at> wrote:
Same error here - any news on this?
This might indicate that the interface genev_sys_6081 is flickering. The reason for this might be a problem during creating the GENEVE tunnels for OVN. Would you please share the last lines of all files in /var/log/openvswitch and the relevant lines of /var/log/messages? If the error is about too many files open, would you please share the ouput of ss -lap lsof on the affected host might point which files are opened. The output of ovs-vsctl list Interface will show the state of the tunnels. Can the affected host reach the other OVN hosts via their remote_ip on udp port 6081? A check by netcat might give a hint: ovs-vsctl list Interface | grep remote_ip nc -zuv <remote_ip> 6081 Does running systemctl restart ovn-controller fix the issue?
_______________________________________________ Users mailing list -- users@ovirt.org To unsubscribe send an email to users-leave@ovirt.org Privacy Statement: https://www.ovirt.org/site/privacy-policy/ oVirt Code of Conduct: https://www.ovirt.org/community/about/community-guidelines/ List Archives: https://lists.ovirt.org/archives/list/users@ovirt.org/message/OCDUMEBUEBIXFV...
participants (4)
-
Dee Slaw
-
Dominik Holler
-
Markus Schaufler
-
Nardus Geldenhuys