[ovirt-users] 3.6 looses network on reboot

Dan Kenigsberg danken at redhat.com
Mon Feb 29 13:58:08 UTC 2016


This sounds very bad. Changing the subject, so the wider, more
problematic issue is visible.

Did any other user see this behavior?

On Mon, Feb 29, 2016 at 06:27:46AM +0000, David LeVene wrote:
> Hi Dan,
> 
> Answers as follows;
> 
> # rpm -qa | grep -i vdsm
> vdsm-jsonrpc-4.17.18-1.el7.noarch
> vdsm-hook-vmfex-4.17.18-1.el7.noarch
> vdsm-infra-4.17.18-1.el7.noarch
> vdsm-4.17.18-1.el7.noarch
> vdsm-python-4.17.18-1.el7.noarch
> vdsm-yajsonrpc-4.17.18-1.el7.noarch
> vdsm-cli-4.17.18-1.el7.noarch
> vdsm-xmlrpc-4.17.18-1.el7.noarch
> vdsm-hook-vmfex-dev-4.17.18-1.el7.noarch
> 
> 
> There was in this folder ifcfg-ovirtmgnt bridge setup, and also route-ovirtmgnt & rule-ovirtmgmt.. but they were removed after the reboot.
> 
> # ls -althr | grep ifcfg
> -rw-r--r--. 1 root root  254 Sep 16 21:21 ifcfg-lo
> -rw-r--r--. 1 root root  120 Feb 25 14:07 ifcfg-enp7s0f0
> -rw-rw-r--. 1 root root  174 Feb 25 14:40 ifcfg-enp6s0
> 
> I think I modified ifcfg-enp6s0 to get networking up again (eg was set to bridge.. but the bridge wasn't configured).. it was a few days ago.. if it's important I can reboot the box again to see what state it comes up with.
> 
> # cat ifcfg-enp6s0
> BOOTPROTO="none"
> IPADDR="10.80.10.117"
> NETMASK="255.255.255.0"
> GATEWAY="10.80.10.1"
> DEVICE="enp6s0"
> HWADDR="00:25:b5:00:0b:4f"
> ONBOOT=yes
> PEERDNS=yes
> PEERROUTES=yes
> MTU=1500
> 
> # cat ifcfg-enp7s0f0
> # Generated by VDSM version 4.17.18-1.el7
> DEVICE=enp7s0f0
> ONBOOT=yes
> MTU=1500
> HWADDR=00:25:b5:00:0b:0f
> NM_CONTROLLED=no
> 
> # find /var/lib/vdsm/persistence
> /var/lib/vdsm/persistence
> /var/lib/vdsm/persistence/netconf
> /var/lib/vdsm/persistence/netconf.1456371473833165545
> /var/lib/vdsm/persistence/netconf.1456371473833165545/nets
> /var/lib/vdsm/persistence/netconf.1456371473833165545/nets/ovirtmgmt
> 
> # cat /var/lib/vdsm/persistence/netconf.1456371473833165545/nets/ovirtmgmt
> {
>     "nic": "enp6s0",
>     "ipaddr": "10.80.10.117",
>     "mtu": "1500",
>     "netmask": "255.255.255.0",
>     "STP": "no",
>     "bridged": "true",
>     "gateway": "10.80.10.1",
>     "defaultRoute": true
> }
> 
> Supervdsm log is attached.

Have you editted ifcfg-ovirtmgmt manually? Can you somehow reproduce it,
and share its content?
Do you have NetworkManager running? which version?

It seems that Vdsm has two bugs: on boot, initscripts end up setting an
ipv6 address that Vdsm never requested.

    restore-net::INFO::2016-02-25 14:14:58,024::vdsm-restore-net-config::261::root::(_find_changed_or_missing) ovirtmgmt is different or missing from persistent configuration. current: {'nic': 'enp6s0', 'dhcpv6': False, 'ipaddr': '10.80.10.117', 'mtu': '1500', 'netmask': '255.255.255.0', 'bootproto': 'none', 'stp': False, 'bridged': True, 'ipv6addr': ['2400:7d00:110:3:225:b5ff:fe00:b4f/64'], 'gateway': '10.80.10.1', 'defaultRoute': True}, persisted: {u'nic': u'enp6s0', 'dhcpv6': False, u'ipaddr': u'10.80.10.117', u'mtu': '1500', u'netmask': u'255.255.255.0', 'bootproto': 'none', 'stp': False, u'bridged': True, u'gateway': u'10.80.10.1', u'defaultRoute': True}


Then, Vdsm tries to drop the
unsolicited address, but fails. Both must be fixed ASAP.

    restore-net::ERROR::2016-02-25 14:14:59,490::__init__::58::root::(__exit__) Failed rollback transaction last known good network.
    Traceback (most recent call last):
      File "/usr/share/vdsm/network/api.py", line 918, in setupNetworks
        keep_bridge=keep_bridge)
      File "/usr/share/vdsm/network/api.py", line 222, in wrapped
        ret = func(**attrs)
      File "/usr/share/vdsm/network/api.py", line 502, in _delNetwork
        configurator.removeQoS(net_ent)
      File "/usr/share/vdsm/network/configurators/__init__.py", line 122, in removeQoS
        qos.remove_outbound(top_device)
      File "/usr/share/vdsm/network/configurators/qos.py", line 60, in remove_outbound
        device, pref=_NON_VLANNED_ID if vlan_tag is None else vlan_tag)
      File "/usr/share/vdsm/network/tc/filter.py", line 31, in delete
        _wrapper.process_request(command)
      File "/usr/share/vdsm/network/tc/_wrapper.py", line 38, in process_request
        raise TrafficControlException(retcode, err, command)
    TrafficControlException: (None, 'Message truncated', ['/usr/sbin/tc', 'filter', 'del', 'dev', 'enp6s0', 'pref', '5000'])

Regards,
Dan.



More information about the Users mailing list