This sounds very bad. Changing the subject, so the wider, more problematic issue is visible. Did any other user see this behavior? On Mon, Feb 29, 2016 at 06:27:46AM +0000, David LeVene wrote:
Hi Dan,
Answers as follows;
# rpm -qa | grep -i vdsm vdsm-jsonrpc-4.17.18-1.el7.noarch vdsm-hook-vmfex-4.17.18-1.el7.noarch vdsm-infra-4.17.18-1.el7.noarch vdsm-4.17.18-1.el7.noarch vdsm-python-4.17.18-1.el7.noarch vdsm-yajsonrpc-4.17.18-1.el7.noarch vdsm-cli-4.17.18-1.el7.noarch vdsm-xmlrpc-4.17.18-1.el7.noarch vdsm-hook-vmfex-dev-4.17.18-1.el7.noarch
There was in this folder ifcfg-ovirtmgnt bridge setup, and also route-ovirtmgnt & rule-ovirtmgmt.. but they were removed after the reboot.
# ls -althr | grep ifcfg -rw-r--r--. 1 root root 254 Sep 16 21:21 ifcfg-lo -rw-r--r--. 1 root root 120 Feb 25 14:07 ifcfg-enp7s0f0 -rw-rw-r--. 1 root root 174 Feb 25 14:40 ifcfg-enp6s0
I think I modified ifcfg-enp6s0 to get networking up again (eg was set to bridge.. but the bridge wasn't configured).. it was a few days ago.. if it's important I can reboot the box again to see what state it comes up with.
# cat ifcfg-enp6s0 BOOTPROTO="none" IPADDR="10.80.10.117" NETMASK="255.255.255.0" GATEWAY="10.80.10.1" DEVICE="enp6s0" HWADDR="00:25:b5:00:0b:4f" ONBOOT=yes PEERDNS=yes PEERROUTES=yes MTU=1500
# cat ifcfg-enp7s0f0 # Generated by VDSM version 4.17.18-1.el7 DEVICE=enp7s0f0 ONBOOT=yes MTU=1500 HWADDR=00:25:b5:00:0b:0f NM_CONTROLLED=no
# find /var/lib/vdsm/persistence /var/lib/vdsm/persistence /var/lib/vdsm/persistence/netconf /var/lib/vdsm/persistence/netconf.1456371473833165545 /var/lib/vdsm/persistence/netconf.1456371473833165545/nets /var/lib/vdsm/persistence/netconf.1456371473833165545/nets/ovirtmgmt
# cat /var/lib/vdsm/persistence/netconf.1456371473833165545/nets/ovirtmgmt { "nic": "enp6s0", "ipaddr": "10.80.10.117", "mtu": "1500", "netmask": "255.255.255.0", "STP": "no", "bridged": "true", "gateway": "10.80.10.1", "defaultRoute": true }
Supervdsm log is attached.
Have you editted ifcfg-ovirtmgmt manually? Can you somehow reproduce it, and share its content? Do you have NetworkManager running? which version? It seems that Vdsm has two bugs: on boot, initscripts end up setting an ipv6 address that Vdsm never requested. restore-net::INFO::2016-02-25 14:14:58,024::vdsm-restore-net-config::261::root::(_find_changed_or_missing) ovirtmgmt is different or missing from persistent configuration. current: {'nic': 'enp6s0', 'dhcpv6': False, 'ipaddr': '10.80.10.117', 'mtu': '1500', 'netmask': '255.255.255.0', 'bootproto': 'none', 'stp': False, 'bridged': True, 'ipv6addr': ['2400:7d00:110:3:225:b5ff:fe00:b4f/64'], 'gateway': '10.80.10.1', 'defaultRoute': True}, persisted: {u'nic': u'enp6s0', 'dhcpv6': False, u'ipaddr': u'10.80.10.117', u'mtu': '1500', u'netmask': u'255.255.255.0', 'bootproto': 'none', 'stp': False, u'bridged': True, u'gateway': u'10.80.10.1', u'defaultRoute': True} Then, Vdsm tries to drop the unsolicited address, but fails. Both must be fixed ASAP. restore-net::ERROR::2016-02-25 14:14:59,490::__init__::58::root::(__exit__) Failed rollback transaction last known good network. Traceback (most recent call last): File "/usr/share/vdsm/network/api.py", line 918, in setupNetworks keep_bridge=keep_bridge) File "/usr/share/vdsm/network/api.py", line 222, in wrapped ret = func(**attrs) File "/usr/share/vdsm/network/api.py", line 502, in _delNetwork configurator.removeQoS(net_ent) File "/usr/share/vdsm/network/configurators/__init__.py", line 122, in removeQoS qos.remove_outbound(top_device) File "/usr/share/vdsm/network/configurators/qos.py", line 60, in remove_outbound device, pref=_NON_VLANNED_ID if vlan_tag is None else vlan_tag) File "/usr/share/vdsm/network/tc/filter.py", line 31, in delete _wrapper.process_request(command) File "/usr/share/vdsm/network/tc/_wrapper.py", line 38, in process_request raise TrafficControlException(retcode, err, command) TrafficControlException: (None, 'Message truncated', ['/usr/sbin/tc', 'filter', 'del', 'dev', 'enp6s0', 'pref', '5000']) Regards, Dan.