4:58 p.m.
This sounds very bad. Changing the subject, so the wider, more
problematic issue is visible.
Did any other user see this behavior?
On Mon, Feb 29, 2016 at 06:27:46AM +0000, David LeVene wrote:
Hi Dan,
Answers as follows;
# rpm -qa | grep -i vdsm
vdsm-jsonrpc-4.17.18-1.el7.noarch
vdsm-hook-vmfex-4.17.18-1.el7.noarch
vdsm-infra-4.17.18-1.el7.noarch
vdsm-4.17.18-1.el7.noarch
vdsm-python-4.17.18-1.el7.noarch
vdsm-yajsonrpc-4.17.18-1.el7.noarch
vdsm-cli-4.17.18-1.el7.noarch
vdsm-xmlrpc-4.17.18-1.el7.noarch
vdsm-hook-vmfex-dev-4.17.18-1.el7.noarch
There was in this folder ifcfg-ovirtmgnt bridge setup, and also route-ovirtmgnt &
rule-ovirtmgmt.. but they were removed after the reboot.
# ls -althr | grep ifcfg
-rw-r--r--. 1 root root 254 Sep 16 21:21 ifcfg-lo
-rw-r--r--. 1 root root 120 Feb 25 14:07 ifcfg-enp7s0f0
-rw-rw-r--. 1 root root 174 Feb 25 14:40 ifcfg-enp6s0
I think I modified ifcfg-enp6s0 to get networking up again (eg was set to bridge.. but
the bridge wasn't configured).. it was a few days ago.. if it's important I can
reboot the box again to see what state it comes up with.
# cat ifcfg-enp6s0
BOOTPROTO="none"
IPADDR="10.80.10.117"
NETMASK="255.255.255.0"
GATEWAY="10.80.10.1"
DEVICE="enp6s0"
HWADDR="00:25:b5:00:0b:4f"
ONBOOT=yes
PEERDNS=yes
PEERROUTES=yes
MTU=1500
# cat ifcfg-enp7s0f0
# Generated by VDSM version 4.17.18-1.el7
DEVICE=enp7s0f0
ONBOOT=yes
MTU=1500
HWADDR=00:25:b5:00:0b:0f
NM_CONTROLLED=no
# find /var/lib/vdsm/persistence
/var/lib/vdsm/persistence
/var/lib/vdsm/persistence/netconf
/var/lib/vdsm/persistence/netconf.1456371473833165545
/var/lib/vdsm/persistence/netconf.1456371473833165545/nets
/var/lib/vdsm/persistence/netconf.1456371473833165545/nets/ovirtmgmt
# cat /var/lib/vdsm/persistence/netconf.1456371473833165545/nets/ovirtmgmt
{
"nic": "enp6s0",
"ipaddr": "10.80.10.117",
"mtu": "1500",
"netmask": "255.255.255.0",
"STP": "no",
"bridged": "true",
"gateway": "10.80.10.1",
"defaultRoute": true
}
Supervdsm log is attached.
Have you editted ifcfg-ovirtmgmt manually? Can you somehow reproduce it,
and share its content?
Do you have NetworkManager running? which version?
It seems that Vdsm has two bugs: on boot, initscripts end up setting an
ipv6 address that Vdsm never requested.
restore-net::INFO::2016-02-25
14:14:58,024::vdsm-restore-net-config::261::root::(_find_changed_or_missing) ovirtmgmt is
different or missing from persistent configuration. current: {'nic':
'enp6s0', 'dhcpv6': False, 'ipaddr': '10.80.10.117',
'mtu': '1500', 'netmask': '255.255.255.0',
'bootproto': 'none', 'stp': False, 'bridged': True,
'ipv6addr': ['2400:7d00:110:3:225:b5ff:fe00:b4f/64'], 'gateway':
'10.80.10.1', 'defaultRoute': True}, persisted: {u'nic':
u'enp6s0', 'dhcpv6': False, u'ipaddr': u'10.80.10.117',
u'mtu': '1500', u'netmask': u'255.255.255.0',
'bootproto': 'none', 'stp': False, u'bridged': True,
u'gateway': u'10.80.10.1', u'defaultRoute': True}
Then, Vdsm tries to drop the
unsolicited address, but fails. Both must be fixed ASAP.
restore-net::ERROR::2016-02-25 14:14:59,490::__init__::58::root::(__exit__) Failed
rollback transaction last known good network.
Traceback (most recent call last):
File "/usr/share/vdsm/network/api.py", line 918, in setupNetworks
keep_bridge=keep_bridge)
File "/usr/share/vdsm/network/api.py", line 222, in wrapped
ret = func(**attrs)
File "/usr/share/vdsm/network/api.py", line 502, in _delNetwork
configurator.removeQoS(net_ent)
File "/usr/share/vdsm/network/configurators/__init__.py", line 122, in
removeQoS
qos.remove_outbound(top_device)
File "/usr/share/vdsm/network/configurators/qos.py", line 60, in
remove_outbound
device, pref=_NON_VLANNED_ID if vlan_tag is None else vlan_tag)
File "/usr/share/vdsm/network/tc/filter.py", line 31, in delete
_wrapper.process_request(command)
File "/usr/share/vdsm/network/tc/_wrapper.py", line 38, in
process_request
raise TrafficControlException(retcode, err, command)
TrafficControlException: (None, 'Message truncated', ['/usr/sbin/tc',
'filter', 'del', 'dev', 'enp6s0', 'pref',
'5000'])
Regards,
Dan.