[ovirt-users] Logical network error

[Moti Asayag]

----- Original Message -----
> From: "Dan Kenigsberg" <danken at redhat.com>
> To: "Maurice James" <mjames at media-node.com>, fromani at redhat.com
> Cc: "Moti Asayag" <masayag at redhat.com>, "users" <users at ovirt.org>, "Antoni Segura Puimedon" <asegurap at redhat.com>
> Sent: Monday, July 21, 2014 6:49:56 PM
> Subject: Re: [ovirt-users] Logical network error
> 
> On Mon, Jul 21, 2014 at 07:35:00AM -0400, Maurice James wrote:
> > 
> > Here are the supervdsm logs
> 
> Hmm, it's a trove of errors and tracebacks; there's a lot to debug, for
> example, here Vdsm was asked to create a 'migration' network on top of
> bond0 that was already used by ovirtmgmt. Engine should have blocked
> that. Moti? 

Obviously should have been blocked from engine side, however there are couple of issues with that migration network
which might contempt the engine caps:

MainProcess|Thread-43::DEBUG::2014-04-11 08:56:24,677::configNetwork::250::root::(addNetwork) validating network...
MainProcess|Thread-43::INFO::2014-04-11 08:56:24,677::configNetwork::272::root::(addNetwork) Adding network migration with vlan=5, bonding=bond0, nics=['em1', 'em2'], bondingOptions=mode=4 miimon=100, mtu=None, bridged=False, defaultRoute=False,options={'implicitBonding': True}
MainProcess|Thread-43::DEBUG::2014-04-11 08:56:24,679::ifcfg::447::root::(_persistentBackup) backing up ifcfg-bond0.5: # original file did not exist

MainProcess|Thread-43::DEBUG::2014-04-11 08:56:24,681::ifcfg::313::root::(writeBackupFile) Persistently backed up /var/lib/vdsm/netconfback/ifcfg-bond0.5 (until next 'set safe config')
MainProcess|Thread-43::DEBUG::2014-04-11 08:56:24,681::ifcfg::537::root::(writeConfFile) Writing to file /etc/sysconfig/network-scripts/ifcfg-bond0.5 configuration:
# Generated by VDSM version 4.14.6-7.git41501b3.el6
DEVICE=bond0.5
ONBOOT=yes
VLAN=yes
DEFROUTE=no
NM_CONTROLLED=no

MainProcess|Thread-43::DEBUG::2014-04-11 08:56:24,684::utils::642::root::(execCmd) '/sbin/ip route show to 0.0.0.0/0 table all' (cwd None)
MainProcess|Thread-43::DEBUG::2014-04-11 08:56:24,690::utils::662::root::(execCmd) SUCCESS: <err> = ''; <rc> = 0
MainProcess|Thread-43::DEBUG::2014-04-11 08:56:24,728::utils::642::root::(execCmd) '/sbin/ip -d link show dev bond0.2' (cwd None)
MainProcess|Thread-43::DEBUG::2014-04-11 08:56:24,735::utils::662::root::(execCmd) SUCCESS: <err> = ''; <rc> = 0
MainProcess|Thread-43::DEBUG::2014-04-11 08:56:24,741::utils::642::root::(execCmd) '/sbin/ip -d link show dev bond0.4' (cwd None)
MainProcess|Thread-43::DEBUG::2014-04-11 08:56:24,747::utils::662::root::(execCmd) SUCCESS: <err> = ''; <rc> = 0
MainProcess|Thread-43::DEBUG::2014-04-11 08:56:24,753::utils::642::root::(execCmd) '/sbin/ip -d link show dev bond0.3' (cwd None)
MainProcess|Thread-43::DEBUG::2014-04-11 08:56:24,759::utils::662::root::(execCmd) SUCCESS: <err> = ''; <rc> = 0
MainProcess|Thread-43::DEBUG::2014-04-11 08:56:24,764::__init__::122::root::(_addSourceRoute) Adding source route bond0.5, None, None, None
MainProcess|Thread-43::ERROR::2014-04-11 08:56:24,764::sourceRoute::68::root::(configure) ipaddr, mask or gateway not received
MainProcess|Thread-43::DEBUG::2014-04-11 08:56:24,764::utils::642::root::(execCmd) '/sbin/ifup bond0.5' (cwd None)
MainProcess|Thread-43::DEBUG::2014-04-11 08:56:24,914::utils::662::root::(execCmd) SUCCESS: <err> = ''; <rc> = 0
MainProcess|Thread-43::DEBUG::2014-04-11 08:56:24,917::libvirtconnection::124::root::(wrapper) Unknown libvirterror: ecode: 43 edom: 19 level: 2 message: Network not found: no network with matching name 'vdsm-migration'
MainProcess|Thread-43::DEBUG::2014-04-11 08:56:24,918::ifcfg::324::root::(_atomicNetworkBackup) Backed up migration
MainProcess|Thread-43::DEBUG::2014-04-11 08:56:24,919::libvirtconnection::124::root::(wrapper) Unknown libvirterror: ecode: 43 edom: 19 level: 2 message: Network not found: no network with matching name 'vdsm-migration'
MainProcess|Thread-43::DEBUG::2014-04-11 08:56:24,919::ifcfg::333::root::(_persistentNetworkBackup) backing up network migration: # original file did not exist

MainProcess|Thread-43::DEBUG::2014-04-11 08:56:24,921::ifcfg::313::root::(writeBackupFile) Persistently backed up /var/lib/vdsm/netconfback/logicalnetworks/migration (until next 'set safe config')
MainProcess|Thread-43::DEBUG::2014-04-11 08:56:25,031::configNetwork::651::setupNetworks::(setupNetworks) Checking connectivity...
MainProcess|Thread-43::DEBUG::2014-04-11 08:56:26,035::supervdsmServer::103::SuperVdsm.ServerCallback::(wrapper) return setupNetworks with None
MainProcess|Thread-43::DEBUG::201

I wasn't able to find the output of getCapabilities for the host in the logs. 
Without 'getCaps' it is hard to know which network configuration was reported to the engine by vdsm.

But if 'vdsm-migration' is not a know network (based on above message), the engine might allow to use the underlying interface.

In addition, there are a lot of setup-networks call which most of them failed due to 'resource unavailable'. 

> 
> MainProcess|Thread-47826::DEBUG::2014-04-11
> 10:00:26,335::configNetwork::589::setupNetworks::(setupNetworks) Setting up
> network according to configuration: networks:{'migration': {'bonding':
> 'bond0', 'bridged': 'false'}}, bondings:{}, options:{'connectivityCheck':
> 'true', 'connectivityTimeout': 120}
> Traceback (most recent call last):
>   File "/usr/share/vdsm/supervdsmServer", line 98, in wrapper
>     res = func(*args, **kwargs)
>   File "/usr/share/vdsm/supervdsmServer", line 202, in setupNetworks
>     return configNetwork.setupNetworks(networks, bondings, **options)
>   File "/usr/share/vdsm/configNetwork.py", line 648, in setupNetworks
>     implicitBonding=True, **d)
>   File "/usr/share/vdsm/configNetwork.py", line 186, in wrapped
>     return func(*args, **kwargs)
>   File "/usr/share/vdsm/configNetwork.py", line 256, in addNetwork
>     bridged)
>   File "/usr/share/vdsm/configNetwork.py", line 170, in
>   _validateInterNetworkCompatibility
>     _validateNoDirectNet(ifaces_bridgeless)
>   File "/usr/share/vdsm/configNetwork.py", line 154, in _validateNoDirectNet
>     (iface, iface_net))
> ConfigNetworkError: (21, "interface 'bond0' already member of network
> 'ovirtmgmt'")
> 
> 
> There are also repeated failed attempts to create a payload disk: did you
> notice when they happen? Could it be related to insufficient disk space?
> 
> MainProcess|clientIFinit::ERROR::2014-03-25
> 22:13:02,529::supervdsmServer::100::SuperVdsm.ServerCallback::(wrapper)
> Error in mkFloppyFs
> Traceback (most recent call last):
>   File "/usr/share/vdsm/supervdsmServer", line 98, in wrapper
>     res = func(*args, **kwargs)
>   File "/usr/share/vdsm/supervdsmServer", line 325, in mkFloppyFs
>     return mkimage.mkFloppyFs(vmId, files, volId)
>   File "/usr/share/vdsm/mkimage.py", line 104, in mkFloppyFs
>     "code %s, out %s\nerr %s" % (rc, out, err))
> OSError: [Errno 5] could not create floppy file: code 1, out mkfs.msdos 3.0.9
> (31 Jan 2010)
> 
> err mkfs.msdos: unable to create
> /var/run/vdsm/payload/94632c2e-28a0-4304-9261-c302e0027604.ecac527e731a2a0057dc6a3ae3df9ba3.img
> 
> 
> In any case, if you manage to reproduce these issues, please open a detailed
> bug entry.
> 
> Regards,
> Dan.
>