feature suggestion: initial generation of management network

[Dan Kenigsberg]

On Tue, Dec 25, 2012 at 09:29:26AM -0500, Simon Grinberg wrote:
> 
> 
> ----- Original Message -----
> > From: "Dan Kenigsberg" <danken at redhat.com>
> > To: "arch" <arch at ovirt.org>
> > Sent: Tuesday, December 25, 2012 2:27:22 PM
> > Subject: feature suggestion: initial generation of management network
> > 
> > Current condition:
> > ==================
> > The management network, named ovirtmgmt, is created during host
> > bootstrap. It consists of a bridge device, connected to the network
> > device that was used to communicate with Engine (nic, bonding or
> > vlan).
> > It inherits its ip settings from the latter device.
> > 
> > Why Is the Management Network Needed?
> > =====================================
> > Understandably, some may ask why do we need to have a management
> > network - why having a host with IPv4 configured on it is not enough.
> > The answer is twofold:
> > 1. In oVirt, a network is an abstraction of the resources required
> > for
> >    connectivity of a host for a specific usage. This is true for the
> >    management network just as it is for VM network or a display
> >    network.
> >    The network entity is the key for adding/changing nics and IP
> >    address.
> > 2. In many occasions (such as small setups) the management network is
> >    used as a VM/display network as well.
> > 
> > Problems in current connectivity:
> > ================================
> > According to alonbl of ovirt-host-deploy fame, and with no conflict
> > to
> > my own experience, creating the management network is the most
> > fragile,
> > error-prone step of bootstrap.
> 
> +1, 
> I've raise that repeatedly in the past, bootstrap should not create the management network but pick up the existing configuration and let the engine override later with it's own configuration if it differs , I'm glad that we finally get to that. 
> 
> > 
> > Currently it always creates a bridged network (even if the DC
> > requires a
> > non-bridged ovirtmgmt), it knows nothing about the defined MTU for
> > ovirtmgmt, it uses ping to guess on top of which device to build (and
> > thus requires Vdsm-to-Engine reverse connectivity), and is the sole
> > remaining user of the addNetwork/vdsm-store-net-conf scripts.
> > 
> > Suggested feature:
> > ==================
> > Bootstrap would avoid creating a management network. Instead, after
> > bootstrapping a host, Engine would send a getVdsCaps probe to the
> > installed host, receiving a complete picture of the network
> > configuration on the host. Among this picture is the device that
> > holds
> > the host's management IP address.
> > 
> > Engine would send setupNetwork command to generate ovirtmgmt with
> > details devised from this picture, and according to the DC definition
> > of
> > ovirtmgmt.  For example, if Vdsm reports:
> > 
> > - vlan bond4.3000 has the host's IP, configured to use dhcp.
> > - bond4 is comprises eth2 and eth3
> > - ovirtmgmt is defined as a VM network with MTU 9000
> > 
> > then Engine sends the likes of:
> >   setupNetworks(ovirtmgmt: {bridged=True, vlan=3000, iface=bond4,
> >                 bonding=bond4: {eth2,eth3}, MTU=9000)
> 
> Just one comment here, 
> In order to save time and confusion - if the ovirtmgmt is defined with default values meaning the user did not bother to touch it, let it pick up the VLAN configuration from the first host added in the Data Center. 
> 
> Otherwise, you may override the host VLAN and loose connectivity.
> 
> This will also solve the situation many users encounter today. 
> 1. The engine in on a host that actually has VLAN defined 
> 2. The ovirtmgmt network was not updated in the DC
> 3. A host, with VLAN already defined is added - everything works fine
> 4. Any number of hosts are now added, again everything seems to work fine. 
> 
> But, now try to use setupNetworks, and you'll find out that you can't do much on the interface that contains the ovirtmgmt since the definition does not match. You can't sync (Since this will remove the VLAN and cause connectivity lose) you can't add more networks on top since it already has non-VLAN network on top according to the DC definition, etc.
> 
> On the other hand you can't update the ovirtmgmt definition on the DC since there are clusters in the DC that use the network.
> 
> The only workaround not involving DB hack to change the VLAN on the network is to:
> 1. Create new DC 
> 2. Do not use the wizard that pops up to create your cluster.
> 3. Modify the ovirtmgmt network to have VLANs 
> 4. Now create a cluster and add your hosts.
> 
> If you insist on using the default DC and cluster then before adding the first host, create an additional DC and move the Default cluster over there. You may then change the network on the Default cluster and then move the Default cluster back
> 
> Both are ugly. And should be solved by the proposal above. 
> 
> We do something similar for the Default cluster CPU level, where we set the intial level based on the first host added to the cluster. 

I'm not sure what Engine has for Default cluster CPU level. But I have
reservation of the hysteresis in your proposal - after a host is added,
the DC cannot forget ovirtmgmt's vlan.

How about letting the admin edit ovirtmgmt's vlan in the DC level, thus
rendering all hosts out-of-sync. The the admin could manually, or
through a script, or in the future through a distributed operation, sync
all the hosts to the definition?

Dan.

> 
> > 
> > A call to setSafeNetConfig would wrap the network configuration up.
> > 
> > Currently, the host underegoes a reboot as the last step of
> > bootstrap.
> > This allows us to verify immediately if the host would be accessible
> > post-boot using its new network configuration. If we want to maintain
> > this, Engine would need to send a fenceNode request.
> > 
> > Benefits:
> > =========
> > - Simplified bootstrapping
> > - Simplified ovirt-node registration (similar ovirtmgmt-generation
> > logic lies there).
> > - Host installation ends with an ovirtmgmt network that matches DC
> >   definition (bridged-ness, mtu, vlan).
> > - vdsm-to-engine connectivity is not required.
> > 
> > Drawbacks:
> > ==========
> > - We need to implement new Engine logic for devising ovirtmgmt
> > definition out of getVdsCaps output.
> 
> That is not a drawback - it's just writing code. Drawback as I see it, is within the suggested feature on the final behaviour and code. 
> 
> > - ... your input is welcome here
> > 
> > Missing:
> > ========
> > A wiki feature page for this new behavior.
> > 
> > 
> > _______________________________________________
> > Arch mailing list
> > Arch at ovirt.org
> > http://lists.ovirt.org/mailman/listinfo/arch
> >