Design issue when using optional networks for administrative usages

[Dan Kenigsberg]

On Tue, Sep 10, 2013 at 10:08:46AM +0300, Livnat Peer wrote:
> On 09/09/2013 07:34 PM, Dan Kenigsberg wrote:
> > On Sun, Sep 08, 2013 at 05:30:20AM -0400, Mike Kolesnik wrote:
> >> Hi, 
> >>
> >> I would like to hear opinions about what I consider a design issue in oVirt. 
> >>
> >> First of all, a short description of the current situation in oVirt 3.3: 
> >> Network is a data-center level entity, representing a L2 broadcast domain. 
> >> Each network can be attached to one or more clusters, where the attachment can have several properties: 
> >> - Required/Optional - Does the network have to be on all hosts or not? 
> >> - Usages (administrative): 
> >> - Display network - used for the display traffic 
> >> - Migration network - used for the migration traffic 
> >>
> >> Now, what bothers me is the affinity between these two properties - if a network is defined "optional", can is be used for an "administrative" usage? 
> >>
> >> Currently I can have the following situation: 
> >> 0. Fresh install with some hosts and a shared storage, and no networks other than default. 
> >> 1. Create a network X. 
> >> 2. Attach to a cluster as "migration", "display", "optional". 
> >> 3. Create a VM in the same cluster. 
> >>
> >> Now all is well and everything is green across the board, BUT: 
> >> 1. The VM can't be run on any host in that cluster if the host doesn't have the display network. 
> >> 2. VM will migrate over the default network if the network is not present on the source host. 
> >> 3. Migration will not work if the network is not present on the destination host. 
> >>
> >> I find this situation very troublesome! 
> >> We give the admin the impression that everything is fine and dandy, but underneath the surface everything is NOT. 
> >>
> >> If we look at the previous points we can see that: 
> >> 1. No VM can run in that cluster, but hosts and network seem A-OK - this is intrinsically awful as we don't reflect the real problem anywhere in the network nor the host statuses but rather postpone it until someone makes an attempt to actually use the VM. 
> >> 2. Migration network is NOT being used, which was obviously not the intent of the admin who set it up. 
> >> 3. There is still an open bug for it ( https://bugzilla.redhat.com/983515 ) and it's unclear as to what should happen, but it would be either what happens in case #1 or in case #2. 
> >>
> >> What I suggest is to have any network with usage be "required". 
> >> This will utilize the existing logic for required networks: 
> >> - Either the network should not be used until its available on all hosts (reflected in the network status being Non-Operational) 
> >> - Or the host should be Non-Operational as it's incapable of running/migrating VMs 
> >>
> >> Therefore reflecting the problem to the admin and giving him a chance to fix it properly, and not hiding the failure until it occurs or doing some unexpected behavior. 
> >>
> >> I would love to hear your thoughts on the subject. 
> > 
> > Some history first. Once upon at time, we wanted an Up host to mean
> > "this host is ready to run any of its cluster's VMs". This meant that if
> > a host lost connectivity to one of the cluster networks, it had to be
> > taken down.
> > 
> > Customers did not like our over protection, so we've introduced
> > non-required networks. When an admin uses this option he says "I know
> > what I'm doing, let me do stuff on this host even if the network is
> > down."
> > 
> > I think that this request is a valid one, even when a network serves
> > other purposes than connecting VMs. When designing migration network,
> > we've decided that if it is missing, migration would be attempted over
> > the management network, as a fallback. I can imagine an admin who says:
> > I don't care much about migrations, most of my VMs are pinned-to-host
> > anyway. so if the migration network is gone, don't make a fuss out of
> > it.
> > 
> 
> I think this is a classic case of flexibility vs. simplicity.
> Usually I'm all for not over-protecting the user but looking on this
> specific case I think it make sense to have migration network as
> required network.
> 
> I think the motivation at the time for setting the migration network as
> optional was to be able to support different migration network to
> different tenets.
> I think this use case is mostly relevant for public cloud which is not
> our target user.
> 
> If a user does not want to use migration or want to use it on rare
> occasions then he does not have to define migration network, he can use
> the defaults which is the using the management network for migration.
> 
> The down side of having the migration network as optional is that if
> someone chooses that option by mistake (or not) he had to read long
> documentation which are complicated and not intuitive to understand what
> is the expected behavior.  (BTW only when I put it in words I realized
> how cumbersome the current behavior is)
> 
> > The use case for letting a host be Up even if its display network is
> > less obvious. But then again, I can think of an admin who uses a vdsm
> > hook to set the display IP of each VM. He does not care if the display
> > network is up or not.
> > 
> 
> I would not design behavior for the case someone is using hooks to
> override the behavior in oVirt.

We do not design a behavior for this case. We already had the behavior
specifically asked by users for VMs. I suggest, again, to be open for
more use cases.

If we see that users ask us to nanny them, and to limit their ability to
choose non-req networks, let's do it. Currently, I do not find the
danger as a real one.

Dan.