feature suggestion: initial generation of management network

[Alon Bar-Lev]

----- Original Message -----
> From: "Dan Kenigsberg" <danken at redhat.com>
> To: "Moti Asayag" <masayag at redhat.com>
> Cc: "arch" <arch at ovirt.org>
> Sent: Wednesday, May 8, 2013 4:35:49 PM
> Subject: Re: feature suggestion: initial generation of management network
> 
> On Tue, May 07, 2013 at 09:31:47AM -0400, Moti Asayag wrote:
> > 
> > 
> > ----- Original Message -----
> > > From: "Omer Frenkel" <ofrenkel at redhat.com>
> > > To: "Moti Asayag" <masayag at redhat.com>
> > > Cc: "arch" <arch at ovirt.org>, "Alon Bar-Lev" <abarlev at redhat.com>
> > > Sent: Tuesday, May 7, 2013 4:11:05 PM
> > > Subject: Re: feature suggestion: initial generation of management network
> > > 
> > > 
> > > 
> > > ----- Original Message -----
> > > > From: "Moti Asayag" <masayag at redhat.com>
> > > > To: "arch" <arch at ovirt.org>
> > > > Cc: "Alon Bar-Lev" <abarlev at redhat.com>
> > > > Sent: Tuesday, May 7, 2013 2:22:19 PM
> > > > Subject: Re: feature suggestion: initial generation of management
> > > > network
> > > > 
> > > > I stumbled upon few issues with the current design while implementing
> > > > it:
> > > > 
> > > > There seems to be a requirement to reboot the host after the
> > > > installation
> > > > is completed in order to assure the host is recoverable.
> > > > 
> > > > Therefore, the building blocks of the installation process of 3.3 are:
> > > > 1. host deploy which installs the host expect configuring its
> > > > management
> > > > network.
> > > > 2. SetupNetwork (and CommitNetworkChanges) - for creating the
> > > > management
> > > > network
> > > > on the host and persisting the network configuration.
> > > > 3. Reboot the host - This is a missing piece. (engine has FenceVds
> > > > command,
> > > > but it
> > > > requires the power management to be configured prior to the
> > > > installation
> > > > and
> > > > might
> > > > be irrelevant for hosts without PM.)
> > > > 
> > > > So, there are couple of issues here:
> > > > 1. How to reboot the host?
> > > > 1.1. By exposing new RebootNode verb in VDSM and invoking it from the
> > > > engine
> > > > 1.2. By opening ssh dialog to the host in order to execute the reboot
> > > > 
> > > 
> > > why not send a reboot flag to the CommitNetworkChanges which is sent
> > > anyway,
> > > one less call (or connection if you choose ssh) and easier to do.
> > > 
> > 
> > Adding a reboot parameter to the CommitNetworkChanges (setSafeNetworkConfig
> > on vdsm side)
> > exceeds its logical scope which is persisting the network changes.
> > 
> > Needless to say if such functionally will be required elsewhere, it
> > couldn't be
> > properly reused if implemented as part of that command.
> > 
> > Adding Dan to comment on this as well.
> 
> Yeah, a "reboot-after-me" flag defies my sense of cleanliness.

Yes.

> If reboot-after-initial-net-config is crucial, we would need to add a
> special verb for that (or use the fenceNode verb if available).
> 
> 
> However, I am not sure that this reboot is unavoidable.
> Originally the reboot had two important goal:
> - make sure that the updated kernel is running
> - make sure that the network, which we tweak during bootstrap, is
>   accessible after boot
> 
> Nowadays, the kernels does not change THAT often, for all ovirt can
> matter. running an oldish kernel is not the end of the world.
> 
> And with Moti's feature implemented, we no longer tweak net config
> blindly during boot. We use a well-define setupNetwork API, with a
> well-tested rollback mechanism.
> 
> The bottom line is, that in my opinion, reboot-after-install can be
> skipped these days.

I agree.

The current design of ovirt-host-deploy fully supports a deployment cycle without requiring a reboot.
We no longer update kernel command-line, nor performing changes without activating them at runtime.
The bridge setup was the one bit that was risky and could not be rolled back cleanly without re-implementation of large chunk of engine logic.

The risk of a deployed system (without bridge) to be unresponsive after reboot is minimum.

1. iptables rules are already active.
2. udev rules are active.
3. vdsm is up.

The major risk is basically if some dependency package was UPDATED during the installation of vdsm, while its service/consumer is already running, then we are running a host with the old software and there is a chance that after reboot with the new software the host will fail.

I think that the decision to reboot host should be delegated to administrator, adding vdsm verb to reboot is usable. This way admin will be able to take a host to maintenance mode and reboot, and we can add checkbox to the add host dialog '[x] reboot', rebooting the host at the end of the sequence. I think the default should be off.

> 
> > 
> > > > 2. When to perform the reboot?
> > > > 2.1. After host deploy, by utilizing the host deploy to perform the
> > > > reboot.
> > > > It requires to configure the network by the monitor when the host is
> > > > detected
> > > > by the engine,
> > > > detached from the installation flow. However it is a step toward the
> > > > non-persistent network feature
> > > > yet to be defined.
> > > > 2.2. After setupNetwork is done and network was configured and
> > > > persisted on
> > > > the host.
> > > > There is no special advantage from recoverable aspect, as setupNetwork
> > > > is
> > > > constantly
> > > > used to persist the network configuration (by the complementary
> > > > CommitNetworkChanges command).
> > > > In case and network configuration fails, VDSM will revert to the last
> > > > well
> > > > known configuration
> > > > - so connectivity with engine should be restored. Design wise, it fits
> > > > to
> > > > configure the management
> > > >  network as part of the installation sequence.
> > > > If the network configuration fails in this context, the host status
> > > > will be
> > > > set to "InstallFailed" rather than "NonOperational",
> > > > as might occur as a result of a failed setupNetwork command.
> > > > 
> > > > 
> > > > Your inputs are welcome.
> > > > 
> > > > Thanks,
> > > > Moti
> _______________________________________________
> Arch mailing list
> Arch at ovirt.org
> http://lists.ovirt.org/mailman/listinfo/arch
>