feature suggestion: initial generation of management network
Alon Bar-Lev
alonbl at redhat.com
Thu May 9 20:24:26 UTC 2013
----- Original Message -----
> From: "Barak Azulay" <bazulay at redhat.com>
> To: "Alon Bar-Lev" <alonbl at redhat.com>
> Cc: "Livnat Peer" <lpeer at redhat.com>, "arch" <arch at ovirt.org>
> Sent: Thursday, May 9, 2013 10:59:59 PM
> Subject: Re: feature suggestion: initial generation of management network
>
>
>
> ----- Original Message -----
> > From: "Alon Bar-Lev" <alonbl at redhat.com>
> > To: "Barak Azulay" <bazulay at redhat.com>
> > Cc: "Livnat Peer" <lpeer at redhat.com>, "arch" <arch at ovirt.org>
> > Sent: Thursday, May 9, 2013 6:36:30 PM
> > Subject: Re: feature suggestion: initial generation of management network
> >
> >
> >
> > ----- Original Message -----
> > > From: "Barak Azulay" <bazulay at redhat.com>
> > > To: "Livnat Peer" <lpeer at redhat.com>
> > > Cc: "arch" <arch at ovirt.org>
> > > Sent: Thursday, May 9, 2013 6:12:38 PM
> > > Subject: Re: feature suggestion: initial generation of management network
> > >
> > > After reading the thread, top posting to give my perspective of the what
> > > I
> > > think would be the best approach:
> > >
> > > Some background:
> > > - The reboot at the end of host deployment came after issues in host
> > > deployment that were discovered after fencing/reboot (which had nothing
> > > to
> > > do with the reboot reason).
> > > - Host deployment takes care of of constructing the bridge (when no
> > > bonding
> > > exist on host).
> > >
> > > So Bottom line:
> > > - I think that both handling the network configuration and reboot should
> > > be
> > > removed from host deployment (I think we all agree on that)
> > > - host deployment should leave the VDSM up and running with everything
> > > configured (including the SSL and SSH keys), and listening to all
> > > networks.
> > > - About the reboot - I'm not sure we still need reboot (this is a
> > > question
> > > even without this discussion)
> > > - Anyway about how to reboot:
> > > - I don't think it should be done through a VDSM command
> > > - There is an open issue already with enabling a few stages of fencing
> > > with
> > > no PowerManagement module but based on SSH,
> > > So a new engine internal command should be introduced like
> > > (runSSHCmdOnVds ... and this command can have 2 variants ... one of
> > > them
> > > is reboot ... (the other is restart VDSM))
> > > - such command will not require a password as the SSH keys should
> > > already
> > > be in place
> >
> > I disagree.
> > After management agent (VDSM in our case) is installed, all communications
> > should be done via that agent.
>
> First I must say that the reboot may not be a must for host deploy, this is
> a legitimate point to discuss
>
> However on the other hand a reboot might still be needed for various reasons,
> If it was that simple we wouldn't have to:
> - use Power Management cards
> - build safety belts around services we depend on
>
> In addition the fact is that VDSM can turn not responsive due to many
> unexpected reasons, and we might still need the ability to force a host to
> reboot (and not through PM).
> On the other hand SSH is very reliable, used anyway by engine for various
> flows (deploy/upgrade/log collection), and for that reason will be
> configured anyway.
>
>
> > Using multi-protocol layers is not wise in this architecture.
>
> We already do, and as it looks like its here to stay - unless we remove the
> host deployment part from ovirt.
>
> > For example, if we switch the direction of engine->vdsm communications we
> > cannot use SSH.
>
> The current plans are to move to a bi directional communication transport on
> top of AMQP/TCP,
> See the work done by Saggi.M
>
> And even if we do - the SSH will still be required.
>
I think it would be mistake to require SSH or relay on communication direction (initiated by the engine for example).
Also the requirement of 'root' user at host is something that should be avoided.
If you do not trust your own agent, then we need to provide fail-safe mechanism for our own core functionality (such as reboot).
SSH is used now only to provision host, as you wrote:
1. Host-deploy.
2. ovirt-node upgrade.
Log collection is irrelevant, as it is not exactly part of the product, and can be easily replaced with any other method of log collections.
Both of the above are host provisioning, which can be easily automated out-side of ovirt domain.
2. Node upgrade - for sure... it can be done manually or via pxe, we provide this within ovirt-engine just for the sake of friendly.
1. host-deploy, can be done via external provisioning framework, such as foreman.
Introducing application dependency in SSH is new introduction, and should be considered carefully.
Thanks!
>
> >
> > >
> > > Thoughts/Implications on Host Life cycle:
> > > - The easiest approach will be to leave the host in Installing phase
> > > throughout this process (all under installVdsCommand), and eventually
> > > move
> > > it to reboot
> > > - If one skips reboot than assuming the cluster has more than one
> > > network,
> > > he
> > > host will imminently move to none-operational ( like today)
> > > - We can take a different approach and add a new status to the Host -
> > > PendingNetworkConfig, which could be executed automatically in the future
> > > with network-labels, ot today simply popup todo dialog in the UI.
> > >
> > >
> > > Thanks
> > > Barak Azulay
> > >
> > >
> > >
> > > ----- Original Message -----
> > > > From: "Livnat Peer" <lpeer at redhat.com>
> > > > To: "Dan Kenigsberg" <danken at redhat.com>, "Barak Azulay"
> > > > <bazulay at redhat.com>
> > > > Cc: "arch" <arch at ovirt.org>
> > > > Sent: Thursday, May 9, 2013 8:42:28 AM
> > > > Subject: Re: feature suggestion: initial generation of management
> > > > network
> > > >
> > > > On 05/08/2013 04:35 PM, Dan Kenigsberg wrote:
> > > > > On Tue, May 07, 2013 at 09:31:47AM -0400, Moti Asayag wrote:
> > > > >>
> > > > >>
> > > > >> ----- Original Message -----
> > > > >>> From: "Omer Frenkel" <ofrenkel at redhat.com>
> > > > >>> To: "Moti Asayag" <masayag at redhat.com>
> > > > >>> Cc: "arch" <arch at ovirt.org>, "Alon Bar-Lev" <abarlev at redhat.com>
> > > > >>> Sent: Tuesday, May 7, 2013 4:11:05 PM
> > > > >>> Subject: Re: feature suggestion: initial generation of management
> > > > >>> network
> > > > >>>
> > > > >>>
> > > > >>>
> > > > >>> ----- Original Message -----
> > > > >>>> From: "Moti Asayag" <masayag at redhat.com>
> > > > >>>> To: "arch" <arch at ovirt.org>
> > > > >>>> Cc: "Alon Bar-Lev" <abarlev at redhat.com>
> > > > >>>> Sent: Tuesday, May 7, 2013 2:22:19 PM
> > > > >>>> Subject: Re: feature suggestion: initial generation of management
> > > > >>>> network
> > > > >>>>
> > > > >>>> I stumbled upon few issues with the current design while
> > > > >>>> implementing
> > > > >>>> it:
> > > > >>>>
> > > > >>>> There seems to be a requirement to reboot the host after the
> > > > >>>> installation
> > > > >>>> is completed in order to assure the host is recoverable.
> > > > >>>>
> > > > >>>> Therefore, the building blocks of the installation process of 3.3
> > > > >>>> are:
> > > > >>>> 1. host deploy which installs the host expect configuring its
> > > > >>>> management
> > > > >>>> network.
> > > > >>>> 2. SetupNetwork (and CommitNetworkChanges) - for creating the
> > > > >>>> management
> > > > >>>> network
> > > > >>>> on the host and persisting the network configuration.
> > > > >>>> 3. Reboot the host - This is a missing piece. (engine has FenceVds
> > > > >>>> command,
> > > > >>>> but it
> > > > >>>> requires the power management to be configured prior to the
> > > > >>>> installation
> > > > >>>> and
> > > > >>>> might
> > > > >>>> be irrelevant for hosts without PM.)
> > > > >>>>
> > > > >>>> So, there are couple of issues here:
> > > > >>>> 1. How to reboot the host?
> > > > >>>> 1.1. By exposing new RebootNode verb in VDSM and invoking it from
> > > > >>>> the
> > > > >>>> engine
> > > > >>>> 1.2. By opening ssh dialog to the host in order to execute the
> > > > >>>> reboot
> > > > >>>>
> > > > >>>
> > > > >>> why not send a reboot flag to the CommitNetworkChanges which is
> > > > >>> sent
> > > > >>> anyway,
> > > > >>> one less call (or connection if you choose ssh) and easier to do.
> > > > >>>
> > > > >>
> > > > >> Adding a reboot parameter to the CommitNetworkChanges
> > > > >> (setSafeNetworkConfig on vdsm side)
> > > > >> exceeds its logical scope which is persisting the network changes.
> > > > >>
> > > > >> Needless to say if such functionally will be required elsewhere, it
> > > > >> couldn't be
> > > > >> properly reused if implemented as part of that command.
> > > > >>
> > > > >> Adding Dan to comment on this as well.
> > > > >
> > > > > Yeah, a "reboot-after-me" flag defies my sense of cleanliness.
> > > > > If reboot-after-initial-net-config is crucial, we would need to add a
> > > > > special verb for that (or use the fenceNode verb if available).
> > > > >
> > > >
> > > > +1
> > > >
> > > > >
> > > > > However, I am not sure that this reboot is unavoidable.
> > > > > Originally the reboot had two important goal:
> > > > > - make sure that the updated kernel is running
> > > > > - make sure that the network, which we tweak during bootstrap, is
> > > > > accessible after boot
> > > > >
> > > > > Nowadays, the kernels does not change THAT often, for all ovirt can
> > > > > matter. running an oldish kernel is not the end of the world.
> > > > >
> > > > > And with Moti's feature implemented, we no longer tweak net config
> > > > > blindly during boot. We use a well-define setupNetwork API, with a
> > > > > well-tested rollback mechanism.
> > > > >
> > > > > The bottom line is, that in my opinion, reboot-after-install can be
> > > > > skipped these days.
> > > > >
> > > >
> > > > Adding Barak to the thread as I think he had some concern about
> > > > removing
> > > > the reboot after install.
> > > >
> > > >
> > > > >>
> > > > >>>> 2. When to perform the reboot?
> > > > >>>> 2.1. After host deploy, by utilizing the host deploy to perform
> > > > >>>> the
> > > > >>>> reboot.
> > > > >>>> It requires to configure the network by the monitor when the host
> > > > >>>> is
> > > > >>>> detected
> > > > >>>> by the engine,
> > > > >>>> detached from the installation flow. However it is a step toward
> > > > >>>> the
> > > > >>>> non-persistent network feature
> > > > >>>> yet to be defined.
> > > > >>>> 2.2. After setupNetwork is done and network was configured and
> > > > >>>> persisted
> > > > >>>> on
> > > > >>>> the host.
> > > > >>>> There is no special advantage from recoverable aspect, as
> > > > >>>> setupNetwork
> > > > >>>> is
> > > > >>>> constantly
> > > > >>>> used to persist the network configuration (by the complementary
> > > > >>>> CommitNetworkChanges command).
> > > > >>>> In case and network configuration fails, VDSM will revert to the
> > > > >>>> last
> > > > >>>> well
> > > > >>>> known configuration
> > > > >>>> - so connectivity with engine should be restored. Design wise, it
> > > > >>>> fits
> > > > >>>> to
> > > > >>>> configure the management
> > > > >>>> network as part of the installation sequence.
> > > > >>>> If the network configuration fails in this context, the host
> > > > >>>> status
> > > > >>>> will
> > > > >>>> be
> > > > >>>> set to "InstallFailed" rather than "NonOperational",
> > > > >>>> as might occur as a result of a failed setupNetwork command.
> > > > >>>>
> > > > >>>>
> > > > >>>> Your inputs are welcome.
> > > > >>>>
> > > > >>>> Thanks,
> > > > >>>> Moti
> > > > > _______________________________________________
> > > > > Arch mailing list
> > > > > Arch at ovirt.org
> > > > > http://lists.ovirt.org/mailman/listinfo/arch
> > > > >
> > > >
> > > > _______________________________________________
> > > > Arch mailing list
> > > > Arch at ovirt.org
> > > > http://lists.ovirt.org/mailman/listinfo/arch
> > > >
> > > >
> > > >
> > > _______________________________________________
> > > Arch mailing list
> > > Arch at ovirt.org
> > > http://lists.ovirt.org/mailman/listinfo/arch
> > >
> >
>
More information about the Arch
mailing list