feature suggestion: migration network

[Dan Kenigsberg]

On Thu, Jan 10, 2013 at 10:45:42AM +0800, Mark Wu wrote:
> On 01/08/2013 10:46 PM, Yaniv Kaul wrote:
> >On 08/01/13 15:04, Dan Kenigsberg wrote:
> >>There's talk about this for ages, so it's time to have proper discussion
> >>and a feature page about it: let us have a "migration" network role, and
> >>use such networks to carry migration data
> >>
> >>When Engine requests to migrate a VM from one node to another, the VM
> >>state (Bios, IO devices, RAM) is transferred over a TCP/IP connection
> >>that is opened from the source qemu process to the destination qemu.
> >>Currently, destination qemu listens for the incoming connection on the
> >>management IP address of the destination host. This has serious
> >>downsides: a "migration storm" may choke the destination's management
> >>interface; migration is plaintext and ovirtmgmt includes Engine which
> >>sits may sit the node cluster.
> >>
> >>With this feature, a cluster administrator may grant the "migration"
> >>role to one of the cluster networks. Engine would use that network's IP
> >>address on the destination host when it requests a migration of a VM.
> >>With proper network setup, migration data would be separated to that
> >>network.
> >>
> >>=== Benefit to oVirt ===
> >>* Users would be able to define and dedicate a separate network for
> >>   migration. Users that need quick migration would use nics with high
> >>   bandwidth. Users who want to cap the bandwidth consumed by migration
> >>   could define a migration network over nics with bandwidth limitation.
> >>* Migration data can be limited to a separate network, that has no
> >>   layer-2 access from Engine
> >>
> >>=== Vdsm ===
> >>The "migrate" verb should be extended with an additional parameter,
> >>specifying the address that the remote qemu process should listen on. A
> >>new argument is to be added to the currently-defined migration
> >>arguments:
> >>* vmId: UUID
> >>* dst: management address of destination host
> >>* dstparams: hibernation volumes definition
> >>* mode: migration/hibernation
> >>* method: rotten legacy
> >>* ''New'': migration uri, according to
> >>http://libvirt.org/html/libvirt-libvirt.html#virDomainMigrateToURI2
> >>such as tcp://<ip of migration network on remote node>
> >>
> >>=== Engine ===
> >>As usual, complexity lies here, and several changes are required:
> >>
> >>1. Network definition.
> >>1.1 A new network role - not unlike "display network" should be
> >>     added.Only one migration network should be defined on a cluster.
> >>1.2 If none is defined, the legacy "use ovirtmgmt for migration"
> >>     behavior would apply.
> >>1.3 A migration network is more likely to be a ''required'' network, but
> >>     a user may opt for non-required. He may face unpleasant
> >>surprises if he
> >>     wants to migrate his machine, but no candidate host has the network
> >>     available.
> >>1.4 The "migration" role can be granted or taken on-the-fly, when hosts
> >>     are active, as long as there are no currently-migrating VMs.
> >>
> >>2. Scheduler
> >>2.1 when deciding which host should be used for automatic
> >>     migration, take into account the existence and availability of the
> >>     migration network on the destination host.
> >>2.2 For manual migration, let user migrate a VM to a host with no
> >>     migration network - if the admin wants to keep jamming the
> >>     management network with migration traffic, let her.
> >>
> >>3. VdsBroker migration verb.
> >>3.1 For the a modern cluster level, with migration network defined on
> >>     the destination host, an additional ''miguri'' parameter
> >>should be added
> >>     to the "migrate" command
> >>
> >>_______________________________________________
> >>Arch mailing list
> >>Arch at ovirt.org
> >>http://lists.ovirt.org/mailman/listinfo/arch
> >
> >How is the authentication of the peers handled? Do we need a cert
> >per each source/destination logical interface?
> >Y.
> In my understanding, using a separate migration network doesn't
> change the current peers authentication.  We still use the URI
> ''qemu+tls://remoeHost/system' to connect the target libvirt service
> if ssl enabled,  and the remote host should be the ip address of
> management interface. But we can choose other interfaces except the
> manage interface to transport the migration data. We just change the
> migrateURI, so the current authentication mechanism should still
> work for this new feature.

vdsm-vdsm and libvirt-libvirt communication is authenticated, but I am
not sure at all that qemu-qemu communication is.

After qemu is sprung up on the destination with
    -incoming <some ip>:<some port> , anything with access to that
address could hijack the process. Our migrateURI starts with "tcp://"
with all the consequences of this. That a good reason to make sure
<some ip> has as limited access as possible.

But maybe I'm wrong here, and libvir-list can show me the light.

Dan.