[Kimchi-devel] [RFC] Live migration backend + API

Fri Oct 16 11:40:07 UTC 2015

On 10/16/2015 08:15 AM, Walter Niklaus wrote:
>
>
> On 15.10.2015 21:18, Daniel Henrique Barboza wrote:
>> Hi,
>>
>> Live migration will be one of the new features for Kimchi 2.0 and 
>> I've been investigating it for a week or so. This is how I'm planning 
>> to implement it:
>>
>> - API:
>>
>> The API will be similar to the API of the 'cloneGuest', but with 
>> extra parameters:
>>
>> url : 'plugins/kimchi/vms/' +'vm_name' + "/migrate"
>> type: 'POST'
>> parameters:
>> * destination: the IP address of the destination host
>> * (optional, not sure if applicable) password: the password to log in 
>> the destination via ssh tunnel
>>
>> In my opinion we can start with a 'migrate' button in the options of 
>> the VM, which will open up a new window to insert the destination IP 
>> address. This of course thinking about the old UI. I believe we can 
>> do whatever it is done with the 'Clone' function in the case of the 
>> new UI, with this extra window.
>>
>> As the migration might take a while, we'll need a progress bar or 
>> something to demonstrate that the progress is still ongoing. Again, 
>> the same thing as clone does :)
> This progress bar may be a little bit tricky because depending on the 
> workload of the VM the amount of dirty pages could increase again at a 
> certain point. But from a user point of view I really would like to 
> get the reality, meaning to see regression as well in order to give me 
> the chance to react.

Note that by 'progress bar' I mean an UI indicator that will simply show 
that the process is still running. I do not believe that the API will 
provide us with a progress status like "27% completed".

>>
>> - backend and core functionality:
>>
>> This is where it gets tricky. Libvirt has *a lot* of ways to do a 
>> live migration between hosts. My approach, at least for this first 
>> attempt, will be focusing in a single way, implement it, stabilize it 
>> and then extend it to the other migration types, if necessary.
>>
> Makes a lot of sense.
>> Same thing goes for the preset up. The only way I managed to do a 
>> live migration was with a NFS storage pool in both hosts, same name 
>> and same NFS server. This shared storage is necessary because the 
>> live migration will *not* copy the disks of the VM, just the RAM 
>> memory. I had to share ssh keys between the hosts to allow for 
>> password less ssh sessions.
>>
> From what I remember there is a validation step before starting 
> migration where libvirt is checking if the resources of the VM are 
> available on the remote host as well. I suspect the check is that 
> simple to define and start the domain in listen mode on the remote 
> host by using the domain xml from the target. And I guess this one 
> would already fail if the artifacts specified in the domain xml are 
> not present.

At first the validations we're going to do will be simple (check if the 
VM is already being migrated, check if it's the same 
hypervisor/architecture at source and destination). We can add more 
validations when it's implemented, like trying to see if the destination 
host has the resources (pci devices, available RAM) to complete the 
migration.

> Live migration has an option to perform volume migration as well ( 
> it's "copy-storage-all" in virsh I think), but I have never tried it out.

Unfortunately this migration type was deprecated starting in RHEL 7:

https://access.redhat.com/solutions/60034

"Live migration with non-shared storage in RHEL6/RHEV3.* is a deprecated 
functionality, but under certain circunstances and with some 
limitations, it works fine. In RHEL7, this feature has been disabled."

>
> Are you planning to support also FCP attached storage ? I guess this 
> one could work:
>   - if both Hosts have access to the LUN
>   - the disk-id used in the domain definition is the same on source 
> and target:
>         - for example by using the multipathing id like 
> 36005076802810d504800000000002c1d
>         - because sda on source could point to a different disk as sda 
> on target.

Any shared storage will do. I gave the NFS example because it was the 
setup I've tested, but
I am sure it will work with LUN/iSCSI or other types.

The idea is that the VM disk volumes must be accessed by the same path 
in both source and destination. The way that this is implemented it's up 
to the user.

>> At first, I want to relieve this first implementation from this pre 
>> config, which will have to be done entirely by the user. If 
>> everything goes well, I can add a few automation in this process 
>> inside the backend before the 2.0 launch.
>>
>> Kimchi internals will be implemented by using async task facilities.
>>
>>
>>
>> Any questions? Comments?
>>
>>
> Are you planning to offer a "cancel live migration" API and task as 
> well ? I'm thinking of the situation where migration will not converge 
> due to high workload on the VM and the user may decide to stop it ?

Good idea. I'll see if it's viable to be in this first attempt.

> Is this migration going to be supported only from one Kimchi managed 
> host to another or could the target host be also a non Kimchi managed 
> host ?

In theory, if you have libvirt on both sides it would work, even without 
kimchi. The only reason I'm thinking about Kimchi to Kimchi at first is 
because of the facilities that the Kimchi Federation can provide to 
exchange user credentials.

> Are we planning to transfer also Kimchi-template data to the target 
> host in case this one is managed by Kimchi ?

That sounds interesting. If Kimchi has this support in the Federation 
already, why not?

Also, let me clarify that when I say 'first attempt' I am not talking 
about the final backend for the 2.0 release. I am hoping that I will be 
able to release at least 2 versions of this backend before the release. 
Ideas like cancelling an ongoing migration sounds useful and I hope we 
can get it done before 2.0.

Daniel

>>
>> Daniel
>>
>>
>> _______________________________________________
>> Kimchi-devel mailing list
>> Kimchi-devel at ovirt.org
>> http://lists.ovirt.org/mailman/listinfo/kimchi-devel
>>
>
> _______________________________________________
> Kimchi-devel mailing list
> Kimchi-devel at ovirt.org
> http://lists.ovirt.org/mailman/listinfo/kimchi-devel
>