[Engine-devel] host cpu feature

[Yaniv Kaul]

On 12/05/2012 07:10 PM, Doron Fediuck wrote:
> ----- Original Message -----
>> From: "Yaniv Kaul" <ykaul at redhat.com>
>> To: "Doron Fediuck" <dfediuck at redhat.com>
>> Cc: "Laszlo Hornyak" <lhornyak at redhat.com>, "engine-devel" <engine-devel at ovirt.org>
>> Sent: Wednesday, December 5, 2012 6:48:33 PM
>> Subject: Re: [Engine-devel] host cpu feature
>>
>> On 12/05/2012 06:46 PM, Doron Fediuck wrote:
>>> ----- Original Message -----
>>>> From: "Laszlo Hornyak" <lhornyak at redhat.com>
>>>> To: "Doron Fediuck" <dfediuck at redhat.com>
>>>> Cc: "engine-devel" <engine-devel at ovirt.org>, "Yaniv Kaul"
>>>> <ykaul at redhat.com>
>>>> Sent: Wednesday, December 5, 2012 6:41:24 PM
>>>> Subject: Re: [Engine-devel] host cpu feature
>>>>
>>>>
>>>>
>>>> ----- Original Message -----
>>>>> From: "Doron Fediuck" <dfediuck at redhat.com>
>>>>> To: "Laszlo Hornyak" <lhornyak at redhat.com>
>>>>> Cc: "engine-devel" <engine-devel at ovirt.org>, "Yaniv Kaul"
>>>>> <ykaul at redhat.com>
>>>>> Sent: Wednesday, December 5, 2012 5:33:35 PM
>>>>> Subject: Re: [Engine-devel] host cpu feature
>>>>>
>>>>>
>>>>>
>>>>> ----- Original Message -----
>>>>>> From: "Laszlo Hornyak" <lhornyak at redhat.com>
>>>>>> To: "Yaniv Kaul" <ykaul at redhat.com>
>>>>>> Cc: "engine-devel" <engine-devel at ovirt.org>
>>>>>> Sent: Wednesday, December 5, 2012 5:24:59 PM
>>>>>> Subject: Re: [Engine-devel] host cpu feature
>>>>>>
>>>>>>
>>>>>> ----- Original Message -----
>>>>>>> From: "Yaniv Kaul" <ykaul at redhat.com>
>>>>>>> To: "Laszlo Hornyak" <lhornyak at redhat.com>
>>>>>>> Cc: "Dan Kenigsberg" <danken at redhat.com>, "engine-devel"
>>>>>>> <engine-devel at ovirt.org>
>>>>>>> Sent: Wednesday, December 5, 2012 4:10:13 PM
>>>>>>> Subject: Re: [Engine-devel] host cpu feature
>>>>>>>
>>>>>>> On 12/05/2012 04:59 PM, Laszlo Hornyak wrote:
>>>>>>>> ----- Original Message -----
>>>>>>>>> From: "Dan Kenigsberg" <danken at redhat.com>
>>>>>>>>> To: "Yaniv Kaul" <ykaul at redhat.com>
>>>>>>>>> Cc: "Laszlo Hornyak" <lhornyak at redhat.com>, "engine-devel"
>>>>>>>>> <engine-devel at ovirt.org>
>>>>>>>>> Sent: Wednesday, December 5, 2012 3:22:18 PM
>>>>>>>>> Subject: Re: [Engine-devel] host cpu feature
>>>>>>>>>
>>>>>>>>> On Wed, Dec 05, 2012 at 04:05:00PM +0200, Yaniv Kaul wrote:
>>>>>>>>>> On 12/05/2012 03:55 PM, Laszlo Hornyak wrote:
>>>>>>>>>>>>>> The nice thing about hostModel (unlike hostPassthrough)
>>>>>>>>>>>>>> is
>>>>>>>>>>>>>> that
>>>>>>>>>>>>>> once
>>>>>>>>>>>>>> we
>>>>>>>>>>>>>> created the VM we can migrate it to stronger hosts, and
>>>>>>>>>>>>>> back
>>>>>>>>>>>>>> to
>>>>>>>>>>>>>> the
>>>>>>>>>>>>>> original host. I suppose that it complicates the
>>>>>>>>>>>>>> scheduler.
>>>>>>>>>>>>> Yes with host-model you get the features that libvirt
>>>>>>>>>>>>> handles.
>>>>>>>>>>>>> In
>>>>>>>>>>>>> such cases the engine could decide, if you want this
>>>>>>>>>>>>> functionality. Well the scheduler architecture is just
>>>>>>>>>>>>> being
>>>>>>>>>>>>> reinvented.
>>>>>>>>>>>>>
>>>>>>>>>>>>> For the host-passthrough, I think the
>>>>>>>>>>>>> AllowMigrateCPUHost
>>>>>>>>>>>>> configuration option would be a simple decision for the
>>>>>>>>>>>>> administrator: set it to true if all hosts are uniform.
>>>>>>>>> If it is THAT simple, Engine could take this decision
>>>>>>>>> without
>>>>>>>>> human
>>>>>>>>> intervension.
>>>>>>>> Is there a way engine can figure out if the cpu-models in all
>>>>>>>> the
>>>>>>>> hosts are the same?
>>>>>>>> I mean even if some host flags are not handled by libvirt and
>>>>>>>> therefore vdsm and engine...
>>>>>>>> So I would really need that permission from the user.
>>>>>>>>
>>>>>>>>>>>>> If it is
>>>>>>>>>>>>> not set to true, then we will not allow migration of
>>>>>>>>>>>>> such
>>>>>>>>>>>>> VMs.
>>>>>>>>>>>> That's not what I understood from libvirt's
>>>>>>>>>>>> documentation.
>>>>>>>>>>>> I
>>>>>>>>>>> You may be right, could you send an URL to that point of
>>>>>>>>>>> the
>>>>>>>>>>> documentation or copy-paste?
>>>>>>>>>> The link I followed from your feature page:
>>>>>>>>>> http://libvirt.org/formatdomain.html#elementsCPU :
>>>>>>>>>>
>>>>>>>>>> host-model
>>>>>>>>>> The host-model mode is essentially a shortcut to copying
>>>>>>>>>> host
>>>>>>>>>> CPU
>>>>>>>>>> definition from capabilities XML into domain XML. Since the
>>>>>>>>>> CPU
>>>>>>>>>> definition is copied just before starting a domain, exactly
>>>>>>>>>> the
>>>>>>>>>> same
>>>>>>>>>> XML can be used on different hosts while still providing
>>>>>>>>>> the
>>>>>>>>>> best
>>>>>>>>>> guest CPU each host supports. Neither match attribute nor
>>>>>>>>>> any
>>>>>>>>>> feature elements can be used in this mode. Specifying CPU
>>>>>>>>>> model
>>>>>>>>>> is
>>>>>>>>>> not supported either, but model's fallback attribute may
>>>>>>>>>> still
>>>>>>>>>> be
>>>>>>>>>> used. Libvirt does not model every aspect of each CPU so
>>>>>>>>>> the
>>>>>>>>>> guest
>>>>>>>>>> CPU will not match the host CPU exactly. On the other hand,
>>>>>>>>>> the
>>>>>>>>>> ABI
>>>>>>>>>> provided to the guest is reproducible. During migration,
>>>>>>>>>> complete
>>>>>>>>>> CPU model definition is transferred to the destination host
>>>>>>>>>> so
>>>>>>>>>> the
>>>>>>>>>> migrated guest will see exactly the same CPU model even if
>>>>>>>>>> the
>>>>>>>>>> destination host contains more capable CPUs for the running
>>>>>>>>>> instance
>>>>>>>>>> of the guest; but shutting down and restarting the guest
>>>>>>>>>> may
>>>>>>>>>> present
>>>>>>>>>> different hardware to the guest according to the
>>>>>>>>>> capabilities
>>>>>>>>>> of
>>>>>>>>>> the
>>>>>>>>>> new host.
>>>>>>>>>> host-passthrough
>>>>>>>>>> With this mode, the CPU visible to the guest should be
>>>>>>>>>> exactly
>>>>>>>>>> the
>>>>>>>>>> same as the host CPU even in the aspects that libvirt does
>>>>>>>>>> not
>>>>>>>>>> understand. Though the downside of this mode is that the
>>>>>>>>>> guest
>>>>>>>>>> environment cannot be reproduced on different hardware.
>>>>>>>>>> Thus,
>>>>>>>>>> if
>>>>>>>>>> you
>>>>>>>>>> hit any bugs, you are on your own.
>>>>>>>>> That's exactly where AllowMigrateCPUHost fits well: when a
>>>>>>>>> user
>>>>>>>>> ticks
>>>>>>>>> this for a cluster he says "yeah, I like to be on my own."
>>>>>>>>>
>>>>>>>> cpu mode="host-passthrough" migration: I talked to the
>>>>>>>> libvirt
>>>>>>>> guys
>>>>>>>> and they said it is OK if the hardware and the software are
>>>>>>>> the
>>>>>>>> same, and it will work, but they would not recommend.
>>>>>>>>
>>>>>>>> So if they do not recommend it, I would drop this from the
>>>>>>>> feature
>>>>>>>> spec. Anyone against it?
>>>>>>>>
>>>>>>>> Laszlo
>>>>>>> I'm a bit against it. I don't see why it's that complicated:
>>>>>>> Allow migration -> use 'host-model'
>>>>>>> Do not allow -> use 'host-passthrough'.
>>>>>> So the actual cpu capabilities would depend not only on the
>>>>>> 'host
>>>>>> cpu' checkbox, but also on the 'pinned to host' checkbox. I
>>>>>> think
>>>>>> this logical trick would be both funny and confusing from the
>>>>>> user's
>>>>>> perspective.
>>>>>>
>>>>>>> The reason of why we need host-passthrough is that otherwise I
>>>>>>> suspect
>>>>>>> we depend on libvirt for newer features to be somehow exposed
>>>>>>> to
>>>>>>> the
>>>>>>> guest (not sure about it).
>>>>>> Yes, with other words: this is a tuning feature.
>>>>>>
>>>>> Let's keep it simple.
>>>>> 1. Please remove AllowMigrateCPUHost. No reason for us to do what
>>>>> libvirt is asking to void.
>>>> With pleasure :)
>>>>
>>>>> 2. host-passthrough should be available only for non migratable
>>>>> VMs.
>>>> Right.
>>>>
>>>> And no host-model support for now, right?
>>>>
>>> Right.
>>> In can later be added if we have a good reasoning for it.
>> Alternative idea, inspired by "Thus, if you hit any bugs, you are on
>> your own" (http://libvirt.org/formatdomain.html#elementsCPU wrt
>> 'host-passthrough'):
>> A config option to determine if we use host-model or
>> host-passthrough.
>> Y.
>>
> I do not think the engine should go to this level.

IMHO, it's not the engine, it's the user - the advanced user. But even 
an advanced user won't go and change VDSM's logic.
Lets hope that host-passthrough is well tested.
Y.

> ie- it can ask for passthrough as a feature, and the
> actual implementation is handled by vdsm.