[ovirt-devel] Discussion VM CPU pinning and NUMA CPU pinning
Gilad Chaplik
gchaplik at redhat.com
Mon May 26 15:37:47 UTC 2014
----- Original Message -----
> From: "Chuan Liao (Jason Liao, HPservers-Core-OE-PSC)" <chuan.liao at hp.com>
> To: devel at ovirt.org
> Cc: "Doron Fediuck" <dfediuck at redhat.com>, "Martin Sivák" <msivak at redhat.com>, "Juan Hernandez"
> <juan.hernandez at redhat.com>, "Malini Rao" <mrao at redhat.com>, "Chegu Vinod" <chegu_vinod at hp.com>, "Xiao-Lei Shi
> (Bruce, HP Servers-PSC-CQ)" <xiao-lei.shi at hp.com>, "Gilad Chaplik" <gchaplik at redhat.com>, "Shang-Chun Liang (David
> Liang, HPservers-Core-OE-PSC)" <shangchun.liang at hp.com>
> Sent: Monday, May 26, 2014 6:11:35 AM
> Subject: RE: Discussion VM CPU pinning and NUMA CPU pinning
>
> Hi All,
>
> After discussion with some community members, we are supposed to make some
> conclusion of this topic.
>
> Change API model object on api.xsd
> 1. remove NumaCPU and NumaCPUs from NumaNode and VirtualNumaNode.
> 2. add Core and Cores under the CPU to show the CPU topology details.
> 3. add the CPU under the NumaNode and VirtualNumaNode to show the Numa inside
> CPU topology details.
>
> The CPU model will defined like this:
> <xs:complexType name="CPU">
> <xs:sequence>
> ...
> <xs:element name="cores" type="Cores" minOccurs="0"/>
> ...
> </xs:complexType>
>
> <xs:complexType name="Cores">
> <xs:sequence>
> <xs:element ref="core" maxOccurs="unbounded"/>
> </xs:sequence>
> </xs:complexType>
>
> <xs:complexType name="Core">
> <xs:attribute name="index" type="xs:int" use="required"/>
> <xs:attribute name="socket" type="xs:int" use="optional"/>
Not a cpu topology expert but isn't core contained within a socket? and the other way around?
but I guess this informative attribute is good enough.
I'm +1.
> </xs:complexType>
>
> Change related backend implementation on NumaMapper and YAML defined files.
>
> We are appreciated that anyone could give us some suggestion and feedback
> before 2014/5/30.
>
> Best Regards,
> Jason Liao
>
> > -----Original Message-----
> > From: Liao, Chuan (Jason Liao, HPservers-Core-OE-PSC)
> > Sent: 2014年5月19日 10:34
> > To: 'Gilad Chaplik'; Liang, Shang-Chun (David Liang, HPservers-Core-OE-PSC)
> > Cc: devel at ovirt.org; Doron Fediuck; Martin Sivák; Juan Hernandez; Malini
> > Rao; Vinod,
> > Chegu; Shi, Xiao-Lei (Bruce, HP Servers-PSC-CQ)
> > Subject: RE: Discussion VM CPU pinning and NUMA CPU pinning
> >
> > Hi Gilad,
> >
> > Thanks for you mentioned. I want clarify some details with you.
> >
> > 1. The NumaCPU model is under the NumaNode and VirutalNumaNode model to
> > show
> > one NUMA node's inside cpu list. ( BE model property: VdsNumaNode.getCpuIds
> > )
> > hosts/{host:id}/numanodes/{numa:id}/numacpus/numacpu
> > [index="{numacpu.index}"]
> > vms/{vm:id}/numanodes/{vnuma:id}/numacpus/numacpu
> > [index="{numacpu.index}"]
> > Notice: This is a completed solution for showing not setting.
> >
> > 2. VM CPU pinning in restful API is a completed solution using
> > vm/{vm:id}/cpu/cputune/vcpupin model.
> >
> > 3. All the discussion is based on we want to do setting CPU pinning inside
> > VirtualNumaNode.
> > The whole solution should contain both UI model solution and restful API
> > solution
> > and backend solution.
> > In restful API solution, we already design a extent point on model NUMACPU
> > could support this. ( add an attribute cpu_set in NUMACPU model )
> >
> > So, I am not sure about your options.
> > User already could do setting CPU pinning for VM now, do we need to do
> > setting CPU
> > pinning inside VirtualiNumaNode in restful API ?
> >
> > Best Regards,
> > Jason Liao
> >
> > > -----Original Message-----
> > > From: Gilad Chaplik [mailto:gchaplik at redhat.com]
> > > Sent: 2014年5月18日 22:22
> > > To: Liao, Chuan (Jason Liao, HPservers-Core-OE-PSC); Liang, Shang-Chun
> > > (David Liang,
> > > HPservers-Core-OE-PSC)
> > > Cc: devel at ovirt.org; Doron Fediuck; Martin Sivák; Juan Hernandez;
> > > Malini Rao; Vinod, Chegu; Shi, Xiao-Lei (Bruce, HP Servers-PSC-CQ)
> > > Subject: Re: Discussion VM CPU pinning and NUMA CPU pinning
> > >
> > > ----- Original Message -----
> > > > From: "Gilad Chaplik" <gchaplik at redhat.com>
> > > > To: "Chuan Liao (Jason Liao, HPservers-Core-OE-PSC)"
> > > > <chuan.liao at hp.com>, "Shang-Chun Liang (David Liang,
> > > > HPservers-Core-OE-PSC)" <shangchun.liang at hp.com>
> > > > Cc: devel at ovirt.org, "Doron Fediuck" <dfediuck at redhat.com>, "Martin
> > > > Sivák"
> > > <msivak at redhat.com>, "Juan Hernandez"
> > > > <juan.hernandez at redhat.com>, "Malini Rao" <mrao at redhat.com>, "Chegu
> > > > Vinod" <chegu_vinod at hp.com>, "Xiao-Lei Shi (Bruce, HP Servers-PSC-CQ)"
> > > > <xiao-lei.shi at hp.com>
> > > > Sent: Sunday, May 18, 2014 5:21:32 PM
> > > > Subject: Re: Discussion VM CPU pinning and NUMA CPU pinning
> > > >
> > > > ----- Original Message -----
> > > > > From: "Chuan Liao (Jason Liao, HPservers-Core-OE-PSC)"
> > > > > <chuan.liao at hp.com>
> > > > > To: "Gilad Chaplik" <gchaplik at redhat.com>
> > > > > Cc: devel at ovirt.org, "Doron Fediuck" <dfediuck at redhat.com>, "Martin
> > > > > Sivák"
> > > > > <msivak at redhat.com>, "Juan Hernandez"
> > > > > <juan.hernandez at redhat.com>, "Malini Rao" <mrao at redhat.com>,
> > > > > "Chegu
> > > Vinod"
> > > > > <chegu_vinod at hp.com>, "Shang-Chun Liang (David Liang,
> > > > > HPservers-Core-OE-PSC)" <shangchun.liang at hp.com>, "Xiao-Lei Shi
> > > > > (Bruce, HP Servers-PSC-CQ)"
> > > > > <xiao-lei.shi at hp.com>
> > > > > Sent: Friday, May 16, 2014 5:30:34 AM
> > > > > Subject: RE: Discussion VM CPU pinning and NUMA CPU pinning
> > > > >
> > > > > Hi Gilad,
> > > > >
> > > > > The restful API model of NUMA CPU and CPU-pinning is named NUMACPU
> > > > > ( Under NumaNode, VirutalNumaNode ) Which is contain some
> > > > > attributes; index (xs:int): Numa CPU index or vNuma CPU index.
> > > > > pin_set (xs:string): Virtual NUMA CPU pin to host NUMA CPU set. (
> > > > > only enabled on VirtualNumaNode )
> > > >
> > > > I think that CPU-pinning new modeling should be coupled with NUMA.
> > >
> > > /s/should/shouldn't
> > >
> > > > also CPU is a Host sub-entity, so there shouldn't be NUMACPU.
> > > >
> > > > >
> > > > > BTW, Could you give us some update about UI layer solution of NUMA
> > > > > CPU-pinning configuration ?
> > > >
> > > > [adding David's response as well, and answering to both questions]
> > > > >
> > > > > Hi Gilad,
> > > > > Thanks for your reply. Did you submit the UI code. We need to know
> > > > > your UI model to consolidate your feedback. Meanwhile, we need to
> > > > > know the current cpu pining solution that include the UI code you
> > > > > are in charging. Then, we will see if we can specific to RESTful.
> > > > >
> > > > > Best Regards,
> > > > > David Liang
> > > >
> > > > As discussed, the CPU pinning modeling in GUI won't be handled in
> > > > first phase (maybe as a 3.5 bug later on).
> > > > The reason RESTful modeling is important now is obvious, GUI can
> > > > wait, so there's shouldn't be any coupling between GUI and RESTful
> > > > impl.
> > > >
> > > > Thanks,
> > > > Gilad.
> > > >
> > > > >
> > > > > Best Regards,
> > > > > Jason Liao
> > > > >
> > > > > > -----Original Message-----
> > > > > > From: Gilad Chaplik [mailto:gchaplik at redhat.com]
> > > > > > Sent: 2014年5月15日 20:07
> > > > > > To: Liao, Chuan (Jason Liao, HPservers-Core-OE-PSC)
> > > > > > Cc: devel at ovirt.org; Doron Fediuck; Martin Sivák; Juan
> > > > > > Hernandez; Malini Rao; Vinod, Chegu; Liang, Shang-Chun (David
> > > > > > Liang, HPservers-Core-OE-PSC); Shi, Xiao-Lei (Bruce, HP
> > > > > > Servers-PSC-CQ)
> > > > > > Subject: Re: Discussion VM CPU pinning and NUMA CPU pinning
> > > > > >
> > > > > > Can you please be more specific on how to model RESTful API,
> > > > > > with regards to CPU and CPU-pinning.
> > > > > >
> > > > > > Thanks,
> > > > > > Gilad.
> > > > > >
> > > > > >
> > > > > > ----- Original Message -----
> > > > > > > From: "Chuan Liao (Jason Liao, HPservers-Core-OE-PSC)"
> > > > > > > <chuan.liao at hp.com>
> > > > > > > To: devel at ovirt.org
> > > > > > > Cc: "Doron Fediuck" <dfediuck at redhat.com>, "Gilad Chaplik"
> > > > > > <gchaplik at redhat.com>, "Martin Sivák" <msivak at redhat.com>,
> > > > > > > "Juan Hernandez" <juan.hernandez at redhat.com>, "Malini Rao"
> > > > > > <mrao at redhat.com>, "Chegu Vinod" <chegu_vinod at hp.com>,
> > > > > > > "Shang-Chun Liang (David Liang, HPservers-Core-OE-PSC)"
> > > > > > <shangchun.liang at hp.com>, "Xiao-Lei Shi (Bruce, HP
> > > > > > > Servers-PSC-CQ)" <xiao-lei.shi at hp.com>
> > > > > > > Sent: Thursday, May 15, 2014 2:50:23 PM
> > > > > > > Subject: Discussion VM CPU pinning and NUMA CPU pinning
> > > > > > >
> > > > > > > Hi All,
> > > > > > >
> > > > > > > Now we are working on the NUMA tune feature development for oVirt
> > > > > > > 3.5.
> > > > > > > This
> > > > > > > feature allow user to configure the vCPU pining according to
> > > > > > > host CPU/NUMA topology to get the best performance for created
> > > > > > > VM. But this will impact the current function of vCPU pining.
> > > > > > > Now, we are looking for a solution to consolidate the conflict.
> > > > > > > We want to start an open discussion about VM CPU pinning (the
> > > > > > > current design) and NUMA CPU pining (new feature we are
> > > > > > > working
> > > > > > > on) in developer list:
> > > > > > >
> > > > > > > Background:
> > > > > > > Concept:
> > > > > > >
> > > > > > > 1. VM CPU pinning: the exist feature in current oVirt, which
> > > > > > > allow
> > > > > > > user
> > > > > > > to configure the VM vCPU pinning over ovirt, user can
> > > > > > > configure vCPU pining independently without NUMA tune.
> > > > > > >
> > > > > > > 2. NUMA CPU pinning: allow user to configure the vCPU pining
> > > > > > > according
> > > > > > > to host NUMA topology to get the best performance for created
> > > > > > > VM
> > > > > > >
> > > > > > > 3. vNode: Virtual NUMA node ( User configured )
> > > > > > >
> > > > > > > 4. pNode: host physical NUMA node ( Get capability from host
> > > > > > > )
> > > > > > > Notice:
> > > > > > >
> > > > > > > 1. NUMA tuning feature and CPU pinning feature are
> > > > > > > individually in
> > > > > > > libvirt ( ovirt backend ).
> > > > > > >
> > > > > > > 2. User could configure VM CPU pining individually without
> > > > > > > NUMA
> > > > > > > tune
> > > > > > > setup.
> > > > > > >
> > > > > > > 3. When configuring NUMA tuning feature, user need to
> > > > > > > configure VM
> > > > > > > NUMA
> > > > > > > tuning ( vNode pinto pNode & tuning mode ) and VM CPU pinning
> > > > > > > ( NUMA included ) to get optimized VM performance, otherwise
> > > > > > > VM will have low performance.
> > > > > > >
> > > > > > > We have two proposal now for this issue. Please give us some
> > > > > > > comments and your feedback, thanks.
> > > > > > >
> > > > > > > Solution 1:
> > > > > > > GUI:
> > > > > > > Transform between the CPU pinning text and a structure which
> > > > > > > will be used in NUMA CPU pinning configure page.
> > > > > > > And then save the CPU pinning text to the current VM CPU
> > > > > > > pinning
> > > > > > > field of
> > > > > > > VM.
> > > > > > > Restful:
> > > > > > > Transform between the CPU pinning text and a structure which
> > > > > > > will be used in restful NUMA CPU pinning.
> > > > > > > And then save the CPU pinning text to the current VM CPU
> > > > > > > pinning field of VM.
> > > > > > > Broker:
> > > > > > > Remove temporary solution(See the current implementation as
> > > > > > > below) and follow previous cpupin configuration procedure.
> > > > > > >
> > > > > > > Solution 2:
> > > > > > > GUI:
> > > > > > > If current VM CPU pinning is configured, when user open NUMA
> > > > > > > CPU pinning configure page, he will get a warning message. If
> > > > > > > he continues to configure NUMA CPU pinning and save the data,
> > > > > > > the current VM CPU pinning configuration will be cleared.
> > > > > > > NUMA CPU pinning configured data will be saved as new
> > > > > > > structure individually without changing current VM CPU pinning
> > > > > > > configuration.
> > > > > > > Restful:
> > > > > > > Configure NUMA CPU pinning and then save the data with the new
> > > > > > > NUMA CPU pinning structure.
> > > > > > > Entity and Database:
> > > > > > > Individually NUMA CPU pinning entities and data structure.
> > > > > > > Broker:
> > > > > > > NUMA CPU pinning configuration will be first considered
> > > > > > > to
> > > > > > > use.
> > > > > > > If
> > > > > > > it's not configured, it will use the current VM CPU
> > > > > > > pinning.
> > > > > > >
> > > > > > > Solution 1 The CPU pining data is consistent but the code
> > > > > > > logic is very complex.
> > > > > > > Solution 2 have better adaptive and better data structure.
> > > > > > > This is the way we preferred.
> > > > > > >
> > > > > > > We are appreciated that anybody could give us your comments or
> > > > > > > the better solution you have.
> > > > > > >
> > > > > > >
> > > > > > >
> > > > > > > The below is the current implementation for your reference:
> > > > > > > VM CPU pinning feature
> > > > > > > GUI:
> > > > > > > User input vCPU pining configure data with formatted text.
> > > > > > > Restful:
> > > > > > > User configure CpuTune and VCpuPin model with mapper to CPU
> > > > > > > pinning text.
> > > > > > > Entity and Database:
> > > > > > > CPU pinning is String property.
> > > > > > > Broker:
> > > > > > > Generate structure of cputune ( libvirt format ) from CPU
> > > > > > > pining string property.
> > > > > > > NUMA tuning feature
> > > > > > > GUI:
> > > > > > > User can drag & drop virtual NUMA node to host NUMA node ( pin
> > > > > > > to or remove pin to ).
> > > > > > > Restful:
> > > > > > > Add/update/remove virtual NUMA node with property of pin to
> > > > > > > host NUMA node index.
> > > > > > > Design NUMACPU model under NUMA node for extend.
> > > > > > > Entity and Database:
> > > > > > > Individually NUMA node entities ( vNode extend pNode ) and
> > > > > > > store procedure.
> > > > > > > Broker:
> > > > > > > Generate structure of numatune, cpu/numa ( libvirt format )
> > > > > > > from NUMA node entities.
> > > > > > > Temporary solution prevent Notice 3
> > > > > > > Broker:
> > > > > > > Generate right structure of cputune ( libivirt format ) from
> > > > > > > NUMA node entities ( vNode pinto pNode )
> > > > > > > Limitation:
> > > > > > > This structure of cputune will not get the best performance of
> > > > > > > vNode.
> > > > > > >
> > > > > > > Best Regards,
> > > > > > > Jason Liao
> > > > > > >
> > > > > > >
> > > > >
>
More information about the Devel
mailing list