Re: [ovirt-devel] Discussion VM CPU pinning and NUMA CPU pinning

From: "Chuan Liao (Jason Liao, HPservers-Core-OE-PSC)" <chuan.liao(a)hp.com&gt; To: devel(a)ovirt.org Cc: "Doron Fediuck" <dfediuck(a)redhat.com&gt;, "Martin Sivák" <msivak(a)redhat.com&gt;, "Juan Hernandez" <juan.hernandez(a)redhat.com&gt;, "Malini Rao" <mrao(a)redhat.com&gt;, "Chegu Vinod" <chegu_vinod(a)hp.com&gt;, "Xiao-Lei Shi (Bruce, HP Servers-PSC-CQ)" <xiao-lei.shi(a)hp.com&gt;, "Gilad Chaplik" <gchaplik(a)redhat.com&gt;, "Shang-Chun Liang (David Liang, HPservers-Core-OE-PSC)" <shangchun.liang(a)hp.com&gt; Sent: Monday, May 26, 2014 6:11:35 AM Subject: RE: Discussion VM CPU pinning and NUMA CPU pinning Hi All, After discussion with some community members, we are supposed to make some conclusion of this topic. Change API model object on api.xsd 1. remove NumaCPU and NumaCPUs from NumaNode and VirtualNumaNode. 2. add Core and Cores under the CPU to show the CPU topology details. 3. add the CPU under the NumaNode and VirtualNumaNode to show the Numa inside CPU topology details. The CPU model will defined like this: <xs:complexType name="CPU"> <xs:sequence> ... <xs:element name="cores" type="Cores" minOccurs="0"/> ... </xs:complexType> <xs:complexType name="Cores"> <xs:sequence> <xs:element ref="core" maxOccurs="unbounded"/> </xs:sequence> </xs:complexType> <xs:complexType name="Core"> <xs:attribute name="index" type="xs:int" use="required"/> <xs:attribute name="socket" type="xs:int" use="optional"/>

</xs:complexType> Change related backend implementation on NumaMapper and YAML defined files. We are appreciated that anyone could give us some suggestion and feedback before 2014/5/30. Best Regards, Jason Liao > -----Original Message----- > From: Liao, Chuan (Jason Liao, HPservers-Core-OE-PSC) > Sent: 2014年5月19日 10:34 > To: 'Gilad Chaplik'; Liang, Shang-Chun (David Liang, HPservers-Core-OE-PSC) > Cc: devel(a)ovirt.org; Doron Fediuck; Martin Sivák; Juan Hernandez; Malini > Rao; Vinod, > Chegu; Shi, Xiao-Lei (Bruce, HP Servers-PSC-CQ) > Subject: RE: Discussion VM CPU pinning and NUMA CPU pinning > > Hi Gilad, > > Thanks for you mentioned. I want clarify some details with you. > > 1. The NumaCPU model is under the NumaNode and VirutalNumaNode model to > show > one NUMA node's inside cpu list. ( BE model property: VdsNumaNode.getCpuIds > ) > hosts/{host:id}/numanodes/{numa:id}/numacpus/numacpu > [index="{numacpu.index}"] > vms/{vm:id}/numanodes/{vnuma:id}/numacpus/numacpu > [index="{numacpu.index}"] > Notice: This is a completed solution for showing not setting. > > 2. VM CPU pinning in restful API is a completed solution using > vm/{vm:id}/cpu/cputune/vcpupin model. > > 3. All the discussion is based on we want to do setting CPU pinning inside > VirtualNumaNode. > The whole solution should contain both UI model solution and restful API > solution > and backend solution. > In restful API solution, we already design a extent point on model NUMACPU > could support this. ( add an attribute cpu_set in NUMACPU model ) > > So, I am not sure about your options. > User already could do setting CPU pinning for VM now, do we need to do > setting CPU > pinning inside VirtualiNumaNode in restful API ? > > Best Regards, > Jason Liao > > > -----Original Message----- > > From: Gilad Chaplik [mailto:gchaplik@redhat.com] > > Sent: 2014年5月18日 22:22 > > To: Liao, Chuan (Jason Liao, HPservers-Core-OE-PSC); Liang, Shang-Chun > > (David Liang, > > HPservers-Core-OE-PSC) > > Cc: devel(a)ovirt.org; Doron Fediuck; Martin Sivák; Juan Hernandez; > > Malini Rao; Vinod, Chegu; Shi, Xiao-Lei (Bruce, HP Servers-PSC-CQ) > > Subject: Re: Discussion VM CPU pinning and NUMA CPU pinning > > > > ----- Original Message ----- > > > From: "Gilad Chaplik" <gchaplik(a)redhat.com&gt; > > > To: "Chuan Liao (Jason Liao, HPservers-Core-OE-PSC)" > > > <chuan.liao(a)hp.com&gt;, "Shang-Chun Liang (David Liang, > > > HPservers-Core-OE-PSC)" <shangchun.liang(a)hp.com&gt; > > > Cc: devel(a)ovirt.org, "Doron Fediuck" <dfediuck(a)redhat.com&gt;, "Martin > > > Sivák" > > <msivak(a)redhat.com&gt;, "Juan Hernandez" > > > <juan.hernandez(a)redhat.com&gt;, "Malini Rao" <mrao(a)redhat.com&gt;, "Chegu > > > Vinod" <chegu_vinod(a)hp.com&gt;, "Xiao-Lei Shi (Bruce, HP Servers-PSC-CQ)" > > > <xiao-lei.shi(a)hp.com&gt; > > > Sent: Sunday, May 18, 2014 5:21:32 PM > > > Subject: Re: Discussion VM CPU pinning and NUMA CPU pinning > > > > > > ----- Original Message ----- > > > > From: "Chuan Liao (Jason Liao, HPservers-Core-OE-PSC)" > > > > <chuan.liao(a)hp.com&gt; > > > > To: "Gilad Chaplik" <gchaplik(a)redhat.com&gt; > > > > Cc: devel(a)ovirt.org, "Doron Fediuck" <dfediuck(a)redhat.com&gt;, "Martin > > > > Sivák" > > > > <msivak(a)redhat.com&gt;, "Juan Hernandez" > > > > <juan.hernandez(a)redhat.com&gt;, "Malini Rao" <mrao(a)redhat.com&gt;, > > > > "Chegu > > Vinod" > > > > <chegu_vinod(a)hp.com&gt;, "Shang-Chun Liang (David Liang, > > > > HPservers-Core-OE-PSC)" <shangchun.liang(a)hp.com&gt;, "Xiao-Lei Shi > > > > (Bruce, HP Servers-PSC-CQ)" > > > > <xiao-lei.shi(a)hp.com&gt; > > > > Sent: Friday, May 16, 2014 5:30:34 AM > > > > Subject: RE: Discussion VM CPU pinning and NUMA CPU pinning > > > > > > > > Hi Gilad, > > > > > > > > The restful API model of NUMA CPU and CPU-pinning is named NUMACPU > > > > ( Under NumaNode, VirutalNumaNode ) Which is contain some > > > > attributes; index (xs:int): Numa CPU index or vNuma CPU index. > > > > pin_set (xs:string): Virtual NUMA CPU pin to host NUMA CPU set. ( > > > > only enabled on VirtualNumaNode ) > > > > > > I think that CPU-pinning new modeling should be coupled with NUMA. > > > > /s/should/shouldn't > > > > > also CPU is a Host sub-entity, so there shouldn't be NUMACPU. > > > > > > > > > > > BTW, Could you give us some update about UI layer solution of NUMA > > > > CPU-pinning configuration ? > > > > > > [adding David's response as well, and answering to both questions] > > > > > > > > Hi Gilad, > > > > Thanks for your reply. Did you submit the UI code. We need to know > > > > your UI model to consolidate your feedback. Meanwhile, we need to > > > > know the current cpu pining solution that include the UI code you > > > > are in charging. Then, we will see if we can specific to RESTful. > > > > > > > > Best Regards, > > > > David Liang > > > > > > As discussed, the CPU pinning modeling in GUI won't be handled in > > > first phase (maybe as a 3.5 bug later on). > > > The reason RESTful modeling is important now is obvious, GUI can > > > wait, so there's shouldn't be any coupling between GUI and RESTful > > > impl. > > > > > > Thanks, > > > Gilad. > > > > > > > > > > > Best Regards, > > > > Jason Liao > > > > > > > > > -----Original Message----- > > > > > From: Gilad Chaplik [mailto:gchaplik@redhat.com] > > > > > Sent: 2014年5月15日 20:07 > > > > > To: Liao, Chuan (Jason Liao, HPservers-Core-OE-PSC) > > > > > Cc: devel(a)ovirt.org; Doron Fediuck; Martin Sivák; Juan > > > > > Hernandez; Malini Rao; Vinod, Chegu; Liang, Shang-Chun (David > > > > > Liang, HPservers-Core-OE-PSC); Shi, Xiao-Lei (Bruce, HP > > > > > Servers-PSC-CQ) > > > > > Subject: Re: Discussion VM CPU pinning and NUMA CPU pinning > > > > > > > > > > Can you please be more specific on how to model RESTful API, > > > > > with regards to CPU and CPU-pinning. > > > > > > > > > > Thanks, > > > > > Gilad. > > > > > > > > > > > > > > > ----- Original Message ----- > > > > > > From: "Chuan Liao (Jason Liao, HPservers-Core-OE-PSC)" > > > > > > <chuan.liao(a)hp.com&gt; > > > > > > To: devel(a)ovirt.org > > > > > > Cc: "Doron Fediuck" <dfediuck(a)redhat.com&gt;, "Gilad Chaplik" > > > > > <gchaplik(a)redhat.com&gt;, "Martin Sivák" <msivak(a)redhat.com&gt;, > > > > > > "Juan Hernandez" <juan.hernandez(a)redhat.com&gt;, "Malini Rao" > > > > > <mrao(a)redhat.com&gt;, "Chegu Vinod" <chegu_vinod(a)hp.com&gt;, > > > > > > "Shang-Chun Liang (David Liang, HPservers-Core-OE-PSC)" > > > > > <shangchun.liang(a)hp.com&gt;, "Xiao-Lei Shi (Bruce, HP > > > > > > Servers-PSC-CQ)" <xiao-lei.shi(a)hp.com&gt; > > > > > > Sent: Thursday, May 15, 2014 2:50:23 PM > > > > > > Subject: Discussion VM CPU pinning and NUMA CPU pinning > > > > > > > > > > > > Hi All, > > > > > > > > > > > > Now we are working on the NUMA tune feature development for oVirt > > > > > > 3.5. > > > > > > This > > > > > > feature allow user to configure the vCPU pining according to > > > > > > host CPU/NUMA topology to get the best performance for created > > > > > > VM. But this will impact the current function of vCPU pining. > > > > > > Now, we are looking for a solution to consolidate the conflict. > > > > > > We want to start an open discussion about VM CPU pinning (the > > > > > > current design) and NUMA CPU pining (new feature we are > > > > > > working > > > > > > on) in developer list: > > > > > > > > > > > > Background: > > > > > > Concept: > > > > > > > > > > > > 1. VM CPU pinning: the exist feature in current oVirt, which > > > > > > allow > > > > > > user > > > > > > to configure the VM vCPU pinning over ovirt, user can > > > > > > configure vCPU pining independently without NUMA tune. > > > > > > > > > > > > 2. NUMA CPU pinning: allow user to configure the vCPU pining > > > > > > according > > > > > > to host NUMA topology to get the best performance for created > > > > > > VM > > > > > > > > > > > > 3. vNode: Virtual NUMA node ( User configured ) > > > > > > > > > > > > 4. pNode: host physical NUMA node ( Get capability from host > > > > > > ) > > > > > > Notice: > > > > > > > > > > > > 1. NUMA tuning feature and CPU pinning feature are > > > > > > individually in > > > > > > libvirt ( ovirt backend ). > > > > > > > > > > > > 2. User could configure VM CPU pining individually without > > > > > > NUMA > > > > > > tune > > > > > > setup. > > > > > > > > > > > > 3. When configuring NUMA tuning feature, user need to > > > > > > configure VM > > > > > > NUMA > > > > > > tuning ( vNode pinto pNode & tuning mode ) and VM CPU pinning > > > > > > ( NUMA included ) to get optimized VM performance, otherwise > > > > > > VM will have low performance. > > > > > > > > > > > > We have two proposal now for this issue. Please give us some > > > > > > comments and your feedback, thanks. > > > > > > > > > > > > Solution 1: > > > > > > GUI: > > > > > > Transform between the CPU pinning text and a structure which > > > > > > will be used in NUMA CPU pinning configure page. > > > > > > And then save the CPU pinning text to the current VM CPU > > > > > > pinning > > > > > > field of > > > > > > VM. > > > > > > Restful: > > > > > > Transform between the CPU pinning text and a structure which > > > > > > will be used in restful NUMA CPU pinning. > > > > > > And then save the CPU pinning text to the current VM CPU > > > > > > pinning field of VM. > > > > > > Broker: > > > > > > Remove temporary solution(See the current implementation as > > > > > > below) and follow previous cpupin configuration procedure. > > > > > > > > > > > > Solution 2: > > > > > > GUI: > > > > > > If current VM CPU pinning is configured, when user open NUMA > > > > > > CPU pinning configure page, he will get a warning message. If > > > > > > he continues to configure NUMA CPU pinning and save the data, > > > > > > the current VM CPU pinning configuration will be cleared. > > > > > > NUMA CPU pinning configured data will be saved as new > > > > > > structure individually without changing current VM CPU pinning > > > > > > configuration. > > > > > > Restful: > > > > > > Configure NUMA CPU pinning and then save the data with the new > > > > > > NUMA CPU pinning structure. > > > > > > Entity and Database: > > > > > > Individually NUMA CPU pinning entities and data structure. > > > > > > Broker: > > > > > > NUMA CPU pinning configuration will be first considered > > > > > > to > > > > > > use. > > > > > > If > > > > > > it's not configured, it will use the current VM CPU > > > > > > pinning. > > > > > > > > > > > > Solution 1 The CPU pining data is consistent but the code > > > > > > logic is very complex. > > > > > > Solution 2 have better adaptive and better data structure. > > > > > > This is the way we preferred. > > > > > > > > > > > > We are appreciated that anybody could give us your comments or > > > > > > the better solution you have. > > > > > > > > > > > > > > > > > > > > > > > > The below is the current implementation for your reference: > > > > > > VM CPU pinning feature > > > > > > GUI: > > > > > > User input vCPU pining configure data with formatted text. > > > > > > Restful: > > > > > > User configure CpuTune and VCpuPin model with mapper to CPU > > > > > > pinning text. > > > > > > Entity and Database: > > > > > > CPU pinning is String property. > > > > > > Broker: > > > > > > Generate structure of cputune ( libvirt format ) from CPU > > > > > > pining string property. > > > > > > NUMA tuning feature > > > > > > GUI: > > > > > > User can drag & drop virtual NUMA node to host NUMA node ( pin > > > > > > to or remove pin to ). > > > > > > Restful: > > > > > > Add/update/remove virtual NUMA node with property of pin to > > > > > > host NUMA node index. > > > > > > Design NUMACPU model under NUMA node for extend. > > > > > > Entity and Database: > > > > > > Individually NUMA node entities ( vNode extend pNode ) and > > > > > > store procedure. > > > > > > Broker: > > > > > > Generate structure of numatune, cpu/numa ( libvirt format ) > > > > > > from NUMA node entities. > > > > > > Temporary solution prevent Notice 3 > > > > > > Broker: > > > > > > Generate right structure of cputune ( libivirt format ) from > > > > > > NUMA node entities ( vNode pinto pNode ) > > > > > > Limitation: > > > > > > This structure of cputune will not get the best performance of > > > > > > vNode. > > > > > > > > > > > > Best Regards, > > > > > > Jason Liao > > > > > > > > > > > > > > > >