[Engine-devel] CPU Pinning @engine
Hi All, Currently the VDSM has a CPU pinning hook. We'd like to add better support of it into the engine itself. Here's a design draft to cover it: http://www.ovirt.org/wiki/Features/Design/cpu-pinning Please review and comment if needed. Thanks! -- /d Never say "OOPS!" always say "Ah, Interesting!"
On 05/17/2012 02:42 PM, Doron Fediuck wrote: > Hi All, > Currently the VDSM has a CPU pinning hook. > We'd like to add better support of it into the engine itself. > Here's a design draft to cover it: http://www.ovirt.org/wiki/Features/Design/cpu-pinning > > Please review and comment if needed. Hi, I have a number of comments: 1. Will the user have any indication for the maximum amount of vCPU's? 2. What happens if a VM is already configured with the same CPU pin info? Will this be shared? 3. Would it possible to extend this to limit the MHz of CPU that can be allocated to a VM? 5. If the VM has more than one core, will each core have to be mapped? 6. "similar to the existing hook Shahar wrote" - can you please put a link the the hook that Shahar wrote. :) This will be helpful for those of us unfamiliar with it. 7. Why will live migration not be supported? What is the gap to have this type of support? Thanks Gary > Thanks!
On 05/17/2012 04:24 PM, Gary Kotton wrote: > On 05/17/2012 02:42 PM, Doron Fediuck wrote: >> Hi All, >> Currently the VDSM has a CPU pinning hook. >> We'd like to add better support of it into the engine itself. >> Here's a design draft to cover it: >> http://www.ovirt.org/wiki/Features/Design/cpu-pinning >> >> Please review and comment if needed. > Hi, > I have a number of comments: > 1. Will the user have any indication for the maximum amount of vCPU's? > 2. What happens if a VM is already configured with the same CPU pin > info? Will this be shared? > 3. Would it possible to extend this to limit the MHz of CPU that can be > allocated to a VM? > 5. If the VM has more than one core, will each core have to be mapped? > 6. "similar to the existing hook Shahar wrote" - can you please put a > link the the hook that Shahar wrote. :) This will be helpful for those > of us unfamiliar with it. > 7. Why will live migration not be supported? What is the gap to have > this type of support? +1 on the above. In addition, it should be nice to reveal the host cpu topology so the user will be able to match the virtual one. Getting the hyperthread topology is important factor as well. Numa is not mentioned at all. It should be highly desirable to expose the numa topology and add numa pinning using numactl. Since you're refactoring this code, adding options for the new numda daemon would be important too. Another important factor is the ability to pin not only the vcpu threads but also to pin 'service threads' in the host, potential to other physical cpus. Examples for such are the qemu io thread, vhost threads, etc. Regards, Dor > Thanks > Gary >> Thanks! > > _______________________________________________ > Engine-devel mailing list > Engine-devel@ovirt.org > http://lists.ovirt.org/mailman/listinfo/engine-devel
----- Original Message ----- > From: "Dor Laor" <dlaor@redhat.com> > To: gkotton@redhat.com > Cc: engine-devel@ovirt.org, "Eduardo Habkost" <ehabkost@redhat.com>, "Gleb Natapov" <gleb@redhat.com> > Sent: Thursday, May 17, 2012 10:04:10 AM > Subject: Re: [Engine-devel] CPU Pinning @engine > > On 05/17/2012 04:24 PM, Gary Kotton wrote: > > On 05/17/2012 02:42 PM, Doron Fediuck wrote: > >> Hi All, > >> Currently the VDSM has a CPU pinning hook. > >> We'd like to add better support of it into the engine itself. > >> Here's a design draft to cover it: > >> http://www.ovirt.org/wiki/Features/Design/cpu-pinning > >> > >> Please review and comment if needed. > > Hi, > > I have a number of comments: > > 1. Will the user have any indication for the maximum amount of > > vCPU's? > > 2. What happens if a VM is already configured with the same CPU pin > > info? Will this be shared? Yes > > 3. Would it possible to extend this to limit the MHz of CPU that > > can be > > allocated to a VM? No, that's not supported today in KVM. In the future we should use cgroups to provide prioritization as part of the future SLA feature. > > 5. If the VM has more than one core, will each core have to be > > mapped? yes > > 6. "similar to the existing hook Shahar wrote" - can you please put > > a > > link the the hook that Shahar wrote. :) This will be helpful for > > those > > of us unfamiliar with it. http://gerrit.ovirt.org/gitweb?p=vdsm.git;a=tree;f=vdsm_hooks/pincpu;h=fd7a1a85912504d6b0cc8ae4e8123d71bf3c2741;hb=HEAD But basically it's the same syntax as libvirt uses today. > > 7. Why will live migration not be supported? What is the gap to > > have > > this type of support? There's been disagreement here. The argument against it is that if you have heterogeneous hardware then it might not work. eg. If I pin on vCPU 10-11 on host A and and host B only has 8 CPUs .... I'd argue that if an admin is doing the pinning then it's up to him to make sure he picks a sensible value > > +1 on the above. > > In addition, it should be nice to reveal the host cpu topology so the > user will be able to match the virtual one. > Getting the hyperthread topology is important factor as well. > > Numa is not mentioned at all. It should be highly desirable to expose > the numa topology and add numa pinning using numactl. > Since you're refactoring this code, adding options for the new numda > daemon would be important too. > > Another important factor is the ability to pin not only the vcpu > threads > but also to pin 'service threads' in the host, potential to other > physical cpus. Examples for such are the qemu io thread, vhost > threads, etc. > > Regards, > Dor > > > Thanks > > Gary > >> Thanks! > > > > _______________________________________________ > > Engine-devel mailing list > > Engine-devel@ovirt.org > > http://lists.ovirt.org/mailman/listinfo/engine-devel > > _______________________________________________ > Engine-devel mailing list > Engine-devel@ovirt.org > http://lists.ovirt.org/mailman/listinfo/engine-devel >
On 17/05/12 17:38, Andrew Cathrow wrote: > > > ----- Original Message ----- >> From: "Dor Laor" <dlaor@redhat.com> >> To: gkotton@redhat.com >> Cc: engine-devel@ovirt.org, "Eduardo Habkost" <ehabkost@redhat.com>, "Gleb Natapov" <gleb@redhat.com> >> Sent: Thursday, May 17, 2012 10:04:10 AM >> Subject: Re: [Engine-devel] CPU Pinning @engine >> >> On 05/17/2012 04:24 PM, Gary Kotton wrote: >>> On 05/17/2012 02:42 PM, Doron Fediuck wrote: >>>> Hi All, >>>> Currently the VDSM has a CPU pinning hook. >>>> We'd like to add better support of it into the engine itself. >>>> Here's a design draft to cover it: >>>> http://www.ovirt.org/wiki/Features/Design/cpu-pinning >>>> >>>> Please review and comment if needed. >>> Hi, >>> I have a number of comments: >>> 1. Will the user have any indication for the maximum amount of >>> vCPU's? > > >>> 2. What happens if a VM is already configured with the same CPU pin >>> info? Will this be shared? > > Yes > >>> 3. Would it possible to extend this to limit the MHz of CPU that >>> can be >>> allocated to a VM? > > No, that's not supported today in KVM. In the future we should use cgroups to provide prioritization as part of the future SLA feature. > > >>> 5. If the VM has more than one core, will each core have to be >>> mapped? > > yes > >>> 6. "similar to the existing hook Shahar wrote" - can you please put >>> a >>> link the the hook that Shahar wrote. :) This will be helpful for >>> those >>> of us unfamiliar with it. > > http://gerrit.ovirt.org/gitweb?p=vdsm.git;a=tree;f=vdsm_hooks/pincpu;h=fd7a1a85912504d6b0cc8ae4e8123d71bf3c2741;hb=HEAD > But basically it's the same syntax as libvirt uses today. > > > >>> 7. Why will live migration not be supported? What is the gap to >>> have >>> this type of support? > > There's been disagreement here. > The argument against it is that if you have heterogeneous hardware then it might not work. > eg. If I pin on vCPU 10-11 on host A and and host B only has 8 CPUs .... > I'd argue that if an admin is doing the pinning then it's up to him to make sure he picks a sensible value > >> >> +1 on the above. >> >> In addition, it should be nice to reveal the host cpu topology so the >> user will be able to match the virtual one. >> Getting the hyperthread topology is important factor as well. >> >> Numa is not mentioned at all. It should be highly desirable to expose >> the numa topology and add numa pinning using numactl. >> Since you're refactoring this code, adding options for the new numda >> daemon would be important too. >> >> Another important factor is the ability to pin not only the vcpu >> threads >> but also to pin 'service threads' in the host, potential to other >> physical cpus. Examples for such are the qemu io thread, vhost >> threads, etc. >> >> Regards, >> Dor >> >>> Thanks >>> Gary >>>> Thanks! >>> +1 on Andrew's answers. Just want to add a general comment- CPU tuning can evolve into something which is much bigger. I choose to start with a humble pinning support, which I know is already being used today using vdsm hook. Having it in the engine (even in a basic mode) will be more friendly and helpful to many users. Going forward we have a lot of grounds to cover in this area, including NUMA and other exciting features, which we'll gradually introduce as the community allows. -- /d "All computers wait at the same speed."
"Live migration will not be supported for such VM's." Migration will work on homogeneous clusters so this should not be enforced (not limited to VMs which are pinned to host) just give a warning. ----- Original Message -----
From: "Doron Fediuck" <dfediuck@redhat.com> To: engine-devel@ovirt.org Sent: Thursday, May 17, 2012 2:42:46 PM Subject: [Engine-devel] CPU Pinning @engine
Hi All, Currently the VDSM has a CPU pinning hook. We'd like to add better support of it into the engine itself. Here's a design draft to cover it: http://www.ovirt.org/wiki/Features/Design/cpu-pinning
Please review and comment if needed.
Thanks! --
/d
Never say "OOPS!" always say "Ah, Interesting!" _______________________________________________ Engine-devel mailing list Engine-devel@ovirt.org http://lists.ovirt.org/mailman/listinfo/engine-devel
On 17/05/12 20:28, Ayal Baron wrote:
"Live migration will not be supported for such VM's."
Migration will work on homogeneous clusters so this should not be enforced (not limited to VMs which are pinned to host) just give a warning.
I agree, but if we wish to ping vCPUx to pCPUy in host a, we cannot ensure to have the same in host b, since it may have pCPUy too busy, so performance will degrade. Also, I hope you saw my general comment about starting with a humble solution and gradually improving. In this context (auto)numa can help us, so we better do numa than handle migration for basic mode, risking performance issues.
----- Original Message -----
From: "Doron Fediuck" <dfediuck@redhat.com> To: engine-devel@ovirt.org Sent: Thursday, May 17, 2012 2:42:46 PM Subject: [Engine-devel] CPU Pinning @engine
Hi All, Currently the VDSM has a CPU pinning hook. We'd like to add better support of it into the engine itself. Here's a design draft to cover it: http://www.ovirt.org/wiki/Features/Design/cpu-pinning
Please review and comment if needed.
Thanks! --
/d
Never say "OOPS!" always say "Ah, Interesting!" _______________________________________________ Engine-devel mailing list Engine-devel@ovirt.org http://lists.ovirt.org/mailman/listinfo/engine-devel
_______________________________________________ Engine-devel mailing list Engine-devel@ovirt.org http://lists.ovirt.org/mailman/listinfo/engine-devel
-- /d "Common sense is not so common." --Voltaire, Dictionnaire Philosophique (1764)
On 05/17/2012 11:15 PM, Doron Fediuck wrote:
On 17/05/12 20:28, Ayal Baron wrote:
"Live migration will not be supported for such VM's."
Migration will work on homogeneous clusters so this should not be enforced (not limited to VMs which are pinned to host) just give a warning.
I agree, but if we wish to ping vCPUx to pCPUy in host a, we cannot ensure to have the same in host b, since it may have pCPUy too busy, so performance will degrade. Also, I hope you saw my general comment
Performance may degrade on any migration to loaded host regardless of pinning. There is not need to forbid migration. Instead, the SLA assurance policy should verify the dedicated resources and match the target w/ the guarantee and decide according to this.
about starting with a humble solution and gradually improving. In this context (auto)numa can help us, so we better do numa than handle migration for basic mode, risking performance issues.
----- Original Message -----
From: "Doron Fediuck"<dfediuck@redhat.com> To: engine-devel@ovirt.org Sent: Thursday, May 17, 2012 2:42:46 PM Subject: [Engine-devel] CPU Pinning @engine
Hi All, Currently the VDSM has a CPU pinning hook. We'd like to add better support of it into the engine itself. Here's a design draft to cover it: http://www.ovirt.org/wiki/Features/Design/cpu-pinning
Please review and comment if needed.
Thanks! --
/d
Never say "OOPS!" always say "Ah, Interesting!" _______________________________________________ Engine-devel mailing list Engine-devel@ovirt.org http://lists.ovirt.org/mailman/listinfo/engine-devel
_______________________________________________ Engine-devel mailing list Engine-devel@ovirt.org http://lists.ovirt.org/mailman/listinfo/engine-devel
On 20/05/12 21:41, Dor Laor wrote:
On 05/17/2012 11:15 PM, Doron Fediuck wrote:
On 17/05/12 20:28, Ayal Baron wrote:
"Live migration will not be supported for such VM's."
Migration will work on homogeneous clusters so this should not be enforced (not limited to VMs which are pinned to host) just give a warning.
I agree, but if we wish to ping vCPUx to pCPUy in host a, we cannot ensure to have the same in host b, since it may have pCPUy too busy, so performance will degrade. Also, I hope you saw my general comment
Performance may degrade on any migration to loaded host regardless of pinning.
There is not need to forbid migration. Instead, the SLA assurance policy should verify the dedicated resources and match the target w/ the guarantee and decide according to this.
Thanks for the input. We may start with migration blocked in a configurable manner, so people who wish migrate to work will simply set the relevant configuration key. As for looking for a matched CPU, will add it as p2.
about starting with a humble solution and gradually improving. In this context (auto)numa can help us, so we better do numa than handle migration for basic mode, risking performance issues.
----- Original Message -----
From: "Doron Fediuck"<dfediuck@redhat.com> To: engine-devel@ovirt.org Sent: Thursday, May 17, 2012 2:42:46 PM Subject: [Engine-devel] CPU Pinning @engine
Hi All, Currently the VDSM has a CPU pinning hook. We'd like to add better support of it into the engine itself. Here's a design draft to cover it: http://www.ovirt.org/wiki/Features/Design/cpu-pinning
Please review and comment if needed.
Thanks! --
/d
Never say "OOPS!" always say "Ah, Interesting!" _______________________________________________ Engine-devel mailing list Engine-devel@ovirt.org http://lists.ovirt.org/mailman/listinfo/engine-devel
_______________________________________________ Engine-devel mailing list Engine-devel@ovirt.org http://lists.ovirt.org/mailman/listinfo/engine-devel
-- /d Never say "OOPS!" always say "Ah, Interesting!"
----- Original Message -----
From: "Doron Fediuck" <dfediuck@redhat.com> To: dlaor@redhat.com Cc: engine-devel@ovirt.org Sent: Sunday, May 20, 2012 5:49:08 PM Subject: Re: [Engine-devel] CPU Pinning @engine
On 20/05/12 21:41, Dor Laor wrote:
On 05/17/2012 11:15 PM, Doron Fediuck wrote:
On 17/05/12 20:28, Ayal Baron wrote:
"Live migration will not be supported for such VM's."
Migration will work on homogeneous clusters so this should not be enforced (not limited to VMs which are pinned to host) just give a warning.
I agree, but if we wish to ping vCPUx to pCPUy in host a, we cannot ensure to have the same in host b, since it may have pCPUy too busy, so performance will degrade. Also, I hope you saw my general comment
Performance may degrade on any migration to loaded host regardless of pinning.
There is not need to forbid migration. Instead, the SLA assurance policy should verify the dedicated resources and match the target w/ the guarantee and decide according to this.
Thanks for the input. We may start with migration blocked in a configurable manner, so people who wish migrate to work will simply set the relevant configuration key. As for looking for a matched CPU, will add it as p2.
We shouldn't block migration If a user wants to not migrate a cpu pinned VM then let them use the non-migratable flag in the UI.
about starting with a humble solution and gradually improving. In this context (auto)numa can help us, so we better do numa than handle migration for basic mode, risking performance issues.
----- Original Message -----
From: "Doron Fediuck"<dfediuck@redhat.com> To: engine-devel@ovirt.org Sent: Thursday, May 17, 2012 2:42:46 PM Subject: [Engine-devel] CPU Pinning @engine
Hi All, Currently the VDSM has a CPU pinning hook. We'd like to add better support of it into the engine itself. Here's a design draft to cover it: http://www.ovirt.org/wiki/Features/Design/cpu-pinning
Please review and comment if needed.
Thanks! --
/d
Never say "OOPS!" always say "Ah, Interesting!" _______________________________________________ Engine-devel mailing list Engine-devel@ovirt.org http://lists.ovirt.org/mailman/listinfo/engine-devel
_______________________________________________ Engine-devel mailing list Engine-devel@ovirt.org http://lists.ovirt.org/mailman/listinfo/engine-devel
--
/d
Never say "OOPS!" always say "Ah, Interesting!" _______________________________________________ Engine-devel mailing list Engine-devel@ovirt.org http://lists.ovirt.org/mailman/listinfo/engine-devel
On 21/05/12 01:45, Andrew Cathrow wrote:
----- Original Message -----
From: "Doron Fediuck" <dfediuck@redhat.com> To: dlaor@redhat.com Cc: engine-devel@ovirt.org Sent: Sunday, May 20, 2012 5:49:08 PM Subject: Re: [Engine-devel] CPU Pinning @engine
On 20/05/12 21:41, Dor Laor wrote:
On 05/17/2012 11:15 PM, Doron Fediuck wrote:
On 17/05/12 20:28, Ayal Baron wrote:
"Live migration will not be supported for such VM's."
Migration will work on homogeneous clusters so this should not be enforced (not limited to VMs which are pinned to host) just give a warning.
I agree, but if we wish to ping vCPUx to pCPUy in host a, we cannot ensure to have the same in host b, since it may have pCPUy too busy, so performance will degrade. Also, I hope you saw my general comment
Performance may degrade on any migration to loaded host regardless of pinning.
There is not need to forbid migration. Instead, the SLA assurance policy should verify the dedicated resources and match the target w/ the guarantee and decide according to this.
Thanks for the input. We may start with migration blocked in a configurable manner, so people who wish migrate to work will simply set the relevant configuration key. As for looking for a matched CPU, will add it as p2.
We shouldn't block migration If a user wants to not migrate a cpu pinned VM then let them use the non-migratable flag in the UI.
We're not blocking migration, we're allowing pinning (for now) only for non-migrative VM's. For p2 we'll verify destination host has the relevant pinning capacity, which will allow pinning for migrative VM's as well.
about starting with a humble solution and gradually improving. In this context (auto)numa can help us, so we better do numa than handle migration for basic mode, risking performance issues.
----- Original Message -----
From: "Doron Fediuck"<dfediuck@redhat.com> To: engine-devel@ovirt.org Sent: Thursday, May 17, 2012 2:42:46 PM Subject: [Engine-devel] CPU Pinning @engine
Hi All, Currently the VDSM has a CPU pinning hook. We'd like to add better support of it into the engine itself. Here's a design draft to cover it: http://www.ovirt.org/wiki/Features/Design/cpu-pinning
Please review and comment if needed.
Thanks! --
/d
Never say "OOPS!" always say "Ah, Interesting!" _______________________________________________ Engine-devel mailing list Engine-devel@ovirt.org http://lists.ovirt.org/mailman/listinfo/engine-devel
_______________________________________________ Engine-devel mailing list Engine-devel@ovirt.org http://lists.ovirt.org/mailman/listinfo/engine-devel
--
/d
Never say "OOPS!" always say "Ah, Interesting!" _______________________________________________ Engine-devel mailing list Engine-devel@ovirt.org http://lists.ovirt.org/mailman/listinfo/engine-devel
-- /d "Forty-two," said Deep Thought, with infinite majesty and calm. --Douglas Adams, The Hitchhiker's Guide to the Galaxy
On 05/21/2012 10:58 AM, Doron Fediuck wrote:
On 21/05/12 01:45, Andrew Cathrow wrote:
----- Original Message -----
From: "Doron Fediuck"<dfediuck@redhat.com> To: dlaor@redhat.com Cc: engine-devel@ovirt.org Sent: Sunday, May 20, 2012 5:49:08 PM Subject: Re: [Engine-devel] CPU Pinning @engine
On 20/05/12 21:41, Dor Laor wrote:
On 05/17/2012 11:15 PM, Doron Fediuck wrote:
On 17/05/12 20:28, Ayal Baron wrote:
"Live migration will not be supported for such VM's."
Migration will work on homogeneous clusters so this should not be enforced (not limited to VMs which are pinned to host) just give a warning.
I agree, but if we wish to ping vCPUx to pCPUy in host a, we cannot ensure to have the same in host b, since it may have pCPUy too busy, so performance will degrade. Also, I hope you saw my general comment
Performance may degrade on any migration to loaded host regardless of pinning.
There is not need to forbid migration. Instead, the SLA assurance policy should verify the dedicated resources and match the target w/ the guarantee and decide according to this.
Thanks for the input. We may start with migration blocked in a configurable manner, so people who wish migrate to work will simply set the relevant configuration key. As for looking for a matched CPU, will add it as p2.
We shouldn't block migration If a user wants to not migrate a cpu pinned VM then let them use the non-migratable flag in the UI.
We're not blocking migration, we're allowing pinning (for now) only for non-migrative VM's. For p2 we'll verify destination host has the relevant pinning capacity, which will allow pinning for migrative VM's as well.
IMHO the order should be the opposite - if the user likes to migrate a pinned VM, let him (you can pop some message if you like).
about starting with a humble solution and gradually improving. In this context (auto)numa can help us, so we better do numa than handle migration for basic mode, risking performance issues.
----- Original Message ----- > From: "Doron Fediuck"<dfediuck@redhat.com> > To: engine-devel@ovirt.org > Sent: Thursday, May 17, 2012 2:42:46 PM > Subject: [Engine-devel] CPU Pinning @engine > > Hi All, > Currently the VDSM has a CPU pinning hook. > We'd like to add better support of it into the engine itself. > Here's a design draft to cover it: > http://www.ovirt.org/wiki/Features/Design/cpu-pinning > > Please review and comment if needed. > > Thanks! > -- > > /d > > Never say "OOPS!" always say "Ah, Interesting!" > _______________________________________________ > Engine-devel mailing list > Engine-devel@ovirt.org > http://lists.ovirt.org/mailman/listinfo/engine-devel > _______________________________________________ Engine-devel mailing list Engine-devel@ovirt.org http://lists.ovirt.org/mailman/listinfo/engine-devel
--
/d
Never say "OOPS!" always say "Ah, Interesting!" _______________________________________________ Engine-devel mailing list Engine-devel@ovirt.org http://lists.ovirt.org/mailman/listinfo/engine-devel
On 21/05/12 11:01, Dor Laor wrote:
On 05/21/2012 10:58 AM, Doron Fediuck wrote:
On 21/05/12 01:45, Andrew Cathrow wrote:
----- Original Message -----
From: "Doron Fediuck"<dfediuck@redhat.com> To: dlaor@redhat.com Cc: engine-devel@ovirt.org Sent: Sunday, May 20, 2012 5:49:08 PM Subject: Re: [Engine-devel] CPU Pinning @engine
On 20/05/12 21:41, Dor Laor wrote:
On 05/17/2012 11:15 PM, Doron Fediuck wrote:
On 17/05/12 20:28, Ayal Baron wrote: > "Live migration will not be supported for such VM's." > > Migration will work on homogeneous clusters so this should not be > enforced (not limited to VMs which are pinned to host) just give > a warning. > I agree, but if we wish to ping vCPUx to pCPUy in host a, we cannot ensure to have the same in host b, since it may have pCPUy too busy, so performance will degrade. Also, I hope you saw my general comment
Performance may degrade on any migration to loaded host regardless of pinning.
There is not need to forbid migration. Instead, the SLA assurance policy should verify the dedicated resources and match the target w/ the guarantee and decide according to this.
Thanks for the input. We may start with migration blocked in a configurable manner, so people who wish migrate to work will simply set the relevant configuration key. As for looking for a matched CPU, will add it as p2.
We shouldn't block migration If a user wants to not migrate a cpu pinned VM then let them use the non-migratable flag in the UI.
We're not blocking migration, we're allowing pinning (for now) only for non-migrative VM's. For p2 we'll verify destination host has the relevant pinning capacity, which will allow pinning for migrative VM's as well.
IMHO the order should be the opposite - if the user likes to migrate a pinned VM, let him (you can pop some message if you like). I'll check it with UI guys (AFAIK engine UI has no pop-ups). Also, I prefer safe than sorry. So start with a safe working pinning for pinned-to-host VMs, and then push the envelope.
about starting with a humble solution and gradually improving. In this context (auto)numa can help us, so we better do numa than handle migration for basic mode, risking performance issues.
> > ----- Original Message ----- >> From: "Doron Fediuck"<dfediuck@redhat.com> >> To: engine-devel@ovirt.org >> Sent: Thursday, May 17, 2012 2:42:46 PM >> Subject: [Engine-devel] CPU Pinning @engine >> >> Hi All, >> Currently the VDSM has a CPU pinning hook. >> We'd like to add better support of it into the engine itself. >> Here's a design draft to cover it: >> http://www.ovirt.org/wiki/Features/Design/cpu-pinning >> >> Please review and comment if needed. >> >> Thanks! >> -- >> >> /d >> >> Never say "OOPS!" always say "Ah, Interesting!" >> _______________________________________________ >> Engine-devel mailing list >> Engine-devel@ovirt.org >> http://lists.ovirt.org/mailman/listinfo/engine-devel >> > _______________________________________________ > Engine-devel mailing list > Engine-devel@ovirt.org > http://lists.ovirt.org/mailman/listinfo/engine-devel
--
/d
Never say "OOPS!" always say "Ah, Interesting!" _______________________________________________ Engine-devel mailing list Engine-devel@ovirt.org http://lists.ovirt.org/mailman/listinfo/engine-devel
-- /d "Air conditioned environment - Do NOT open Windows!"
On 05/21/2012 11:08 AM, Doron Fediuck wrote:
On 21/05/12 11:01, Dor Laor wrote:
On 05/21/2012 10:58 AM, Doron Fediuck wrote:
On 21/05/12 01:45, Andrew Cathrow wrote:
----- Original Message -----
From: "Doron Fediuck"<dfediuck@redhat.com> To: dlaor@redhat.com Cc: engine-devel@ovirt.org Sent: Sunday, May 20, 2012 5:49:08 PM Subject: Re: [Engine-devel] CPU Pinning @engine
On 20/05/12 21:41, Dor Laor wrote:
On 05/17/2012 11:15 PM, Doron Fediuck wrote: > On 17/05/12 20:28, Ayal Baron wrote: >> "Live migration will not be supported for such VM's." >> >> Migration will work on homogeneous clusters so this should not be >> enforced (not limited to VMs which are pinned to host) just give >> a warning. >> > I agree, but if we wish to ping vCPUx to pCPUy in host a, we > cannot ensure to have the same in host b, > since it may have pCPUy too busy, so performance will degrade. > Also, I hope you saw my general comment Performance may degrade on any migration to loaded host regardless of pinning.
There is not need to forbid migration. Instead, the SLA assurance policy should verify the dedicated resources and match the target w/ the guarantee and decide according to this.
Thanks for the input. We may start with migration blocked in a configurable manner, so people who wish migrate to work will simply set the relevant configuration key. As for looking for a matched CPU, will add it as p2. We shouldn't block migration If a user wants to not migrate a cpu pinned VM then let them use the non-migratable flag in the UI.
We're not blocking migration, we're allowing pinning (for now) only for non-migrative VM's. For p2 we'll verify destination host has the relevant pinning capacity, which will allow pinning for migrative VM's as well.
IMHO the order should be the opposite - if the user likes to migrate a pinned VM, let him (you can pop some message if you like). I'll check it with UI guys (AFAIK engine UI has no pop-ups). Also, I prefer safe than sorry. So start with a safe working pinning for pinned-to-host VMs, and then push the envelope.
Safe is letting the user migrate. It may or may not succeed (I'd worry it may put a host into Error state - I hope we've removed this state). Sorry is when you wish you had done so initially. ;-) Y.
> about starting with a humble solution and gradually improving. In > this context (auto)numa can help us, > so we better do numa than handle migration for basic mode, risking > performance issues. > >> ----- Original Message ----- >>> From: "Doron Fediuck"<dfediuck@redhat.com> >>> To: engine-devel@ovirt.org >>> Sent: Thursday, May 17, 2012 2:42:46 PM >>> Subject: [Engine-devel] CPU Pinning @engine >>> >>> Hi All, >>> Currently the VDSM has a CPU pinning hook. >>> We'd like to add better support of it into the engine itself. >>> Here's a design draft to cover it: >>> http://www.ovirt.org/wiki/Features/Design/cpu-pinning >>> >>> Please review and comment if needed. >>> >>> Thanks! >>> -- >>> >>> /d >>> >>> Never say "OOPS!" always say "Ah, Interesting!" >>> _______________________________________________ >>> Engine-devel mailing list >>> Engine-devel@ovirt.org >>> http://lists.ovirt.org/mailman/listinfo/engine-devel >>> >> _______________________________________________ >> Engine-devel mailing list >> Engine-devel@ovirt.org >> http://lists.ovirt.org/mailman/listinfo/engine-devel >
--
/d
Never say "OOPS!" always say "Ah, Interesting!" _______________________________________________ Engine-devel mailing list Engine-devel@ovirt.org http://lists.ovirt.org/mailman/listinfo/engine-devel
On 21/05/12 11:16, Yaniv Kaul wrote:
On 05/21/2012 11:08 AM, Doron Fediuck wrote:
On 21/05/12 11:01, Dor Laor wrote:
On 05/21/2012 10:58 AM, Doron Fediuck wrote:
On 21/05/12 01:45, Andrew Cathrow wrote:
----- Original Message -----
From: "Doron Fediuck"<dfediuck@redhat.com> To: dlaor@redhat.com Cc: engine-devel@ovirt.org Sent: Sunday, May 20, 2012 5:49:08 PM Subject: Re: [Engine-devel] CPU Pinning @engine
On 20/05/12 21:41, Dor Laor wrote: > On 05/17/2012 11:15 PM, Doron Fediuck wrote: >> On 17/05/12 20:28, Ayal Baron wrote: >>> "Live migration will not be supported for such VM's." >>> >>> Migration will work on homogeneous clusters so this should not be >>> enforced (not limited to VMs which are pinned to host) just give >>> a warning. >>> >> I agree, but if we wish to ping vCPUx to pCPUy in host a, we >> cannot ensure to have the same in host b, >> since it may have pCPUy too busy, so performance will degrade. >> Also, I hope you saw my general comment > Performance may degrade on any migration to loaded host regardless > of pinning. > > There is not need to forbid migration. Instead, the SLA assurance > policy should verify the dedicated resources and match the target > w/ the guarantee and decide according to this. > Thanks for the input. We may start with migration blocked in a configurable manner, so people who wish migrate to work will simply set the relevant configuration key. As for looking for a matched CPU, will add it as p2. We shouldn't block migration If a user wants to not migrate a cpu pinned VM then let them use the non-migratable flag in the UI.
We're not blocking migration, we're allowing pinning (for now) only for non-migrative VM's. For p2 we'll verify destination host has the relevant pinning capacity, which will allow pinning for migrative VM's as well.
IMHO the order should be the opposite - if the user likes to migrate a pinned VM, let him (you can pop some message if you like). I'll check it with UI guys (AFAIK engine UI has no pop-ups). Also, I prefer safe than sorry. So start with a safe working pinning for pinned-to-host VMs, and then push the envelope.
Safe is letting the user migrate. It may or may not succeed (I'd worry it may put a host into Error state - I hope we've removed this state). This is a pure speculation. If I'm pinning a specific VM to a specific host, and its vCPUS into specific pCPUs, how can live migration be considered a safe thing? If I pinned to host, this is a violation of my request. Now, let's say I didn't pin to host, but I ask for specific vCPUS pinning into specific pCPUs. For now, there's no guarantee the destination host will have a relevant CPU capacity (maybe even topology), so CPU pinning may not work, which is also a violation of my request. Is live migration really the safe thing to do here??? Why not alert me so I could manually migrate to where I think is better? Having that said, I'm reminding you the somewhere earlier in this thread I wrote we'll make the constraint of cpu pinning allowed only on pinned-to-host VMs configurable, so people who wish to walk on wild side, can give it a go.
Sorry is when you wish you had done so initially. ;-) In this case, I disagree as this is configurable.
Y.
>> about starting with a humble solution and gradually improving. In >> this context (auto)numa can help us, >> so we better do numa than handle migration for basic mode, risking >> performance issues. >> >>> ----- Original Message ----- >>>> From: "Doron Fediuck"<dfediuck@redhat.com> >>>> To: engine-devel@ovirt.org >>>> Sent: Thursday, May 17, 2012 2:42:46 PM >>>> Subject: [Engine-devel] CPU Pinning @engine >>>> >>>> Hi All, >>>> Currently the VDSM has a CPU pinning hook. >>>> We'd like to add better support of it into the engine itself. >>>> Here's a design draft to cover it: >>>> http://www.ovirt.org/wiki/Features/Design/cpu-pinning >>>> >>>> Please review and comment if needed. >>>> >>>> Thanks! >>>> -- >>>> >>>> /d >>>> >>>> Never say "OOPS!" always say "Ah, Interesting!" >>>> _______________________________________________ >>>> Engine-devel mailing list >>>> Engine-devel@ovirt.org >>>> http://lists.ovirt.org/mailman/listinfo/engine-devel >>>> >>> _______________________________________________ >>> Engine-devel mailing list >>> Engine-devel@ovirt.org >>> http://lists.ovirt.org/mailman/listinfo/engine-devel >>
--
/d
Never say "OOPS!" always say "Ah, Interesting!" _______________________________________________ Engine-devel mailing list Engine-devel@ovirt.org http://lists.ovirt.org/mailman/listinfo/engine-devel
-- /d Why doesn't DOS ever say "EXCELLENT command or filename!"
On 05/21/2012 11:53 AM, Doron Fediuck wrote:
On 21/05/12 11:16, Yaniv Kaul wrote:
On 05/21/2012 11:08 AM, Doron Fediuck wrote:
On 21/05/12 11:01, Dor Laor wrote:
On 05/21/2012 10:58 AM, Doron Fediuck wrote:
On 21/05/12 01:45, Andrew Cathrow wrote:
----- Original Message ----- > From: "Doron Fediuck"<dfediuck@redhat.com> > To: dlaor@redhat.com > Cc: engine-devel@ovirt.org > Sent: Sunday, May 20, 2012 5:49:08 PM > Subject: Re: [Engine-devel] CPU Pinning @engine > > On 20/05/12 21:41, Dor Laor wrote: >> On 05/17/2012 11:15 PM, Doron Fediuck wrote: >>> On 17/05/12 20:28, Ayal Baron wrote: >>>> "Live migration will not be supported for such VM's." >>>> >>>> Migration will work on homogeneous clusters so this should not be >>>> enforced (not limited to VMs which are pinned to host) just give >>>> a warning. >>>> >>> I agree, but if we wish to ping vCPUx to pCPUy in host a, we >>> cannot ensure to have the same in host b, >>> since it may have pCPUy too busy, so performance will degrade. >>> Also, I hope you saw my general comment >> Performance may degrade on any migration to loaded host regardless >> of pinning. >> >> There is not need to forbid migration. Instead, the SLA assurance >> policy should verify the dedicated resources and match the target >> w/ the guarantee and decide according to this. >> > Thanks for the input. > We may start with migration blocked in a configurable manner, so > people who wish migrate to work > will simply set the relevant configuration key. > As for looking for a matched CPU, will add it as p2. We shouldn't block migration If a user wants to not migrate a cpu pinned VM then let them use the non-migratable flag in the UI.
We're not blocking migration, we're allowing pinning (for now) only for non-migrative VM's. For p2 we'll verify destination host has the relevant pinning capacity, which will allow pinning for migrative VM's as well. IMHO the order should be the opposite - if the user likes to migrate a pinned VM, let him (you can pop some message if you like). I'll check it with UI guys (AFAIK engine UI has no pop-ups). Also, I prefer safe than sorry. So start with a safe working pinning for pinned-to-host VMs, and then push the envelope. Safe is letting the user migrate. It may or may not succeed (I'd worry it may put a host into Error state - I hope we've removed this state). This is a pure speculation. If I'm pinning a specific VM to a specific host, and its vCPUS into specific pCPUs, how can live migration be considered a safe thing? If I pinned to host, this is a violation of my request. Now, let's say I didn't pin to host, but I ask for specific vCPUS pinning into specific pCPUs. For now, there's no guarantee the destination host will have a relevant CPU capacity (maybe even topology), so CPU pinning may not work, which is also a violation of my request. Is live migration really the safe thing to do here??? Why not alert me so I could manually migrate to where I think is better? Having that said, I'm reminding you the somewhere earlier in this thread I wrote we'll make the constraint of cpu pinning allowed only on pinned-to-host VMs configurable, so people who wish to walk on wild side, can give it a go.
I think the consensus of the thread is that you should allow pin vCPUs to pCPUs even if the VM is not pinned to host. If the user wants to ensure it will not migrate, he should mark the VM as non migratable. Otherwise, it may migrate, and will succeed - if it can meet the same pinning requirements on the destination, or fail - and then no harm done. If migration succeeds without meeting the pinning requirements on the destination, I see this as a libvirt bug. Y.
Sorry is when you wish you had done so initially. ;-) In this case, I disagree as this is configurable.
Y.
>>> about starting with a humble solution and gradually improving. In >>> this context (auto)numa can help us, >>> so we better do numa than handle migration for basic mode, risking >>> performance issues. >>> >>>> ----- Original Message ----- >>>>> From: "Doron Fediuck"<dfediuck@redhat.com> >>>>> To: engine-devel@ovirt.org >>>>> Sent: Thursday, May 17, 2012 2:42:46 PM >>>>> Subject: [Engine-devel] CPU Pinning @engine >>>>> >>>>> Hi All, >>>>> Currently the VDSM has a CPU pinning hook. >>>>> We'd like to add better support of it into the engine itself. >>>>> Here's a design draft to cover it: >>>>> http://www.ovirt.org/wiki/Features/Design/cpu-pinning >>>>> >>>>> Please review and comment if needed. >>>>> >>>>> Thanks! >>>>> -- >>>>> >>>>> /d >>>>> >>>>> Never say "OOPS!" always say "Ah, Interesting!" >>>>> _______________________________________________ >>>>> Engine-devel mailing list >>>>> Engine-devel@ovirt.org >>>>> http://lists.ovirt.org/mailman/listinfo/engine-devel >>>>> >>>> _______________________________________________ >>>> Engine-devel mailing list >>>> Engine-devel@ovirt.org >>>> http://lists.ovirt.org/mailman/listinfo/engine-devel > -- > > /d > > Never say "OOPS!" always say "Ah, Interesting!" > _______________________________________________ > Engine-devel mailing list > Engine-devel@ovirt.org > http://lists.ovirt.org/mailman/listinfo/engine-devel >
On 21/05/12 12:18, Yaniv Kaul wrote:
On 05/21/2012 11:53 AM, Doron Fediuck wrote:
On 21/05/12 11:16, Yaniv Kaul wrote:
On 05/21/2012 11:08 AM, Doron Fediuck wrote:
On 21/05/12 11:01, Dor Laor wrote:
On 05/21/2012 10:58 AM, Doron Fediuck wrote:
On 21/05/12 01:45, Andrew Cathrow wrote: > ----- Original Message ----- >> From: "Doron Fediuck"<dfediuck@redhat.com> >> To: dlaor@redhat.com >> Cc: engine-devel@ovirt.org >> Sent: Sunday, May 20, 2012 5:49:08 PM >> Subject: Re: [Engine-devel] CPU Pinning @engine >> >> On 20/05/12 21:41, Dor Laor wrote: >>> On 05/17/2012 11:15 PM, Doron Fediuck wrote: >>>> On 17/05/12 20:28, Ayal Baron wrote: >>>>> "Live migration will not be supported for such VM's." >>>>> >>>>> Migration will work on homogeneous clusters so this should not be >>>>> enforced (not limited to VMs which are pinned to host) just give >>>>> a warning. >>>>> >>>> I agree, but if we wish to ping vCPUx to pCPUy in host a, we >>>> cannot ensure to have the same in host b, >>>> since it may have pCPUy too busy, so performance will degrade. >>>> Also, I hope you saw my general comment >>> Performance may degrade on any migration to loaded host regardless >>> of pinning. >>> >>> There is not need to forbid migration. Instead, the SLA assurance >>> policy should verify the dedicated resources and match the target >>> w/ the guarantee and decide according to this. >>> >> Thanks for the input. >> We may start with migration blocked in a configurable manner, so >> people who wish migrate to work >> will simply set the relevant configuration key. >> As for looking for a matched CPU, will add it as p2. > We shouldn't block migration > If a user wants to not migrate a cpu pinned VM then let them use the non-migratable flag in the UI. > We're not blocking migration, we're allowing pinning (for now) only for non-migrative VM's. For p2 we'll verify destination host has the relevant pinning capacity, which will allow pinning for migrative VM's as well. IMHO the order should be the opposite - if the user likes to migrate a pinned VM, let him (you can pop some message if you like). I'll check it with UI guys (AFAIK engine UI has no pop-ups). Also, I prefer safe than sorry. So start with a safe working pinning for pinned-to-host VMs, and then push the envelope. Safe is letting the user migrate. It may or may not succeed (I'd worry it may put a host into Error state - I hope we've removed this state). This is a pure speculation. If I'm pinning a specific VM to a specific host, and its vCPUS into specific pCPUs, how can live migration be considered a safe thing? If I pinned to host, this is a violation of my request. Now, let's say I didn't pin to host, but I ask for specific vCPUS pinning into specific pCPUs. For now, there's no guarantee the destination host will have a relevant CPU capacity (maybe even topology), so CPU pinning may not work, which is also a violation of my request. Is live migration really the safe thing to do here??? Why not alert me so I could manually migrate to where I think is better? Having that said, I'm reminding you the somewhere earlier in this thread I wrote we'll make the constraint of cpu pinning allowed only on pinned-to-host VMs configurable, so people who wish to walk on wild side, can give it a go.
I think the consensus of the thread is that you should allow pin vCPUs to pCPUs even if the VM is not pinned to host. If the user wants to ensure it will not migrate, he should mark the VM as non migratable. Otherwise, it may migrate, and will succeed - if it can meet the same pinning requirements on the destination, or fail - and then no harm done. If migration succeeds without meeting the pinning requirements on the destination, I see this as a libvirt bug. Y. Basically I already said cpu-pin is allowed for migrative VM's once configuring it. So basically what you're saying is that if such migration works, while loosing cpu-pin, it's a libvirt bug. I can live with that. I hope Daniel agrees with you.
Sorry is when you wish you had done so initially. ;-) In this case, I disagree as this is configurable.
Y.
>>>> about starting with a humble solution and gradually improving. In >>>> this context (auto)numa can help us, >>>> so we better do numa than handle migration for basic mode, risking >>>> performance issues. >>>> >>>>> ----- Original Message ----- >>>>>> From: "Doron Fediuck"<dfediuck@redhat.com> >>>>>> To: engine-devel@ovirt.org >>>>>> Sent: Thursday, May 17, 2012 2:42:46 PM >>>>>> Subject: [Engine-devel] CPU Pinning @engine >>>>>> >>>>>> Hi All, >>>>>> Currently the VDSM has a CPU pinning hook. >>>>>> We'd like to add better support of it into the engine itself. >>>>>> Here's a design draft to cover it: >>>>>> http://www.ovirt.org/wiki/Features/Design/cpu-pinning >>>>>> >>>>>> Please review and comment if needed. >>>>>> >>>>>> Thanks! >>>>>> -- >>>>>> >>>>>> /d >>>>>> >>>>>> Never say "OOPS!" always say "Ah, Interesting!" >>>>>> _______________________________________________ >>>>>> Engine-devel mailing list >>>>>> Engine-devel@ovirt.org >>>>>> http://lists.ovirt.org/mailman/listinfo/engine-devel >>>>>> >>>>> _______________________________________________ >>>>> Engine-devel mailing list >>>>> Engine-devel@ovirt.org >>>>> http://lists.ovirt.org/mailman/listinfo/engine-devel >> -- >> >> /d >> >> Never say "OOPS!" always say "Ah, Interesting!" >> _______________________________________________ >> Engine-devel mailing list >> Engine-devel@ovirt.org >> http://lists.ovirt.org/mailman/listinfo/engine-devel >>
-- /d "The answer, my friend, is blowing in the wind" --Bob Dylan, Blowin' in the Wind (1963)
On 21/05/12 12:40, Doron Fediuck wrote:
On 21/05/12 12:18, Yaniv Kaul wrote:
On 05/21/2012 11:53 AM, Doron Fediuck wrote:
On 21/05/12 11:16, Yaniv Kaul wrote:
On 05/21/2012 11:08 AM, Doron Fediuck wrote:
On 21/05/12 11:01, Dor Laor wrote:
On 05/21/2012 10:58 AM, Doron Fediuck wrote: > On 21/05/12 01:45, Andrew Cathrow wrote: >> ----- Original Message ----- >>> From: "Doron Fediuck"<dfediuck@redhat.com> >>> To: dlaor@redhat.com >>> Cc: engine-devel@ovirt.org >>> Sent: Sunday, May 20, 2012 5:49:08 PM >>> Subject: Re: [Engine-devel] CPU Pinning @engine >>> >>> On 20/05/12 21:41, Dor Laor wrote: >>>> On 05/17/2012 11:15 PM, Doron Fediuck wrote: >>>>> On 17/05/12 20:28, Ayal Baron wrote: >>>>>> "Live migration will not be supported for such VM's." >>>>>> >>>>>> Migration will work on homogeneous clusters so this should not be >>>>>> enforced (not limited to VMs which are pinned to host) just give >>>>>> a warning. >>>>>> >>>>> I agree, but if we wish to ping vCPUx to pCPUy in host a, we >>>>> cannot ensure to have the same in host b, >>>>> since it may have pCPUy too busy, so performance will degrade. >>>>> Also, I hope you saw my general comment >>>> Performance may degrade on any migration to loaded host regardless >>>> of pinning. >>>> >>>> There is not need to forbid migration. Instead, the SLA assurance >>>> policy should verify the dedicated resources and match the target >>>> w/ the guarantee and decide according to this. >>>> >>> Thanks for the input. >>> We may start with migration blocked in a configurable manner, so >>> people who wish migrate to work >>> will simply set the relevant configuration key. >>> As for looking for a matched CPU, will add it as p2. >> We shouldn't block migration >> If a user wants to not migrate a cpu pinned VM then let them use the non-migratable flag in the UI. >> > We're not blocking migration, we're allowing pinning (for now) only for non-migrative VM's. > For p2 we'll verify destination host has the relevant pinning capacity, which will allow > pinning for migrative VM's as well. IMHO the order should be the opposite - if the user likes to migrate a pinned VM, let him (you can pop some message if you like). I'll check it with UI guys (AFAIK engine UI has no pop-ups). Also, I prefer safe than sorry. So start with a safe working pinning for pinned-to-host VMs, and then push the envelope. Safe is letting the user migrate. It may or may not succeed (I'd worry it may put a host into Error state - I hope we've removed this state). This is a pure speculation. If I'm pinning a specific VM to a specific host, and its vCPUS into specific pCPUs, how can live migration be considered a safe thing? If I pinned to host, this is a violation of my request. Now, let's say I didn't pin to host, but I ask for specific vCPUS pinning into specific pCPUs. For now, there's no guarantee the destination host will have a relevant CPU capacity (maybe even topology), so CPU pinning may not work, which is also a violation of my request. Is live migration really the safe thing to do here??? Why not alert me so I could manually migrate to where I think is better? Having that said, I'm reminding you the somewhere earlier in this thread I wrote we'll make the constraint of cpu pinning allowed only on pinned-to-host VMs configurable, so people who wish to walk on wild side, can give it a go.
I think the consensus of the thread is that you should allow pin vCPUs to pCPUs even if the VM is not pinned to host. If the user wants to ensure it will not migrate, he should mark the VM as non migratable. Otherwise, it may migrate, and will succeed - if it can meet the same pinning requirements on the destination, or fail - and then no harm done. If migration succeeds without meeting the pinning requirements on the destination, I see this as a libvirt bug. Y. Basically I already said cpu-pin is allowed for migrative VM's once configuring it. So basically what you're saying is that if such migration works, while loosing cpu-pin, it's a libvirt bug. I can live with that. I hope Daniel agrees with you.
Design updated, so pinning is not blocked, unless user will block it (pin to host, etc). UI will be updated shortly. Please review- http://www.ovirt.org/wiki/Features/Design/cpu-pinning
Sorry is when you wish you had done so initially. ;-) In this case, I disagree as this is configurable.
Y.
>>>>> about starting with a humble solution and gradually improving. In >>>>> this context (auto)numa can help us, >>>>> so we better do numa than handle migration for basic mode, risking >>>>> performance issues. >>>>> >>>>>> ----- Original Message ----- >>>>>>> From: "Doron Fediuck"<dfediuck@redhat.com> >>>>>>> To: engine-devel@ovirt.org >>>>>>> Sent: Thursday, May 17, 2012 2:42:46 PM >>>>>>> Subject: [Engine-devel] CPU Pinning @engine >>>>>>> >>>>>>> Hi All, >>>>>>> Currently the VDSM has a CPU pinning hook. >>>>>>> We'd like to add better support of it into the engine itself. >>>>>>> Here's a design draft to cover it: >>>>>>> http://www.ovirt.org/wiki/Features/Design/cpu-pinning >>>>>>> >>>>>>> Please review and comment if needed. >>>>>>> >>>>>>> Thanks! >>>>>>> -- >>>>>>> >>>>>>> /d >>>>>>> >>>>>>> Never say "OOPS!" always say "Ah, Interesting!" >>>>>>> _______________________________________________ >>>>>>> Engine-devel mailing list >>>>>>> Engine-devel@ovirt.org >>>>>>> http://lists.ovirt.org/mailman/listinfo/engine-devel >>>>>>> >>>>>> _______________________________________________ >>>>>> Engine-devel mailing list >>>>>> Engine-devel@ovirt.org >>>>>> http://lists.ovirt.org/mailman/listinfo/engine-devel >>> -- >>> >>> /d >>> >>> Never say "OOPS!" always say "Ah, Interesting!" >>> _______________________________________________ >>> Engine-devel mailing list >>> Engine-devel@ovirt.org >>> http://lists.ovirt.org/mailman/listinfo/engine-devel >>>
-- /d "Hi, my name is Any Key. Please don't hit me!"
<SNIP>
Design updated, so pinning is not blocked, unless user will block it (pin to host, etc). UI will be updated shortly. Please review- http://www.ovirt.org/wiki/Features/Design/cpu-pinning
What happens if VM is not pinned to host but the other checkbox with the very long description is checked (i.e. only user initiated migrations), upon startup the system would automatically choose a host but from that point on not migrate unless user chooses to do so? The different combinations of the following leave me feeling that this is over complicated (or presented in a confusing manner): 1. Specific host 2. Run VM on Selected Host 3. Allow VM migration only upon Administrator... a. Is 1 supposed to be "Preferred host" ? i.e. should run on it if possible but not mandate it (and auto migrate is enabled and also probably also auto migrate back to it when possible) b. if using 1 + 2 - Vm is pinned to host? ("Run VM on Selected Host" is the wrong terminology imo) c. if using 1 + 3 - VM preferred host is specified (could start on different host in case of pressure) and no auto migrate (if started on non preferred host then will not auto-migrate to preferred host) d. ~1 (not specific host) + 3 - start wherever but only user initiated migrations are allowed. e. I assume 2+3 cannot coexist as well as ~1+2 f. ~1 + ~3 - Regular VM (system chooses where to run it and it auto migrates).
On 05/24/2012 10:02 AM, Ayal Baron wrote:
<SNIP>
Design updated, so pinning is not blocked, unless user will block it (pin to host, etc). UI will be updated shortly. Please review- http://www.ovirt.org/wiki/Features/Design/cpu-pinning
What happens if VM is not pinned to host but the other checkbox with the very long description is checked (i.e. only user initiated migrations), upon startup the system would automatically choose a host but from that point on not migrate unless user chooses to do so?
The different combinations of the following leave me feeling that this is over complicated (or presented in a confusing manner): 1. Specific host 2. Run VM on Selected Host 3. Allow VM migration only upon Administrator...
I'm didn't fully understand the above options - what's the diff between 1 and 2? You're right it's all pretty complicated. The friendliest thing one can do (but takes time) is a graphical presentation: The GUI presents various icons that represents all the host in the cluster (when there are plenty of hosts, it can bunch them together and write the number underneath). When a user selects a pinning string, the GUI automatically shows the group of hosts that match this pinning (removes the ones not relevant). Eventually the user can choose whether there are enough matching hosts to allow migration+cpu_pinning or only admin manual migration. About the pinning string, it's gets messy defining exclusion (can you exclude several pcups?). Again GUI should come to the rescue and you should have a drag and drop graphical tool where you drag vcpus on a group of pcpus. I've done a similar thing in a previous company I worked for and it was really simple and neat for the user.
a. Is 1 supposed to be "Preferred host" ? i.e. should run on it if possible but not mandate it (and auto migrate is enabled and also probably also auto migrate back to it when possible) b. if using 1 + 2 - Vm is pinned to host? ("Run VM on Selected Host" is the wrong terminology imo) c. if using 1 + 3 - VM preferred host is specified (could start on different host in case of pressure) and no auto migrate (if started on non preferred host then will not auto-migrate to preferred host) d. ~1 (not specific host) + 3 - start wherever but only user initiated migrations are allowed. e. I assume 2+3 cannot coexist as well as ~1+2 f. ~1 + ~3 - Regular VM (system chooses where to run it and it auto migrates).
_______________________________________________ Engine-devel mailing list Engine-devel@ovirt.org http://lists.ovirt.org/mailman/listinfo/engine-devel
This is good feedback - unrelated to the vCPU pining discussion tough. Few points: 1. Migration options are relevant only when specific host is selected. 2. "run on selected host" - means pin ONLY to this host, if unavailable, the VM won't run, and no migration will occur. 3. "allow migration" - means admin can manually migrate this VM to any other host. AFAIK, no logic for migration back currently exists. Miki ----- Original Message -----
From: "Dor Laor" <dlaor@redhat.com> To: "Ayal Baron" <abaron@redhat.com> Cc: engine-devel@ovirt.org Sent: Thursday, May 24, 2012 10:25:35 AM Subject: Re: [Engine-devel] CPU Pinning @engine
On 05/24/2012 10:02 AM, Ayal Baron wrote:
<SNIP>
Design updated, so pinning is not blocked, unless user will block it (pin to host, etc). UI will be updated shortly. Please review- http://www.ovirt.org/wiki/Features/Design/cpu-pinning
What happens if VM is not pinned to host but the other checkbox with the very long description is checked (i.e. only user initiated migrations), upon startup the system would automatically choose a host but from that point on not migrate unless user chooses to do so?
The different combinations of the following leave me feeling that this is over complicated (or presented in a confusing manner): 1. Specific host 2. Run VM on Selected Host 3. Allow VM migration only upon Administrator...
I'm didn't fully understand the above options - what's the diff between 1 and 2?
You're right it's all pretty complicated.
The friendliest thing one can do (but takes time) is a graphical presentation:
The GUI presents various icons that represents all the host in the cluster (when there are plenty of hosts, it can bunch them together and write the number underneath). When a user selects a pinning string, the GUI automatically shows the group of hosts that match this pinning (removes the ones not relevant).
Eventually the user can choose whether there are enough matching hosts to allow migration+cpu_pinning or only admin manual migration.
About the pinning string, it's gets messy defining exclusion (can you exclude several pcups?).
Again GUI should come to the rescue and you should have a drag and drop graphical tool where you drag vcpus on a group of pcpus.
I've done a similar thing in a previous company I worked for and it was really simple and neat for the user.
a. Is 1 supposed to be "Preferred host" ? i.e. should run on it if possible but not mandate it (and auto migrate is enabled and also probably also auto migrate back to it when possible) b. if using 1 + 2 - Vm is pinned to host? ("Run VM on Selected Host" is the wrong terminology imo) c. if using 1 + 3 - VM preferred host is specified (could start on different host in case of pressure) and no auto migrate (if started on non preferred host then will not auto-migrate to preferred host) d. ~1 (not specific host) + 3 - start wherever but only user initiated migrations are allowed. e. I assume 2+3 cannot coexist as well as ~1+2 f. ~1 + ~3 - Regular VM (system chooses where to run it and it auto migrates).
_______________________________________________ Engine-devel mailing list Engine-devel@ovirt.org http://lists.ovirt.org/mailman/listinfo/engine-devel
_______________________________________________ Engine-devel mailing list Engine-devel@ovirt.org http://lists.ovirt.org/mailman/listinfo/engine-devel
On 24/05/12 10:25, Dor Laor wrote:
On 05/24/2012 10:02 AM, Ayal Baron wrote:
<SNIP>
Design updated, so pinning is not blocked, unless user will block it (pin to host, etc). UI will be updated shortly. Please review- http://www.ovirt.org/wiki/Features/Design/cpu-pinning
What happens if VM is not pinned to host but the other checkbox with the very long description is checked (i.e. only user initiated migrations), upon startup the system would automatically choose a host but from that point on not migrate unless user chooses to do so?
The different combinations of the following leave me feeling that this is over complicated (or presented in a confusing manner): 1. Specific host 2. Run VM on Selected Host 3. Allow VM migration only upon Administrator...
I'm didn't fully understand the above options - what's the diff between 1 and 2?
You're right it's all pretty complicated.
The friendliest thing one can do (but takes time) is a graphical presentation:
The GUI presents various icons that represents all the host in the cluster (when there are plenty of hosts, it can bunch them together and write the number underneath). When a user selects a pinning string, the GUI automatically shows the group of hosts that match this pinning (removes the ones not relevant).
Eventually the user can choose whether there are enough matching hosts to allow migration+cpu_pinning or only admin manual migration.
About the pinning string, it's gets messy defining exclusion (can you exclude several pcups?).
Again GUI should come to the rescue and you should have a drag and drop graphical tool where you drag vcpus on a group of pcpus.
I've done a similar thing in a previous company I worked for and it was really simple and neat for the user.
I agree with Dor. Let's keep it simple; this feature handles CPU pinning only. Anything related to migration should be determined by the various existing _host_ pinning options, which should already be there.
From CPU-pinning point of view, migration is failed by libvirt if destination topology cannot host such pinning policy (we verified it).
I also agree UI will do magic here, but that's for next phases.
a. Is 1 supposed to be "Preferred host" ? i.e. should run on it if possible but not mandate it (and auto migrate is enabled and also probably also auto migrate back to it when possible) b. if using 1 + 2 - Vm is pinned to host? ("Run VM on Selected Host" is the wrong terminology imo) c. if using 1 + 3 - VM preferred host is specified (could start on different host in case of pressure) and no auto migrate (if started on non preferred host then will not auto-migrate to preferred host) d. ~1 (not specific host) + 3 - start wherever but only user initiated migrations are allowed. e. I assume 2+3 cannot coexist as well as ~1+2 f. ~1 + ~3 - Regular VM (system chooses where to run it and it auto migrates).
_______________________________________________ Engine-devel mailing list Engine-devel@ovirt.org http://lists.ovirt.org/mailman/listinfo/engine-devel
-- /d "Air conditioned environment - Do NOT open Windows!"
----- Original Message -----
From: "Dor Laor" <dlaor@redhat.com> To: "Doron Fediuck" <dfediuck@redhat.com> Cc: engine-devel@ovirt.org Sent: Monday, May 21, 2012 11:01:58 AM Subject: Re: [Engine-devel] CPU Pinning @engine
On 05/21/2012 10:58 AM, Doron Fediuck wrote:
On 21/05/12 01:45, Andrew Cathrow wrote:
----- Original Message -----
From: "Doron Fediuck"<dfediuck@redhat.com> To: dlaor@redhat.com Cc: engine-devel@ovirt.org Sent: Sunday, May 20, 2012 5:49:08 PM Subject: Re: [Engine-devel] CPU Pinning @engine
On 20/05/12 21:41, Dor Laor wrote:
On 05/17/2012 11:15 PM, Doron Fediuck wrote:
On 17/05/12 20:28, Ayal Baron wrote: > "Live migration will not be supported for such VM's." > > Migration will work on homogeneous clusters so this should not > be > enforced (not limited to VMs which are pinned to host) just > give > a warning. > I agree, but if we wish to ping vCPUx to pCPUy in host a, we cannot ensure to have the same in host b, since it may have pCPUy too busy, so performance will degrade. Also, I hope you saw my general comment
Performance may degrade on any migration to loaded host regardless of pinning.
There is not need to forbid migration. Instead, the SLA assurance policy should verify the dedicated resources and match the target w/ the guarantee and decide according to this.
Thanks for the input. We may start with migration blocked in a configurable manner, so people who wish migrate to work will simply set the relevant configuration key. As for looking for a matched CPU, will add it as p2.
We shouldn't block migration If a user wants to not migrate a cpu pinned VM then let them use the non-migratable flag in the UI.
We're not blocking migration, we're allowing pinning (for now) only for non-migrative VM's. For p2 we'll verify destination host has the relevant pinning capacity, which will allow pinning for migrative VM's as well.
IMHO the order should be the opposite - if the user likes to migrate a pinned VM, let him (you can pop some message if you like). +1.
about starting with a humble solution and gradually improving. In this context (auto)numa can help us, so we better do numa than handle migration for basic mode, risking performance issues.
> > ----- Original Message ----- >> From: "Doron Fediuck"<dfediuck@redhat.com> >> To: engine-devel@ovirt.org >> Sent: Thursday, May 17, 2012 2:42:46 PM >> Subject: [Engine-devel] CPU Pinning @engine >> >> Hi All, >> Currently the VDSM has a CPU pinning hook. >> We'd like to add better support of it into the engine itself. >> Here's a design draft to cover it: >> http://www.ovirt.org/wiki/Features/Design/cpu-pinning >> >> Please review and comment if needed. >> >> Thanks! >> -- >> >> /d >> >> Never say "OOPS!" always say "Ah, Interesting!" >> _______________________________________________ >> Engine-devel mailing list >> Engine-devel@ovirt.org >> http://lists.ovirt.org/mailman/listinfo/engine-devel >> > _______________________________________________ > Engine-devel mailing list > Engine-devel@ovirt.org > http://lists.ovirt.org/mailman/listinfo/engine-devel
--
/d
Never say "OOPS!" always say "Ah, Interesting!" _______________________________________________ Engine-devel mailing list Engine-devel@ovirt.org http://lists.ovirt.org/mailman/listinfo/engine-devel
_______________________________________________ Engine-devel mailing list Engine-devel@ovirt.org http://lists.ovirt.org/mailman/listinfo/engine-devel
Hi Doron, one suggestion regarding the ui, we can reuse the key-value widget we did in Custom Properties Sheet (http://www.ovirt.org/wiki/Features/CustomPropertiesSheet) instead of the text box ([vcpu - dropdown][value - textbox][+][-]). just need to know the number of vcpus (keys in that case). Thanks, Gilad. ----- Original Message -----
From: "Doron Fediuck" <dfediuck@redhat.com> To: engine-devel@ovirt.org Sent: Thursday, May 17, 2012 2:42:46 PM Subject: [Engine-devel] CPU Pinning @engine
Hi All, Currently the VDSM has a CPU pinning hook. We'd like to add better support of it into the engine itself. Here's a design draft to cover it: http://www.ovirt.org/wiki/Features/Design/cpu-pinning
Please review and comment if needed.
Thanks! --
/d
Never say "OOPS!" always say "Ah, Interesting!" _______________________________________________ Engine-devel mailing list Engine-devel@ovirt.org http://lists.ovirt.org/mailman/listinfo/engine-devel
On 21/05/12 08:32, Gilad Chaplik wrote:
Hi Doron,
one suggestion regarding the ui, we can reuse the key-value widget we did in Custom Properties Sheet (http://www.ovirt.org/wiki/Features/CustomPropertiesSheet) instead of the text box ([vcpu - dropdown][value - textbox][+][-]). just need to know the number of vcpus (keys in that case).
The vcpu number changes based on the relevant VM, and we should consider a more generic case of vcpu set. So, although the idea is nice, we can't use it.
Thanks, Gilad.
----- Original Message -----
From: "Doron Fediuck" <dfediuck@redhat.com> To: engine-devel@ovirt.org Sent: Thursday, May 17, 2012 2:42:46 PM Subject: [Engine-devel] CPU Pinning @engine
Hi All, Currently the VDSM has a CPU pinning hook. We'd like to add better support of it into the engine itself. Here's a design draft to cover it: http://www.ovirt.org/wiki/Features/Design/cpu-pinning
Please review and comment if needed.
Thanks! --
/d
Never say "OOPS!" always say "Ah, Interesting!" _______________________________________________ Engine-devel mailing list Engine-devel@ovirt.org http://lists.ovirt.org/mailman/listinfo/engine-devel
-- /d "Forty-two," said Deep Thought, with infinite majesty and calm. --Douglas Adams, The Hitchhiker's Guide to the Galaxy
participants (8)
-
Andrew Cathrow -
Ayal Baron -
Dor Laor -
Doron Fediuck -
Gary Kotton -
Gilad Chaplik -
Miki Kenneth -
Yaniv Kaul