[Engine-devel] CPU Overcommit Feature
Hi, I've been working on a feature to allow CPU Overcommitment of hosts in a cluster. This first stage allows the engine to consider host cpu threads as cores for the purposes of VM resource allocation. This wiki page has further details, your comments are welcome! http://www.ovirt.org/Features/cpu_overcommit Thanks in advance, Greg
----- Original Message -----
From: "Greg Padgett" <gpadgett@redhat.com> To: "engine-devel" <engine-devel@ovirt.org> Sent: Monday, December 17, 2012 4:37:57 PM Subject: [Engine-devel] CPU Overcommit Feature
Hi,
I've been working on a feature to allow CPU Overcommitment of hosts in a cluster. This first stage allows the engine to consider host cpu threads as cores for the purposes of VM resource allocation.
This wiki page has further details, your comments are welcome! http://www.ovirt.org/Features/cpu_overcommit
Basically looking good. Hyperthread though is vendor specific. For AMD it's Clustered Multi-Thread while for Intel it's Hyper-Thread Official name is simultaneous multithreading (SMT) but no one outside of the academy will recognize that. in libvirt if I read it right it's <attribute name='thread_siblings'> So why not just call it threads. We'll have cpuSockets, cpiCores, and cpuThreads, should be clear when in CPU context. In the GUI just change hyperthreads to CPU threads. While in the tool tip explain that it's either AMD Clustered Multi-Thread or Intel Hyperthread
Thanks in advance, Greg _______________________________________________ Engine-devel mailing list Engine-devel@ovirt.org http://lists.ovirt.org/mailman/listinfo/engine-devel
... and let's not call this CPU overcommit feature. It's nothing like that - it's "Hyperthread handling" ----- Original Message -----
From: "Simon Grinberg" <simon@redhat.com> To: "Greg Padgett" <gpadgett@redhat.com> Cc: "engine-devel" <engine-devel@ovirt.org> Sent: Monday, December 17, 2012 1:13:03 PM Subject: Re: [Engine-devel] CPU Overcommit Feature
----- Original Message -----
From: "Greg Padgett" <gpadgett@redhat.com> To: "engine-devel" <engine-devel@ovirt.org> Sent: Monday, December 17, 2012 4:37:57 PM Subject: [Engine-devel] CPU Overcommit Feature
Hi,
I've been working on a feature to allow CPU Overcommitment of hosts in a cluster. This first stage allows the engine to consider host cpu threads as cores for the purposes of VM resource allocation.
This wiki page has further details, your comments are welcome! http://www.ovirt.org/Features/cpu_overcommit
Basically looking good. Hyperthread though is vendor specific.
For AMD it's Clustered Multi-Thread while for Intel it's Hyper-Thread Official name is simultaneous multithreading (SMT) but no one outside of the academy will recognize that.
in libvirt if I read it right it's <attribute name='thread_siblings'>
So why not just call it threads. We'll have cpuSockets, cpiCores, and cpuThreads, should be clear when in CPU context.
In the GUI just change hyperthreads to CPU threads. While in the tool tip explain that it's either AMD Clustered Multi-Thread or Intel Hyperthread
Thanks in advance, Greg _______________________________________________ Engine-devel mailing list Engine-devel@ovirt.org http://lists.ovirt.org/mailman/listinfo/engine-devel
_______________________________________________ Engine-devel mailing list Engine-devel@ovirt.org http://lists.ovirt.org/mailman/listinfo/engine-devel
On 12/17/2012 05:52 PM, Andrew Cathrow wrote:
... and let's not call this CPU overcommit feature. It's nothing like that - it's "Hyperthread handling"
----- Original Message -----
From: "Simon Grinberg" <simon@redhat.com> To: "Greg Padgett" <gpadgett@redhat.com> Cc: "engine-devel" <engine-devel@ovirt.org> Sent: Monday, December 17, 2012 1:13:03 PM Subject: Re: [Engine-devel] CPU Overcommit Feature
----- Original Message -----
From: "Greg Padgett" <gpadgett@redhat.com> To: "engine-devel" <engine-devel@ovirt.org> Sent: Monday, December 17, 2012 4:37:57 PM Subject: [Engine-devel] CPU Overcommit Feature
Hi,
I've been working on a feature to allow CPU Overcommitment of hosts in a cluster. This first stage allows the engine to consider host cpu threads as cores for the purposes of VM resource allocation.
This wiki page has further details, your comments are welcome! http://www.ovirt.org/Features/cpu_overcommit
Basically looking good. Hyperthread though is vendor specific.
For AMD it's Clustered Multi-Thread while for Intel it's Hyper-Thread Official name is simultaneous multithreading (SMT) but no one outside of the academy will recognize that.
in libvirt if I read it right it's <attribute name='thread_siblings'>
So why not just call it threads. We'll have cpuSockets, cpiCores, and cpuThreads, should be clear when in CPU context.
In the GUI just change hyperthreads to CPU threads. While in the tool tip explain that it's either AMD Clustered Multi-Thread or Intel Hyperthread
Thanks in advance, Greg _______________________________________________ Engine-devel mailing list Engine-devel@ovirt.org http://lists.ovirt.org/mailman/listinfo/engine-devel
_______________________________________________ Engine-devel mailing list Engine-devel@ovirt.org http://lists.ovirt.org/mailman/listinfo/engine-devel
Thanks Simon and Andrew. I've moved the wiki page [1] (with a redirect at the old name), updated the terms within to not be vendor-specific, and will do the same with the implementation. [1] http://www.ovirt.org/Features/cpu_thread_handling
On 12/17/2012 07:13 PM, Simon Grinberg wrote:
----- Original Message -----
From: "Greg Padgett" <gpadgett@redhat.com> To: "engine-devel" <engine-devel@ovirt.org> Sent: Monday, December 17, 2012 4:37:57 PM Subject: [Engine-devel] CPU Overcommit Feature
Hi,
I've been working on a feature to allow CPU Overcommitment of hosts in a cluster. This first stage allows the engine to consider host cpu threads as cores for the purposes of VM resource allocation.
This wiki page has further details, your comments are welcome! http://www.ovirt.org/Features/cpu_overcommit
Basically looking good. Hyperthread though is vendor specific.
For AMD it's Clustered Multi-Thread while for Intel it's Hyper-Thread Official name is simultaneous multithreading (SMT) but no one outside of the academy will recognize that.
in libvirt if I read it right it's <attribute name='thread_siblings'>
So why not just call it threads. We'll have cpuSockets, cpiCores, and cpuThreads, should be clear when in CPU context.
In the GUI just change hyperthreads to CPU threads. While in the tool tip explain that it's either AMD Clustered Multi-Thread or Intel Hyperthread
Does this affect only the number of potential vCpus for the guests or does this also have an impact on the actual scheduling? So far I always disabled HT out of fear that a 2 vCpu guest might actually be scheduled to run in 2 threads of the same core but now I'm not so sure anymore. In the HT case does KVM know that two threads belong to the same core and will it only schedule its vCpus on distinct cores? Is there some documentation about this somewhere? Regards, Dennis
----- Original Message -----
From: "Dennis Jacobfeuerborn" <dennisml@conversis.de> To: engine-devel@ovirt.org Sent: Tuesday, December 18, 2012 12:30:34 PM Subject: Re: [Engine-devel] CPU Overcommit Feature
On 12/17/2012 07:13 PM, Simon Grinberg wrote:
----- Original Message -----
From: "Greg Padgett" <gpadgett@redhat.com> To: "engine-devel" <engine-devel@ovirt.org> Sent: Monday, December 17, 2012 4:37:57 PM Subject: [Engine-devel] CPU Overcommit Feature
Hi,
I've been working on a feature to allow CPU Overcommitment of hosts in a cluster. This first stage allows the engine to consider host cpu threads as cores for the purposes of VM resource allocation.
This wiki page has further details, your comments are welcome! http://www.ovirt.org/Features/cpu_overcommit
Basically looking good. Hyperthread though is vendor specific.
For AMD it's Clustered Multi-Thread while for Intel it's Hyper-Thread Official name is simultaneous multithreading (SMT) but no one outside of the academy will recognize that.
in libvirt if I read it right it's <attribute name='thread_siblings'>
So why not just call it threads. We'll have cpuSockets, cpiCores, and cpuThreads, should be clear when in CPU context.
In the GUI just change hyperthreads to CPU threads. While in the tool tip explain that it's either AMD Clustered Multi-Thread or Intel Hyperthread
Does this affect only the number of potential vCpus for the guests or does this also have an impact on the actual scheduling? So far I always disabled HT out of fear that a 2 vCpu guest might actually be scheduled to run in 2 threads of the same core but now I'm not so sure anymore. In the HT case does KVM know that two threads belong to the same core and will it only schedule its vCpus on distinct cores? Is there some documentation about this somewhere?
This is about the maximum number of vCPUs we can give to a VM. If the machine has 32 Physical cores that are hyperthreaded then do we say the max number of vCPUs for a single VM is 32 or 64.
Regards, Dennis
_______________________________________________ Engine-devel mailing list Engine-devel@ovirt.org http://lists.ovirt.org/mailman/listinfo/engine-devel
On 12/18/2012 06:33 PM, Andrew Cathrow wrote:
----- Original Message -----
From: "Dennis Jacobfeuerborn" <dennisml@conversis.de> To: engine-devel@ovirt.org Sent: Tuesday, December 18, 2012 12:30:34 PM Subject: Re: [Engine-devel] CPU Overcommit Feature
On 12/17/2012 07:13 PM, Simon Grinberg wrote:
----- Original Message -----
From: "Greg Padgett" <gpadgett@redhat.com> To: "engine-devel" <engine-devel@ovirt.org> Sent: Monday, December 17, 2012 4:37:57 PM Subject: [Engine-devel] CPU Overcommit Feature
Hi,
I've been working on a feature to allow CPU Overcommitment of hosts in a cluster. This first stage allows the engine to consider host cpu threads as cores for the purposes of VM resource allocation.
This wiki page has further details, your comments are welcome! http://www.ovirt.org/Features/cpu_overcommit
Basically looking good. Hyperthread though is vendor specific.
For AMD it's Clustered Multi-Thread while for Intel it's Hyper-Thread Official name is simultaneous multithreading (SMT) but no one outside of the academy will recognize that.
in libvirt if I read it right it's <attribute name='thread_siblings'>
So why not just call it threads. We'll have cpuSockets, cpiCores, and cpuThreads, should be clear when in CPU context.
In the GUI just change hyperthreads to CPU threads. While in the tool tip explain that it's either AMD Clustered Multi-Thread or Intel Hyperthread
Does this affect only the number of potential vCpus for the guests or does this also have an impact on the actual scheduling? So far I always disabled HT out of fear that a 2 vCpu guest might actually be scheduled to run in 2 threads of the same core but now I'm not so sure anymore. In the HT case does KVM know that two threads belong to the same core and will it only schedule its vCpus on distinct cores? Is there some documentation about this somewhere?
This is about the maximum number of vCPUs we can give to a VM. If the machine has 32 Physical cores that are hyperthreaded then do we say the max number of vCPUs for a single VM is 32 or 64.
If the actual scheduling of vCPUs cannot distinguish between threads and cores then why would you even want to limit yourself to 32 in you example? In that case the scheduling might end up being inefficient no matter how many vCPUs you assign to a guest so why restrict the user? (You might of course want to limit the user for policy reasons but that has nothing to to with the thread/core topic.) On the other hand if KVM does only schedule the vCPUs on distinct cores and extending the count from 32 to 64 implies that this distinction is to be disabled then this will have a performance impact for the guest. In that case I might want to limit the guests to just the 32 physical cores so two vCPUs of a single guest don't get scheduled on two threads of the same core. I've never really looked that closely into the scheduling issue but it might play a role here so I asked if someone had any pointers to infos about how exactly KVM makes its scheduling decisions. Regards, Dennis
----- Original Message -----
From: "Dennis Jacobfeuerborn" <dennisml@conversis.de> To: "Andrew Cathrow" <acathrow@redhat.com> Cc: engine-devel@ovirt.org Sent: Tuesday, December 18, 2012 7:59:26 PM Subject: Re: [Engine-devel] CPU Overcommit Feature
On 12/18/2012 06:33 PM, Andrew Cathrow wrote:
----- Original Message -----
From: "Dennis Jacobfeuerborn" <dennisml@conversis.de> To: engine-devel@ovirt.org Sent: Tuesday, December 18, 2012 12:30:34 PM Subject: Re: [Engine-devel] CPU Overcommit Feature
On 12/17/2012 07:13 PM, Simon Grinberg wrote:
----- Original Message -----
From: "Greg Padgett" <gpadgett@redhat.com> To: "engine-devel" <engine-devel@ovirt.org> Sent: Monday, December 17, 2012 4:37:57 PM Subject: [Engine-devel] CPU Overcommit Feature
Hi,
I've been working on a feature to allow CPU Overcommitment of hosts in a cluster. This first stage allows the engine to consider host cpu threads as cores for the purposes of VM resource allocation.
This wiki page has further details, your comments are welcome! http://www.ovirt.org/Features/cpu_overcommit
Basically looking good. Hyperthread though is vendor specific.
For AMD it's Clustered Multi-Thread while for Intel it's Hyper-Thread Official name is simultaneous multithreading (SMT) but no one outside of the academy will recognize that.
in libvirt if I read it right it's <attribute name='thread_siblings'>
So why not just call it threads. We'll have cpuSockets, cpiCores, and cpuThreads, should be clear when in CPU context.
In the GUI just change hyperthreads to CPU threads. While in the tool tip explain that it's either AMD Clustered Multi-Thread or Intel Hyperthread
Does this affect only the number of potential vCpus for the guests or does this also have an impact on the actual scheduling? So far I always disabled HT out of fear that a 2 vCpu guest might actually be scheduled to run in 2 threads of the same core but now I'm not so sure anymore. In the HT case does KVM know that two threads belong to the same core and will it only schedule its vCpus on distinct cores? Is there some documentation about this somewhere?
This is about the maximum number of vCPUs we can give to a VM. If the machine has 32 Physical cores that are hyperthreaded then do we say the max number of vCPUs for a single VM is 32 or 64.
If the actual scheduling of vCPUs cannot distinguish between threads and cores then why would you even want to limit yourself to 32 in you example? In that case the scheduling might end up being inefficient no matter how many vCPUs you assign to a guest so why restrict the user? (You might of course want to limit the user for policy reasons but that has nothing to to with the thread/core topic.)
On the other hand if KVM does only schedule the vCPUs on distinct cores and extending the count from 32 to 64 implies that this distinction is to be disabled then this will have a performance impact for the guest. In that case I might want to limit the guests to just the 32 physical cores so two vCPUs of a single guest don't get scheduled on two threads of the same core.
I've never really looked that closely into the scheduling issue but it might play a role here so I asked if someone had any pointers to infos about how exactly KVM makes its scheduling decisions.
Regards, Dennis
Dennis, first of all every virtual cpu is basically a qemu-thread which can run on any cpu-thread. The scheduling is done by the kernel scheduler, while we may control it using cpu pinning. ie- you may ask for specific vcpu to run on a specific thread which is from the OS point of view another core. Indeed there are cases where this is not recommended, but other cases where this will actually give you a performance boost, as L1 cache is being shared by the sibling threads. So it's really up to you to test your workload and decide id you wish to utilize it or not. We're giving you powerful tools, and you can decide if and how to use it. Doron
On 12/18/2012 07:33 PM, Doron Fediuck wrote:
----- Original Message -----
From: "Dennis Jacobfeuerborn" <dennisml@conversis.de> To: "Andrew Cathrow" <acathrow@redhat.com> Cc: engine-devel@ovirt.org Sent: Tuesday, December 18, 2012 7:59:26 PM Subject: Re: [Engine-devel] CPU Overcommit Feature
On 12/18/2012 06:33 PM, Andrew Cathrow wrote:
----- Original Message -----
From: "Dennis Jacobfeuerborn" <dennisml@conversis.de> To: engine-devel@ovirt.org Sent: Tuesday, December 18, 2012 12:30:34 PM Subject: Re: [Engine-devel] CPU Overcommit Feature
On 12/17/2012 07:13 PM, Simon Grinberg wrote:
----- Original Message -----
From: "Greg Padgett" <gpadgett@redhat.com> To: "engine-devel" <engine-devel@ovirt.org> Sent: Monday, December 17, 2012 4:37:57 PM Subject: [Engine-devel] CPU Overcommit Feature
Hi,
I've been working on a feature to allow CPU Overcommitment of hosts in a cluster. This first stage allows the engine to consider host cpu threads as cores for the purposes of VM resource allocation.
This wiki page has further details, your comments are welcome! http://www.ovirt.org/Features/cpu_overcommit
Basically looking good. Hyperthread though is vendor specific.
For AMD it's Clustered Multi-Thread while for Intel it's Hyper-Thread Official name is simultaneous multithreading (SMT) but no one outside of the academy will recognize that.
in libvirt if I read it right it's <attribute name='thread_siblings'>
So why not just call it threads. We'll have cpuSockets, cpiCores, and cpuThreads, should be clear when in CPU context.
In the GUI just change hyperthreads to CPU threads. While in the tool tip explain that it's either AMD Clustered Multi-Thread or Intel Hyperthread
Does this affect only the number of potential vCpus for the guests or does this also have an impact on the actual scheduling? So far I always disabled HT out of fear that a 2 vCpu guest might actually be scheduled to run in 2 threads of the same core but now I'm not so sure anymore. In the HT case does KVM know that two threads belong to the same core and will it only schedule its vCpus on distinct cores? Is there some documentation about this somewhere?
This is about the maximum number of vCPUs we can give to a VM. If the machine has 32 Physical cores that are hyperthreaded then do we say the max number of vCPUs for a single VM is 32 or 64.
If the actual scheduling of vCPUs cannot distinguish between threads and cores then why would you even want to limit yourself to 32 in you example? In that case the scheduling might end up being inefficient no matter how many vCPUs you assign to a guest so why restrict the user? (You might of course want to limit the user for policy reasons but that has nothing to to with the thread/core topic.)
On the other hand if KVM does only schedule the vCPUs on distinct cores and extending the count from 32 to 64 implies that this distinction is to be disabled then this will have a performance impact for the guest. In that case I might want to limit the guests to just the 32 physical cores so two vCPUs of a single guest don't get scheduled on two threads of the same core.
I've never really looked that closely into the scheduling issue but it might play a role here so I asked if someone had any pointers to infos about how exactly KVM makes its scheduling decisions.
Regards, Dennis
Dennis, first of all every virtual cpu is basically a qemu-thread which can run on any cpu-thread. The scheduling is done by the kernel scheduler, while we may control it using cpu pinning. ie- you may ask for specific vcpu to run on a specific thread which is from the OS point of view another core. Indeed there are cases where this is not recommended, but other cases where this will actually give you a performance boost, as L1 cache is being shared by the sibling threads. So it's really up to you to test your workload and decide id you wish to utilize it or not. We're giving you powerful tools, and you can decide if and how to use it.
What I'm trying to get at is this: Isn't the "Count threads as physical cores" setting superfluous? If HT is disabled on the node this doesn't do anything anyway but if it is enabled what is to be gained by disabling this option? As far as I can see this makes the UI more complicated for no apparent reason. Regards, Dennis
----- Original Message -----
From: "Dennis Jacobfeuerborn" <dennisml@conversis.de> To: "Doron Fediuck" <dfediuck@redhat.com> Cc: engine-devel@ovirt.org, "Andrew Cathrow" <acathrow@redhat.com> Sent: Wednesday, December 19, 2012 1:49:24 PM Subject: Re: [Engine-devel] CPU Overcommit Feature
On 12/18/2012 07:33 PM, Doron Fediuck wrote:
----- Original Message -----
From: "Dennis Jacobfeuerborn" <dennisml@conversis.de> To: "Andrew Cathrow" <acathrow@redhat.com> Cc: engine-devel@ovirt.org Sent: Tuesday, December 18, 2012 7:59:26 PM Subject: Re: [Engine-devel] CPU Overcommit Feature
On 12/18/2012 06:33 PM, Andrew Cathrow wrote:
----- Original Message -----
From: "Dennis Jacobfeuerborn" <dennisml@conversis.de> To: engine-devel@ovirt.org Sent: Tuesday, December 18, 2012 12:30:34 PM Subject: Re: [Engine-devel] CPU Overcommit Feature
On 12/17/2012 07:13 PM, Simon Grinberg wrote:
----- Original Message ----- > From: "Greg Padgett" <gpadgett@redhat.com> > To: "engine-devel" <engine-devel@ovirt.org> > Sent: Monday, December 17, 2012 4:37:57 PM > Subject: [Engine-devel] CPU Overcommit Feature > > Hi, > > I've been working on a feature to allow CPU Overcommitment of > hosts > in a > cluster. This first stage allows the engine to consider host > cpu > threads > as cores for the purposes of VM resource allocation. > > This wiki page has further details, your comments are welcome! > http://www.ovirt.org/Features/cpu_overcommit
Basically looking good. Hyperthread though is vendor specific.
For AMD it's Clustered Multi-Thread while for Intel it's Hyper-Thread Official name is simultaneous multithreading (SMT) but no one outside of the academy will recognize that.
in libvirt if I read it right it's <attribute name='thread_siblings'>
So why not just call it threads. We'll have cpuSockets, cpiCores, and cpuThreads, should be clear when in CPU context.
In the GUI just change hyperthreads to CPU threads. While in the tool tip explain that it's either AMD Clustered Multi-Thread or Intel Hyperthread
Does this affect only the number of potential vCpus for the guests or does this also have an impact on the actual scheduling? So far I always disabled HT out of fear that a 2 vCpu guest might actually be scheduled to run in 2 threads of the same core but now I'm not so sure anymore. In the HT case does KVM know that two threads belong to the same core and will it only schedule its vCpus on distinct cores? Is there some documentation about this somewhere?
This is about the maximum number of vCPUs we can give to a VM. If the machine has 32 Physical cores that are hyperthreaded then do we say the max number of vCPUs for a single VM is 32 or 64.
If the actual scheduling of vCPUs cannot distinguish between threads and cores then why would you even want to limit yourself to 32 in you example? In that case the scheduling might end up being inefficient no matter how many vCPUs you assign to a guest so why restrict the user? (You might of course want to limit the user for policy reasons but that has nothing to to with the thread/core topic.)
On the other hand if KVM does only schedule the vCPUs on distinct cores and extending the count from 32 to 64 implies that this distinction is to be disabled then this will have a performance impact for the guest. In that case I might want to limit the guests to just the 32 physical cores so two vCPUs of a single guest don't get scheduled on two threads of the same core.
I've never really looked that closely into the scheduling issue but it might play a role here so I asked if someone had any pointers to infos about how exactly KVM makes its scheduling decisions.
Regards, Dennis
Dennis, first of all every virtual cpu is basically a qemu-thread which can run on any cpu-thread. The scheduling is done by the kernel scheduler, while we may control it using cpu pinning. ie- you may ask for specific vcpu to run on a specific thread which is from the OS point of view another core. Indeed there are cases where this is not recommended, but other cases where this will actually give you a performance boost, as L1 cache is being shared by the sibling threads. So it's really up to you to test your workload and decide id you wish to utilize it or not. We're giving you powerful tools, and you can decide if and how to use it.
What I'm trying to get at is this: Isn't the "Count threads as physical cores" setting superfluous?
If HT is disabled on the node this doesn't do anything anyway but if it is enabled what is to be gained by disabling this option? As far as I can see this makes the UI more complicated for no apparent reason.
Regards, Dennis
Hi Dennis. Let's take it one at a time; UI wise, this is a cluster level policy, and falls into the optimization tab. So it shouldn't complicate things unless you're looking for specific optimization. In which-case, I'd expect you to know what HT is all about, and what should be the proper settings. Additionally, you now may observe the HT status in the general sub-tab of the hosts. Functionality wise, some users actually expects 8 cores to be counted as 16 when HT is on, since they need to utilize it based on their experience and tests. The efficiency factor HT will provide is something which varies from one workload to another and from one hardware to another as already mentioned. The default state of HT optimization is off, so it shouldn't bother you if you're not interested in such optimization. Once you start utilizing it, if you turn off the HT in one or more of the cluster hosts (BIOS setting, requires reboot!), vdsm will report it to the engine, so the correct number of cores will be used when scheduling a target host for running or migrating a VM. Hope this helps, Doron
On Mon, Dec 17, 2012 at 09:37:57AM -0500, Greg Padgett wrote:
Hi,
I've been working on a feature to allow CPU Overcommitment of hosts in a cluster. This first stage allows the engine to consider host cpu threads as cores for the purposes of VM resource allocation.
This wiki page has further details, your comments are welcome! http://www.ovirt.org/Features/cpu_overcommit
I've commented about the vdsm/engine API on http://gerrit.ovirt.org/#/c/10144/ but it is probably better to reiterate it here. The suggested API is tightly coupled with an ugly hack we pushed to vdsm in order not to solve the issue properly on the first strike. If we had not have report_host_threads_as_cores, I think we'd have a simpler API reporting only cpuThreads and cpuCores; with no funny boolean flags. Let us strive to that position as much as we can. How about asking whoever used report_host_threads_as_cores to unset it once they install Engine 3.2 ? I think that these are very few people, that would not mind this very much. If this is impossible, I'd add a cpuCores2, always reporting the true number, to be used by new Engines. We may even report it only on the very few cases of report_host_threads_as_cores being set. Dan.
----- Original Message -----
From: "Dan Kenigsberg" <danken@redhat.com> To: "Greg Padgett" <gpadgett@redhat.com> Cc: "engine-devel" <engine-devel@ovirt.org>, vdsm-devel@fedorahosted.org Sent: Wednesday, December 19, 2012 3:59:11 PM Subject: Re: [Engine-devel] CPU Overcommit Feature
On Mon, Dec 17, 2012 at 09:37:57AM -0500, Greg Padgett wrote:
Hi,
I've been working on a feature to allow CPU Overcommitment of hosts in a cluster. This first stage allows the engine to consider host cpu threads as cores for the purposes of VM resource allocation.
This wiki page has further details, your comments are welcome! http://www.ovirt.org/Features/cpu_overcommit
I've commented about the vdsm/engine API on http://gerrit.ovirt.org/#/c/10144/ but it is probably better to reiterate it here.
The suggested API is tightly coupled with an ugly hack we pushed to vdsm in order not to solve the issue properly on the first strike.
If we had not have report_host_threads_as_cores, I think we'd have a simpler API reporting only cpuThreads and cpuCores; with no funny boolean flags.
Let us strive to that position as much as we can.
How about asking whoever used report_host_threads_as_cores to unset it once they install Engine 3.2 ? I think that these are very few people, that would not mind this very much.
If this is impossible, I'd add a cpuCores2, always reporting the true number, to be used by new Engines. We may even report it only on the very few cases of report_host_threads_as_cores being set.
Dan.
Hi Dan, Thanks for the review. I agree simply reporting cores and threads would be the right solution. However, when you have hyperthreading turned off you get cores=threads. This is the same situation you have when hyperthreading turned on, and someone used the vdsm configuration of reporting threads as cores. So the engine won't know the real status of the host. We need to be able to tell the difference. So this moves us to cpuCores2 suggestion. This is one possibility (cpuRealCores?), and the alternative is an indication of vdsm config (true/false) which may be removed in the future. I suspect over time cpu and cpu2 will confuse people. So I'd suggest having the boolean and removing it along with the vdsm configuration in the next ovirt version. Doron
On Wed, Dec 19, 2012 at 09:53:15AM -0500, Doron Fediuck wrote:
----- Original Message -----
From: "Dan Kenigsberg" <danken@redhat.com> To: "Greg Padgett" <gpadgett@redhat.com> Cc: "engine-devel" <engine-devel@ovirt.org>, vdsm-devel@fedorahosted.org Sent: Wednesday, December 19, 2012 3:59:11 PM Subject: Re: [Engine-devel] CPU Overcommit Feature
On Mon, Dec 17, 2012 at 09:37:57AM -0500, Greg Padgett wrote:
Hi,
I've been working on a feature to allow CPU Overcommitment of hosts in a cluster. This first stage allows the engine to consider host cpu threads as cores for the purposes of VM resource allocation.
This wiki page has further details, your comments are welcome! http://www.ovirt.org/Features/cpu_overcommit
I've commented about the vdsm/engine API on http://gerrit.ovirt.org/#/c/10144/ but it is probably better to reiterate it here.
The suggested API is tightly coupled with an ugly hack we pushed to vdsm in order not to solve the issue properly on the first strike.
If we had not have report_host_threads_as_cores, I think we'd have a simpler API reporting only cpuThreads and cpuCores; with no funny boolean flags.
Let us strive to that position as much as we can.
How about asking whoever used report_host_threads_as_cores to unset it once they install Engine 3.2 ? I think that these are very few people, that would not mind this very much.
If this is impossible, I'd add a cpuCores2, always reporting the true number, to be used by new Engines. We may even report it only on the very few cases of report_host_threads_as_cores being set.
Dan.
Hi Dan, Thanks for the review.
I agree simply reporting cores and threads would be the right solution. However, when you have hyperthreading turned off you get cores=threads. This is the same situation you have when hyperthreading turned on, and someone used the vdsm configuration of reporting threads as cores.
So the engine won't know the real status of the host.
This is not surprising, as report_host_threads_as_cores means in blunt English "lie to Engine about the number of cores". The newly suggested flag says "don't believe what I said in cpuCores, since I'm lying". Next thing we'd have is another flag saying that the former flag was a lie, and cpuCores is actually trustworthy. Instead of dancing this dance, I suggest to stop lying. report_host_threads_as_cores was a hack to assist a older Engine versions. Engine users that needed it had to set it out-of-band on their hosts. Now if they upgrade their Engine, they can -- as easily -- reset that value. If they forget, nothing devastating happens beyond Engine assuming that hyperthreading is off. Please consider this suggestion. I find it the simplest for all involved parties. Dan.
We need to be able to tell the difference. So this moves us to cpuCores2 suggestion. This is one possibility (cpuRealCores?), and the alternative is an indication of vdsm config (true/false) which may be removed in the future. I suspect over time cpu and cpu2 will confuse people.
So I'd suggest having the boolean and removing it along with the vdsm configuration in the next ovirt version.
On 12/20/2012 09:43 AM, Dan Kenigsberg wrote:
On Wed, Dec 19, 2012 at 09:53:15AM -0500, Doron Fediuck wrote:
----- Original Message -----
From: "Dan Kenigsberg" <danken@redhat.com> To: "Greg Padgett" <gpadgett@redhat.com> Cc: "engine-devel" <engine-devel@ovirt.org>, vdsm-devel@fedorahosted.org Sent: Wednesday, December 19, 2012 3:59:11 PM Subject: Re: [Engine-devel] CPU Overcommit Feature
On Mon, Dec 17, 2012 at 09:37:57AM -0500, Greg Padgett wrote:
Hi,
I've been working on a feature to allow CPU Overcommitment of hosts in a cluster. This first stage allows the engine to consider host cpu threads as cores for the purposes of VM resource allocation.
This wiki page has further details, your comments are welcome! http://www.ovirt.org/Features/cpu_overcommit
I've commented about the vdsm/engine API on http://gerrit.ovirt.org/#/c/10144/ but it is probably better to reiterate it here.
The suggested API is tightly coupled with an ugly hack we pushed to vdsm in order not to solve the issue properly on the first strike.
If we had not have report_host_threads_as_cores, I think we'd have a simpler API reporting only cpuThreads and cpuCores; with no funny boolean flags.
Let us strive to that position as much as we can.
How about asking whoever used report_host_threads_as_cores to unset it once they install Engine 3.2 ? I think that these are very few people, that would not mind this very much.
If this is impossible, I'd add a cpuCores2, always reporting the true number, to be used by new Engines. We may even report it only on the very few cases of report_host_threads_as_cores being set.
Dan.
Hi Dan, Thanks for the review.
I agree simply reporting cores and threads would be the right solution. However, when you have hyperthreading turned off you get cores=threads. This is the same situation you have when hyperthreading turned on, and someone used the vdsm configuration of reporting threads as cores.
So the engine won't know the real status of the host.
This is not surprising, as report_host_threads_as_cores means in blunt English "lie to Engine about the number of cores". The newly suggested flag says "don't believe what I said in cpuCores, since I'm lying". Next thing we'd have is another flag saying that the former flag was a lie, and cpuCores is actually trustworthy.
Instead of dancing this dance, I suggest to stop lying.
report_host_threads_as_cores was a hack to assist a older Engine versions. Engine users that needed it had to set it out-of-band on their hosts. Now if they upgrade their Engine, they can -- as easily -- reset that value.
If they forget, nothing devastating happens beyond Engine assuming that hyperthreading is off.
Please consider this suggestion. I find it the simplest for all involved parties.
the only problem is the new vdsm doesn't know which engine may be using it. if engine would say "getVdsCaps engineVersion=3.2", then vdsm could know engine no longer needs lying to and ignore the flag, re-using same field.
Dan.
We need to be able to tell the difference. So this moves us to cpuCores2 suggestion. This is one possibility (cpuRealCores?), and the alternative is an indication of vdsm config (true/false) which may be removed in the future. I suspect over time cpu and cpu2 will confuse people.
So I'd suggest having the boolean and removing it along with the vdsm configuration in the next ovirt version.
vdsm-devel mailing list vdsm-devel@lists.fedorahosted.org https://lists.fedorahosted.org/mailman/listinfo/vdsm-devel
On Thu, Dec 20, 2012 at 10:23:55AM +0200, Itamar Heim wrote:
On 12/20/2012 09:43 AM, Dan Kenigsberg wrote:
On Wed, Dec 19, 2012 at 09:53:15AM -0500, Doron Fediuck wrote:
----- Original Message -----
From: "Dan Kenigsberg" <danken@redhat.com> To: "Greg Padgett" <gpadgett@redhat.com> Cc: "engine-devel" <engine-devel@ovirt.org>, vdsm-devel@fedorahosted.org Sent: Wednesday, December 19, 2012 3:59:11 PM Subject: Re: [Engine-devel] CPU Overcommit Feature
On Mon, Dec 17, 2012 at 09:37:57AM -0500, Greg Padgett wrote:
Hi,
I've been working on a feature to allow CPU Overcommitment of hosts in a cluster. This first stage allows the engine to consider host cpu threads as cores for the purposes of VM resource allocation.
This wiki page has further details, your comments are welcome! http://www.ovirt.org/Features/cpu_overcommit
I've commented about the vdsm/engine API on http://gerrit.ovirt.org/#/c/10144/ but it is probably better to reiterate it here.
The suggested API is tightly coupled with an ugly hack we pushed to vdsm in order not to solve the issue properly on the first strike.
If we had not have report_host_threads_as_cores, I think we'd have a simpler API reporting only cpuThreads and cpuCores; with no funny boolean flags.
Let us strive to that position as much as we can.
How about asking whoever used report_host_threads_as_cores to unset it once they install Engine 3.2 ? I think that these are very few people, that would not mind this very much.
If this is impossible, I'd add a cpuCores2, always reporting the true number, to be used by new Engines. We may even report it only on the very few cases of report_host_threads_as_cores being set.
Dan.
Hi Dan, Thanks for the review.
I agree simply reporting cores and threads would be the right solution. However, when you have hyperthreading turned off you get cores=threads. This is the same situation you have when hyperthreading turned on, and someone used the vdsm configuration of reporting threads as cores.
So the engine won't know the real status of the host.
This is not surprising, as report_host_threads_as_cores means in blunt English "lie to Engine about the number of cores". The newly suggested flag says "don't believe what I said in cpuCores, since I'm lying". Next thing we'd have is another flag saying that the former flag was a lie, and cpuCores is actually trustworthy.
Instead of dancing this dance, I suggest to stop lying.
report_host_threads_as_cores was a hack to assist a older Engine versions. Engine users that needed it had to set it out-of-band on their hosts. Now if they upgrade their Engine, they can -- as easily -- reset that value.
If they forget, nothing devastating happens beyond Engine assuming that hyperthreading is off.
Please consider this suggestion. I find it the simplest for all involved parties.
the only problem is the new vdsm doesn't know which engine may be using it. if engine would say "getVdsCaps engineVersion=3.2", then vdsm could know engine no longer needs lying to and ignore the flag, re-using same field.
Note that I do not suggest to drop report_host_threads_as_cores now. I am suggesting to keep on lying even to new Engine. If someone thinks that lying is bad, he should reset report_host_threads_as_cores. It seems to me that the suggested API is being coerced by a very limited use case, that is not going to be really harmed by a straight-forward API. Dan.
----- Original Message -----
From: "Dan Kenigsberg" <danken@redhat.com> To: "Itamar Heim" <iheim@redhat.com> Cc: "engine-devel" <engine-devel@ovirt.org>, vdsm-devel@fedorahosted.org Sent: Thursday, December 20, 2012 11:55:10 AM Subject: Re: [Engine-devel] [vdsm] CPU Overcommit Feature
On Thu, Dec 20, 2012 at 10:23:55AM +0200, Itamar Heim wrote:
On 12/20/2012 09:43 AM, Dan Kenigsberg wrote:
On Wed, Dec 19, 2012 at 09:53:15AM -0500, Doron Fediuck wrote:
----- Original Message -----
From: "Dan Kenigsberg" <danken@redhat.com> To: "Greg Padgett" <gpadgett@redhat.com> Cc: "engine-devel" <engine-devel@ovirt.org>, vdsm-devel@fedorahosted.org Sent: Wednesday, December 19, 2012 3:59:11 PM Subject: Re: [Engine-devel] CPU Overcommit Feature
On Mon, Dec 17, 2012 at 09:37:57AM -0500, Greg Padgett wrote:
Hi,
I've been working on a feature to allow CPU Overcommitment of hosts in a cluster. This first stage allows the engine to consider host cpu threads as cores for the purposes of VM resource allocation.
This wiki page has further details, your comments are welcome! http://www.ovirt.org/Features/cpu_overcommit
I've commented about the vdsm/engine API on http://gerrit.ovirt.org/#/c/10144/ but it is probably better to reiterate it here.
The suggested API is tightly coupled with an ugly hack we pushed to vdsm in order not to solve the issue properly on the first strike.
If we had not have report_host_threads_as_cores, I think we'd have a simpler API reporting only cpuThreads and cpuCores; with no funny boolean flags.
Let us strive to that position as much as we can.
How about asking whoever used report_host_threads_as_cores to unset it once they install Engine 3.2 ? I think that these are very few people, that would not mind this very much.
If this is impossible, I'd add a cpuCores2, always reporting the true number, to be used by new Engines. We may even report it only on the very few cases of report_host_threads_as_cores being set.
Dan.
Hi Dan, Thanks for the review.
I agree simply reporting cores and threads would be the right solution. However, when you have hyperthreading turned off you get cores=threads. This is the same situation you have when hyperthreading turned on, and someone used the vdsm configuration of reporting threads as cores.
So the engine won't know the real status of the host.
This is not surprising, as report_host_threads_as_cores means in blunt English "lie to Engine about the number of cores". The newly suggested flag says "don't believe what I said in cpuCores, since I'm lying". Next thing we'd have is another flag saying that the former flag was a lie, and cpuCores is actually trustworthy.
Instead of dancing this dance, I suggest to stop lying.
report_host_threads_as_cores was a hack to assist a older Engine versions. Engine users that needed it had to set it out-of-band on their hosts. Now if they upgrade their Engine, they can -- as easily -- reset that value.
If they forget, nothing devastating happens beyond Engine assuming that hyperthreading is off.
Please consider this suggestion. I find it the simplest for all involved parties.
the only problem is the new vdsm doesn't know which engine may be using it. if engine would say "getVdsCaps engineVersion=3.2", then vdsm could know engine no longer needs lying to and ignore the flag, re-using same field.
Note that I do not suggest to drop report_host_threads_as_cores now. I am suggesting to keep on lying even to new Engine. If someone thinks that lying is bad, he should reset report_host_threads_as_cores.
It seems to me that the suggested API is being coerced by a very limited use case, that is not going to be really harmed by a straight-forward API.
Dan.
Dan, Did some further checking, and we can go with it; So basically now we add cpuThreads. Additionally, if the report_host_threads_as_cores is turned on, an additional cpuCoresReal will be reported. Since this is a 3.2 cluster feature, the engine will make sure to update the DB with the real numbers, and older clusters will use the information vdsm reports, while blocking the engine side optimization. An update path is already available here: http://gerrit.ovirt.org/#/c/10144/4 I hope that for cluster 3.3 we'll be able to drop the vdsm configuration. A review would be highly appreciated due to the holidays proximity. Doron
----- Original Message -----
From: "Doron Fediuck" <dfediuck@redhat.com> To: "Dan Kenigsberg" <danken@redhat.com> Cc: "engine-devel" <engine-devel@ovirt.org>, vdsm-devel@fedorahosted.org Sent: Thursday, December 20, 2012 2:14:45 PM Subject: Re: [Engine-devel] [vdsm] CPU Overcommit Feature
----- Original Message -----
From: "Dan Kenigsberg" <danken@redhat.com> To: "Itamar Heim" <iheim@redhat.com> Cc: "engine-devel" <engine-devel@ovirt.org>, vdsm-devel@fedorahosted.org Sent: Thursday, December 20, 2012 11:55:10 AM Subject: Re: [Engine-devel] [vdsm] CPU Overcommit Feature
On Thu, Dec 20, 2012 at 10:23:55AM +0200, Itamar Heim wrote:
On 12/20/2012 09:43 AM, Dan Kenigsberg wrote:
On Wed, Dec 19, 2012 at 09:53:15AM -0500, Doron Fediuck wrote:
----- Original Message -----
From: "Dan Kenigsberg" <danken@redhat.com> To: "Greg Padgett" <gpadgett@redhat.com> Cc: "engine-devel" <engine-devel@ovirt.org>, vdsm-devel@fedorahosted.org Sent: Wednesday, December 19, 2012 3:59:11 PM Subject: Re: [Engine-devel] CPU Overcommit Feature
On Mon, Dec 17, 2012 at 09:37:57AM -0500, Greg Padgett wrote: >Hi, > >I've been working on a feature to allow CPU Overcommitment of >hosts >in a cluster. This first stage allows the engine to consider >host >cpu threads as cores for the purposes of VM resource >allocation. > >This wiki page has further details, your comments are >welcome! >http://www.ovirt.org/Features/cpu_overcommit
I've commented about the vdsm/engine API on http://gerrit.ovirt.org/#/c/10144/ but it is probably better to reiterate it here.
The suggested API is tightly coupled with an ugly hack we pushed to vdsm in order not to solve the issue properly on the first strike.
If we had not have report_host_threads_as_cores, I think we'd have a simpler API reporting only cpuThreads and cpuCores; with no funny boolean flags.
Let us strive to that position as much as we can.
How about asking whoever used report_host_threads_as_cores to unset it once they install Engine 3.2 ? I think that these are very few people, that would not mind this very much.
If this is impossible, I'd add a cpuCores2, always reporting the true number, to be used by new Engines. We may even report it only on the very few cases of report_host_threads_as_cores being set.
Dan.
Hi Dan, Thanks for the review.
I agree simply reporting cores and threads would be the right solution. However, when you have hyperthreading turned off you get cores=threads. This is the same situation you have when hyperthreading turned on, and someone used the vdsm configuration of reporting threads as cores.
So the engine won't know the real status of the host.
This is not surprising, as report_host_threads_as_cores means in blunt English "lie to Engine about the number of cores". The newly suggested flag says "don't believe what I said in cpuCores, since I'm lying". Next thing we'd have is another flag saying that the former flag was a lie, and cpuCores is actually trustworthy.
Instead of dancing this dance, I suggest to stop lying.
report_host_threads_as_cores was a hack to assist a older Engine versions. Engine users that needed it had to set it out-of-band on their hosts. Now if they upgrade their Engine, they can -- as easily -- reset that value.
If they forget, nothing devastating happens beyond Engine assuming that hyperthreading is off.
Please consider this suggestion. I find it the simplest for all involved parties.
the only problem is the new vdsm doesn't know which engine may be using it. if engine would say "getVdsCaps engineVersion=3.2", then vdsm could know engine no longer needs lying to and ignore the flag, re-using same field.
Note that I do not suggest to drop report_host_threads_as_cores now. I am suggesting to keep on lying even to new Engine. If someone thinks that lying is bad, he should reset report_host_threads_as_cores.
It seems to me that the suggested API is being coerced by a very limited use case, that is not going to be really harmed by a straight-forward API.
Dan.
Dan, Did some further checking, and we can go with it; So basically now we add cpuThreads. Additionally, if the report_host_threads_as_cores is turned on, an additional cpuCoresReal will be reported.
No need for that. There is only one problematic state where VDSM cheats and reports cores == threads. This does not happen by mistake, the user specifically asked for it. The above condition above also happens if threads is really off or the processor does not have threads, so it's a state we need to handle in any case. So just report threads=off, whenever cores == threads and treat it as such. If the user is unhappy then he should just turn off the cheat mode.
Since this is a 3.2 cluster feature, the engine will make sure to update the DB with the real numbers, and older clusters will use the information vdsm reports, while blocking the engine side optimization. An update path is already available here: http://gerrit.ovirt.org/#/c/10144/4
I hope that for cluster 3.3 we'll be able to drop the vdsm configuration. A review would be highly appreciated due to the holidays proximity.
Doron _______________________________________________ Engine-devel mailing list Engine-devel@ovirt.org http://lists.ovirt.org/mailman/listinfo/engine-devel
----- Original Message -----
From: "Simon Grinberg" <simon@redhat.com> To: "Doron Fediuck" <dfediuck@redhat.com> Cc: "engine-devel" <engine-devel@ovirt.org>, vdsm-devel@fedorahosted.org Sent: Thursday, December 20, 2012 4:56:14 PM Subject: Re: [Engine-devel] [vdsm] CPU Overcommit Feature
----- Original Message -----
From: "Doron Fediuck" <dfediuck@redhat.com> To: "Dan Kenigsberg" <danken@redhat.com> Cc: "engine-devel" <engine-devel@ovirt.org>, vdsm-devel@fedorahosted.org Sent: Thursday, December 20, 2012 2:14:45 PM Subject: Re: [Engine-devel] [vdsm] CPU Overcommit Feature
----- Original Message -----
From: "Dan Kenigsberg" <danken@redhat.com> To: "Itamar Heim" <iheim@redhat.com> Cc: "engine-devel" <engine-devel@ovirt.org>, vdsm-devel@fedorahosted.org Sent: Thursday, December 20, 2012 11:55:10 AM Subject: Re: [Engine-devel] [vdsm] CPU Overcommit Feature
On Thu, Dec 20, 2012 at 10:23:55AM +0200, Itamar Heim wrote:
On 12/20/2012 09:43 AM, Dan Kenigsberg wrote:
On Wed, Dec 19, 2012 at 09:53:15AM -0500, Doron Fediuck wrote:
----- Original Message ----- >From: "Dan Kenigsberg" <danken@redhat.com> >To: "Greg Padgett" <gpadgett@redhat.com> >Cc: "engine-devel" <engine-devel@ovirt.org>, >vdsm-devel@fedorahosted.org >Sent: Wednesday, December 19, 2012 3:59:11 PM >Subject: Re: [Engine-devel] CPU Overcommit Feature > >On Mon, Dec 17, 2012 at 09:37:57AM -0500, Greg Padgett >wrote: >>Hi, >> >>I've been working on a feature to allow CPU Overcommitment >>of >>hosts >>in a cluster. This first stage allows the engine to >>consider >>host >>cpu threads as cores for the purposes of VM resource >>allocation. >> >>This wiki page has further details, your comments are >>welcome! >>http://www.ovirt.org/Features/cpu_overcommit > >I've commented about the vdsm/engine API on >http://gerrit.ovirt.org/#/c/10144/ but it is probably better >to >reiterate it here. > >The suggested API is tightly coupled with an ugly hack we >pushed >to >vdsm >in order not to solve the issue properly on the first >strike. > >If we had not have report_host_threads_as_cores, I think >we'd >have a >simpler API reporting only cpuThreads and cpuCores; with no >funny >boolean flags. > >Let us strive to that position as much as we can. > >How about asking whoever used report_host_threads_as_cores >to >unset >it >once they install Engine 3.2 ? I think that these are very >few >people, >that would not mind this very much. > >If this is impossible, I'd add a cpuCores2, always reporting >the >true >number, to be used by new Engines. We may even report it >only >on >the >very few cases of report_host_threads_as_cores being set. > >Dan.
Hi Dan, Thanks for the review.
I agree simply reporting cores and threads would be the right solution. However, when you have hyperthreading turned off you get cores=threads. This is the same situation you have when hyperthreading turned on, and someone used the vdsm configuration of reporting threads as cores.
So the engine won't know the real status of the host.
This is not surprising, as report_host_threads_as_cores means in blunt English "lie to Engine about the number of cores". The newly suggested flag says "don't believe what I said in cpuCores, since I'm lying". Next thing we'd have is another flag saying that the former flag was a lie, and cpuCores is actually trustworthy.
Instead of dancing this dance, I suggest to stop lying.
report_host_threads_as_cores was a hack to assist a older Engine versions. Engine users that needed it had to set it out-of-band on their hosts. Now if they upgrade their Engine, they can -- as easily -- reset that value.
If they forget, nothing devastating happens beyond Engine assuming that hyperthreading is off.
Please consider this suggestion. I find it the simplest for all involved parties.
the only problem is the new vdsm doesn't know which engine may be using it. if engine would say "getVdsCaps engineVersion=3.2", then vdsm could know engine no longer needs lying to and ignore the flag, re-using same field.
Note that I do not suggest to drop report_host_threads_as_cores now. I am suggesting to keep on lying even to new Engine. If someone thinks that lying is bad, he should reset report_host_threads_as_cores.
It seems to me that the suggested API is being coerced by a very limited use case, that is not going to be really harmed by a straight-forward API.
Dan.
Dan, Did some further checking, and we can go with it; So basically now we add cpuThreads. Additionally, if the report_host_threads_as_cores is turned on, an additional cpuCoresReal will be reported.
No need for that. There is only one problematic state where VDSM cheats and reports cores == threads. This does not happen by mistake, the user specifically asked for it.
The above condition above also happens if threads is really off or the processor does not have threads, so it's a state we need to handle in any case.
So just report threads=off, whenever cores == threads and treat it as such. If the user is unhappy then he should just turn off the cheat mode.
ack. * <3.2 clusters are not supported. * >=3.2 clusters will assume HT off unless cores < threads. Some release note is needed for this, so users will be advised to turn off vdsm cheating when upgrading to new 3.2 engine, in case they happen to use it.
Since this is a 3.2 cluster feature, the engine will make sure to update the DB with the real numbers, and older clusters will use the information vdsm reports, while blocking the engine side optimization. An update path is already available here: http://gerrit.ovirt.org/#/c/10144/4
I hope that for cluster 3.3 we'll be able to drop the vdsm configuration. A review would be highly appreciated due to the holidays proximity.
Doron
On Thu, Dec 20, 2012 at 10:17:50AM -0500, Doron Fediuck wrote:
----- Original Message -----
From: "Simon Grinberg" <simon@redhat.com> To: "Doron Fediuck" <dfediuck@redhat.com> Cc: "engine-devel" <engine-devel@ovirt.org>, vdsm-devel@fedorahosted.org Sent: Thursday, December 20, 2012 4:56:14 PM Subject: Re: [Engine-devel] [vdsm] CPU Overcommit Feature
----- Original Message -----
From: "Doron Fediuck" <dfediuck@redhat.com> To: "Dan Kenigsberg" <danken@redhat.com> Cc: "engine-devel" <engine-devel@ovirt.org>, vdsm-devel@fedorahosted.org Sent: Thursday, December 20, 2012 2:14:45 PM Subject: Re: [Engine-devel] [vdsm] CPU Overcommit Feature
----- Original Message -----
From: "Dan Kenigsberg" <danken@redhat.com> To: "Itamar Heim" <iheim@redhat.com> Cc: "engine-devel" <engine-devel@ovirt.org>, vdsm-devel@fedorahosted.org Sent: Thursday, December 20, 2012 11:55:10 AM Subject: Re: [Engine-devel] [vdsm] CPU Overcommit Feature
On Thu, Dec 20, 2012 at 10:23:55AM +0200, Itamar Heim wrote:
On 12/20/2012 09:43 AM, Dan Kenigsberg wrote:
On Wed, Dec 19, 2012 at 09:53:15AM -0500, Doron Fediuck wrote: > >----- Original Message ----- >>From: "Dan Kenigsberg" <danken@redhat.com> >>To: "Greg Padgett" <gpadgett@redhat.com> >>Cc: "engine-devel" <engine-devel@ovirt.org>, >>vdsm-devel@fedorahosted.org >>Sent: Wednesday, December 19, 2012 3:59:11 PM >>Subject: Re: [Engine-devel] CPU Overcommit Feature >> >>On Mon, Dec 17, 2012 at 09:37:57AM -0500, Greg Padgett >>wrote: >>>Hi, >>> >>>I've been working on a feature to allow CPU Overcommitment >>>of >>>hosts >>>in a cluster. This first stage allows the engine to >>>consider >>>host >>>cpu threads as cores for the purposes of VM resource >>>allocation. >>> >>>This wiki page has further details, your comments are >>>welcome! >>>http://www.ovirt.org/Features/cpu_overcommit >> >>I've commented about the vdsm/engine API on >>http://gerrit.ovirt.org/#/c/10144/ but it is probably better >>to >>reiterate it here. >> >>The suggested API is tightly coupled with an ugly hack we >>pushed >>to >>vdsm >>in order not to solve the issue properly on the first >>strike. >> >>If we had not have report_host_threads_as_cores, I think >>we'd >>have a >>simpler API reporting only cpuThreads and cpuCores; with no >>funny >>boolean flags. >> >>Let us strive to that position as much as we can. >> >>How about asking whoever used report_host_threads_as_cores >>to >>unset >>it >>once they install Engine 3.2 ? I think that these are very >>few >>people, >>that would not mind this very much. >> >>If this is impossible, I'd add a cpuCores2, always reporting >>the >>true >>number, to be used by new Engines. We may even report it >>only >>on >>the >>very few cases of report_host_threads_as_cores being set. >> >>Dan. > >Hi Dan, >Thanks for the review. > >I agree simply reporting cores and threads would be the right >solution. >However, when you have hyperthreading turned off you get >cores=threads. >This is the same situation you have when hyperthreading >turned >on, and >someone used the vdsm configuration of reporting threads as >cores. > >So the engine won't know the real status of the host.
This is not surprising, as report_host_threads_as_cores means in blunt English "lie to Engine about the number of cores". The newly suggested flag says "don't believe what I said in cpuCores, since I'm lying". Next thing we'd have is another flag saying that the former flag was a lie, and cpuCores is actually trustworthy.
Instead of dancing this dance, I suggest to stop lying.
report_host_threads_as_cores was a hack to assist a older Engine versions. Engine users that needed it had to set it out-of-band on their hosts. Now if they upgrade their Engine, they can -- as easily -- reset that value.
If they forget, nothing devastating happens beyond Engine assuming that hyperthreading is off.
Please consider this suggestion. I find it the simplest for all involved parties.
the only problem is the new vdsm doesn't know which engine may be using it. if engine would say "getVdsCaps engineVersion=3.2", then vdsm could know engine no longer needs lying to and ignore the flag, re-using same field.
Note that I do not suggest to drop report_host_threads_as_cores now. I am suggesting to keep on lying even to new Engine. If someone thinks that lying is bad, he should reset report_host_threads_as_cores.
It seems to me that the suggested API is being coerced by a very limited use case, that is not going to be really harmed by a straight-forward API.
Dan.
Dan, Did some further checking, and we can go with it; So basically now we add cpuThreads. Additionally, if the report_host_threads_as_cores is turned on, an additional cpuCoresReal will be reported.
No need for that. There is only one problematic state where VDSM cheats and reports cores == threads. This does not happen by mistake, the user specifically asked for it.
The above condition above also happens if threads is really off or the processor does not have threads, so it's a state we need to handle in any case.
So just report threads=off, whenever cores == threads and treat it as such. If the user is unhappy then he should just turn off the cheat mode.
ack.
* <3.2 clusters are not supported. * >=3.2 clusters will assume HT off unless cores < threads.
Some release note is needed for this, so users will be advised to turn off vdsm cheating when upgrading to new 3.2 engine, in case they happen to use it.
Plain and simple. vdsm side accepted to master branch. Please backport it to the ovirt-3.2 branch in order to get into the release.
participants (7)
-
Andrew Cathrow -
Dan Kenigsberg -
Dennis Jacobfeuerborn -
Doron Fediuck -
Greg Padgett -
Itamar Heim -
Simon Grinberg