Adding Memory Overcommitment Manager (MOM) to oVirt
Hi all, I would like propose adding the Memory Overcommitment Manager (MOM) project to oVirt. MOM is a daemon that is used to apply dynamic host-level and VM-level tuning according to a customizable policy. Today MOM can control memory ballooning and KSM. Please see the following links for more information: Project wiki: http://github.com/aglitke/mom/wiki Source code: git://github.com/aglitke/mom.git KVM Forum presentation and video: http://www.linux-kvm.org/wiki/images/e/e8/2010-forum-litke-kvmforum2010.pdf http://vimeo.com/15223652 developerWorks article: http://www.ibm.com/developerworks/linux/library/l-overcommit-kvm-resources/ In my view, MOM could be nicely integrated with oVirt-node and VDSM. I am looking forward to your comments and am ready to address some next steps. Thanks! -- Adam Litke <agl@us.ibm.com> IBM Linux Technology Center
On 09/26/2011 11:01 PM, Adam Litke wrote:
Hi all,
I would like propose adding the Memory Overcommitment Manager (MOM) project to oVirt. MOM is a daemon that is used to apply dynamic host-level and VM-level tuning according to a customizable policy. Today MOM can control memory ballooning and KSM. Please see the following links for more information:
Project wiki: http://github.com/aglitke/mom/wiki
Source code: git://github.com/aglitke/mom.git
KVM Forum presentation and video: http://www.linux-kvm.org/wiki/images/e/e8/2010-forum-litke-kvmforum2010.pdf http://vimeo.com/15223652
developerWorks article: http://www.ibm.com/developerworks/linux/library/l-overcommit-kvm-resources/
In my view, MOM could be nicely integrated with oVirt-node and VDSM. I am looking forward to your comments and am ready to address some next steps.
Thanks!
+1 for mom, hope to see a dad too :)
On 09/26/2011 03:19 PM, Dor Laor wrote:
On 09/26/2011 11:01 PM, Adam Litke wrote:
Hi all,
I would like propose adding the Memory Overcommitment Manager (MOM) project to oVirt. MOM is a daemon that is used to apply dynamic host-level and VM-level tuning according to a customizable policy. Today MOM can control memory ballooning and KSM. Please see the following links for more information:
Project wiki: http://github.com/aglitke/mom/wiki
Source code: git://github.com/aglitke/mom.git
KVM Forum presentation and video: http://www.linux-kvm.org/wiki/images/e/e8/2010-forum-litke-kvmforum2010.pdf http://vimeo.com/15223652
developerWorks article: http://www.ibm.com/developerworks/linux/library/l-overcommit-kvm-resources/
In my view, MOM could be nicely integrated with oVirt-node and VDSM. I am looking forward to your comments and am ready to address some next steps.
Thanks!
+1 for mom, hope to see a dad too :)
Data Acceleration Director? Joking aside, would MOM just be integrated directly into VDSM or would it remain as a separate project? I can envision a world where MOM is just a feature of VDSM. Regards, Anthony Liguori
_______________________________________________ Board mailing list Board@ovirt.org http://lists.ovirt.org/mailman/listinfo/board
On 09/26/2011 04:26 PM, Anthony Liguori wrote:
On 09/26/2011 03:19 PM, Dor Laor wrote:
On 09/26/2011 11:01 PM, Adam Litke wrote:
Hi all,
I would like propose adding the Memory Overcommitment Manager (MOM) project to oVirt. MOM is a daemon that is used to apply dynamic host-level and VM-level tuning according to a customizable policy. Today MOM can control memory ballooning and KSM. Please see the following links for more information:
Project wiki: http://github.com/aglitke/mom/wiki
Source code: git://github.com/aglitke/mom.git
KVM Forum presentation and video: http://www.linux-kvm.org/wiki/images/e/e8/2010-forum-litke-kvmforum2010.pdf
developerWorks article: http://www.ibm.com/developerworks/linux/library/l-overcommit-kvm-resources/
In my view, MOM could be nicely integrated with oVirt-node and VDSM. I am looking forward to your comments and am ready to address some next steps.
Thanks!
+1 for mom, hope to see a dad too :)
Data Acceleration Director?
Joking aside, would MOM just be integrated directly into VDSM or would it remain as a separate project?
I can envision a world where MOM is just a feature of VDSM.
The key question is how do we integrate it into the stack, - is it integrated into WDSM as Anthony mentions - or is a project that WDSM uses to provide MOM - etc. Carl.
----- Original Message -----
On 09/26/2011 04:26 PM, Anthony Liguori wrote:
On 09/26/2011 03:19 PM, Dor Laor wrote:
On 09/26/2011 11:01 PM, Adam Litke wrote:
Hi all,
I would like propose adding the Memory Overcommitment Manager (MOM) project to oVirt. MOM is a daemon that is used to apply dynamic host-level and VM-level tuning according to a customizable policy. Today MOM can control memory ballooning and KSM. Please see the following links for more information:
Project wiki: http://github.com/aglitke/mom/wiki
Source code: git://github.com/aglitke/mom.git
KVM Forum presentation and video: http://www.linux-kvm.org/wiki/images/e/e8/2010-forum-litke-kvmforum2010.pdf
developerWorks article: http://www.ibm.com/developerworks/linux/library/l-overcommit-kvm-resources/
In my view, MOM could be nicely integrated with oVirt-node and VDSM. I am looking forward to your comments and am ready to address some next steps.
Thanks!
+1 for mom, hope to see a dad too :)
Data Acceleration Director?
Joking aside, would MOM just be integrated directly into VDSM or would it remain as a separate project?
I can envision a world where MOM is just a feature of VDSM.
The key question is how do we integrate it into the stack, - is it integrated into WDSM as Anthony mentions - or is a project that WDSM uses to provide MOM - etc.
First of all I think this is great! Thanks Adam! Wrt the question above, the way I see vdsm going forward is a lot more policy management, especially once we start talking about SLAs. This will involve a lot more than just memory policies, and although I may be impartial here, I think that this should fall under vdsm or under a policy engine project (bigger scope than just memory) which vdsm would use. To elaborate, in addition to memory, we need to manage policies around network, storage and cpu and manipulate cgroups, tc, page cache etc. If we take page cache as an example, it affects memory on the machine hence, MOM would like to tweak it (also in the preso), but we already have recent results from the perf team to change the defaults here to significantly increase I/O throughput, improve fairness and reduce effect of buffered writers on VMs and we know that going forward we'll have to tweak these parameters dynamically to adjust to varying workloads, esp if we ever plan on making good the claim for a hybrid server. Having separate projects tweaking the same params will not lead to good results.
Carl.
_______________________________________________ Board mailing list Board@ovirt.org http://lists.ovirt.org/mailman/listinfo/board
On Mon, Sep 26, 2011 at 05:28:54PM -0400, Ayal Baron wrote:
----- Original Message -----
On 09/26/2011 04:26 PM, Anthony Liguori wrote:
On 09/26/2011 03:19 PM, Dor Laor wrote:
On 09/26/2011 11:01 PM, Adam Litke wrote:
Hi all,
I would like propose adding the Memory Overcommitment Manager (MOM) project to oVirt. MOM is a daemon that is used to apply dynamic host-level and VM-level tuning according to a customizable policy. Today MOM can control memory ballooning and KSM. Please see the following links for more information:
Project wiki: http://github.com/aglitke/mom/wiki
Source code: git://github.com/aglitke/mom.git
KVM Forum presentation and video: http://www.linux-kvm.org/wiki/images/e/e8/2010-forum-litke-kvmforum2010.pdf
developerWorks article: http://www.ibm.com/developerworks/linux/library/l-overcommit-kvm-resources/
In my view, MOM could be nicely integrated with oVirt-node and VDSM. I am looking forward to your comments and am ready to address some next steps.
Thanks!
+1 for mom, hope to see a dad too :)
Data Acceleration Director?
Joking aside, would MOM just be integrated directly into VDSM or would it remain as a separate project?
I can envision a world where MOM is just a feature of VDSM.
The key question is how do we integrate it into the stack, - is it integrated into WDSM as Anthony mentions - or is a project that WDSM uses to provide MOM - etc.
First of all I think this is great! Thanks Adam! Wrt the question above, the way I see vdsm going forward is a lot more policy management, especially once we start talking about SLAs. This will involve a lot more than just memory policies, and although I may be impartial here, I think that this should fall under vdsm or under a policy engine project (bigger scope than just memory) which vdsm would use. To elaborate, in addition to memory, we need to manage policies around network, storage and cpu and manipulate cgroups, tc, page cache etc. If we take page cache as an example, it affects memory on the machine hence, MOM would like to tweak it (also in the preso), but we already have recent results from the perf team to change the defaults here to significantly increase I/O throughput, improve fairness and reduce effect of buffered writers on VMs and we know that going forward we'll have to tweak these parameters dynamically to adjust to varying workloads, esp if we ever plan on making good the claim for a hybrid server. Having separate projects tweaking the same params will not lead to good results.
Ayal, I definitely agree. I must admit that the project name MOM (while a nice acronym), implies a restricted scope (memory only) but the architecture of MOM does not limit its functionality to memory tuning only. I assert that MOM can easily become the policy engine piece that you are speaking of. The framework is very extensible (a set of Collectors that expose system state, a set of Controllers that expose "tuning knobs", and a policy that reads the state and manipulates the knobs. Adding control for cgroups is as simple as writing a cgroups Collector and Controller. -- Adam Litke <agl@us.ibm.com> IBM Linux Technology Center
On 09/27/2011 12:52 AM, Adam Litke wrote:
First of all I think this is great! Thanks Adam! Wrt the question above, the way I see vdsm going forward is a lot more policy management, especially once we start talking about SLAs. This will involve a lot more than just memory policies, and although I may be impartial here, I think that this should fall under vdsm or under a policy engine project (bigger scope than just memory) which vdsm would use. To elaborate, in addition to memory, we need to manage policies around network, storage and cpu and manipulate cgroups, tc, page cache etc. If we take page cache as an example, it affects memory on the machine hence, MOM would like to tweak it (also in the preso), but we already have recent results from the perf team to change the defaults here to significantly increase I/O throughput, improve fairness and reduce effect of buffered writers on VMs and we know that going forward we'll have to tweak these parameters dynamically to adjust to varying workloads, esp if we ever plan on making good the claim for a hybrid server. Having separate projects tweaking the same params will not lead to good results. Ayal, I definitely agree. I must admit that the project name MOM (while a nice acronym), implies a restricted scope (memory only) but the architecture of MOM does not limit its functionality to memory tuning only. I assert that MOM can easily become the policy engine piece that you are speaking of.
The framework is very extensible (a set of Collectors that expose system state, a set of Controllers that expose "tuning knobs", and a policy that reads the state and manipulates the knobs. Adding control for cgroups is as simple as writing a cgroups Collector and Controller.
It would be nice to agree on the initial (more limited scope maybe) for the code within vdsm so the work be able to be done soonish. Since vdsm is a relevantly a new open source project there will be many future changes and imho there is no need to wait for the perfect policy engine framework.
----- Original Message -----
On 09/27/2011 12:52 AM, Adam Litke wrote:
First of all I think this is great! Thanks Adam! Wrt the question above, the way I see vdsm going forward is a lot more policy management, especially once we start talking about SLAs. This will involve a lot more than just memory policies, and although I may be impartial here, I think that this should fall under vdsm or under a policy engine project (bigger scope than just memory) which vdsm would use. To elaborate, in addition to memory, we need to manage policies around network, storage and cpu and manipulate cgroups, tc, page cache etc. If we take page cache as an example, it affects memory on the machine hence, MOM would like to tweak it (also in the preso), but we already have recent results from the perf team to change the defaults here to significantly increase I/O throughput, improve fairness and reduce effect of buffered writers on VMs and we know that going forward we'll have to tweak these parameters dynamically to adjust to varying workloads, esp if we ever plan on making good the claim for a hybrid server. Having separate projects tweaking the same params will not lead to good results. Ayal, I definitely agree. I must admit that the project name MOM (while a nice acronym), implies a restricted scope (memory only) but the architecture of MOM does not limit its functionality to memory tuning only. I assert that MOM can easily become the policy engine piece that you are speaking of.
The framework is very extensible (a set of Collectors that expose system state, a set of Controllers that expose "tuning knobs", and a policy that reads the state and manipulates the knobs. Adding control for cgroups is as simple as writing a cgroups Collector and Controller.
It would be nice to agree on the initial (more limited scope maybe) for the code within vdsm so the work be able to be done soonish. Since vdsm is a relevantly a new open source project there will be many future changes and imho there is no need to wait for the perfect policy engine framework.
The project can initially contain only memory management, that is not an issue whatsoever. But the project mission should be correct from the start so that design decisions are made with the full picture in mind.
Am 27.09.2011 um 08:56 schrieb "Ayal Baron" <abaron@redhat.com>:
----- Original Message -----
On 09/27/2011 12:52 AM, Adam Litke wrote:
First of all I think this is great! Thanks Adam! Wrt the question above, the way I see vdsm going forward is a lot more policy management, especially once we start talking about SLAs. This will involve a lot more than just memory policies, and although I may be impartial here, I think that this should fall under vdsm or under a policy engine project (bigger scope than just memory) which vdsm would use. To elaborate, in addition to memory, we need to manage policies around network, storage and cpu and manipulate cgroups, tc, page cache etc. If we take page cache as an example, it affects memory on the machine hence, MOM would like to tweak it (also in the preso), but we already have recent results from the perf team to change the defaults here to significantly increase I/O throughput, improve fairness and reduce effect of buffered writers on VMs and we know that going forward we'll have to tweak these parameters dynamically to adjust to varying workloads, esp if we ever plan on making good the claim for a hybrid server. Having separate projects tweaking the same params will not lead to good results. Ayal, I definitely agree. I must admit that the project name MOM (while a nice acronym), implies a restricted scope (memory only) but the architecture of MOM does not limit its functionality to memory tuning only. I assert that MOM can easily become the policy engine piece that you are speaking of.
The framework is very extensible (a set of Collectors that expose system state, a set of Controllers that expose "tuning knobs", and a policy that reads the state and manipulates the knobs. Adding control for cgroups is as simple as writing a cgroups Collector and Controller.
It would be nice to agree on the initial (more limited scope maybe) for the code within vdsm so the work be able to be done soonish. Since vdsm is a relevantly a new open source project there will be many future changes and imho there is no need to wait for the perfect policy engine framework.
The project can initially contain only memory management, that is not an issue whatsoever. But the project mission should be correct from the start so that design decisions are made with the full picture in mind.
Let me try to grasp the full picture here. VDSM is basically the 'node controller', managing all active VMs on a system, correct? So this is the instance that watches things on node granularity. MOM watches resources on node granularity, interacting only with vdsm and the VM(?) to manage overcommit of memory. Why should this be separate from VDSM? Does it buy us anything as a separate project? Is the interface cut here helpful in any way? I don't want to see 50 separate daemons that in reality are a single entity but have to be artificially maintained separate, because someone thought it's nice to split things into projects instead of submodules of a project. So what's the benefit of a separate MOM vs an integrated MOM inside of VDSM? Alex
----- Original Message -----
From: "Alexander Graf" <agraf@suse.de> To: "Ayal Baron" <abaron@redhat.com> Cc: "<board@ovirt.org>" <board@ovirt.org> Sent: Tuesday, September 27, 2011 3:28:29 AM Subject: Re: Adding Memory Overcommitment Manager (MOM) to oVirt
Am 27.09.2011 um 08:56 schrieb "Ayal Baron" <abaron@redhat.com>:
----- Original Message -----
On 09/27/2011 12:52 AM, Adam Litke wrote:
First of all I think this is great! Thanks Adam! Wrt the question above, the way I see vdsm going forward is a lot more policy management, especially once we start talking about SLAs. This will involve a lot more than just memory policies, and although I may be impartial here, I think that this should fall under vdsm or under a policy engine project (bigger scope than just memory) which vdsm would use. To elaborate, in addition to memory, we need to manage policies around network, storage and cpu and manipulate cgroups, tc, page cache etc. If we take page cache as an example, it affects memory on the machine hence, MOM would like to tweak it (also in the preso), but we already have recent results from the perf team to change the defaults here to significantly increase I/O throughput, improve fairness and reduce effect of buffered writers on VMs and we know that going forward we'll have to tweak these parameters dynamically to adjust to varying workloads, esp if we ever plan on making good the claim for a hybrid server. Having separate projects tweaking the same params will not lead to good results. Ayal, I definitely agree. I must admit that the project name MOM (while a nice acronym), implies a restricted scope (memory only) but the architecture of MOM does not limit its functionality to memory tuning only. I assert that MOM can easily become the policy engine piece that you are speaking of.
The framework is very extensible (a set of Collectors that expose system state, a set of Controllers that expose "tuning knobs", and a policy that reads the state and manipulates the knobs. Adding control for cgroups is as simple as writing a cgroups Collector and Controller.
It would be nice to agree on the initial (more limited scope maybe) for the code within vdsm so the work be able to be done soonish. Since vdsm is a relevantly a new open source project there will be many future changes and imho there is no need to wait for the perfect policy engine framework.
The project can initially contain only memory management, that is not an issue whatsoever. But the project mission should be correct from the start so that design decisions are made with the full picture in mind.
Let me try to grasp the full picture here. VDSM is basically the 'node controller', managing all active VMs on a system, correct? So this is the instance that watches things on node granularity.
MOM watches resources on node granularity, interacting only with vdsm and the VM(?) to manage overcommit of memory. Why should this be separate from VDSM? Does it buy us anything as a separate project? Is the interface cut here helpful in any way?
I don't want to see 50 separate daemons that in reality are a single entity but have to be artificially maintained separate, because someone thought it's nice to split things into projects instead of submodules of a project.
So what's the benefit of a separate MOM vs an integrated MOM inside of VDSM?
I'd agree this is really VM policy which should be handled by VDSM.
Alex _______________________________________________ Board mailing list Board@ovirt.org http://lists.ovirt.org/mailman/listinfo/board
So what's the benefit of a separate MOM vs an integrated MOM inside of VDSM?
I'd agree this is really VM policy which should be handled by VDSM.
It's pretty simple. MOM does something valuable today that VDSM doesn't do. An integrated MOM inside of VDSM doesn't exist. If VDSM wants to incorporate MOM that's great. But how is the existence of VDSM an argument against contributing MOM source code to the oVirt community? Am I missing something? On a related point, there are good examples that argue for consolidating function inside a single daemon, and there are good counterexamples. It's not always true that every node policy function should be integrated within a single daemon. It probably makes sense in this case. But again, VDSM doesn't do what MOM does today, which is another argument in favor of contributing MOM and letting the community work with it. Thanks, Mike Mike Day IBM Distinguished Engineer Chief Virtualization Architect, Open Systems Development Cell: +1 919 371-8786 | mdday@us.ibm.com http://code.ncultra.org
On 09/27/2011 09:43 AM, Michael D Day wrote:
So what's the benefit of a separate MOM vs an integrated MOM inside of VDSM?
I'd agree this is really VM policy which should be handled by VDSM.
It's pretty simple. MOM does something valuable today that VDSM doesn't do. An integrated MOM inside of VDSM doesn't exist. If VDSM wants to incorporate MOM that's great. But how is the existence of VDSM an argument against contributing MOM source code to the oVirt community? Am I missing something?
On a related point, there are good examples that argue for consolidating function inside a single daemon, and there are good counterexamples. It's not always true that every node policy function should be integrated within a single daemon. It probably makes sense in this case. But again, VDSM doesn't do what MOM does today, which is another argument in favor of contributing MOM and letting the community work with it.
I think the debate has ascertained that we want it. The question is how we integrate it... (used by VDSM or additional daemon) I like Dor's suggestion. Let's let the guys figure out the best way to do an initial integration given where we are at, we play with it, and if in future we want to evolve the integration we can. Carl. Carl.
On Tue, Sep 27, 2011 at 09:53:00AM -0400, Carl Trieloff wrote:
On 09/27/2011 09:43 AM, Michael D Day wrote:
So what's the benefit of a separate MOM vs an integrated MOM inside of VDSM?
I'd agree this is really VM policy which should be handled by VDSM.
It's pretty simple. MOM does something valuable today that VDSM doesn't do. An integrated MOM inside of VDSM doesn't exist. If VDSM wants to incorporate MOM that's great. But how is the existence of VDSM an argument against contributing MOM source code to the oVirt community? Am I missing something?
On a related point, there are good examples that argue for consolidating function inside a single daemon, and there are good counterexamples. It's not always true that every node policy function should be integrated within a single daemon. It probably makes sense in this case. But again, VDSM doesn't do what MOM does today, which is another argument in favor of contributing MOM and letting the community work with it.
I think the debate has ascertained that we want it. The question is how we integrate it... (used by VDSM or additional daemon) I like Dor's suggestion. Let's let the guys figure out the best way to do an initial integration given where we are at, we play with it, and if in future we want to evolve the integration we can.
I completely agree and (boldly perhaps) assume we are driving at a consensus here. In that case, what are the next steps for MOM from an oVirt project perspective? For Hosting, the project happily resides on github. I created a mailing list using Google Groups, but discussions have mostly been had on other projects' lists. Although I am not completely opposed to renaming it, I am not sure it's worth the effort if we can see uses for this project outside of the oVirt umbrella. Perhaps we can revisit that in the future as things shake out. A decent next step would be to get MOM included into Fedora. I have a stalled request that I could use some help moving along: https://bugzilla.redhat.com/show_bug.cgi?id=638647 The package is ready (spec file has been approved) but needs a Fedora package sponsor who is willing to guide it through the remaining steps of the process. Anything else? -- Adam Litke <agl@us.ibm.com> IBM Linux Technology Center
On 09/27/2011 10:32 AM, Adam Litke wrote:
On 09/27/2011 09:43 AM, Michael D Day wrote:
So what's the benefit of a separate MOM vs an integrated MOM inside of VDSM? I'd agree this is really VM policy which should be handled by VDSM. It's pretty simple. MOM does something valuable today that VDSM doesn't do. An integrated MOM inside of VDSM doesn't exist. If VDSM wants to incorporate MOM that's great. But how is the existence of VDSM an argument against contributing MOM source code to the oVirt community? Am I missing something?
On a related point, there are good examples that argue for consolidating function inside a single daemon, and there are good counterexamples. It's not always true that every node policy function should be integrated within a single daemon. It probably makes sense in this case. But again, VDSM doesn't do what MOM does today, which is another argument in favor of contributing MOM and letting the community work with it.
I think the debate has ascertained that we want it. The question is how we integrate it... (used by VDSM or additional daemon) I like Dor's suggestion. Let's let the guys figure out the best way to do an initial integration given where we are at, we play with it, and if in future we want to evolve the integration we can. I completely agree and (boldly perhaps) assume we are driving at a consensus here. In that case, what are the next steps for MOM from an oVirt project
On Tue, Sep 27, 2011 at 09:53:00AM -0400, Carl Trieloff wrote: perspective?
For Hosting, the project happily resides on github. I created a mailing list using Google Groups, but discussions have mostly been had on other projects' lists. Although I am not completely opposed to renaming it, I am not sure it's worth the effort if we can see uses for this project outside of the oVirt umbrella. Perhaps we can revisit that in the future as things shake out.
A decent next step would be to get MOM included into Fedora. I have a stalled request that I could use some help moving along: https://bugzilla.redhat.com/show_bug.cgi?id=638647 The package is ready (spec file has been approved) but needs a Fedora package sponsor who is willing to guide it through the remaining steps of the process.
Anything else?
I'll be sending out a vote mail to include it shortly. Thanks for being patient as we get our processes worked out. After the vote, we can get infra requires / lists / projects page updated etc and whatever else needs needs to be done. Carl.
On 27.09.2011, at 15:43, Michael D Day wrote:
So what's the benefit of a separate MOM vs an integrated MOM inside of VDSM?
I'd agree this is really VM policy which should be handled by VDSM.
It's pretty simple. MOM does something valuable today that VDSM doesn't do. An integrated MOM inside of VDSM doesn't exist. If VDSM wants to incorporate MOM that's great. But how is the existence of VDSM an argument against contributing MOM source code to the oVirt community? Am I missing something?
Oh, sorry if that's what it felt like I was suggesting. It's not at all. I was just wondering if it makes more sense to integrate MOM into VDSM vs. keeping both separate projects that don't share code.
On a related point, there are good examples that argue for consolidating function inside a single daemon, and there are good counterexamples. It's not always true that every node policy function should be integrated within a single daemon. It probably makes sense in this case. But again, VDSM doesn't do what MOM does today, which is another argument in favor of contributing MOM and letting the community work with it.
I fully agree. This is more of a "big picture" discussion again, not a nack or anything even remotely resembling such :) Alex
----- Original Message -----
From: "Michael D Day" <mdday@us.ibm.com> To: "Andrew Cathrow" <acathrow@redhat.com> Cc: "Alexander Graf" <agraf@suse.de>, "board" <board@ovirt.org>, board-bounces@ovirt.org Sent: Tuesday, September 27, 2011 9:43:39 AM Subject: Re: Adding Memory Overcommitment Manager (MOM) to oVirt
So what's the benefit of a separate MOM vs an integrated MOM inside of VDSM?
I'd agree this is really VM policy which should be handled by VDSM.
It's pretty simple. MOM does something valuable today that VDSM doesn't do. An integrated MOM inside of VDSM doesn't exist. If VDSM wants to incorporate MOM that's great. But how is the existence of VDSM an argument against contributing MOM source code to the oVirt community? Am I missing something?
Sorry, I should have given a multi line reply, I was thinking a few steps ahead. Bringing MOM under the oVirt umbrella would be a great thing - whether that's as a standalone project or as part of VDSM. Personally I believe it will/should end up as part of VDSM, but either way - if VDSM leverages MOM (like it leverages libvirt) or if it gets merged into VDSM this should happen under the oVirt umbrella. Adding MOM to oVirt would be great and also helps to show momentum, maybe it remains standalone or maybe it becomes part of VDSM, either way the path forward should start with adding the project into oVirt. Aic
On a related point, there are good examples that argue for consolidating function inside a single daemon, and there are good counterexamples. It's not always true that every node policy function should be integrated within a single daemon. It probably makes sense in this case. But again, VDSM doesn't do what MOM does today, which is another argument in favor of contributing MOM and letting the community work with it.
Thanks,
Mike
Mike Day IBM Distinguished Engineer Chief Virtualization Architect, Open Systems Development Cell: +1 919 371-8786 | mdday@us.ibm.com http://code.ncultra.org
On 09/27/2011 08:59 AM, Andrew Cathrow wrote:
Sorry, I should have given a multi line reply, I was thinking a few steps ahead.
Bringing MOM under the oVirt umbrella would be a great thing - whether that's as a standalone project or as part of VDSM.
Personally I believe it will/should end up as part of VDSM, but either way - if VDSM leverages MOM (like it leverages libvirt) or if it gets merged into VDSM this should happen under the oVirt umbrella.
Adding MOM to oVirt would be great and also helps to show momentum, maybe it remains standalone or maybe it becomes part of VDSM, either way the path forward should start with adding the project into oVirt.
I think this discussion highlights the need for a bit more structure for adding a sub project. I would like to propose the following: 1) Project posts an overview page, including scope and short term roadmap to ovirt.org wiki. That page is then copied as a mail to board@ovirt 2) There is an official comment period that lasts for a fixed period of time (2-4 weeks?). This time period allows for questions to be enhanced, and the wiki to be improved. 3) At any point in this time period, the proposal can be officially withdrawn. 4) At the end of the period, there is a final call for comments before a vote takes place. Thoughts? Regards, Anthony Liguori
Aic
On a related point, there are good examples that argue for consolidating function inside a single daemon, and there are good counterexamples. It's not always true that every node policy function should be integrated within a single daemon. It probably makes sense in this case. But again, VDSM doesn't do what MOM does today, which is another argument in favor of contributing MOM and letting the community work with it.
Thanks,
Mike
Mike Day IBM Distinguished Engineer Chief Virtualization Architect, Open Systems Development Cell: +1 919 371-8786 | mdday@us.ibm.com http://code.ncultra.org
_______________________________________________ Board mailing list Board@ovirt.org http://lists.ovirt.org/mailman/listinfo/board
On 09/27/2011 10:27 AM, Anthony Liguori wrote:
I think this discussion highlights the need for a bit more structure for adding a sub project. I would like to propose the following:
1) Project posts an overview page, including scope and short term roadmap to ovirt.org wiki. That page is then copied as a mail to board@ovirt
2) There is an official comment period that lasts for a fixed period of time (2-4 weeks?). This time period allows for questions to be enhanced, and the wiki to be improved.
3) At any point in this time period, the proposal can be officially withdrawn.
4) At the end of the period, there is a final call for comments before a vote takes place.
I like. Don't think we need 2-4 weeks. maybe say discuss till basic agreement is reached. could be 24 hours, could be weeks. Let's also add the basic questions for the new project to answer, and turn this into an additional page on the site. Carl.
On 09/27/2011 09:32 AM, Carl Trieloff wrote:
On 09/27/2011 10:27 AM, Anthony Liguori wrote:
I think this discussion highlights the need for a bit more structure for adding a sub project. I would like to propose the following:
1) Project posts an overview page, including scope and short term roadmap to ovirt.org wiki. That page is then copied as a mail to board@ovirt
2) There is an official comment period that lasts for a fixed period of time (2-4 weeks?). This time period allows for questions to be enhanced, and the wiki to be improved.
3) At any point in this time period, the proposal can be officially withdrawn.
4) At the end of the period, there is a final call for comments before a vote takes place.
I like. Don't think we need 2-4 weeks. maybe say discuss till basic agreement is reached. could be 24 hours, could be weeks.
The other thing I was thinking is that could have Acked-by's and wait to get N Acked-by's before going to a vote. The reason for having a time period was to give people time to actually look at code and read proposals instead of just immediately jumping into a heated debate. I think forcing a little think time is a good thing.
Let's also add the basic questions for the new project to answer, and turn this into an additional page on the site.
Do we have a wiki ETA? I'd be happy to start putting together questions. Regards, Anthony Liguori
Carl. _______________________________________________ Board mailing list Board@ovirt.org http://lists.ovirt.org/mailman/listinfo/board
On 09/27/2011 10:43 AM, Anthony Liguori wrote:
The other thing I was thinking is that could have Acked-by's and wait to get N Acked-by's before going to a vote.
The reason for having a time period was to give people time to actually look at code and read proposals instead of just immediately jumping into a heated debate. I think forcing a little think time is a good thing.
yes, my point is more around, do it when ready, versus on a specific timeline.
Let's also add the basic questions for the new project to answer, and turn this into an additional page on the site.
Do we have a wiki ETA? I'd be happy to start putting together questions.
wiki is live. ovirt.org/wiki Carl.
On Tue, Sep 27, 2011 at 11:03:01AM -0400, Carl Trieloff wrote:
wiki is live.
ovirt.org/wiki
Great! I would like to create a page to discuss MOM oVirt integration. Could I have an account from an existing account holder (username: aglitke) please? Thanks! -- Adam Litke <agl@us.ibm.com> IBM Linux Technology Center
----- Original Message -----
On Tue, Sep 27, 2011 at 11:03:01AM -0400, Carl Trieloff wrote:
wiki is live.
ovirt.org/wiki
Great! I would like to create a page to discuss MOM oVirt integration. Could I have an account from an existing account holder (username: aglitke) please? Thanks!
Excellent as I have some questions and would also like some time to examine the code (first code, then questions).
-- Adam Litke <agl@us.ibm.com> IBM Linux Technology Center
_______________________________________________ Board mailing list Board@ovirt.org http://lists.ovirt.org/mailman/listinfo/board
* Anthony Liguori <aliguori@us.ibm.com> [2011-09-27 09:30]:
On 09/27/2011 08:59 AM, Andrew Cathrow wrote:
Sorry, I should have given a multi line reply, I was thinking a few steps ahead.
Bringing MOM under the oVirt umbrella would be a great thing - whether that's as a standalone project or as part of VDSM.
Personally I believe it will/should end up as part of VDSM, but either way - if VDSM leverages MOM (like it leverages libvirt) or if it gets merged into VDSM this should happen under the oVirt umbrella.
Adding MOM to oVirt would be great and also helps to show momentum, maybe it remains standalone or maybe it becomes part of VDSM, either way the path forward should start with adding the project into oVirt.
I think this discussion highlights the need for a bit more structure for adding a sub project. I would like to propose the following:
1) Project posts an overview page, including scope and short term roadmap to ovirt.org wiki. That page is then copied as a mail to board@ovirt
2) There is an official comment period that lasts for a fixed period of time (2-4 weeks?). This time period allows for questions to be enhanced, and the wiki to be improved.
3) At any point in this time period, the proposal can be officially withdrawn.
4) At the end of the period, there is a final call for comments before a vote takes place.
Thoughts?
I like the structured approach here. +1
Regards,
Anthony Liguori
Aic
On a related point, there are good examples that argue for consolidating function inside a single daemon, and there are good counterexamples. It's not always true that every node policy function should be integrated within a single daemon. It probably makes sense in this case. But again, VDSM doesn't do what MOM does today, which is another argument in favor of contributing MOM and letting the community work with it.
Thanks,
Mike
Mike Day IBM Distinguished Engineer Chief Virtualization Architect, Open Systems Development Cell: +1 919 371-8786 | mdday@us.ibm.com http://code.ncultra.org
_______________________________________________ Board mailing list Board@ovirt.org http://lists.ovirt.org/mailman/listinfo/board
_______________________________________________ Board mailing list Board@ovirt.org http://lists.ovirt.org/mailman/listinfo/board
-- Ryan Harper Software Engineer; Linux Technology Center IBM Corp., Austin, Tx ryanh@us.ibm.com
----- Original Message -----
So what's the benefit of a separate MOM vs an integrated MOM inside of VDSM?
I'd agree this is really VM policy which should be handled by VDSM.
It's pretty simple. MOM does something valuable today that VDSM doesn't do. An integrated MOM inside of VDSM doesn't exist. If VDSM wants to incorporate MOM that's great. But how is the existence of VDSM an argument against contributing MOM source code to the oVirt community? Am I missing something?
No one is against accepting MOM into oVirt, on the contrary, we really want it. The only question is do we want to assimilate the code into vdsm or manage it as a separate project. And the question here for me is just whether this would have other consumers (in which case a separate project could make sense) or would vdsm be the only user (in which case it at most should be a submodule).
On a related point, there are good examples that argue for consolidating function inside a single daemon, and there are good counterexamples. It's not always true that every node policy function should be integrated within a single daemon. It probably makes sense in this case. But again, VDSM doesn't do what MOM does today, which is another argument in favor of contributing MOM and letting the community work with it.
vdsm today does integrate (in a degenerated way) with ksm today (which is part of what mom does), as mentioned before we're going to tweak page cache settings for 3.0GA and in general vdsm does a lot of policy management and will do a lot more. Obviously if vdsm did everything that mom does, then mom wouldn't be interesting at all.
Thanks,
Mike
Mike Day IBM Distinguished Engineer Chief Virtualization Architect, Open Systems Development Cell: +1 919 371-8786 | mdday@us.ibm.com http://code.ncultra.org
_______________________________________________ Board mailing list Board@ovirt.org http://lists.ovirt.org/mailman/listinfo/board
Ayal Baron <abaron@redhat.com> wrote on 09/27/2011 10:02:23 AM:
No one is against accepting MOM into oVirt, on the contrary, we really want it. The only question is do we want to assimilate the code into vdsm or manage it as a separate project.
Thanks for the clarification folks, I read in-between the lines of Andy's single-line response :-). Thanks, Mike Mike Day IBM Distinguished Engineer Chief Virtualization Architect, Open Systems Development Cell: +1 919 371-8786 | mdday@us.ibm.com http://code.ncultra.org
On 09/27/2011 10:07 AM, Michael D Day wrote:
Ayal Baron <abaron@redhat.com> wrote on 09/27/2011 10:02:23 AM:
No one is against accepting MOM into oVirt, on the contrary, we really want it. The only question is do we want to assimilate the code into vdsm or manage it as a separate project.
Thanks for the clarification folks, I read in-between the lines of Andy's single-line response :-).
Andy has a 3? week old and is not sleeping :-) -- expect short answers, long ones require a good nights rest... Carl.
On Tue, Sep 27, 2011 at 10:02:23AM -0400, Ayal Baron wrote:
----- Original Message -----
So what's the benefit of a separate MOM vs an integrated MOM inside of VDSM?
I'd agree this is really VM policy which should be handled by VDSM.
It's pretty simple. MOM does something valuable today that VDSM doesn't do. An integrated MOM inside of VDSM doesn't exist. If VDSM wants to incorporate MOM that's great. But how is the existence of VDSM an argument against contributing MOM source code to the oVirt community? Am I missing something?
No one is against accepting MOM into oVirt, on the contrary, we really want it. The only question is do we want to assimilate the code into vdsm or manage it as a separate project. And the question here for me is just whether this would have other consumers (in which case a separate project could make sense) or would vdsm be the only user (in which case it at most should be a submodule).
I think the best way to start is to begin with the projects as they are today (separate entities). I do think that we will find uses for MOM outside of a VDSM scope. For example, if MOM learns to manage storage and/or networking tunables, we might want to deploy it onto an appliance where there are no virtual machines. -- Adam Litke <agl@us.ibm.com> IBM Linux Technology Center
----- Original Message -----
On Mon, Sep 26, 2011 at 05:28:54PM -0400, Ayal Baron wrote:
----- Original Message -----
On 09/26/2011 04:26 PM, Anthony Liguori wrote:
On 09/26/2011 03:19 PM, Dor Laor wrote:
On 09/26/2011 11:01 PM, Adam Litke wrote:
Hi all,
I would like propose adding the Memory Overcommitment Manager (MOM) project to oVirt. MOM is a daemon that is used to apply dynamic host-level and VM-level tuning according to a customizable policy. Today MOM can control memory ballooning and KSM. Please see the following links for more information:
Project wiki: http://github.com/aglitke/mom/wiki
Source code: git://github.com/aglitke/mom.git
KVM Forum presentation and video: http://www.linux-kvm.org/wiki/images/e/e8/2010-forum-litke-kvmforum2010.pdf
developerWorks article: http://www.ibm.com/developerworks/linux/library/l-overcommit-kvm-resources/
In my view, MOM could be nicely integrated with oVirt-node and VDSM. I am looking forward to your comments and am ready to address some next steps.
Thanks!
+1 for mom, hope to see a dad too :)
Data Acceleration Director?
Joking aside, would MOM just be integrated directly into VDSM or would it remain as a separate project?
I can envision a world where MOM is just a feature of VDSM.
The key question is how do we integrate it into the stack, - is it integrated into WDSM as Anthony mentions - or is a project that WDSM uses to provide MOM - etc.
First of all I think this is great! Thanks Adam! Wrt the question above, the way I see vdsm going forward is a lot more policy management, especially once we start talking about SLAs. This will involve a lot more than just memory policies, and although I may be impartial here, I think that this should fall under vdsm or under a policy engine project (bigger scope than just memory) which vdsm would use. To elaborate, in addition to memory, we need to manage policies around network, storage and cpu and manipulate cgroups, tc, page cache etc. If we take page cache as an example, it affects memory on the machine hence, MOM would like to tweak it (also in the preso), but we already have recent results from the perf team to change the defaults here to significantly increase I/O throughput, improve fairness and reduce effect of buffered writers on VMs and we know that going forward we'll have to tweak these parameters dynamically to adjust to varying workloads, esp if we ever plan on making good the claim for a hybrid server. Having separate projects tweaking the same params will not lead to good results.
Ayal, I definitely agree. I must admit that the project name MOM (while a nice acronym), implies a restricted scope (memory only) but the architecture of MOM does not limit its functionality to memory tuning only. I assert that MOM can easily become the policy engine piece that you are speaking of.
The framework is very extensible (a set of Collectors that expose system state, a set of Controllers that expose "tuning knobs", and a policy that reads the state and manipulates the knobs. Adding control for cgroups is as simple as writing a cgroups Collector and Controller.
Great, so let's rename it and I'm all for it being a subproject (the name also reminds me too much of an M$ thingy). We have a ton of ideas around this. An interesting question though is would you want a separate mailing list? initially at least, I believe that vdsm would be driving the requirements and it would just be a lot easier using the vdsm mailing list for that.
-- Adam Litke <agl@us.ibm.com> IBM Linux Technology Center
On Mon, Sep 26, 2011 at 04:45:34PM -0400, Carl Trieloff wrote:
The key question is how do we integrate it into the stack, - is it integrated into WDSM as Anthony mentions - or is a project that WDSM uses to provide MOM
I like Ayal's suggestion that MOM is a sub-project that provides all integrated dynamic tuning that is required for an oVirt host. This would then be used/started by VDSM and VDSM can control/update the policy being used at any given time. -- Adam Litke <agl@us.ibm.com> IBM Linux Technology Center
participants (9)
-
Adam Litke -
Alexander Graf -
Andrew Cathrow -
Anthony Liguori -
Ayal Baron -
Carl Trieloff -
Dor Laor -
Michael D Day -
Ryan Harper