Adding Memory Overcommitment Manager (MOM) to oVirt

[Ayal Baron]

----- Original Message -----
> On Mon, Sep 26, 2011 at 05:28:54PM -0400, Ayal Baron wrote:
> > 
> > 
> > ----- Original Message -----
> > > On 09/26/2011 04:26 PM, Anthony Liguori wrote:
> > > > On 09/26/2011 03:19 PM, Dor Laor wrote:
> > > >> On 09/26/2011 11:01 PM, Adam Litke wrote:
> > > >>> Hi all,
> > > >>>
> > > >>> I would like propose adding the Memory Overcommitment Manager
> > > >>> (MOM)
> > > >>> project to
> > > >>> oVirt. MOM is a daemon that is used to apply dynamic
> > > >>> host-level
> > > >>> and
> > > >>> VM-level
> > > >>> tuning according to a customizable policy. Today MOM can
> > > >>> control
> > > >>> memory
> > > >>> ballooning and KSM. Please see the following links for more
> > > >>> information:
> > > >>>
> > > >>> Project wiki:
> > > >>> http://github.com/aglitke/mom/wiki
> > > >>>
> > > >>> Source code:
> > > >>> git://github.com/aglitke/mom.git
> > > >>>
> > > >>> KVM Forum presentation and video:
> > > >>> http://www.linux-kvm.org/wiki/images/e/e8/2010-forum-litke-kvmforum2010.pdf
> > > >>>
> > > >>> http://vimeo.com/15223652
> > > >>>
> > > >>> developerWorks article:
> > > >>> http://www.ibm.com/developerworks/linux/library/l-overcommit-kvm-resources/
> > > >>>
> > > >>>
> > > >>> In my view, MOM could be nicely integrated with oVirt-node
> > > >>> and
> > > >>> VDSM.
> > > >>> I am
> > > >>> looking forward to your comments and am ready to address some
> > > >>> next
> > > >>> steps.
> > > >>>
> > > >>> Thanks!
> > > >>>
> > > >>
> > > >> +1 for mom, hope to see a dad too :)
> > > >
> > > > Data Acceleration Director?
> > > >
> > > > Joking aside, would MOM just be integrated directly into VDSM
> > > > or
> > > > would
> > > > it remain as a separate project?
> > > >
> > > > I can envision a world where MOM is just a feature of VDSM.
> > > 
> > > The key question is how do we integrate it into the stack,
> > >  - is it integrated into WDSM as Anthony mentions
> > >  - or is a project that WDSM uses to provide MOM
> > >  - etc.
> > 
> > First of all I think this is great!  Thanks Adam!
> > Wrt the question above, the way I see vdsm going forward is a lot
> > more policy
> > management, especially once we start talking about SLAs. This will
> > involve a
> > lot more than just memory policies, and although I may be impartial
> > here, I
> > think that this should fall under vdsm or under a policy engine
> > project
> > (bigger scope than just memory) which vdsm would use.  To
> > elaborate, in
> > addition to memory, we need to manage policies around network,
> > storage and cpu
> > and manipulate cgroups, tc, page cache etc. If we take page cache
> > as an
> > example, it affects memory on the machine hence, MOM would like to
> > tweak it
> > (also in the preso), but we already have recent results from the
> > perf team to
> > change the defaults here to significantly increase I/O throughput,
> > improve
> > fairness and reduce effect of buffered writers on VMs and we know
> > that going
> > forward we'll have to tweak these parameters dynamically to adjust
> > to varying
> > workloads, esp if we ever plan on making good the claim for a
> > hybrid server.
> > Having separate projects tweaking the same params will not lead to
> > good
> > results.
> 
> Ayal, I definitely agree.  I must admit that the project name MOM
> (while a nice
> acronym), implies a restricted scope (memory only) but the
> architecture of MOM
> does not limit its functionality to memory tuning only.  I assert
> that MOM can
> easily become the policy engine piece that you are speaking of.
> 
> The framework is very extensible (a set of Collectors that expose
> system state,
> a set of Controllers that expose "tuning knobs", and a policy that
> reads
> the state and manipulates the knobs.  Adding control for cgroups is
> as simple as
> writing a cgroups Collector and Controller.

Great, so let's rename it and I'm all for it being a subproject (the name also reminds me too much of an M$ thingy).
We have a ton of ideas around this.
An interesting question though is would you want a separate mailing list? initially at least, I believe that vdsm would be driving the requirements and it would just be a lot easier using the vdsm mailing list for that.

> 
> --
> Adam Litke <agl at us.ibm.com>
> IBM Linux Technology Center
> 
>