[SCALE][RFC] ksmd bound to one core
Hi, currently ksmd is a single process and is thus bound to one core. This leads to some scaling problems such as: If you got a lot of vms on one host with huge amounts of ram you can observe that the cpu usage by ksmd goes easily to 100%. I wonder if ksmd could not be split up in child/worker threads, thus enabling higher density of vms on one host. or can this just be tweaked by altering values in /etc/ksmtuned.conf ? What do you think? -- Mit freundlichen Grüßen / Regards Sven Kieske Systemadministrator Mittwald CM Service GmbH & Co. KG Königsberger Straße 6 32339 Espelkamp T: +49-5772-293-100 F: +49-5772-293-333 https://www.mittwald.de Geschäftsführer: Robert Meyer St.Nr.: 331/5721/1033, USt-IdNr.: DE814773217, HRA 6640, AG Bad Oeynhausen Komplementärin: Robert Meyer Verwaltungs GmbH, HRB 13260, AG Bad Oeynhausen
On 04/11/14 13:11 +0100, Sven Kieske wrote:
Hi,
currently ksmd is a single process and is thus bound to one core.
This leads to some scaling problems such as:
If you got a lot of vms on one host with huge amounts of ram you can observe that the cpu usage by ksmd goes easily to 100%.
I wonder if ksmd could not be split up in child/worker threads, thus enabling higher density of vms on one host.
It's likely going to be trickier than you imagine with the added locking that would be required to synchronize the ksmd threads.
or can this just be tweaked by altering values in /etc/ksmtuned.conf ?
I don't think it can.
What do you think?
Interesting idea. -- Adam Litke
On 05.11.2014 21:07, Adam Litke wrote:
On 04/11/14 13:11 +0100, Sven Kieske wrote:
Hi,
currently ksmd is a single process and is thus bound to one core.
This leads to some scaling problems such as:
If you got a lot of vms on one host with huge amounts of ram you can observe that the cpu usage by ksmd goes easily to 100%. I wonder what would be the benefit here... I think spending CPU cycles on something like memory compression is not what (most) users would do. Already I think this is an annoyance; maybe thats why the process is niced to +5. A multi process daemon would require careful confining (with cgroups). To my understanding the for KSM to really work well is to have many (idle / high mem) guests which are quite similar?
I wonder if ksmd could not be split up in child/worker threads, thus enabling higher density of vms on one host.
It's likely going to be trickier than you imagine with the added locking that would be required to synchronize the ksmd threads. Actually I was refraining to answer Svens question since I lack some knowledge here. But I guessed synchronization was the reason for the single threadted design. Some serous work needs to be done on KSM do achieve this goal. I was thinking about some map/reduce aglo to do that... OTOH mangeling with mem pages of guests very requires special care.
or can this just be tweaked by altering values in /etc/ksmtuned.conf ?
I don't think it can.
What do you think?
Interesting idea.
-- Daniel Helgenberger m box bewegtbild GmbH P: +49/30/2408781-22 F: +49/30/2408781-10 ACKERSTR. 19 D-10115 BERLIN www.m-box.de www.monkeymen.tv Geschäftsführer: Martin Retschitzegger / Michaela Göllner Handeslregister: Amtsgericht Charlottenburg / HRB 112767
On Nov 5, 2014, at 23:29 , Daniel Helgenberger <daniel.helgenberger@m-box.de> wrote:
On 05.11.2014 21:07, Adam Litke wrote:
On 04/11/14 13:11 +0100, Sven Kieske wrote:
Hi,
currently ksmd is a single process and is thus bound to one core.
This leads to some scaling problems such as:
If you got a lot of vms on one host with huge amounts of ram you can observe that the cpu usage by ksmd goes easily to 100%. I wonder what would be the benefit here... I think spending CPU cycles on something like memory compression is not what (most) users would do.
Indeed. I would say it is more beneficial to spend time on better analysis of what regions of memory you want to analyze. Instead of this brute force approach trying to scan the whole hosts memory aggressively. E.g. We don't even have ksm per VM yet (but it's in the plan). Even that would help a lot as you can then enable it only where it makes sense (pool deployment with many same/idle VMs) Thanks, michal
Already I think this is an annoyance; maybe thats why the process is niced to +5. A multi process daemon would require careful confining (with cgroups). To my understanding the for KSM to really work well is to have many (idle / high mem) guests which are quite similar?
I wonder if ksmd could not be split up in child/worker threads, thus enabling higher density of vms on one host.
It's likely going to be trickier than you imagine with the added locking that would be required to synchronize the ksmd threads. Actually I was refraining to answer Svens question since I lack some knowledge here. But I guessed synchronization was the reason for the single threadted design. Some serous work needs to be done on KSM do achieve this goal. I was thinking about some map/reduce aglo to do that... OTOH mangeling with mem pages of guests very requires special care.
or can this just be tweaked by altering values in /etc/ksmtuned.conf ?
I don't think it can.
What do you think?
Interesting idea.
-- Daniel Helgenberger m box bewegtbild GmbH
P: +49/30/2408781-22 F: +49/30/2408781-10
ACKERSTR. 19 D-10115 BERLIN
www.m-box.de www.monkeymen.tv
Geschäftsführer: Martin Retschitzegger / Michaela Göllner Handeslregister: Amtsgericht Charlottenburg / HRB 112767 _______________________________________________ Devel mailing list Devel@ovirt.org http://lists.ovirt.org/mailman/listinfo/devel
participants (4)
-
Adam Litke -
Daniel Helgenberger -
Michal Skrivanek -
Sven Kieske