10:39 p.m.
Hello,
Not sure if it was already discussed, but I'd like to talk about oVirt
"Shared Memory".
Webadmin has the Shared Memory percent information in host general tab
[1]. Initially, I thought that Shared Memory was the KSM de-duplication
results. But comparing with my KSM stats, it does not make sense. My env:
3,5 GB virt-host.
6 identical VMs running with 1GB RAM each.
Webadmin host details:
Memory Sharing: Active
Shared Memory: 0%
KSM - How many shared pages are being used:
$ cat /sys/kernel/mm/ksm/pages_sharing
109056
KSM - How many more sites are sharing them i.e. how much saved
$ cat /sys/kernel/mm/ksm/pages_sharing
560128
Converting to Mbytes:
$ echo $(( (109056 * $(getconf PAGE_SIZE)) / (1024 * 1024) ))
426
$ echo $(( ( 560128 * $(getconf PAGE_SIZE)) / (1024 * 1024) ))
2188
With those KSM results, I could expect something but 0 in "Shared Memory".
Tracing the origin of "Shared Memory" in oVirt code, I realized it's
coming from memShared value (from getVdsStats vdsm command), which is
provided in Mbytes:
$ vdsClient -s 192.168.10.250 getVdsStats | grep memShared
memShared = 9
Finding memShared function in vdsm, we have:
$VDSM_ROOT/vdsm/API.py
...
stats['memShared'] = self._memShared() / Mbytes
...
def _memShared(self):
"""
Return an approximation of memory shared by VMs thanks to KSM.
"""
shared = 0
for v in self._cif.vmContainer.values():
if v.conf['pid'] == '0':
continue
try:
statmfile = file('/proc/' + v.conf['pid'] +
'/statm')
shared += int(statmfile.read().split()[2]) * PAGE_SIZE_BYTES
except:
pass
return shared
...
memShared is calculated adding the shared pages value (3rd field) from
/proc/<VM_PID>/statm file from all running VMs, converting to bytes
through PAGE_SIZE value and transforming to Mbytes at the end. This
field in statm file currently is (it changed along kernel history) the
count of pages instantiated in the process address space which are
shared with a file, including executable, library or shared memory.
Despite vdsm code comment, KSM shared pages are not accounted here. KSM
works de-duplicating and sharing memory pages without process awareness.
Engine is calculating the percent considering total physical memory -
memSize value from getVdsCapabilities vdsm command:
$ vdsClient -s 192.168.10.250 getVdsCapabilities | grep memSize
memSize = 3574
Calculating the percent:
$ echo "scale=2; 9 * 100 / 3574" | bc
.25
So, we have arround 0,25%, rounded to 0%. "Shared Memory" is coherent,
but not reflecting the real page deduplication benefits. And unnoticed
administrators - me included - are led to think that Shared Memory is
related with KSM results.
IMO, memShared is not providing any representative information. On the
other hand, the missing KSM results are really important to oVirt
administrators, providing information about how much memory they are
over committed by, for capacity management, and how much money oVirt is
saving in memory.
In order to offer those KSM stats to engine, I sent a patch [2] (waiting
approval) adding "ksmShared" and "ksmSharing" values to vdsm
getVdsStats
command in a standard way, with key names that fits with existing KSM
ones (ksmCpu, ksmPages and ksmState).
Before patch:
$ vdsClient -s 192.168.10.250 getVdsStats | grep ksm
ksmCpu = 1 -> the ksmd process cpu load
ksmPages = 664 -> pages to scan before ksmd goes to sleep
ksmState = True -> is ksm running?
With the patch:
$ vdsClient -s 192.168.10.250 getVdsStats | grep ksm
ksmCpu = 1
ksmPages = 664
ksmShared = 426 -> how many Mbytes of memory are being shared
ksmSharing = 2188 -> how many more sites are sharing them
i.e. how much Mbytes are being saved
ksmState = True
Finally my questions:
1 - Is sharedMem value (from /proc/PID/statm) that significant to be
kept in host details? If yes, why?
2 - What about - and how (%, Mb, ...) - to add the vdsm new ksmShared
and ksmSharing stats in host details?
Sorry about my long history. I look forward to hearing your comments.
All the best,
--
Pahim
[1] -
http://www.pahim.org/wp-content/uploads/2012/05/Screenshot-oVirt-Enterpri...
[2] - http://gerrit.ovirt.org/4755
