On 10/18/2012 02:50 PM, Itamar Heim wrote:

On 10/18/2012 02:43 PM, ly pan wrote:

...

Hi:

This is my test environment: hardware: Dell PowerEdge 2710 ,Memory 48G Software: OVirt engine 3.0 ,VDSM 4.9.4 ,kernel 2.6.32-279.2.1.el6.x86_64

I create 100 vms from pool(each vm has 512M memory and has 1M memory garanteed, with guest_overhead = 65 and reserved_mem = 256), but only 83 vms turn out to run. When I run the 84th vm, engine says there is not enough memory. However at this time KSM is enabled and 15G free memory is still left on the host.

I checked the code and found out that before running a vm, the function hasMemoryToRunVM() in RunVmCommandBase would decide whether there is enough memory in the selected host to run it. The Algorithm in the function is like this: mem_avaliable >= mem_required = curVds.getmem_commited() + curVds.getpending_vmem_size() + curVds.getguest_overhead() + curVds.getreserved_mem() + vm.getMinAllocatedMem();

And here is my question: curVds.getmem_commited() is caculated from the memory of vm

iirc, getmem_committed is the amount of memory actually used on the host, as vdsm reports. its actually the sum of static memory per running VM on the host(as mentioned below)

...

assigned by user. But when the host is running with KSM enabled, the actual memory of each running vm would be smaller than the memory the user assigned to. In this case, the following may occur: engine says there be no more memroy to run any vm while the host has much free space to run more vms(in my case, 15G).

Therefore I think the actual memory size reported by vdsm(The KSM shared memory) should be taken into account when calculating curVds.getmem_commited(). Any ideas are welcome.

in short we rather depend on static resource allocation - for scheduling new VMs we have to guarantee the amount of memory defined for them is reserved. Free Memory by KSM is subjected to reallocation - say you updated one of the VMs OS, the pages are no longer equal and more memory is allocated for them.

_______________________________________________ Engine-devel mailing list Engine-devel@ovirt.org http://lists.ovirt.org/mailman/listinfo/engine-devel

Hi Doron, Sorry for the so late reply. These days I looked into the code again and found it's my setting not rightly configured. 1. You opinion about reserved_mem is right. 2. Yes, pending_vmem_size is 0 before the 84th runs. 3. memory_over_commit is where my problem is. In webadmin UI, I set the cluster's memory optimization to none, thus memory_over_commit is 100 in DB(the dafault value is 100 in my env). when I set the cluster's mem_opt to 'for server load', memory_over_commit grows to 150 and when I set to 'for desktop load', memory_over_commit grows to 200. Both 'for server load' and 'for desktop load' have let the vdsMemLimit grows dramaticaly, thus the 84th vm managed to run. So I realized the memory optimization option in cluster is for KSM function. Thank you. ly pan 2012/10/24 Doron Fediuck <dfediuck@redhat.com>:

----- Original Message -----

...

From: "ly pan" <plysab@gmail.com> To: engine-devel@ovirt.org Sent: Thursday, October 18, 2012 2:43:35 PM Subject: [Engine-devel] Question about hasMemoryToRunVM() in RunVmCommandBase

Hi:

This is my test environment: hardware: Dell PowerEdge 2710 ,Memory 48G Software: OVirt engine 3.0 ,VDSM 4.9.4 ,kernel 2.6.32-279.2.1.el6.x86_64

I create 100 vms from pool(each vm has 512M memory and has 1M memory garanteed, with guest_overhead = 65 and reserved_mem = 256), but only 83 vms turn out to run. When I run the 84th vm, engine says there is not enough memory. However at this time KSM is enabled and 15G free memory is still left on the host.

I checked the code and found out that before running a vm, the function hasMemoryToRunVM() in RunVmCommandBase would decide whether there is enough memory in the selected host to run it. The Algorithm in the function is like this: mem_avaliable >= mem_required = curVds.getmem_commited() + curVds.getpending_vmem_size() + curVds.getguest_overhead() + curVds.getreserved_mem() + vm.getMinAllocatedMem();

And here is my question: curVds.getmem_commited() is caculated from the memory of vm assigned by user. But when the host is running with KSM enabled, the actual memory of each running vm would be smaller than the memory the user assigned to. In this case, the following may occur: engine says there be no more memroy to run any vm while the host has much free space to run more vms(in my case, 15G).

Therefore I think the actual memory size reported by vdsm(The KSM shared memory) should be taken into account when calculating curVds.getmem_commited(). Any ideas are welcome.

Hi Ly, Sorry for the delay. I'd like to look into this issue. In the meanwhile a few relevant points;

1. You're saying that reserved_mem=256. This is not accurate, since it's reported by vds as host_mem_reserve(256) + extra_mem_reserve(65) = 321

2. I'm assuming you retried running the 84th vm, just to make sure your pending_vmem_size is 0 once all 83 VMs are running.

3. So assuming (2) is correct, I'd like to ask what's the max_vds_memory_over_commit you have for the cluster? By default it's set to 120, and in such a case the 84th VM should still be running.

Here's a calculation sample based on your data, and the above 3 issues:

vdsCurrentMem = 83 * (512M+65) = 47,891 0 65 321 1 ======= 48,278

vdsMemLimit = 120/100 * 47,321.1328125 // reported by vdsm: 'cat /proc/meminfo | grep MemTotal' / 1024 ========= 56,785.359375

So vdsCurrentMem <= vdsMemLimit should be true unless you changed something such as the MemTotal or the memory_over_commit. You can try and change memory_over_commit let us know how it works.

Doron