Hi: This is my test environment: hardware: Dell PowerEdge 2710 ,Memory 48G Software: OVirt engine 3.0 ,VDSM 4.9.4 ,kernel 2.6.32-279.2.1.el6.x86_64 I create 100 vms from pool(each vm has 512M memory and has 1M memory garanteed, with guest_overhead = 65 and reserved_mem = 256), but only 83 vms turn out to run. When I run the 84th vm, engine says there is not enough memory. However at this time KSM is enabled and 15G free memory is still left on the host. I checked the code and found out that before running a vm, the function hasMemoryToRunVM() in RunVmCommandBase would decide whether there is enough memory in the selected host to run it. The Algorithm in the function is like this: mem_avaliable >= mem_required = curVds.getmem_commited() + curVds.getpending_vmem_size() + curVds.getguest_overhead() + curVds.getreserved_mem() + vm.getMinAllocatedMem(); And here is my question: curVds.getmem_commited() is caculated from the memory of vm

assigned by user. But when the host is running with KSM enabled, the actual memory of each running vm would be smaller than the memory the user assigned to. In this case, the following may occur: engine says there be no more memroy to run any vm while the host has much free space to run more vms(in my case, 15G). Therefore I think the actual memory size reported by vdsm(The KSM shared memory) should be taken into account when calculating curVds.getmem_commited(). Any ideas are welcome.

From: "ly pan" <plysab(a)gmail.com&gt; To: engine-devel(a)ovirt.org Sent: Thursday, October 18, 2012 2:43:35 PM Subject: [Engine-devel] Question about hasMemoryToRunVM() in RunVmCommandBase Hi: This is my test environment: hardware: Dell PowerEdge 2710 ,Memory 48G Software: OVirt engine 3.0 ,VDSM 4.9.4 ,kernel 2.6.32-279.2.1.el6.x86_64 I create 100 vms from pool(each vm has 512M memory and has 1M memory garanteed, with guest_overhead = 65 and reserved_mem = 256), but only 83 vms turn out to run. When I run the 84th vm, engine says there is not enough memory. However at this time KSM is enabled and 15G free memory is still left on the host. I checked the code and found out that before running a vm, the function hasMemoryToRunVM() in RunVmCommandBase would decide whether there is enough memory in the selected host to run it. The Algorithm in the function is like this: mem_avaliable >= mem_required = curVds.getmem_commited() + curVds.getpending_vmem_size() + curVds.getguest_overhead() + curVds.getreserved_mem() + vm.getMinAllocatedMem(); And here is my question: curVds.getmem_commited() is caculated from the memory of vm assigned by user. But when the host is running with KSM enabled, the actual memory of each running vm would be smaller than the memory the user assigned to. In this case, the following may occur: engine says there be no more memroy to run any vm while the host has much free space to run more vms(in my case, 15G). Therefore I think the actual memory size reported by vdsm(The KSM shared memory) should be taken into account when calculating curVds.getmem_commited(). Any ideas are welcome.

From: "ly pan" <plysab(a)gmail.com&gt; To: "Doron Fediuck" <dfediuck(a)redhat.com&gt; Cc: engine-devel(a)ovirt.org Sent: Wednesday, October 31, 2012 12:32:48 PM Subject: Re: [Engine-devel] Question about hasMemoryToRunVM() in RunVmCommandBase Hi Doron, Sorry for the so late reply. These days I looked into the code again and found it's my setting not rightly configured. 1. You opinion about reserved_mem is right. 2. Yes, pending_vmem_size is 0 before the 84th runs. 3. memory_over_commit is where my problem is. In webadmin UI, I set the cluster's memory optimization to none, thus memory_over_commit is 100 in DB(the dafault value is 100 in my env). when I set the cluster's mem_opt to 'for server load', memory_over_commit grows to 150 and when I set to 'for desktop load', memory_over_commit grows to 200. Both 'for server load' and 'for desktop load' have let the vdsMemLimit grows dramaticaly, thus the 84th vm managed to run. So I realized the memory optimization option in cluster is for KSM function. Thank you. ly pan 2012/10/24 Doron Fediuck <dfediuck(a)redhat.com&gt;: > ----- Original Message ----- >> From: "ly pan" <plysab(a)gmail.com&gt; >> To: engine-devel(a)ovirt.org >> Sent: Thursday, October 18, 2012 2:43:35 PM >> Subject: [Engine-devel] Question about hasMemoryToRunVM() in >> RunVmCommandBase >> >> Hi: >> >> This is my test environment: >> hardware: Dell PowerEdge 2710 ,Memory 48G >> Software: OVirt engine 3.0 ,VDSM 4.9.4 ,kernel >> 2.6.32-279.2.1.el6.x86_64 >> >> >> I create 100 vms from pool(each vm has 512M memory and has >> 1M memory garanteed, with guest_overhead = 65 and reserved_mem = >> 256), >> but only 83 vms >> turn out to run. When I run the 84th vm, engine says there is not >> enough memory. However at this time KSM is enabled and 15G free >> memory >> is still left on the host. >> >> I checked the code and found out that before running a vm, the >> function hasMemoryToRunVM() in RunVmCommandBase would decide >> whether >> there is enough memory in the selected host to run it. >> The Algorithm in the function is like this: >> mem_avaliable >= mem_required = curVds.getmem_commited() + >> curVds.getpending_vmem_size() >> + curVds.getguest_overhead() + curVds.getreserved_mem() + >> vm.getMinAllocatedMem(); >> >> And here is my question: curVds.getmem_commited() is caculated >> from >> the memory of vm >> assigned by user. But when the host is running with KSM enabled, >> the >> actual memory of each >> running vm would be smaller than the memory the user assigned to. >> In >> this case, the following >> may occur: engine says there be no more memroy to run any vm while >> the >> host has much free >> space to run more vms(in my case, 15G). >> >> >> Therefore I think the actual memory size reported by vdsm(The KSM >> shared memory) should be taken into account when calculating >> curVds.getmem_commited(). Any ideas are welcome. > > Hi Ly, > Sorry for the delay. I'd like to look into this issue. > In the meanwhile a few relevant points; > > 1. You're saying that reserved_mem=256. This is not accurate, since > it's > reported by vds as host_mem_reserve(256) + extra_mem_reserve(65) = > 321 > > 2. I'm assuming you retried running the 84th vm, just to make sure > your > pending_vmem_size is 0 once all 83 VMs are running. > > 3. So assuming (2) is correct, I'd like to ask what's the > max_vds_memory_over_commit > you have for the cluster? By default it's set to 120, and in such a > case the 84th > VM should still be running. > > Here's a calculation sample based on your data, and the above 3 > issues: > > vdsCurrentMem = > 83 * (512M+65) = 47,891 > 0 > 65 > 321 > 1 > ======= > 48,278 > > vdsMemLimit = > 120/100 > * > 47,321.1328125 // reported by vdsm: 'cat /proc/meminfo | grep > MemTotal' / 1024 > ========= > 56,785.359375 > > So vdsCurrentMem <= vdsMemLimit should be true unless you changed > something > such as the MemTotal or the memory_over_commit. You can try and > change memory_over_commit > let us know how it works. > > Doron _______________________________________________ Engine-devel mailing list Engine-devel(a)ovirt.org http://lists.ovirt.org/mailman/listinfo/engine-devel