11:33 a.m.
Roy Golan has submitted this change and it was merged.
Change subject: core: throttle running of VMs (#843058)
......................................................................
core: throttle running of VMs (#843058)
https://bugzilla.redhat.com/show_bug.cgi?id=843058
Bulk running of VMs regulary falls short in running all VMs. The reason is
the VdsSelector counts and reserves memory using:
a) host commited memory: # of VMs * each VM static mem
b) pending VM count memory : # of VMs to run * their memory count ; a shared state
variable
pending VM count is increased by the end of RunVm and decreased by VdsUpdateRunTimeInfo
i.e two independent processes, one as a command thread and the other is a scheduled quartz
job thread.
As long as the VdsUpdateRunTimeInfo doesn't run, theres an overflow in the
calculation
of the free memory to run VM which causes a host to be falsely not selected to run the
VM.
The throttler here is using the shared-state between the VdsUpdateRunTimeInfo and the
RunVmCommands, the
pending VM count and is using a wait/notify semantics to let the VdsUpdateRunTimeInfo
interleave,
if we don't have enough memory run the VM.
- Tests -
setup: 1 Host with 128Gb mem with 120 VMs and cluster 200% over commit
1 Host with 8Gb mem, 3.0 pool and cluster, 10 VMs
2 Host with 4Gb mem, 3.1 pool and cluster, 5 VMs
on the 128Gb host I activated 100 VMs with 3G and 82 of them got online. without the fix I
hit 51 limit.
with no over commit I was able to run also 67 and without the fix also 50
multiple run 10 VMs - prior to the fix the 9th and 10th can't locate a host to run
on.
after the fix all 10 VMs complete the run. In the 9th VM run the selector waits for the
update to decrease memory
and continue after less than a second.
migrate VMs back and forth of 5 VMs with no issues.
VmPool - same scheduling effects.
Change-Id: I076ede6cba919bc61f7546d7b29ef436eb6d3375
Signed-off-by: Roy Golan <rgolan(a)redhat.com>
---
M backend/manager/dbscripts/upgrade/pre_upgrade/0000_config.sql
M
backend/manager/modules/bll/src/main/java/org/ovirt/engine/core/bll/MigrateVmCommand.java
A
backend/manager/modules/bll/src/main/java/org/ovirt/engine/core/bll/NonWaitingDelayer.java
M backend/manager/modules/bll/src/main/java/org/ovirt/engine/core/bll/RunVmCommand.java
M
backend/manager/modules/bll/src/main/java/org/ovirt/engine/core/bll/RunVmCommandBase.java
A backend/manager/modules/bll/src/main/java/org/ovirt/engine/core/bll/RunVmDelayer.java
M
backend/manager/modules/bll/src/main/java/org/ovirt/engine/core/bll/VdsEventListener.java
A
backend/manager/modules/bll/src/main/java/org/ovirt/engine/core/bll/VdsFreeMemoryChecker.java
M backend/manager/modules/bll/src/main/java/org/ovirt/engine/core/bll/VdsSelector.java
M
backend/manager/modules/bll/src/main/java/org/ovirt/engine/core/bll/VmPoolCommandBase.java
M
backend/manager/modules/bll/src/test/java/org/ovirt/engine/core/bll/RunVmCommandTest.java
M
backend/manager/modules/common/src/main/java/org/ovirt/engine/core/common/businessentities/IVdsAsyncCommand.java
M
backend/manager/modules/common/src/main/java/org/ovirt/engine/core/common/config/ConfigValues.java
M
backend/manager/modules/vdsbroker/src/main/java/org/ovirt/engine/core/vdsbroker/VdsManager.java
A
backend/manager/modules/vdsbroker/src/main/java/org/ovirt/engine/core/vdsbroker/VdsMonitor.java
15 files changed, 242 insertions(+), 33 deletions(-)
Approvals:
Roy Golan: Verified; Looks good to me, approved
--
To view, visit http://gerrit.ovirt.org/7204
To unsubscribe, visit http://gerrit.ovirt.org/settings
Gerrit-MessageType: merged
Gerrit-Change-Id: I076ede6cba919bc61f7546d7b29ef436eb6d3375
Gerrit-PatchSet: 10
Gerrit-Project: ovirt-engine
Gerrit-Branch: master
Gerrit-Owner: Roy Golan <rgolan(a)redhat.com>
Gerrit-Reviewer: Allon Mureinik <amureini(a)redhat.com>
Gerrit-Reviewer: Barak Azulay <bazulay(a)redhat.com>
Gerrit-Reviewer: Doron Fediuck <dfediuck(a)redhat.com>
Gerrit-Reviewer: Michael Kublin <mkublin(a)redhat.com>
Gerrit-Reviewer: Moti Asayag <masayag(a)redhat.com>
Gerrit-Reviewer: Omer Frenkel <ofrenkel(a)redhat.com>
Gerrit-Reviewer: Roy Golan <rgolan(a)redhat.com>
Gerrit-Reviewer: Yair Zaslavsky <yzaslavs(a)redhat.com>
Gerrit-Reviewer: oVirt Jenkins CI Server